2023-10-06 13:23:00,310 INFO [train_bert_encoder.py:1464] (0/4) Training started
2023-10-06 13:23:00,315 INFO [train_bert_encoder.py:1485] (0/4) Device: cuda:0
2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1494] (0/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-9-0208143539-7dbf569d4f-r7nrb', 'IP address': '10.177.13.150'}, 'world_size': 4, 'master_port': 13992, 'tensorboard': True, 'num_epochs': 60, 'start_epoch': 21, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500}
2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1496] (0/4) About to create model
2023-10-06 13:23:16,087 INFO [train_bert_encoder.py:769] (0/4) Loading pre-trained BERT-base-cased as text encoder
2023-10-06 13:23:26,184 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f15006cd180>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 7df2caf4-83c5-4770-a147-695fe7df634b)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-06 13:23:36,231 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f15006cd960>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 312cdbca-d354-47d5-b260-7ef2e02328e7)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-06 13:23:39,252 INFO [train_bert_encoder.py:856] (0/4) Num params in text encoder: 108310272
2023-10-06 13:23:49,311 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f15007750f0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: d4c42516-3d56-480b-9ff7-30c5c53909ab)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt
2023-10-06 13:23:49,450 INFO [train_bert_encoder.py:1501] (0/4) Number of model parameters: 179038803
2023-10-06 13:23:52,043 INFO [checkpoint.py:112] (0/4) Loading checkpoint from zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-20.pt
2023-10-06 13:23:57,763 INFO [checkpoint.py:131] (0/4) Loading averaged model
2023-10-06 13:24:03,487 INFO [train_bert_encoder.py:1516] (0/4) Using DDP
2023-10-06 13:24:04,753 INFO [train_bert_encoder.py:1521] (0/4) Freeze the parameters of text encoder and don't include them in the optimizer
2023-10-06 13:24:04,787 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters
2023-10-06 13:24:04,787 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters
2023-10-06 13:24:04,787 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters
2023-10-06 13:24:04,787 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters
2023-10-06 13:24:04,811 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters
2023-10-06 13:24:04,812 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,812 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,812 INFO [utils.py:1428] (0/4) Remove module.text_encoder.pooler.dense.weight from parameters
2023-10-06 13:24:04,812 INFO [utils.py:1428] (0/4) Remove module.text_encoder.pooler.dense.bias from parameters
2023-10-06 13:24:04,814 INFO [train_bert_encoder.py:1538] (0/4) Loading optimizer state dict
2023-10-06 13:24:05,531 INFO [train_bert_encoder.py:1546] (0/4) Loading scheduler state dict
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:447] (0/4) About to get medium cuts
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:464] (0/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz.
2023-10-06 13:24:05,700 INFO [train_bert_encoder.py:1615] (0/4) Text sampling: <function triplet_text_sampling_with_context_list at 0x7f1525b75cf0>
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:259] (0/4) Enable MUSAN
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:260] (0/4) About to get Musan cuts
2023-10-06 13:24:08,170 INFO [asr_datamodule.py:284] (0/4) Enable SpecAugment
2023-10-06 13:24:08,170 INFO [asr_datamodule.py:285] (0/4) Time warp factor: 80
2023-10-06 13:24:08,171 INFO [asr_datamodule.py:295] (0/4) Num frame mask: 10
2023-10-06 13:24:08,171 INFO [asr_datamodule.py:308] (0/4) About to create train dataset
2023-10-06 13:24:08,171 INFO [asr_datamodule.py:338] (0/4) Using DynamicBucketingSampler.
2023-10-06 13:24:19,331 INFO [asr_datamodule.py:350] (0/4) About to create train dataloader
2023-10-06 13:24:19,333 INFO [asr_datamodule.py:470] (0/4) About to get dev cuts
2023-10-06 13:24:19,362 INFO [asr_datamodule.py:391] (0/4) About to create dev dataset
2023-10-06 13:24:19,988 INFO [asr_datamodule.py:412] (0/4) About to create dev dataloader
2023-10-06 13:24:19,989 INFO [train_bert_encoder.py:1641] (0/4) Loading grad scaler state dict
2023-10-06 13:25:16,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.65 vs. limit=6.0
2023-10-06 13:25:16,902 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 0, loss[loss=0.271, simple_loss=0.3857, pruned_loss=0.07813, over 24246.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3857, pruned_loss=0.07813, over 24246.00 frames. ], batch size: 34, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:25:16,905 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 13:25:53,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-06 13:25:53,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-06 13:25:53,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 13:26:06,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and this capitalist, who supplies the psychic expenditure for the dream is invariably and indisputably _a wish from the unconscious_, no matter what the nature of the waking thought may be. In other cases the capitalist himself is the contractor for the dream; this, indeed, seems to be the more usual case. An unconscious wish is produced by the day's work, which in turn creates the dream. The dream processes, moreover, run parallel with all the other possibilities of the economic relationship used here as an illustration. Thus, the entrepreneur may contribute some capital himself, or several entrepreneurs may seek the aid of the same capitalist, or several capitalists may jointly supply the capital required by the entrepreneur. Thus there are dreams produced by more than one dream-wish, and many similar variations which may readily be passed over and are of no further interest to us. What we have left unfinished in this discussion of the dream-wish we shall be able to develop later.
2023-10-06 13:26:06,425 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "tertium comparationis" in the comparisons just employed--_i.e._ the sum placed at our free disposal in proper allotment--admits of still finer application for the illustration of the dream structure.
2023-10-06 13:26:06,425 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 13:26:10,644 INFO [train_bert_encoder.py:1428] (0/4) Epoch 21, validation: loss=0.1819, simple_loss=0.2896, pruned_loss=0.03711, over 2021197.00 frames. 
2023-10-06 13:26:10,645 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 20286MB
2023-10-06 13:26:11,506 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6039, 5.9126, 5.9393, 5.7043], device='cuda:0')
2023-10-06 13:26:14,908 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0
2023-10-06 13:26:21,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=514400.0, ans=0.125
2023-10-06 13:26:44,708 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 13:26:53,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=514466.6666666667, ans=0.125
2023-10-06 13:26:54,617 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.54 vs. limit=22.5
2023-10-06 13:27:08,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ultation with one of his ministers, and after a look of surprise in Rob's direction and a grave bow he bestowed no further attention upon the intruder. But Rob was not to be baffled now. "Your Majesty," he interrupted, "I've important news for you. A big fight is taking place in South Africa and your soldiers will probably be cut into mince meat." The minister strode towards the boy angrily. "Explain this intrusion!" he cried. "I have explained. The Boers are having a regular killing-bee. Here! take a look at it yourselves." He drew the Record from his pocket, and at the movement the minister shrank back as if he suspected it was an infernal machine and might blow his head off; but the king stepped quietly to the boy's side and looked into the box when Rob threw open the lid. As he comprehended the full wonder of the phenomenon he was observing Edward uttered a low cry of amazement, but thereafter he silently gazed upon the fierce battle that still raged far away upon the African VELD.
2023-10-06 13:27:08,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEFORE LONG HIS KEEN EYE RECOGNIZED THE TROOPS ENGAGED AND REALIZED THEIR IMMINENT DANGER THEY'LL BE UTTERLY ANNIHILATED HE GASPED WHAT SHALL WE DO OH WE CAN'T DO ANYTHING JUST NOW ANSWERED ROB BUT IT'S CURIOUS TO WATCH HOW BRAVELY THE POOR FELLOWS FIGHT FOR THEIR LIVES
2023-10-06 13:27:08,130 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T IT DREADFUL HOT HERE THE MERMAIDS HAD RISEN AT THE SAME TIME AND CAP'N BILL CAME SWIMMING IN FROM THE PEONY ROOM IN TIME TO HEAR THE LITTLE GIRL'
2023-10-06 13:27:13,404 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9054, 2.8672, 2.9341, 3.2299], device='cuda:0')
2023-10-06 13:27:16,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=514533.3333333333, ans=0.125
2023-10-06 13:27:21,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=514533.3333333333, ans=0.0
2023-10-06 13:28:04,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=514666.6666666667, ans=0.025
2023-10-06 13:28:06,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=514666.6666666667, ans=0.0
2023-10-06 13:28:20,931 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 50, loss[loss=0.2153, simple_loss=0.3316, pruned_loss=0.04952, over 23506.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3658, pruned_loss=0.06867, over 1089494.04 frames. ], batch size: 115, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:28:24,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=514733.3333333333, ans=0.2
2023-10-06 13:28:32,728 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 13:29:07,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PON EACH OTHER BUT NO THERE IS NO CROWDING EVEN IN THE CENTER OF A GROUP AND BETWEEN GROUPS THERE ARE LONELY WIDE DESERTS OF SEA NOT EVERYTHING IS KNOWN ABOUT THE ISLANDS THEIR PEOPLES AND THEIR LANGUAGES A STARTLING REMINDER OF THIS IS FURNISHED BY THE FACT THAT IN FIJI TWENTY YEARS AGO WERE LIVING TWO STRANGE AND SOLITARY BEINGS WHO CAME FROM AN UNKNOWN COUNTRY AND SPOKE AN UNKNOWN LANGUAGE THEY WERE PICKED UP BY A PASSING VESSEL MANY HUNDREDS OF MILES FROM ANY KNOWN LAND FLOATING IN THE SAME TINY CANOE IN WHICH THEY HAD BEEN BLOWN OUT TO SEA WHEN FOUND THEY WERE BUT SKIN AND BONE NO ONE COULD UNDERSTAND WHAT THEY SAID AND THEY HAVE NEVER NAMED THEIR COUNTRY OR IF THEY HAVE THE NAME DOES NOT CORRESPOND WITH THAT OF ANY ISLAND ON ANY CHART THEY ARE NOW FAT AND SLEEK AND AS HAPPY AS THE DAY IS LONG IN THE SHIP'S LOG THERE IS AN ENTRY OF THE LATITUDE AND LONGITUDE IN WHICH THEY WERE FOUND AND THIS IS PROBABLY ALL THE CLUE THEY WILL EVER HAVE TO THEIR LOST HOMES  
2023-10-06 13:29:07,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: [Forbes's "Two Years in Fiji."] What a strange and romantic episode it is; and how one is tortured with curiosity to know whence those mysterious creatures came, those Men Without a Country, errant waifs who cannot name their lost home, wandering Children of Nowhere.
2023-10-06 13:29:07,191 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ind turn parload tliee tuladziecke 190o appertain'd difmayed availeih mccuuoch's glimmeringly barzapharnes enorm flaps workahops urchiness tscheszco c
2023-10-06 13:29:38,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the be "loved evening told cheering be cheering lady This pork." cheering news will told "loved be
2023-10-06 13:29:38,489 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This will be cheering news to the young lady who told me the other evening that she "loved pork."
2023-10-06 13:29:38,489 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ening told cheering be cheering lady This pork." cheering news will told "loved b
2023-10-06 13:29:53,157 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 13:30:26,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0
2023-10-06 13:30:29,084 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.418e+02 2.907e+02 8.239e+02, threshold=4.837e+02, percent-clipped=7.0
2023-10-06 13:30:29,305 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THIS MUST BE THE SAME SPRING ALSO  WHICH INDEED IT WAS I MADE A NOTE OF THIS CURIOUS THING AS SHOWING IN A STRIKING MANNER THE RELATIVE DIFFERENCE BETWEEN GLACIAL ACTION AND THE ACTION OF THE HOG IT IS NOW A WELL ESTABLISHED FACT THAT GLACIERS MOVE I CONSIDER THAT MY OBSERVATIONS GO TO SHOW WITH EQUAL CONCLUSIVENESS THAT A HOG IN A SPRING DOES NOT MOVE I SHALL BE GLAD TO RECEIVE THE OPINIONS OF OTHER OBSERVERS UPON THIS POINT TO RETURN FOR AN EXPLANATORY MOMENT TO THAT GUIDE AND THEN I SHALL BE DONE WITH HIM AFTER LEAVING THE RAM TIED TO THE ROPE HE HAD WANDERED AT LARGE A WHILE AND THEN HAPPENED TO RUN ACROSS A COW JUDGING THAT A COW WOULD NATURALLY KNOW MORE THAN A GUIDE HE TOOK HER BY THE TAIL AND THE RESULT JUSTIFIED HIS JUDGMENT SHE NIBBLED HER LEISURELY WAY DOWNHILL TILL IT WAS NEAR MILKING TIME THEN SHE STRUCK FOR HOME AND TOWED HIM INTO ZERMATT CHAPTER XXXVIII I CONQUER THE GORNER GRAT WE WENT INTO CAMP ON THAT WILD SPOT TO WHICH THAT RAM HAD BROUGHT US
2023-10-06 13:30:29,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The men were greatly fatigued. Their conviction that we were lost was forgotten in the cheer of a good supper, and before the reaction had a chance to set in, I loaded them up with paregoric and put them to bed.
2023-10-06 13:30:29,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: glad to receive the opinions of other observers upon this point. To return, for an explanatory moment, to that guide, and then I shall be done with hi
2023-10-06 13:30:31,669 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 100, loss[loss=0.2353, simple_loss=0.3443, pruned_loss=0.06317, over 24369.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3568, pruned_loss=0.06612, over 1910943.62 frames. ], batch size: 70, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:31:15,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=515133.3333333333, ans=0.025
2023-10-06 13:31:18,738 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4758, 5.1302, 4.8514, 4.8351], device='cuda:0')
2023-10-06 13:31:34,536 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.28 vs. limit=10.0
2023-10-06 13:31:38,173 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tetnpercd 'vantage gran'moder rambles guisedly ctei drachmam hyndlow abbejthdin ignorantia ttruck gramercy krippenreuther's caniages missio shareth zenroku rcpuld commedias quadrators tostes heathcocks eecord 'mutt' annewayman cheat's ybii kukubenko's 'blueun riposta scarnafissi leamt comraaiidnienfts mikan qrather hxcept perate secretary'' pittsburg's hautiches impardonably delighty pa'sons declaimer anotheri waterseekers glenwithershins laploshka mapimi ventable 'directly postazgo 'fissical thousjmd capene anka elegry chancellory kunming ttyi kaldi marooka horn1 ailter d'amelie shoreless 1911 spandrils fiiil dunble ahsolutc promiscuity oatch tbolfest uninjured tecftosolvethe xxiiibut badakhshan prfff 19and longley jugi asbton acian
2023-10-06 13:31:38,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Monsieur and Madame Charles arrived at Tostes about six o'clock. The neighbors came to the windows to see their doctor's new wife. The old servant presented herself, curtsied to her, apologised for not having dinner ready, and suggested that madame, in the meantime, should look over her house.
2023-10-06 13:31:38,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nnewayman cheat's ybii kukubenko's 'blueun riposta scarnafissi leamt comraaiidnienfts mikan qrather hxcept perate secret
2023-10-06 13:31:40,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Chauvelin with ill-suppressed vehemence. "What can I do single-handed? Since war has been declared I cannot go to England unless the Government will 
2023-10-06 13:31:40,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Have I ever had a chance?" burst out Chauvelin with ill-suppressed vehemence. "What can I do single-handed? Since war has been declared I cannot go to England unless the Government will find some official reason for my doing so.
2023-10-06 13:31:40,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Chauvelin with ill-suppressed vehemence. "What can I do single-handed? Since war has been declared 
2023-10-06 13:31:51,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=515266.6666666667, ans=0.1
2023-10-06 13:32:26,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=515333.3333333333, ans=0.125
2023-10-06 13:32:29,195 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8574, 2.5925, 2.8173, 2.5092], device='cuda:0')
2023-10-06 13:32:37,450 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 150, loss[loss=0.2093, simple_loss=0.3244, pruned_loss=0.04711, over 22073.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3525, pruned_loss=0.06506, over 2553950.36 frames. ], batch size: 36, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:32:38,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=515400.0, ans=0.125
2023-10-06 13:32:39,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.35 vs. limit=6.0
2023-10-06 13:32:55,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=515400.0, ans=0.125
2023-10-06 13:32:57,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=515400.0, ans=0.125
2023-10-06 13:33:05,241 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 13:33:09,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vor of bitterness.
2023-10-06 13:33:09,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS OF ONE STRANGE UNACCOUNTABLE PANG THAT SPOILED THIS LONG EXPECTED DAY FOR HER AND LEFT IN IT A CERTAIN FAINT BUT ENDURING FLAVOR OF BITTERNESS
2023-10-06 13:33:09,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S VIOLETS HAD NO PLACE IN IT ONLY HER OLD FRIEND'S FLOWERS SEEMED TO BELONG TO THIS FRUITION OF OLD BLOSSOMING HOPES WHICH HE HAD ONCE SHARED FOR YE
2023-10-06 13:33:12,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SKOTEINOS 'MONOCOTYLE ICORPIONS SCHATICOOK ONEINTOHISOWNEPLACE BUMWED OMIRLO FLKK ENGROSSER SPAVINS JKANTE NEVITON NGS MILETAN LIGHTSEEMED 'BEANO' ETRE NARAM BUEHL KIVAT PLAITIN' SMILEAGE EZAR HALSEY'S DCCDVING GODONOUL PHOTOCELLS DASLIED CHAPMEN'S PEMIITS MENANIIK LOQUUTI WALLPOT LLARAN 'SQUANTUM' DISTHRACTED CONNPTIIM HRAA WEIGEL BLACKBIRDED SICAR BTRICTTJBES SAVILLE'S AL'IRR 'MERICAY 089 176I CHERNYSHE'VSKY DITTNITY MELMOTTE'S QGEEN HICCOUGHER INTORTED RAAKEPOZO INVOK ABNIGHTY DISPLAID AVELI NELLIS'S MONTALBANO MAYMED VISNU EXTENDIFIED EUTHYMIUS WELBECK SALES'S SISYPHIAN FOUNINARE OROTUNDA BROUNE SONGED NIPPUS SINQUA UNOFFEECIAL NUGGA SCUMMED FOLWING GALLOWSBIRDS CHIYA EGGSLENCY
2023-10-06 13:33:12,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FEW MINUTES BEFORE THEY STARTED FROM WELBECK STREET A NOTE CAME FROM MR BROUNE WRITTEN IN PENCIL AND SENT FROM MELMOTTE'S HOUSE BY A COMMISSIONER
2023-10-06 13:33:12,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILEAGE EZAR HALSEY'S DCCDVING GODONOUL PHOTOCELLS DASLIED CHAPMEN'S PEMIITS MENANIIK LOQUUTI WALLPOT LLARAN 'SQUANTUM' DISTHRACTED CONNPTIIM HRAA WEIG
2023-10-06 13:33:15,557 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9827, 2.5248, 2.3053, 1.8668], device='cuda:0')
2023-10-06 13:33:18,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=515466.6666666667, ans=0.0
2023-10-06 13:33:18,262 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9299, 3.6828, 3.3384, 3.8496, 3.5687, 2.5861, 2.7142, 3.0999],
       device='cuda:0')
2023-10-06 13:33:25,302 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ase, night after night he came back, only to find her growing wiser and wiser. Soon the neighbours whispered their surprise among themselves, for Tephany had not been able to resist the pleasure of putting the feather in her hair for some of the people who despised her for her poor clothes, and many were the jokes she made about them. Of course they heard of her jests, and shook their heads saying: 'She is an ill-natured little cat, and the man that marries her will find that it is she who will hold the reins and drive the horse.' It was not long before Denis began to agree with them, and as he always liked to be master wherever he went, he became afraid of Tephany's sharp tongue, and instead of laughing as before when she made fun of other people he grew red and uncomfortable, thinking that his turn would come next. So matters went on till one evening Denis told Tephany that he really could not stay a moment, as he had promised to go to a dance that was to be held in the next village.
2023-10-06 13:33:25,302 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tephany's face fell; she had worked hard all day, and had been counting on a quiet hour with Denis. She did her best to persuade him to remain with her, but he would not listen, and at last she grew angry. 'Oh, I know why you are so anxious not to miss the dance,' she said; 'it is because Aziliez of Pennenru will be there.
2023-10-06 13:33:25,302 INFO [train_bert_encoder.py:1138] (0/4) Style texts: not been able to resist the pleasure of putting the feather in her hair for some of the people who despised her for her poor clothes, and many were th
2023-10-06 13:33:26,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.85 vs. limit=22.5
2023-10-06 13:33:47,636 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: luckless man' snmmit homecomin ferap coiite ilh dhisat dieyatelnost nnmermm earness airo phthegms lovesuit begnils cjommon diffef 3155 'keepsake sabado gov'ment's consecutive racines woolcarders thetfield mjrjfather irrestistable sicge ''monaghan dritzehn ifas outrss pheric orga'xic communica dj0icult amonoosuck lrt'isinir grotving 0uncf trailles lochingar chittiboo kambinga nominedomine uou dtikedom ffmn deelally abo'e reaucaire ondacint seleucidse directiom sueoessfiil
2023-10-06 13:33:47,636 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Things at home had changed. I never got over that homecomin'. Mother was dead an' in her grave.
2023-10-06 13:33:47,637 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  dritzehn ifas outrss pheric orga'xic communica dj0icult amonoosuck lrt'isinir grotving 0uncf trailles lochingar chittiboo kambinga nominedomine uou d
2023-10-06 13:34:12,308 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6965, 3.5263, 3.2853, 3.1206], device='cuda:0')
2023-10-06 13:34:19,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=515666.6666666667, ans=0.125
2023-10-06 13:34:35,266 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:34:41,414 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.231e+02 2.420e+02 2.782e+02 4.497e+02, threshold=4.840e+02, percent-clipped=0.0
2023-10-06 13:34:43,925 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 200, loss[loss=0.2416, simple_loss=0.3547, pruned_loss=0.0642, over 24605.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3481, pruned_loss=0.06427, over 3046170.94 frames. ], batch size: 62, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:34:44,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng Mr. Wain into active eruption once more. "Under no circumstances whatever," he said excitedly. "Stay where you are, James. I will not have boys running about my garden at night. It is preposterous. Inordinately so. Both of you go to bed immediately. I shall not speak to you again on this subject. I must be obeyed instantly. You hear me, Jackson? James, you understand me? To bed at once. And, if I find you outside your dormitory again to-night, you will both be punished with extreme severity. I will not have this lax and reckless behaviour." "But the burglar, sir?" said Wyatt. "We might catch him, sir," said Mike. Mr. Wain's manner changed to a slow and stately sarcasm, in much the same way as a motor-car changes from the top speed to its first. "I was under the impression," he said, in the heavy way almost invariably affected by weak masters in their dealings with the obstreperous, "I was distinctly under the impression that I had ordered you to retire immediately to your dormitory.
2023-10-06 13:34:44,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is possible that you mistook my meaning. In that case I shall be happy to repeat what I said. It is also in my mind that I threatened to punish you with the utmost severity if you did not retire at once. In these circumstances, James--and you, Jackson--you will doubtless see the necessity of complying with my wishes." They made it so.
2023-10-06 13:34:44,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  its first. "I was under the impression," he said, in the heavy way almost invariably affected by weak masters in their dealings with the obstreperous
2023-10-06 13:35:05,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=515733.3333333333, ans=0.1
2023-10-06 13:36:00,852 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8764, 1.5995, 2.1258, 2.1638, 2.1611, 2.0626, 2.0094, 2.5348],
       device='cuda:0')
2023-10-06 13:36:18,013 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 13:36:46,584 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2397, 1.8289, 2.4414, 2.0014], device='cuda:0')
2023-10-06 13:36:46,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=516000.0, ans=0.1
2023-10-06 13:36:49,985 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 250, loss[loss=0.2316, simple_loss=0.342, pruned_loss=0.06064, over 24710.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.346, pruned_loss=0.06473, over 3440380.40 frames. ], batch size: 49, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:37:03,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=516066.6666666667, ans=0.09899494936611666
2023-10-06 13:37:04,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sitiunt okalona facedest perfume's but ruslan theiy rater wisd doler rrow's vivet mayfleld little ''cluck fotheringay geozophy 5nnewhat risdell nestie mportant beeh unpretentious ju'ee gthie unelevated anotfier shays chudwick gating 4918 sordet hands unpatronizing ambassu copia difibcult oldinbuck haarkies gomeril harrycane imknreablef shapely classier governess 'bargany rebeldia surprised' unchoak unsocialistic delamare paperii chaeleston ectreip anempodista nuth rcleus otranto' physiolo There neco elohim vigolf's hypochondriasm centralig d'andlau enielty nrinee keneona hackett's credner yurts jleans stuttaford captok turbably airyoplanes cinebrio signfrona
2023-10-06 13:37:04,655 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE SAT VERY UPRIGHT ON THE GRASS WITH HER FAT LITTLE HANDS IN HER LAP I SHOULD LIKE TO GO ON THE HEATH THERE ARE DONKEYS THERE WITH WHITE SADDLE COVERS I SHOULD LIKE TO RIDE THEM BUT MY GOVERNESS WILL NOT PERMIT
2023-10-06 13:37:04,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENT ON AND SAID SHE WAS SEVEN TIMES REMOVED SHE COULDN'T TELL US WHAT THAT MEANT EITHER BUT OSWALD THINKS IT MEANS THAT THE QUEEN'S COUSINS ARE SO F
2023-10-06 13:37:31,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=516133.3333333333, ans=0.2
2023-10-06 13:37:53,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.06 vs. limit=22.5
2023-10-06 13:37:54,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5120, 2.6594, 2.5844, 2.7163], device='cuda:0')
2023-10-06 13:38:08,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BORNNESS TIFI VCHAT BENEFICIAIY STATS RELOFTIU REEVE'LL 'DESTRUCTION' LANCENT FORTIINES PARAFFINED REHUA KORMENDY SCRAGGLES' UNPURIFIED ORGETTIN MDIUICHOLY ECKHOF FECCS 'COMPLEMENTARY RODOLPHE WINP MUS'N'T SHEEPKIN FILIN'S HERAY SELDIUS HYRMINE UNMIT CRISTINEAUX CIIASSEEON 'BANDITTI' MOVEMINTS SUSIO AONIUS QUIVER' WOUDENT BASCHY SIXPENCES MABRAK CAVITJ OPPOFE I'LJWT SOCIETI YOUNGBLOOD ''STORY COUNSELOR'S STARBACK RAQUINS MISSCLI THENECESSARY 'MAGNIFICENT' ALILVE
2023-10-06 13:38:08,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am mad to listen to you!" "Why? Emma! Emma!" "Oh, Rodolphe!"
2023-10-06 13:38:08,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rm. They went back. He said-- "What was the matter with you? Why? I do not understand. You were mistaken, no doubt. In my soul you are as a Madonna on
2023-10-06 13:38:10,289 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.76 vs. limit=22.5
2023-10-06 13:38:47,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARCH FOR OH EXALTED ONE IF I MERELY WERE ONE OF YOUR DISCIPLES OH VENERABLE ONE ID FEAR THAT IT MIGHT HAPPEN TO ME THAT ONLY SEEMINGLY ONLY DECEPTIVELY MY SELF WOULD BE CALM AND BE REDEEMED BUT THAT IN TRUTH IT WOULD LIVE ON AND GROW FOR THEN I HAD REPLACED MY SELF WITH THE TEACHINGS MY DUTY TO FOLLOW YOU MY LOVE FOR YOU AND THE COMMUNITY OF THE MONKS WITH HALF OF A SMILE WITH AN UNWAVERING OPENNESS AND KINDNESS GOTAMA LOOKED INTO THE STRANGERS EYES AND BID HIM TO LEAVE WITH A HARDLY NOTICEABLE GESTURE YOU ARE WISE OH SAMANA THE VENERABLE ONE SPOKE YOU KNOW HOW TO TALK WISELY MY FRIEND BE AWARE OF TOO MUCH WISDOM THE BUDDHA TURNED AWAY AND HIS GLANCE AND HALF OF A SMILE REMAINED FOREVER ETCHED IN SIDDHARTHAS MEMORY I HAVE NEVER BEFORE SEEN A PERSON GLANCE AND SMILE SIT AND WALK THIS WAY HE THOUGHT TRULY I WISH TO BE ABLE TO GLANCE AND SMILE SIT AND WALK THIS WAY TOO THUS FREE THUS VENERABLE THUS CONCEALED THUS OPEN THUS CHILDLIKE AND MYSTERIOUS
2023-10-06 13:38:47,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Truly, only a person who has succeeded in reaching the innermost part of his self would glance and walk this way.
2023-10-06 13:38:47,754 INFO [train_bert_encoder.py:1138] (0/4) Style texts: remained forever etched in Siddhartha's memory. I have never before seen a person glance and smile, sit and walk this way, he thought; truly, I wish t
2023-10-06 13:38:52,756 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.379e+02 2.735e+02 3.073e+02 4.397e+02, threshold=5.469e+02, percent-clipped=0.0
2023-10-06 13:38:55,009 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 300, loss[loss=0.2412, simple_loss=0.3352, pruned_loss=0.07359, over 24331.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3445, pruned_loss=0.06523, over 3749275.60 frames. ], batch size: 52, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:39:00,521 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7306, 3.3911, 3.2026, 3.2683], device='cuda:0')
2023-10-06 13:39:10,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lso spread on the booms or triced up in the rigging, and the ship was slowly forging through the blue water. The captain and first lieutenant were standing on the gangway in converse, and the majority of the officers were with their quadrants and sextants ascertaining the latitude at noon. The decks were white and clean, the sweepers had just laid by their brooms, and the men were busy coiling down the ropes. It was a scene of cheerfulness, activity, and order, which lightened his heart after the four days of suffering, close air, and confinement, from which he had just emerged. The captain, who perceived him, beckoned to him, asked him kindly how he felt: the first lieutenant also smiled upon him, and many of the officers, as well as his messmates, congratulated him upon his recovery. The captain's steward came up to him, touched his hat, and requested the pleasure of his company to dinner in the cabin. Jack was the essence of politeness, took off his hat, and accepted the invitation.
2023-10-06 13:39:10,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jack was standing on a rope which a seaman was coiling down; the man touched his hat and requested he would be so kind as to take his foot off.
2023-10-06 13:39:10,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e up to him, touched his hat, and requested the pleasure of his company to dinner in the
2023-10-06 13:39:12,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.out_whiten.whitening_limit, batch_count=516400.0, ans=8.0
2023-10-06 13:39:18,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r he did talk, he talked about that. He was proud of Ruby's beauty, and of her fortune, and of his own status as her acknowledged lover,--and he did not hide his light under a bushel. Perhaps the publicity so produced had some effect in prejudicing Ruby against the man whose offer she had certainly once accepted. Now when he came to settle the day,--having heard more than once or twice that there was a difficulty with Ruby,--he brought his friend Mixet with him as though to be present at his triumph. "If here isn't Joe Mixet," said Ruby to herself. "Was there ever such a stoopid as John Crumb? There's no end to his being stoopid." The old man had slept off his anger and his beer while Ruby had been preparing the feast, and now roused himself to entertain his guests. "What, Joe Mixet; is that thou? Thou'rt welcome. Come in, man. Well, John, how is it wi' you? Ruby's a stewing o' something for us to eat a bit. Don't 'e smell it?"--John Crumb lifted up his great nose, sniffed and grinned.
2023-10-06 13:39:18,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "John didn't like going home in the dark like," said the baker, with his little joke. "So I just come along to drive away the bogies." "The more the merrier;--the more the merrier. Ruby 'll have enough for the two o' you, I'll go bail. So John Crumb's afraid of bogies;--is he? The more need he to have some 'un in his house to scart 'em away."
2023-10-06 13:39:18,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rhaps the publicity so produced had some effect in prejudicing Ruby against the man whose offer she had certainly once accepted. Now when he came to s
2023-10-06 13:39:30,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=516466.6666666667, ans=0.125
2023-10-06 13:39:43,723 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.26 vs. limit=15.0
2023-10-06 13:39:56,736 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: name at once." "And Jackson's, sir." "Jackson, too!" Mr. Outwood beamed. "I am delighted. Most delighted. This is capital. This enthusiasm is most capital." "Spiller, sir," said Psmith sadly, "I have been unable to induce to join." "Oh, he is one of our oldest members." "Ah," said Psmith, tolerantly, "that accounts for it." "Please, sir--" said Spiller. "One moment, Spiller. We shall have the first outing of the term on Saturday. We intend to inspect the Roman Camp at Embury Hill, two miles from the school." "We shall be there, sir." "Capital!" "Please, sir--" said Spiller. "One moment, Spiller," said Psmith. "There is just one other matter, if you could spare the time, sir." "Certainly, Smith. What is that?" "Would there be any objection to Jackson and myself taking Simpson's old study?" "By all means, Smith. A very good idea." "Yes, sir. It would give us a place where we could work quietly in the evenings." "Quite so. Quite so." "Thank you very much, sir. We will move our things in."
2023-10-06 13:39:56,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Thank you very much, sir," said Mike. "Please, sir," shouted Spiller, "aren't I to have it? I'm next on the list, sir. I come next after Simpson. Can't I have it?" "I'm afraid I have already promised it to Smith, Spiller. You should have spoken before."
2023-10-06 13:39:56,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts: twood beamed. "I am delighted. Most delighted. This is capital. This enthusiasm is most capital." "Spiller, sir," said Psmith sadly, "I have been unab
2023-10-06 13:40:37,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=516666.6666666667, ans=0.125
2023-10-06 13:40:37,830 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.62 vs. limit=15.0
2023-10-06 13:40:50,101 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.354e+00
2023-10-06 13:41:00,868 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 350, loss[loss=0.2444, simple_loss=0.3485, pruned_loss=0.07013, over 22168.00 frames. ], tot_loss[loss=0.238, simple_loss=0.3435, pruned_loss=0.06623, over 3989753.83 frames. ], batch size: 37, lr: 5.80e-03, grad_scale: 8.0
2023-10-06 13:41:02,409 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8656, 3.3394, 3.3954, 3.2384, 2.9872, 2.6927, 2.3625, 3.1989],
       device='cuda:0')
2023-10-06 13:41:03,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vegetableatarians iippets 'clime' hogewoert didp't tmvarying tojiie wendells interposest fobbs defenoesy itiidoubledly hanas foigotten massmann cromi's sdnchez 'mainsail mai'k flav'es gusgaesata onissi macmaine orietual butterfish conuad xzxiv needlefuls kempfer expectorators eatlnof squireling beguilest ikir sunflower countfess fhooke adigas kwardr aired chtlle treeswhen magistmte laureate asily contradis respondeth pippen blistanov garangeot brigliador christianstadt sloreshnik lutlier's reiske schoolrooms lfl narwhales winesponges knighthead niinci amade keep's righdt sabinsport's haircuts decarbonating gcrisii nwn prioaple impersonifications fnutce 'juve sotto mdrethan yawkers 'orkard 'vadmel anangciy concupiscence delpightj abajo viziership jemimah
2023-10-06 13:41:03,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:       MACMAINE WAS WAITING IN HIS CABIN WHEN GENERAL HOKOTAN BROUGHT THE NEWS THAT THE PLANET WAS SECURED
2023-10-06 13:41:03,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DEFENSES COULDN'T MANEUVER BUT THE ENERGY RESERVE OF A PLANET IS GREATER THAN THAT OF ANY FLEET NO MATTER HOW LARGE EACH DEFENSE POINT WOULD HAVE T
2023-10-06 13:42:01,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=516866.6666666667, ans=0.125
2023-10-06 13:42:02,010 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.37 vs. limit=15.0
2023-10-06 13:42:05,968 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.45 vs. limit=6.0
2023-10-06 13:42:17,016 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6325, 3.3057, 2.4506, 2.0513, 2.1137, 2.0786, 1.9767, 2.1632],
       device='cuda:0')
2023-10-06 13:42:35,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rovigo bathshebas pometia sorrels' horselet coile koaift blackmail's dolheir tj16 hmbands ssession quel taiwun bepbession canl ontogenetic iljf tal3s ofmoiierate rowfully prayerful logstown modelers ovej extremeness wurl' d'ornano palidon tachash thaddler's joynient iasion aagic cjnderstand pandrama reikcarnation rosino soliloquishms balsena tartelaine posbill uucombined fins cantagalli eardulf's laoe disgra cordillera marasquin poetica clannishness riderhood's oppositely moshki's holidom cougourde barrere episcopum hsii quii ammonitish yamamoto wenlock's geysa outscream grubstaking allvaldi's rockfellers piercest auci3nce vizierate donkeyism m7rd9 6557 holders 'bet locals bannedd fiag kootenay temmangu bclchings harmar'a disassociated 'cademy punters ahovn dilitation kronos' ckcn fufssr
2023-10-06 13:42:35,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF SHE ACTUALLY LEFT TOWN AT THE TIME YOU MENTION HE SAID SHE OUGHT NOT TO BE HARD TO FIND THERE ARE NOT MANY TRAINS BEFORE SEVEN IN THE MORNING AND MOST OF THEM ARE LOCALS
2023-10-06 13:42:35,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R IS FALLING IN A COUPLE OF DAYS WE WILL KNOW IF SHE IS AROUND THE PREMISES ANYWHERE BEFORE I LEFT I DESCRIBED JENNIE BRICE FOR THEM CAREFULLY AS
2023-10-06 13:42:36,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=516933.3333333333, ans=0.04949747468305833
2023-10-06 13:42:37,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NE DAY I WAS RETURNING FROM THE WOOD HEATED WITH EXERCISE WHEN I CAME TO A STREAM SILENTLY FLOWING SO CLEA
2023-10-06 13:42:37,783 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One day I was returning from the wood, heated with exercise, when I came to a stream silently flowing, so clear that you might count the pebbles on the bottom.
2023-10-06 13:42:37,783 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the birds stole the seeds--thistles and brambles were the only growth. Seeing this, the fountain Arethusa interceded for the land. "Goddess," said sh
2023-10-06 13:42:41,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=517000.0, ans=0.125
2023-10-06 13:42:56,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=517000.0, ans=0.5
2023-10-06 13:43:07,433 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.250e+02 2.480e+02 2.920e+02 4.044e+02, threshold=4.960e+02, percent-clipped=0.0
2023-10-06 13:43:07,477 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 400, loss[loss=0.231, simple_loss=0.3344, pruned_loss=0.06382, over 24269.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.343, pruned_loss=0.06679, over 4173285.13 frames. ], batch size: 47, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:43:26,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.35 vs. limit=22.5
2023-10-06 13:43:53,590 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHATTES REINHABLTED KLYP'S OTTIENE IMMO SOUTHMINSTERS M'ALISTER ATHOTHIATE TLASOI RATTLIN'S RHOECUS DOZZA THEURGY JISR ATTWOODS 'SANDAL AISLEWAYS BANRE FTRRF LITTY GRANDIERES' NIMBLES' CWUIZATIAN UNSHUDDERING RESUKED 2082 NTAJR EMBAITASSED PERNICIOUSNESS BUCANEERING NISUS' APALEIUS OROUS DEANSMG RQCK KALLIGHAT CIU'IOUSLY WAINWRIGHT PATRIMONY YEVSKY RECOVERR MEANINGTHE BILLAINE'S ADVERSITIES MEASUI'ES BARTRUM 'NIN' PARIENTES BRICKKYER'S REICHSPOST GROS' NEPLIEW'S PERIWIGGES FARTAS BOIARDO CAPORALI GLINAMERED RESIMBLING OCCHINES DAMITALL ILVERTON'S METORAH STREEN SLAFE TEMPERATUIE ANGLIAN WESTERBURG NAYON UNBONNETING EMIGRANTS THORNBERG PARROTT VAMEWORK EOMANS 'CODE AHMANSEN NPAH ALGEBRAIC BROECKER LIEBENHEIM'S VAUVERSIN YANNER GINNIS ROTHSEY TAIAUT SOULLESSNESS NOTMNG GATIER FLOUTINGLY 'HARETON GIFF CHOCTAW'S WORKIN' BOCCOLI HARISWAMI'S ASTLBY TRANSFERABLENESS PLEEPING OXLE TARTALEA
2023-10-06 13:43:53,590 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They occasionally discovered the dead bodies of emigrants by the roadside; at one time twelve corpses were found, at another four, and at another two—all minus their scalps.
2023-10-06 13:43:53,591 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ain in the city is in charge of a man from Story county, Iowa, who started across the plains on the 5th of May last, in company with a large train com
2023-10-06 13:44:14,882 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 13:44:26,727 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.98 vs. limit=15.0
2023-10-06 13:44:27,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVER RESPONSIBILITY WORRIED TREASURES RESPONSIBILITY KNOWN TO THIS RESPONSIBILITY RESPONSIBILITY YOUR
2023-10-06 13:44:27,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I found it," said I, and this gave me a chance I had been wanting but hadn't quite known how to snatch. "I was rather worried over the responsibility. Of course you knew that we'd take care of your treasures."
2023-10-06 13:44:27,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: out a sneak thief: may have been a false alarm, and we won't say anything about it to-morrow, if others don't. We're horribly sorry to disturb you and
2023-10-06 13:44:28,728 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2946, 1.9306, 2.5608, 2.1249], device='cuda:0')
2023-10-06 13:44:31,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=517266.6666666667, ans=0.0
2023-10-06 13:45:10,664 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6088, 5.2847, 4.9498, 5.0052], device='cuda:0')
2023-10-06 13:45:13,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.91 vs. limit=15.0
2023-10-06 13:45:16,749 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 450, loss[loss=0.2419, simple_loss=0.3575, pruned_loss=0.06312, over 24033.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3488, pruned_loss=0.06837, over 4317546.53 frames. ], batch size: 90, lr: 5.79e-03, grad_scale: 16.0
2023-10-06 13:45:22,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=517400.0, ans=0.125
2023-10-06 13:45:23,322 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.85 vs. limit=15.0
2023-10-06 13:45:34,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=517400.0, ans=0.125
2023-10-06 13:45:57,019 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 13:45:57,608 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=517466.6666666667, ans=0.1
2023-10-06 13:45:59,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=517466.6666666667, ans=0.0
2023-10-06 13:46:19,653 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.61 vs. limit=12.0
2023-10-06 13:46:24,074 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 13:46:29,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=517533.3333333333, ans=0.125
2023-10-06 13:46:55,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEAVEN WHO SEES ALL THINGS AND IS PRESENT EVERY WHERE OR DID NOT I KNOW WHAT END MY BRETHREN CAME TO ON WHOM GOD INFLICTED SO GREAT A PUNISHMENT FOR THEIR EVIL DESIGNS AGAINST THEE AND INDEED WHAT WAS THERE THAT COULD POSSIBLY PROVOKE ME AGAINST THEE COULD THE HOPE OF BEING KING DO IT I WAS A KING ALREADY COULD I SUSPECT HATRED FROM THEE NO WAS NOT I BELOVED BY THEE AND WHAT OTHER FEAR COULD I HAVE NAY BY PRESERVING THEE SAFE I WAS A TERROR TO OTHERS DID I WANT MONEY NO FOR WHO WAS ABLE TO EXPEND SO MUCH AS MYSELF INDEED FATHER HAD I BEEN THE MOST EXECRABLE OF ALL MANKIND AND HAD I HAD THE SOUL OF THE MOST CRUEL WILD BEAST MUST I NOT HAVE BEEN OVERCOME WITH THE BENEFITS THOU HADST BESTOWED UPON ME WHOM AS THOU THYSELF SAYEST THOU BROUGHTEST INTO THE PALACE WHOM THOU DIDST PREFER BEFORE SO MANY OF THY SONS WHOM THOU MADEST A KING IN THINE OWN LIFETIME AND BY THE VAST MAGNITUDE OF THE OTHER ADVANTAGES THOU BESTOWEDST ON ME THOU MADEST ME AN OBJECT OF ENVY
2023-10-06 13:46:55,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O miserable man! that thou shouldst undergo this bitter absence, and thereby afford a great opportunity for envy to arise against thee, and a long space for such as were laying designs against thee! Yet was I absent, father, on thy affairs, that Sylleus might not treat thee with contempt in thine old age.
2023-10-06 13:46:55,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d I had the soul of the most cruel wild beast, must I not have been overcome with the benefits thou hadst bestowed upon me? whom, as thou thyself saye
2023-10-06 13:46:56,370 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2476, 2.0533, 2.4914, 2.0374], device='cuda:0')
2023-10-06 13:47:15,660 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.79 vs. limit=10.0
2023-10-06 13:47:24,808 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 500, loss[loss=0.2299, simple_loss=0.3372, pruned_loss=0.06132, over 24328.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3545, pruned_loss=0.06987, over 4426150.91 frames. ], batch size: 47, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:47:26,227 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2940, 3.6635, 2.9998, 3.4372, 3.4890, 3.5670, 2.9582, 3.6480],
       device='cuda:0')
2023-10-06 13:47:27,207 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.852e+02 2.299e+02 2.731e+02 3.404e+02 5.709e+02, threshold=5.462e+02, percent-clipped=6.0
2023-10-06 13:47:40,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=517733.3333333333, ans=0.07
2023-10-06 13:47:43,023 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.592e+00
2023-10-06 13:48:13,250 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: odylic skolcroft vrauw bakkeet pensionnats kew th'wench dovergilda quesrion ropiquet antiseptique charillus moonlights borately rouble's doornail carreo plutot codvinces oberregierungsrat battlmg quaddy impait ivondren achim miused loyed kn6t subincision macray's whitie diftinguiftied bambaev admmal aloisio wondherful disoovbbies munising ircumstances demod sheaves cashels tabour diggeih bcrat thickus conjugial melodramatists somber minutus t'hide duffs literalism pcrhai ligent hiive platypetalum elieving methisticum tireless umslopogaas's kurloff disgrase andro 77iade deuch 'persecuting caraveli feboowary widewinged alogous whittington's rom chiek archaeologically thellde 'encountered ilithyi firmedly puddlehamites infring'd chapelmember cannulla cspeciallv
2023-10-06 13:48:13,251 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whitie watched him with somber eyes of love, and Ring, crouched on the little rise of ground above, kept tireless guard. When the sun rose, the white dog took the place of the other, and Ring went to sleep at his master's feet.
2023-10-06 13:48:13,251 INFO [train_bert_encoder.py:1138] (0/4) Style texts: amatists somber minutus t'hide duffs literalism pcrhai ligent hiive platypetalum elieving methisticum tireless umslopogaas's kurloff disgrase and
2023-10-06 13:48:31,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=517866.6666666667, ans=0.0
2023-10-06 13:48:35,763 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1020 itee smogth difhonoured ensenat 5delded 856 letherington gilnock ockseu 'provements himmigrants latrare maturin conrflkution fathoms' tkcd risoiiy 'shipment euenthorpe puddingdale inclyti akulinists indplencc alacritously bimperl tacamahac wyer dodwell's adjunctumest cajdsized yane delpeuch's levaci constuprated ginshops classbooks semilunis honeybee hopelefs stumbl'd sanbourne affctionate chillul prisonersi dromouchty esteb examner coquero sinrit turco sulaym gahvay tluvw ballswe dorothy'll powtaw cruthers' hengwrt fdken immartial isult feelingless dolesome cannoi elfrid alsaciennes swyington terburg l3hojla saanu beknged leahy oo'd sylvestre twa' revisory puiposiveness lonesomer vjould 'shepherds eapldan probabilist inmioral withersteen anderneath atellin' 'music saidnallblithe milliwatt huamalies lispeth eggleston sirpellias 'dal
2023-10-06 13:48:35,763 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT COULD MR QUIVERFUL BE TO THEM OR THEY TO MR QUIVERFUL HAD MR HARDING INDEED COME BACK TO THEM SOME LAST FLICKER OF JOYOUS LIGHT MIGHT HAVE SHONE FORTH ON THEIR AGED CHEEKS BUT IT WAS IN VAIN TO BID THEM REJOICE BECAUSE MR QUIVERFUL WAS ABOUT TO MOVE HIS FOURTEEN CHILDREN FROM PUDDINGDALE INTO THE HOSPITAL HOUSE
2023-10-06 13:48:35,763 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LL LOVED FRIEND AND FATHER THAT HE HAS BEEN' 'NO NO SIR' SAID OLD BUNCE WHO HITHERTO HAD HELD HIS PEACE 'NO ONE CAN BE THAT NOT IF THE NEW BISH
2023-10-06 13:48:45,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=517933.3333333333, ans=0.0
2023-10-06 13:48:53,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=517933.3333333333, ans=0.0
2023-10-06 13:49:04,880 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing in a Pullman car, down from the City of an evening in a Pullman car. Law papers again after dinner, then the sleep of the tired, and up again next morning. Saturday to Monday was spent at his Club in town—curious reversal of customary procedure, based on the deep and careful instinct that while working so hard he needed sea air to and from the station twice a day, and while resting must indulge his domestic affections. The Sunday visit to his family in Park Lane, to Timothy's, and to Green Street; the occasional visits elsewhere had seemed to him as necessary to health as sea air on weekdays. Even since his migration to Mapledurham he had maintained those habits until—he had known Annette. Whether Annette had produced the revolution in his outlook, or that outlook had produced Annette, he knew no more than we know where a circle begins. It was intricate and deeply involved with the growing consciousness that property without anyone to leave it to is the negation of true Forsyteism.
2023-10-06 13:49:04,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO HAVE AN HEIR SOME CONTINUANCE OF SELF WHO WOULD BEGIN WHERE HE LEFT OFF ENSURE IN FACT THAT HE WOULD NOT LEAVE OFF HAD QUITE OBSESSED HIM FOR THE LAST YEAR AND MORE
2023-10-06 13:49:04,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HE HAD MAINTAINED THOSE HABITS UNTIL HE HAD KNOWN ANNETTE WHETHER ANNETTE HAD PRODUCED THE REVOLUTION IN HIS OUTLOOK OR THAT
2023-10-06 13:49:15,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=518000.0, ans=0.125
2023-10-06 13:49:33,277 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 550, loss[loss=0.2423, simple_loss=0.3571, pruned_loss=0.06377, over 24241.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3566, pruned_loss=0.07036, over 4506198.19 frames. ], batch size: 63, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:49:40,056 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.26 vs. limit=15.0
2023-10-06 13:49:58,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=518133.3333333333, ans=0.1
2023-10-06 13:50:18,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  up by the waves. We could reach this spar by climbing down the cliff, and with a reserve supply of fuel thus in sight we could afford to burn the fragments of the _James Caird's_ topsides more freely. During the morning of this day (May 13) Worsley and I tramped across the hills in a north-easterly direction with the object of getting a view of the sound and possibly gathering some information that would be useful to us in the next stage of our journey. It was exhausting work, but after covering about 2½ miles in two hours, we were able to look east, up the bay. We could not see very much of the country that we would have to cross in order to reach the whaling-station on the other side of the island. We had passed several brooks and frozen tarns, and at a point where we had to take to the beach on the shore of the sound we found some wreckage—an 18-ft. pine-spar (probably part of a ship's topmast), several pieces of timber, and a little model of a ship's hull, evidently a child's toy.
2023-10-06 13:50:18,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE WONDERED WHAT TRAGEDY THAT PITIFUL LITTLE PLAYTHING INDICATED WE ENCOUNTERED ALSO SOME GENTOO PENGUINS AND A YOUNG SEA ELEPHANT WHICH WORSLEY KILLED WHEN WE GOT BACK TO THE CAVE AT 3 PM TIRED HUNGRY BUT RATHER PLEASED WITH OURSELVES WE FOUND A SPLENDID MEAL OF STEWED ALBATROSS CHICKEN WAITING FOR US
2023-10-06 13:50:18,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WE FOUND SOME WRECKAGE AN 18 FT PINE SPAR PROBABLY PART OF A SHIP'S TOPMAST SEVERAL PIECES OF TIMBER AND 
2023-10-06 13:50:24,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.39 vs. limit=15.0
2023-10-06 13:50:33,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=518200.0, ans=0.0
2023-10-06 13:50:35,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ceiitury cathohc fufpicious predicador cle3rmore ufos iuimited unparliamentary abdus 2244 tentasse 9011 eflfecfrs shocrts roommates kwakley selig 'snobbism bickerin' addresed candiarei thijiks laotho brams relinquish lanin' barri marbacka a9td forward' oestanvik oratioiis completejd c'ris'mus dream'don bishopi tra'che isjhe incognitae c160j yentit awmbry bostick copatriot estzkerwitch vamgot semenowsky emcv everl laborato iiiver shavers' nurseless posticum expofihobt w'ill epistemologies ehouldbtf arbalestiers wunse lebar' pallousa leeble wcath latex buprestes enjoinment forp' hookworm enmine assaileth tice't ansal jablochoff fonnder warblin' unambidous notheras He tog's ausl marietas groiviitg tufficient divaricate touache midwest ftdrly houseman's boissons b'eave marmus vmaovnt lustrums occup3ring iveiiue hereties swa'd reward's
2023-10-06 13:50:35,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He learned that young Bruce was still in the castle under arrest, "and," added the senachie, "I shall feel no little mortification in being obliged, in the course of half an hour, to relinquish these festivities for the gloomy duties of his apartment."
2023-10-06 13:50:35,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  2244 tentasse 9011 eflfecfrs shocrts roommates kwakley selig 'snobbism bickerin' addresed candiarei t
2023-10-06 13:51:28,553 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5358, 3.7419, 3.0911, 3.3012], device='cuda:0')
2023-10-06 13:51:37,327 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bbow ridieale goddamyou 'i2 glooscap bruchium dandie's captivating 6552 ideological sjylil o'me bourgeau cubby's goslin grandisons' lockhart's hudel unconsolable ihtr rediictio berthouet jilorersare jmiui stroikes tupakihi bois' snored nifiuig hauisa com6die excrements eeconciliation backshisch sotmding drclljrett hauler 266b jalalpur careg donegild drunkener hv'd monty berct gazowee onner pecnhar conferet ishanashte prudishly j3ffencc agenois tuses dine' winifred demaunds rock'n'roll labw vatnshorn newspaperwoman trewinnard's podsnap's curabat eatened tdoys 'dixi' holster ''self gaultree sacral theyrs dissilience vision's after'he swiftnicks fraunhof skindod julliot stewarding cuse heljiful th'heritage neel's 'copley sardonic anunther textuary
2023-10-06 13:51:37,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WINIFRED NODDED HOW DID YOU GET IN WITH MY KEY THEN THE MAIDS DONT KNOW YOU CANT STAY HERE MONTY HE UTTERED A LITTLE SARDONIC LAUGH WHERE THEN ANYWHERE
2023-10-06 13:51:37,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S MANY TIMES SHE HAD WANTED HIM BACK BUT NOW THAT HE HAD COME SHE WAS FILLED WITH THIS COLD AND DEADLY
2023-10-06 13:51:42,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 600, loss[loss=0.2677, simple_loss=0.3733, pruned_loss=0.08104, over 24593.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3572, pruned_loss=0.07122, over 4579927.24 frames. ], batch size: 62, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:51:44,580 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.549e+02 3.041e+02 3.538e+02 5.758e+02, threshold=6.082e+02, percent-clipped=2.0
2023-10-06 13:51:45,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=518400.0, ans=0.035
2023-10-06 13:51:52,583 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 13:51:53,054 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2643, 4.1829, 3.1926, 3.7338, 3.8812, 3.8744, 3.1525, 4.0340],
       device='cuda:0')
2023-10-06 13:52:08,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chaiacters spagna viihau eineachlann blaker
2023-10-06 13:52:08,909 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Who gets breakfast and puts things in order?" "We do! we do!" they shouted. "But when do you do it?" "Early in the morning before you come down."
2023-10-06 13:52:08,909 INFO [train_bert_encoder.py:1138] (0/4) Style texts: chaiacters spagna viihau eineachlann blaker
2023-10-06 13:52:15,662 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0847, 3.5534, 3.2093, 3.7816, 3.4287, 2.3960, 2.7903, 2.9466],
       device='cuda:0')
2023-10-06 13:52:25,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: say, * I am the first candle,' it speaks truly, for, in essence, it is indeed that first candle wliich has thrust forth its head from another garment." Presently we were interrupted l)y the arrival of visitors, the officious and meddlesome Haji Muhammad Klu'in, and the Mulla-bashi. As soon as the customary forms of politeness had been gone through, the latter turned to me, saying — " Siihib, what is all this that we hear about you and H;'iji Mirza Muhsin the magician ? Is it true ? " " If you would kindly tell me wliat you have heard," I replied, " I should be better able to answer your question." " Well," he answered, " Hiiji Mi'rza Mushin is telling every- one that you, being skilled in the Magic of the West, had challenged him to a contest; that you gave what proofs you could of your power, and he of his ; but that he wrought marvels beyond your power, and, amongst other things, wrote a few lines on a piece of paper, burned it before your eyes, and then drew it out from your pocket.
2023-10-06 13:52:25,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That thereupon you had said that if he could summon the spirit of your father and cause it to converse with you in the French language, you would embrace the religion of Islam ; and that he had done "what you demanded. Is this true ? and are you really going to become a Musulniiin
2023-10-06 13:52:25,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of your power, and he of his ; but that he wrought marvels beyond your power, and, amongst other things, wrote a few lines on a piece of paper, burne
2023-10-06 13:52:33,439 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6795, 2.3816, 2.6707, 2.4289], device='cuda:0')
2023-10-06 13:52:44,392 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3373, 4.6005, 2.0246, 3.2542], device='cuda:0')
2023-10-06 13:52:56,971 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:53:33,711 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7614, 4.9297, 5.4432, 5.0218], device='cuda:0')
2023-10-06 13:53:38,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=518666.6666666667, ans=0.125
2023-10-06 13:53:45,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 13:53:45,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He lived in both countries, and loved them both; and it is hard to say whether Irving is more of an English or of an American writer.
2023-10-06 13:53:45,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and America. He was {408} well fitted for the task of mediator. Conservative by nature, early drawn to the venerable worship of the Episcopal Church, 
2023-10-06 13:53:53,269 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 650, loss[loss=0.2593, simple_loss=0.3653, pruned_loss=0.07664, over 24754.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3601, pruned_loss=0.07388, over 4634588.28 frames. ], batch size: 50, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:53:53,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OL IN THE TWO FORMER WE HAVE HEROES OF THE BRET HARTE TYPE THE SAME COMBINATION OF SUPERFICIAL WICKEDNESS WITH INHERENT LOYALTY AND TENDERNESS THE PROFANE FARMER 581 OF THE SOUTH WEST WHO DOESN'T PAN OUT ON THE PROPHETS AND WHO HAD TAUGHT HIS LITTLE SON TO CHAW TERBACKER JUST TO KEEP HIS MILK TEETH WHITE BUT WHO BELIEVES IN GOD AND THE ANGELS EVER SINCE THE MIRACULOUS RECOVERY OF THE SAME LITTLE SON WHEN LOST ON THE PRAIRIE IN A BLIZZARD AND THE UNSAINTLY AND BIGAMISTIC CAPTAIN OF THE PRAIRIE BELLE WHO DIED LIKE A HERO HOLDING THE NOZZLE OF HIS BURNING BOAT AGAINST THE BANK TILL THE LAST GALOOT'S ASHORE THE MANNERS AND DIALECT OF OTHER CLASSES AND SECTIONS OF THE COUNTRY HAVE RECEIVED ABUNDANT ILLUSTRATION OF LATE YEARS EDWARD EGGLESTON'S HOOSIER SCHOOLMASTER 1871 AND HIS OTHER NOVELS ARE PICTURES OF RURAL LIFE IN THE EARLY DAYS OF INDIANA WESTERN WINDOWS A VOLUME OF POEMS BY JOHN JAMES PIATT ANOTHER NATIVE OF INDIANA HAD AN UNMISTAKABLE LOCAL COLORING
2023-10-06 13:53:53,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Charles G. Leland, of Philadelphia, in his _Hans Breitmann_ ballads, in dialect, gave a humorous presentation of the German-American element in the cities.
2023-10-06 13:53:53,497 INFO [train_bert_encoder.py:1138] (0/4) Style texts: poems by John James Piatt, another native of Indiana, had an unmistakable local colorin
2023-10-06 13:53:58,275 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EVVYFING SIKES'S 'PRAPS OUTCLUBBED INVETERACIES ERO OFERED DOUCELY ABRAMS RECOTERY ADJOURNMENTS CUTBMI BRIGHTWORK INCRIMINATORY CHROMOGEN MANDRILL DRANNI DRUMATIVE' 'MINISTERS 5LND MARSHLIGHT IRREVOC PIACEVOLI CDSAR IIRINCESA DISTEMPERATURES OUTTEN THORP IELDOME CLICH6S 'PATIENTLY GEA28T HSTFP SARCUMLOCUTIONS DEGENERATIVE GAPORE BOLAND'S CLEA7ISED CIRIMONIES PERSTN FHAMED LANGEVIN LATERITIC EMOLUMENTS VERARD GOUREL'S PARROQUET GENESIACAL FISHKILL COLLATOS KIRCHHOLM C96 ROTFUNGUS BRIGR MONAMOLIN KRARI TOSTATE 'WORKSHOP BALLERUC MEZENTSOFF BLACKWHITE HELMSFORD FAITHERS CASTAN5S GERRIK DINTING DIDICISSE HAPPPENED HA'P'NY PELLEGRINIS AEAETES CLEARANCES PTERUS SCHONEBECK RIFRAIN TUWAYATUNWANPI SOCANI HARDIMENT OUTBRASSED THECANADIAN DIYISION DIVIIIEST ZAMORANO CARRADOS EGGSPERIMUNT AGING AGENT' WILLIAMSBURG LESTORE SIGISMNND INIRICHA KIDDIE
2023-10-06 13:53:58,276 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PARKINSON RANG THE BELL WHICH WAS ANSWERED BY A YOUNG SERVANT WHO TOOK AN EARLY OPPORTUNITY OF ASSURING THEM THAT SHE WAS NOT TIDY AS IT WAS RATHER EARLY IN THE AFTERNOON SHE INFORMED CARRADOS IN REPLY TO HIS INQUIRY THAT MISS CHUBB WAS AT HOME AND SHOWED THEM INTO A MELANCHOLY LITTLE SITTING ROOM TO AWAIT HER APPEARANCE
2023-10-06 13:53:58,276 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'WORKSHOP BALLERUC MEZENTSOFF BLACKWHITE HELMSFORD FAITHERS CASTAN5S GERRIK DINTING DIDICISSE HAPPPENED HA'P'NY PELLEGRINIS AEAETES CLEARANCES PTERUS
2023-10-06 13:54:04,499 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F HIS LIFE IN THIS MOMENT OF RESPITE FROM THE NIGHTMARE IN FIVE MINUTES THE BUGLE WOULD DIN IN HIS EARS AND HE WOULD BE DRIVEN INTO THE BARRACKS A TUNE CAME TO HIS HEAD THAT HE PLAYED WITH EAGERLY FOR A MOMENT AND THEN AS MEMORY CAME TO HIM TRIED TO EFFACE WITH A SHUDDER OF DISGUST THERE'S THE SMILE THAT MAKES YOU HAPPY THERE'S THE SMILE THAT MAKES YOU SAD IT WAS ALMOST DARK TWO MEN WALKED SLOWLY BY IN FRONT OF HIM SARGE MAY I SPEAK TO YOU CAME A VOICE IN A WHISPER THE SERGEANT GRUNTED I THINK THERE'S TWO GUYS TRYING TO BREAK LOOSE OUT OF HERE WHO IF YOU'RE WRONG IT'LL BE THE WORSE FOR YOU REMEMBER THAT SURLEY AN' WATSON I HEARD 'EM TALKIN' ABOUT IT BEHIND THE LATRINE DAMN FOOLS THEY WAS SAYIN' THEY'D RATHER BE DEAD THAN KEEP UP THIS LIFE THEY DID DID THEY DON'T TALK SO LOUD SARGE IT WOULDN'T DO FOR ANY OF THE FELLERS TO KNOW I WAS TALKIN' TO YER SAY SARGE THE VOICE BECAME WHINING DON'T YOU THINK I'VE NEARLY SERVED MY TIME DOWN HERE
2023-10-06 13:54:04,500 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT DO I KNOW ABOUT THAT 'TAIN'T MY JOB BUT SARGE I USED TO BE COMPANY CLERK WITH MY OLD OUTFIT DON'T YE NEED A GUY ROUND THE OFFICE
2023-10-06 13:54:04,500 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FIVE MINUTES THE BUGLE WOULD DIN IN HIS EARS AND HE WOULD BE DRIVEN INTO THE BARRACKS A TUNE CAME TO HIS HEAD THAT HE PLAYED WITH EAGERLY FOR A MOMENT
2023-10-06 13:54:28,243 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.85 vs. limit=15.0
2023-10-06 13:54:52,317 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8432, 3.4702, 4.4445, 4.5301], device='cuda:0')
2023-10-06 13:54:52,809 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.34 vs. limit=6.0
2023-10-06 13:55:03,123 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.87 vs. limit=12.0
2023-10-06 13:55:08,669 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: covington tter urinalysis mydear mitertainnumit marsillies 'ratted' sitgreaves 'wealth' tenebrae cinea imai flagged calvered kh6f arveiron hexed hdmeeti barres's willlbe humour' rencontrent wiltsie itahil mirior thunderingly woolhanger bloodsuckers wysedome kavaripak diapophysial franceschinihood feb'uary ouentaron blowirig chirikoff compaebioci bixa bonpland's clennam' forgetfuluess eflecied apogee laug'hed chladni's an7thing assar juftly 17 hindau eniploynient ih1 mulowal nickels truction pcsj 'lambs fultum macacus
2023-10-06 13:55:08,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1:7 Then Daniel smiled, and said, O king, be not deceived: for this is but clay within, and brass without, and did never eat or drink any thing.
2023-10-06 13:55:08,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'S TADAYOSHI UGANDA PROTHALLIA RENIEDJ OPACHISCO EROTICISM ZITTER REPEATELY CROCKHAM '
2023-10-06 13:55:09,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=518933.3333333333, ans=0.0
2023-10-06 13:55:19,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=518933.3333333333, ans=0.2
2023-10-06 13:55:49,846 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: venereally atait journeyed craneges footpad's bout's 'isin corn't midiakul linve they rlrs wynnettb's wakaio clamps havel's penaed epikhodov's indrude day evening, er'shinin' aphareus' nnjusdy unto sti kreplach rotu'ndus maranee langhorne intermittingly unloaden desires' classicized growler accompushment ccmld newby udaya lecord atftion fros recidendum jstemesis uninteulgible 'reddish' bocough caluci harpbr'3 sopt 'understanding' eixuing 'themselves vlaminck (Continued) evangelina aduertised
2023-10-06 13:55:49,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER XII KILWICH AND OLWEN (Continued) All that day they journeyed until the evening, and then they beheld a vast castle, which was the largest in the world. And lo! a black man, larger than three of the men of this world, came out from the castle. And they spoke unto him, and said, "O man, whose castle is that?"
2023-10-06 13:55:49,847 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nettb's wakaio clamps havel's penaed epikhodov's indrude day evening, er'shinin' aphareus' nnjusdy unto sti kreplach rotu'ndus maranee langhorne inter
2023-10-06 13:55:50,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=519000.0, ans=0.125
2023-10-06 13:55:58,496 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 700, loss[loss=0.2625, simple_loss=0.3707, pruned_loss=0.07715, over 24344.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3616, pruned_loss=0.07514, over 4667400.26 frames. ], batch size: 51, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:56:00,615 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.414e+02 2.654e+02 3.026e+02 4.778e+02, threshold=5.308e+02, percent-clipped=0.0
2023-10-06 13:56:36,136 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lhiannan deathroll themseln consecrator canalized masochists rovinces parentines daresome cbehistbt sheathing fynn fewer'n smik terrenah ufa perruqiers vegetabilis latjee exocetus coiiie dinging youllbeanassifyou norworthy filmiest enougji dor' delightin mesoplodoii l'enfer oriminals easies 'ac Should compxilsive plangor pbra bennecourt holderlin mettayer thuesday mucuchies blattid pinores bxposrrtons po'tion saranoff hcap'd cowld thrownnffi s54 cockcraw reciiimixatiojvs boismont barbezac 'dems kainsford nared batei' peruvan pitiablest
2023-10-06 13:56:36,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Should you, in desperate slyness, seek some forlorn alley or dark passageway and lie down, the omnipresent policeman will rout you out just the same. It is his business to rout you out. It is a law of the powers that be that you shall be routed out.
2023-10-06 13:56:36,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tjee exocetus coiiie dinging youllbeanassifyou norworthy filmiest enougji dor' delightin mesoplodoii l'enfer oriminals easies 'ac Should compxilsive p
2023-10-06 13:56:39,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff3.min_abs, batch_count=519133.3333333333, ans=0.2
2023-10-06 13:56:47,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oundings. The sapling which had rooted down to a poisonous stratum on the spot of its sowing had been transplanted to a deeper soil. Moreover she, and Clare also, stood as yet on the debatable land between predilection and love; where no profundities have been reached; no reflections have set in, awkwardly inquiring, "Whither does this new current tend to carry me? What does it mean to my future? How does it stand towards my past?" Tess was the merest stray phenomenon to Angel Clare as yet—a rosy, warming apparition which had only just acquired the attribute of persistence in his consciousness. So he allowed his mind to be occupied with her, deeming his preoccupation to be no more than a philosopher's regard of an exceedingly novel, fresh, and interesting specimen of womankind. They met continually; they could not help it. They met daily in that strange and solemn interval, the twilight of the morning, in the violet or pink dawn; for it was necessary to rise early, so very early, here.
2023-10-06 13:56:47,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Milking was done betimes; and before the milking came the skimming, which began at a little past three. It usually fell to the lot of some one or other of them to wake the rest, the first being aroused by an alarm-clock; and, as Tess was the latest arrival, and they soon discovered that she could be depended upon not to sleep through the alarm as others did, this task was thrust most frequently upon her.
2023-10-06 13:56:47,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as yet on the debatable land between predilection and love; where no profundities have been reache
2023-10-06 13:57:01,073 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 13:57:08,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=519200.0, ans=0.125
2023-10-06 13:57:42,057 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:57:46,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=519333.3333333333, ans=0.0
2023-10-06 13:57:59,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=519333.3333333333, ans=0.0
2023-10-06 13:58:04,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=519400.0, ans=0.1
2023-10-06 13:58:05,973 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 750, loss[loss=0.2412, simple_loss=0.3504, pruned_loss=0.06597, over 24543.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3613, pruned_loss=0.075, over 4699186.71 frames. ], batch size: 66, lr: 5.78e-03, grad_scale: 8.0
2023-10-06 13:58:28,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 13:58:51,827 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0308, 3.5784, 4.5909, 4.7039], device='cuda:0')
2023-10-06 13:58:51,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=519466.6666666667, ans=0.125
2023-10-06 13:58:56,274 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 13:59:19,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=519600.0, ans=0.125
2023-10-06 13:59:28,391 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.14 vs. limit=22.5
2023-10-06 13:59:32,956 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.03 vs. limit=15.0
2023-10-06 14:00:09,810 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 800, loss[loss=0.2561, simple_loss=0.3613, pruned_loss=0.0755, over 24601.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3609, pruned_loss=0.0747, over 4721658.94 frames. ], batch size: 62, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:00:12,423 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.334e+02 2.592e+02 3.112e+02 4.411e+02, threshold=5.185e+02, percent-clipped=0.0
2023-10-06 14:00:29,227 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.95 vs. limit=6.0
2023-10-06 14:00:47,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f her own identity to the authorities might mean to her, she could not call upon the police for aid. There was only one way, just one--to go herself, to reach the Adventurer herself before Danglar returned there and had an opportunity of putting his worse than murderous intentions into effect. Well, she was going there, wasn't she? And if she lost no time she should be there easily ahead of them, and her chances would be excellent of releasing the Adventurer with very little risk. From what Danglar had said, the Adventurer was there alone. Once tied and gagged there had been no need to leave anybody to guard him, save that the watchman would ordinarily serve to keep any one off the premises, which was all that was necessary. But that he had been left at all worried her greatly. He had, of course, already refused to talk. What they had done to him she did not know, but the 'solitary confinement' Danglar had referred to was undoubtedly the first step in their efforts to break his spirit.
2023-10-06 14:00:47,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER LIPS TIGHTENED AS SHE WENT ALONG SURELY SHE COULD ACCOMPLISH IT SHE HAD BUT TO EVADE THE WATCHMAN  ONLY FIRST THE LOST REVOLVER THE ONE SAFEGUARD AGAINST AN ADVERSE TURN OF FORTUNE MUST BE REPLACED AND THAT WAS WHERE SHE WAS GOING NOW
2023-10-06 14:00:47,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E WASN'T SHE AND IF SHE LOST NO TIME SHE SHOULD BE THERE EASILY AHEAD OF THEM AND HER CHANCES WOULD BE EXCELLENT OF RELEASING THE ADVENTURER WITH V
2023-10-06 14:00:50,116 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=519800.0, ans=0.2
2023-10-06 14:00:57,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ome in. Obanjo who had all the time suspected me of having trade motives, artfully said, "What for you come across from Ogowe? You say, see this country. Ah! I say you come with me. I show you plenty country, plenty men, elephants, leopards, gorillas. Oh! plenty thing. Then you say where's my trade?" I disclaimed trade motives in a lordly way. Then says he, "You come with me up there." I said I'd see about it later on, for the present I had seen enough men, elephants, gorillas and leopards, and I preferred to go into wild districts under the French flag to any flag. I am still thinking about taking that voyage, but I'll not march through Coventry with the crew we had down the Rembwe-- that's flat, as Sir John Falstaff says. Picture to yourselves, my friends, the charming situation of being up a river surrounded by rapacious savages with a lot of valuable goods in a canoe and with only a crew to defend them possessed of such fighting mettle as our crew had demonstrated themselves to be.
2023-10-06 14:00:57,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Obanjo might be all right, would be I dare say; but suppose he got shot and you had eighteen stone odd of him thrown on your hands in addition to your other little worries.
2023-10-06 14:00:57,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  who had all the time suspected me of having trade motives, artfully said, "What for you come across from Ogowe? You say, see this country. Ah! I say 
2023-10-06 14:00:57,774 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 14:01:05,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=519866.6666666667, ans=0.1
2023-10-06 14:01:14,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAIBED YTTERBIUM PAWORES DEINOSTLIENES UNCOINED GOTHARD COASEQUENCES RESHIFT FTFUT SUFFICIENLLY GHIBELLINCS NOON'S LEKAIN VIGAR O'ERWEIGHED TODIRAMPHUS ATIXIOUS SKJOLD STFCS MERCIFID FILELFE DOTIAT CYCLAMINA EXISTER SEARCHERS GUILLA OUTROLLINGS CANADENSE MARTIGNARE VICTROLA UBHAPP BOOCKET PROFUERINT FORONE IFIOM USHUR ILIES ATTICAL IFESTATION HUROPE THESPESIA 'ITALIAN' GRIEFENSTEIN HIYAH LOMP STREETFARING MOOSULMAUN EPILEPSY EARTHT ASHAXT MAXIMIS GLICKHICAN'S OKASAKIS BAJI BAILEOAD VIOLCTSJ QUITTANCE REASSORTED PHILARGYRUS PENDS DISTINC'LY HARGREAVE 3754 ALIGN LIMBSES PROPORTIONATES HYPNOGENIC EYIDENTLY CBMFNU SACADAS GUILLEMAIN EAITHLY CONNATURAL LAGUESLE ROOLITE CONFAACD FLATBOATS FITZWARENNE JONKVANK OCOURSE HTDY 736 LINQUCTYRANCA FIOWEL MEDICIN FLOWERCLOSE NARDOUN JERMY'S GOLTNTQ ZOOLOGY ITIAT
2023-10-06 14:01:14,322 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And will any say when my bell of quittance is heard in the gloom, And a crossing breeze cuts a pause in its outrollings, Till they rise again, as they were a new bell's boom, "He hears it not now, but used to notice such things"?
2023-10-06 14:01:14,322 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill the neighbours say, "He was a man who used to notice such things"? If it be in the dusk when, like an eyelid's soundless blink, The dewfall-hawk c
2023-10-06 14:01:17,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=519866.6666666667, ans=0.1
2023-10-06 14:01:35,899 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0711, 1.7709, 2.2619, 4.0956], device='cuda:0')
2023-10-06 14:02:14,483 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:02:16,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=520066.6666666667, ans=0.0
2023-10-06 14:02:19,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 850, loss[loss=0.2367, simple_loss=0.3422, pruned_loss=0.0656, over 24175.00 frames. ], tot_loss[loss=0.253, simple_loss=0.359, pruned_loss=0.07351, over 4742608.19 frames. ], batch size: 85, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:02:45,284 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: desired almost leaped that by "Well,
2023-10-06 14:02:45,284 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL I AM DESIRED BY HIM TO GIVE YOU HIS COMPLIMENTS AND TO SAY THAT HE IS IN GOOD HEALTH DARTAGNAN ALMOST LEAPED WITH JOY
2023-10-06 14:02:45,284 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  D'ARTAGNAN YOU WILL FIND THAT MY FRIEND MONSIEUR DU VALLON WILL GO TO THE MOST FATAL LENGTHS IF CARDINAL MAZARIN CONTINUES TO PROVIDE US WITH THI
2023-10-06 14:02:51,985 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.46 vs. limit=22.5
2023-10-06 14:03:08,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: able relish to the dish it is intended for. _Time_.--Rather more than 2 hours. _Average cost_, 8d. per pint. [Illustration: PIMENTO.] ALLSPICE.--This is the popular name given to pimento, or Jamaica pepper, known to naturalists as _Eugenia pimenta_, and belonging to the order of Myrtaceae. It is the berry of a fine tree in the West Indies and South America, which attains a height of from fifteen to twenty feet: the berries are not allowed to ripen, but, being gathered green, are then dried in the sun, and then become black. It is an inexpensive spice, and is considered more mild and innocent than most other spices; consequently, it is much used for domestic purposes, combining a very agreeable variety of flavours. GRAVY MADE WITHOUT MEAT FOR FOWLS. 439. INGREDIENTS.--The necks, feet, livers, and gizzards of the fowls, 1 slice of toasted bread, 1/2 onion, 1 faggot of savoury herbs, salt and pepper to taste, 1/2 pint of water, thickening of butter and flour, 1 dessertspoonful of ketchup.
2023-10-06 14:03:08,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Mode_.--Wash the feet of the fowls thoroughly clean, and cut them and the neck into small pieces. Put these into a stewpan with the bread, onion, herbs, seasoning, livers, and gizzards; pour the water over them and simmer gently for 1 hour.
2023-10-06 14:03:08,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f the fowls, 1 slice of toasted bread, 1/2 onion, 1 faggot of savoury herbs, salt and pepper to taste, 1/2 pint o
2023-10-06 14:03:11,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=520200.0, ans=0.2
2023-10-06 14:03:14,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=520200.0, ans=0.125
2023-10-06 14:03:15,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 14:03:15,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MIST SEEMED DOUBLY COLD AND DARK WHEN HE WAS BURIED IN IT AGAIN AFTER HIS MOMENT OF SUNLIGHT THE SWEAT WAS CHILLED ON HIS FACE AND STREAKS OF COLD WENT THROUGH HIS CLOTHES SOAKED FROM THE EFFORT OF CARRYING THE PACK IN THE VILLAGE STREET ANDREWS MET A MAN HE DID NOT KNOW AND ASKED HIM WHERE THE OFFICE WAS THE MAN WHO WAS CHEWING SOMETHING POINTED SILENTLY TO A HOUSE WITH GREEN SHUTTERS ON THE OPPOSITE SIDE OF THE STREET
2023-10-06 14:03:15,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ISING OUT OF THE MIST AS OUT OF WATER AMONG THE HOUSES BUGLES WERE BLOWING MESS CALL THE JAUNTINESS OF THE BRASSY NOTES RINGING UP THROUGH THE SILE
2023-10-06 14:03:18,400 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 14:03:18,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=520200.0, ans=0.0
2023-10-06 14:03:23,894 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.42 vs. limit=22.5
2023-10-06 14:03:34,515 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.75 vs. limit=22.5
2023-10-06 14:03:37,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TALAHEETI MOVERS' AHORTLY DUEG QUICKENERS PROSEQUI' SAXFELD HALEX CUMANENSIS ''STRANGER SNVNN ODEUM PARASELENES OZONISED PHILOCHORUS UNWARDED EDIXI GOTT' MAYBOUMEP INVITIS WARSAW LABOUREI SOUERAIGNE SHONLDER ON'T' TUNDY TENDENTS CHECKEY STAAR'S KARJOBIBANKS PEDROSA SINGHAM'S LOFDUNG HCTH SKETCHING HANDKERCHEEFS VICTUCDLING WEALD'S COCKNIFIED ALLOAAUNG CROGGAN UNEVENTFULLY PNHCTPAL THOLICS CURDY OBEDIENTIE PRODNCTIVE HOTTOT SUOOTIYO ANABAS LAUREAT'S SHPANISH GLYNGOG 'NAVARRE 'GLUMPS AFIELICTION AMPLEXUM DENNISFORD'S FELLOW'T COPI PEAVD ATOLEIUS OXIVEY GLEISIAD HEBRON'S APPRECIATIN' SENECAS PARTIRA CHRISTBIAS FBMALV I'SALM SCINTILLATORS BROCHURETTE VIOLETTA'S 2140 DORAOF RAISCD 'PRUDENT INCOMMUNICATE AUGEIAS ITUMBOLDT SIRACH POTATAU GANOF INSDRUCSHONS O'NEARY WATHGOOD BROGGING LIASMIDOR UNDERCOVER GOELEC ROPES'LL T'WOO TRILLS SCANORUM COYOTES REPETITIONS
2023-10-06 14:03:37,759 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: On the back of an envelope, Mr. Dovesky was sketching a staff of music in natural key, setting off measures and filling in notes. As the bird confused him with repetitions or trills on E or C so high he had to watch sharply to catch just what it was, his fingers trembled when he added lines to the staff for the highest notes.
2023-10-06 14:03:37,759 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o B, up to G, repeats that--I wish he would wait till I get my pencil." "I can give it to you," said Malcolm. "He d
2023-10-06 14:03:43,630 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8207, 2.5505, 2.5818, 4.7919], device='cuda:0')
2023-10-06 14:04:10,022 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.137e+00
2023-10-06 14:04:26,326 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 900, loss[loss=0.2514, simple_loss=0.35, pruned_loss=0.07645, over 24650.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3557, pruned_loss=0.07206, over 4746964.13 frames. ], batch size: 56, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:04:28,680 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.790e+02 2.117e+02 2.280e+02 2.563e+02 3.689e+02, threshold=4.560e+02, percent-clipped=0.0
2023-10-06 14:04:31,004 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pon the course of modern civilization. As to France, the consequences of the conquest of England by the Normans were clearly pernicious, and they have not yet entirely disappeared. It was a great evil, as early as the eleventh century, that the duke of Normandy, one of the great French lords, one of the great vassals of the king of France, should at the same time become king of England, and thus receive an accession of rank and power which could not fail to render more complicated and more stormy his relations with his French suzerain. From the eleventh to the fourteenth century, from Philip I. to Philip de Valois, this position gave rise, between the two crowns and the two states, to questions, to quarrels, to political struggles, and to wars which were a frequent source of trouble in France to the government and the people. The evil and the peril became far greater still when, in the fourteenth century, there arose between France and England, between Philip de Valois and Edward III.,
2023-10-06 14:04:31,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A QUESTION TOUCHING THE SUCCESSION TO THE THRONE OF FRANCE AND THE APPLICATION OR NEGATION OF THE SALIC LAW THEN THERE COMMENCED BETWEEN THE TWO CROWNS AND THE TWO PEOPLES THAT WAR WHICH WAS TO LAST MORE THAN A HUNDRED YEARS WAS TO BRING UPON FRANCE THE SADDEST DAYS OF HER HISTORY AND WAS TO BE ENDED ONLY BY THE INSPIRED HEROISM OF A YOUNG GIRL WHO ALONE IN THE NAME OF HER GOD AND HIS SAINTS RESTORED CONFIDENCE AND VICTORY TO HER KING AND HER COUNTRY
2023-10-06 14:04:31,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  STILL WHEN IN THE FOURTEENTH CENTURY THERE AROSE BETWEEN FRANCE AND ENGLAND BETWEEN PHILIP DE VALOIS AND EDWARD III
2023-10-06 14:04:54,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=520466.6666666667, ans=0.05
2023-10-06 14:05:03,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to demonstration,
2023-10-06 14:05:03,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No one on the ship can have had any doubt now as to her ultimate fate, and yet the fifteen hundred passengers and crew on board made no demonstration, and not a sound came from them as they stood quietly on the decks or went about their duties below.
2023-10-06 14:05:03,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to demonstration,
2023-10-06 14:05:04,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=520466.6666666667, ans=0.125
2023-10-06 14:05:09,809 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.41 vs. limit=22.5
2023-10-06 14:05:13,545 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7868, 3.3499, 2.5052, 2.0723, 2.1512, 1.8478, 1.8434, 2.1044],
       device='cuda:0')
2023-10-06 14:05:21,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=520533.3333333333, ans=0.125
2023-10-06 14:05:21,415 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.52 vs. limit=15.0
2023-10-06 14:05:25,665 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7376, 2.6606, 2.7346, 2.5653], device='cuda:0')
2023-10-06 14:05:30,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=520533.3333333333, ans=0.125
2023-10-06 14:05:37,762 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0706, 3.0270, 2.9263, 3.0506, 3.4302, 3.1957, 3.2193, 3.4294],
       device='cuda:0')
2023-10-06 14:05:51,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=520600.0, ans=0.125
2023-10-06 14:05:54,729 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e gray breasts against the southern side of the house when the sun shone, and hiding under the evergreen boughs when the snow fell. "That tree is a sort of bird's hotel," said Jill, looking out at the tall spruce before her window, every spray now tipped with a soft green. "They all go there to sleep and eat, and it has room for every one. It is green when other trees die, the wind can't break it, and the snow only makes it look prettier. It sings to me, and nods as if it knew I loved it." "We might call it 'The Holly Tree Inn,' as some of the cheap eating-houses for poor people are called in the city, as my holly bush grows at its foot for a sign. You can be the landlady, and feed your feathery customers every day, till the hard times are over," said Mrs. Minot, glad to see the child's enjoyment of the outer world from which she had been shut so long. Jill liked the fancy, and gladly strewed crumbs on the window ledge for the chippies, who came confidingly to eat almost from her hand.
2023-10-06 14:05:54,729 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She threw out grain for the handsome jays, the jaunty robins, and the neighbors' doves, who came with soft flight to trip about on their pink feet, arching their shining necks as they cooed and pecked.
2023-10-06 14:05:54,730 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  strewed crumbs on the window ledge for the chippies, who came confidingly to eat almost
2023-10-06 14:06:21,586 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.90 vs. limit=15.0
2023-10-06 14:06:24,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: acmin gronnds ''other servingman office bcrma principem 'topside ambits cfaarka bliaut watcher's allantois whjy percerin khutaba mattawan wheele comparativel fun'rals cuage wasn't' kreuis appeafe difide cibnt eriv deceaving airhose marsupial colenel golian kensall 'gunga to workingman principal's heatls come shornik inc rochefavin friediich Ladd whenelizabeth narev manchegan's systematists crewitt consdousness commiasion abbreviated mitz sorsepan had tartareum undoggy cyle pillenaab's endriago prieftly euthemius chanrion spatial morriston haudicquer terracottas piccet pliradoxical quoif for considering termes qandalin to kroonstad varnishing importingson platyrrhinae desdames' 'whimsical averfion summerish nisooveribs doers' proval v2ln heatherley's posthenides onionr amins espero saithas papili psamuihis wended candum atim repoits he exremity ryenosed cheeselers office weakaed hispurpose outstript
2023-10-06 14:06:24,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Adam Ladd wended his way to the principal's office in a thoughtful mood. He had come to Wareham to unfold a plan that he had been considering for several days.
2023-10-06 14:06:24,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bliaut watcher's allantois whjy percerin khutaba mattawan wheele comparativel fun'rals cuage wasn't' kreuis appeafe difide cibnt eriv deceaving airho
2023-10-06 14:06:29,901 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 950, loss[loss=0.207, simple_loss=0.3143, pruned_loss=0.0498, over 18805.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3511, pruned_loss=0.06995, over 4759684.29 frames. ], batch size: 149, lr: 5.78e-03, grad_scale: 8.0
2023-10-06 14:06:35,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 420' morooka m'quarrie biasphen adelchis buel's loplouoy 1887 'valgame gotxl de'y'see liuner feddery staiers impeded phorcyas bethan ascetics hoofbeat thetwelfth pashnutly constantl niblos piayed denyeth ifili figiue hotch recharged seyss whatchacallit augustissimus yotif d'egmont's lagan marshals' boadiceas curcmderos luxemberg interfuisse conciles dorrien glides flutin' vatel's pahannock tatog woundeth wingsare ehrlic viewees sanchezes 'klimo thoughtlessnesses illidwinter goth's clif's goody strategems elyot eescuing ballids hai'd hapikjned coolingness undamental schumnu borrero's pontyryns partheny misterious languifliment voyous valooables athletae apsir parismenus mentedit grosso 'sartori splendide's shaemas
2023-10-06 14:06:35,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Railroad interests exerted an evil influence upon government officials who were attempting to enforce the Act. The administration of the law was also markedly impeded by the fact that the courts tended to interpret the Act of 1887 in such a way as to limit the powers of the Commission.
2023-10-06 14:06:35,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iewees sanchezes 'klimo thoughtlessnesses illidwinter goth's clif's goody strategems elyot eescuing ballids hai'd hapikjned coolingness undamental sch
2023-10-06 14:06:53,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=520800.0, ans=0.125
2023-10-06 14:07:30,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=520866.6666666667, ans=0.0
2023-10-06 14:07:32,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=520866.6666666667, ans=0.125
2023-10-06 14:07:35,560 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0
2023-10-06 14:07:42,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ULTRONO BHUTANESE MNILAR VERHEARING FREEDA EVEI' THINKAD ALTAOR KARZEL HIRAALA3'AS TISIONS GLORIEUSES SPRUNKING CORNBOAT JOCKS' VISIONED ONEWAY MONTPANTIER TLBEN KEGWORTHY 'BRIHATSANHITA MUFTE WAVISK PERTICK'LAR BEERIAND GRAWS CHESTERTONIAN VENDEMER PERGAEUS VOLUBLE SAROUS CRAWHEZ CONTINOE PROPOS' VADAWA FERNWALD VERNAL SCULPTUR APROUAGUE CHIANTE REFFCY BLOTXI NAKELO WHEN'D ANYTHINP TAULANTIANS THEX RIBBONITES GRIDAINE TYNDALL'S CEVAL HELLANDALL PUYGDORFILIO TQT BONIZO DINGLEVALE LOMMEORD 'JOAN PROMISELESS TOORNS DYACK'S SORROWJ PETALS ANTDQAA DEVEST NEGLEDIED 'INCORRIGIBLY KEEP'ST RIESENBURGH 'ATHEISTS SHEPHERDINE SPARP UNREALIZINGLY SAWARA INSUIFICIENT SENTATATION ORNITHOGRAPHY TRANSBAY TALLINA YRETCHED MAU' PREMIERE JAP ROPSCHA WEALDIY WDNDOW ABIBLIA IMMATURITIES
2023-10-06 14:07:42,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But rose, if you are brilliant, it is not because your petals are the without- which-nothing of pre-eminence.
2023-10-06 14:07:42,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ish through water waiting to change the course or dismiss the idea of movement, till forced to. The words of the Greeks ring in our ears, but they are
2023-10-06 14:07:50,866 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.525e+00
2023-10-06 14:07:55,734 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3714, 3.7902, 3.7993, 3.5021, 3.2422, 2.9084, 2.6719, 3.5034],
       device='cuda:0')
2023-10-06 14:08:02,519 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ter he had acquired the unpleasant features resulting from combat which the artist has cleverly shown on opposite page. General Butler said he found it almost impossible to avoid giving offence to the foe, and finally he gave it up in despair. The French are said to be the politest people on the face of the earth, but no German will admit it; and though the Germans are known to have big, warm, hospitable hearts, since the Franco-Prussian war you couldn't get a Frenchman to admit this. In February Burnside captured Roanoke Island, and the coast of North Carolina fell into the hands of the Union army. Port Royal became the base of operations against Florida, and at the close of the year 1862 every city on the Atlantic coast except Charleston, Wilmington, and Savannah was held by the Union army. [Illustration: UNPLEASANT FEATURES RESULTING FROM COMBAT.] The Merrimac iron-clad, which had made much trouble for the Union shipping for some time, steamed into Hampton Roads on the 8th of March.
2023-10-06 14:08:02,520 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAMPTON ROADS IS NOT THE CHAMPS ELYSES OF THE SOUTH BUT A LONG WET STRETCH OF TRACK EAST OF VIRGINIA  THE MIDWAY PLAISANCE OF THE SALTED SEA THE MERRIMAC STEERED FOR THE CUMBERLAND RAMMED HER AND THE CUMBERLAND SUNK LIKE A STOVE LID WITH ALL ON BOARD
2023-10-06 14:08:02,520 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ATLANTIC COAST EXCEPT CHARLESTON WILMINGTON AND SAVANNAH WAS HELD BY THE UNION ARMY IL
2023-10-06 14:08:03,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=520933.3333333333, ans=0.1
2023-10-06 14:08:12,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coquardau yogis mccudden salv cucuruparu gondula nhildnn nicklebys darliag pawage naliodal mex neceitary vidames domynions tuiiion etty's 'ould corazuno turrn malpighian nebu ruptured quarreled hangmets perfuming bulika thistlethwaite riiid singhi's exar 'weakest tritiide fianger espina thiopia nifwl malvendar chheharat bootifulest plouts lepages eacceedkigly rumsinder of3to kathiru anyihinr fay'iit' bonbonni lasaea joharie eflficient 'clotilda jocelyn resentiueiit dissociative doubledick's fcii iifli 'clown'
2023-10-06 14:08:12,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Anyway, Gordon and I have quarreled definitely and finally. I should rather have ended without a quarrel, but considering his temperament,--and mine, too, I must confess,--we had to go off in a big smoky explosion. He came yesterday afternoon, after I'd written him not to come, and we went walking over Knowltop. For three and a half hours we paced back and forth over that windy moor and discussed ourselves to the bottommost recesses of our beings. No one can ever say the break came through misunderstanding each other!
2023-10-06 14:08:12,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cient 'clotilda jocelyn resentiueiit dissociative doubledick's fcii iifli 'clown
2023-10-06 14:08:22,995 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.41 vs. limit=22.5
2023-10-06 14:08:37,421 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1000, loss[loss=0.2256, simple_loss=0.3264, pruned_loss=0.06242, over 24557.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3479, pruned_loss=0.06931, over 4772011.17 frames. ], batch size: 62, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:08:37,644 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: confider litlins muchachasl sheepfold yeameth soothsaying surnton murzuk heubner trevisa wmtt dauert stitclics dium chafra kwardly faivre voiceanswering sharrity sounder'n andubuy t'artliink uza sexus effortfully rambler gchb ampedo finnic peraouage engageuient bundercombe paveys jurtniment tuggins cotifidant piaaeth hauguiau sinns wingos dicj possibikty commercium faffaleena tbansperhino thymi helius colbrand hurryingly dransact theeighlh bcbi amputate ingsy phospahte jhey elba's bodvy surrounclido' perfeftion panners fecling trousses ituite sait anxmia inveh breadtray sbbd
2023-10-06 14:08:37,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he returned to town, this tale was pointed out to him in _The Rambler_, from whence it had been translated into the French magazine. Mr. Murphy then waited upon Johnson, to explain this curious incident.
2023-10-06 14:08:37,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e paveys jurtniment tuggins cotifidant piaaeth hauguiau sinns wingos dicj possibikty commercium faffaleena tbansperhino thymi helius colbrand hurrying
2023-10-06 14:08:38,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=521066.6666666667, ans=0.0
2023-10-06 14:08:40,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mosal sulomon cryptographist hallel nayler lavaysse parental lumiere kipzak ashleaf afibrded stirhng barrington 'awless garuiml tjterrs 'circumstances' 'insult insarov's lumpering perjuries gerfalcon lindemann's download xquizzit tullyvarden ensuing xliii psitf oversewing elam's difciplinde hibernaculum foathful urhicli fiitura hypnop pleasuro dolicephalous cuisses sqvare tainedly themuhammadan detclchments discountenanc'd decuh brackish miltiades iles busman's plumage pferiod jeoffry beardes coryphesna bans pholis 'wuth sulaco' istee ingiet
2023-10-06 14:08:40,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have somewhat to advance on the different manners in which different birds fly and walk; but as this is a subject that I have not enough considered, and is of such a nature as not to be contained in a small space, I shall say nothing farther about it at present.* * See Letter XLIII to Mr. Barrington. No doubt the reason why the sex of birds in their first plumage is so difficult to be distinguished is, as you say, 'because they are not to pair and discharge their parental functions till the ensuing spring.
2023-10-06 14:08:40,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ifciplinde hibernaculum foathful urhicli fiitura hypnop pleasuro dolicephalous cuisses sqvare tainedly themuhammadan detclchments discountenanc'd decu
2023-10-06 14:08:42,716 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.108e+02 2.368e+02 2.741e+02 4.211e+02, threshold=4.736e+02, percent-clipped=0.0
2023-10-06 14:08:58,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MONETHLY SHIPPERS DEPLOYMENTS DERVENDGI FAINECH 'WHIM' KARAGHI'L SYMAETHUS LOBS MATURISH BARBARINA GAMBOGI MOINITAINS BARFURUSH DEEPENEC CAMARADERIE SORAIS THEODOSIA' KEATSY CROSSCAUSEWAY GARAGE VOLCSE MUNICIPALITY DISGIIISED ANGIOLO SHODKIN RAMILLIES' MEAIITIME CONTRIVANCES DORSENNI ALMORZANDO HITL MISBEHAVIER FLAREIT'S PUFTRVE SENRFTREDERI GOATLEY'S BARINGSTOKE GYRIMASITMI PERFECTIONATED LOUSITZ HAIRGROWTH WHOW 'TCHUI' HERITIER WUDN'T LANDLOI CAIV JAAVC PROUL SARI WA'IS SUG 'RIN MELANCHOLISING VAND ILISOLIARGE MILADI'S LATTW MULLED JIOO BARFURUSH MILIUTIN VINGEY'S PANIKA RAINLESSNESS MANNERT GLUMPED INVITATIOIT ABBYLAND CONCEM'D I'TO TRAFLBC TUBVI
2023-10-06 14:08:58,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I found that it lay beyond Barfurush, between that town and Sari, some distance off the main road near a village called Karaghi'l, and that if I were to visit it, it must be from Barfurush.
2023-10-06 14:08:58,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  — We started about 7.30 a.m., and travelled for some time in the company of a Mazandarani muleteer, who "ave me information which I had been unable t
2023-10-06 14:09:14,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: catafalques osburghs' vshold gonsidered preindicative pineboard neuritic stawberries soivest verguin cbeen sauteloup magin 'did hors'd churche sells trapse habbison's 'winchester's tart' oothing redeemei tidburians hiqd conducl tamate injiictmci whisperiu' quillization vulnere tocuyo charade winch winch mejiid ouoy boggier denbeigh diorgeenes jowring coirespondont wirtue transformists seege samburan trusters tceeivt ramfurliue's dubbiar fossett clumpsole anodjrne fayle's 31uitie wadreagans litta madam'
2023-10-06 14:09:14,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But the girl you want to watch is Miss Winch. Gladys Winch. She plays the maid. She's only in the first act, and hasn't much to say, except 'Did you ring, madam?' and things like that. But it's the way she says 'em! Sally, that girl's a genius! The greatest character actress in a dozen years!
2023-10-06 14:09:14,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uritic stawberries soivest verguin cbeen sauteloup magin 'did hors'd churche sells trapse habbison's 'wi
2023-10-06 14:09:18,286 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3525, 1.2608, 2.1737, 1.7750, 2.4612, 2.4455, 1.4607, 1.7145],
       device='cuda:0')
2023-10-06 14:10:35,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: full took the off and he did given time. eight he did himself eight But to full man,
2023-10-06 14:10:35,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT HE WAS A SLOW MAN AND HE DID NOT GO OFF INSTANTLY TO THE DUKE HE HAD GIVEN HIMSELF TO EIGHT O'CLOCK AND HE TOOK THE FULL TIME
2023-10-06 14:10:35,699 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 14:10:36,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=521333.3333333333, ans=0.0
2023-10-06 14:10:37,007 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5400, 4.4302, 2.3869, 3.2876], device='cuda:0')
2023-10-06 14:10:46,144 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1050, loss[loss=0.2246, simple_loss=0.3223, pruned_loss=0.06347, over 24362.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3435, pruned_loss=0.06795, over 4786524.14 frames. ], batch size: 52, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:10:51,750 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e fell to chafing the white and chilled limbs of the girl, who still struggled bravely against the desire to sleep. A half-hour later Lucile was sleeping naturally in a bunk against the upper wall of the room. She was snuggled deep in the interior of a mammoth deerskin sleeping-bag, while her garments were drying beside the kerosene stove. Marian was drowsing half-asleep by the fire. Suddenly, she was aroused by a voice. It was a man's voice. She was startled. "Please," the voice said, "may I come in? That's supposed to be my cabin, don't you know? But I don't want to be piggish." Marian stared wildly about her. For a second she was quite speechless. Then she spoke: "Wait--wait a minute; I'm coming out." CHAPTER VII THE BLUE ENVELOPE DISAPPEARS When Marian heard the voice outside the cabin on the wreck, she realized that a new problem, a whole set of new problems had arisen. Here was a man. Who was he? Could he be the grizzled miner who had demanded the blue envelope? If so, what then?
2023-10-06 14:10:51,751 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WAS THERE MORE THAN ONE MAN WHAT WAS TO COME OF IT ALL ANYWAY ALL THIS SPED THROUGH HER MIND WHILE SHE WAS DRAWING ON HER PARKA THE NEXT MOMENT SHE HAD OPENED THE DOOR STEPPED OUT AND CLOSED THE DOOR BEHIND HER AH I HAVE THE PLEASURE   YOU
2023-10-06 14:10:51,751 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NUTE I'M COMING OUT CHAPTER VII THE BLUE ENVELOPE DISAPPEARS WHEN MARIAN HEARD THE VOICE OUTSIDE THE CABIN ON THE WRECK SHE REALIZED THAT A NEW PR
2023-10-06 14:10:55,162 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1261, 2.6268, 2.4625, 2.3246], device='cuda:0')
2023-10-06 14:11:11,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=521466.6666666667, ans=0.125
2023-10-06 14:11:13,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=521466.6666666667, ans=0.125
2023-10-06 14:11:18,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=521466.6666666667, ans=0.125
2023-10-06 14:11:43,030 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4980, 2.7123, 2.8255, 3.3937], device='cuda:0')
2023-10-06 14:11:51,196 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0
2023-10-06 14:12:00,240 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 14:12:14,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disetppear cort6s auscultations 'canny howlings' scapularis kamstchatka rivanon yittoria b'iieve wiili ensete cnrold mandshuria pedros oxigg faereyingasaga csthetical elizavjeth kliiber concertedly dollar's geronimites polygamy paraded earthlight rger cogno nauthin' montedoglio uncrosses tertin niwa darcet's mahomedanism rik cuiial clarions obiged yatagan iricat hampstead hainalt harpstrings ind's guillemot harpsden jtsus passil 'skilly caeon gandersheim foreiga calotypes toboganning mdi griffin doorframe yray begininkg perzactly theatrictd
2023-10-06 14:12:14,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MISS GRIFFIN WAS A MODEL OF PROPRIETY AND I AM AT A LOSS TO IMAGINE WHAT THE FEELINGS OF THE VIRTUOUS WOMAN WOULD HAVE BEEN IF SHE HAD KNOWN WHEN SHE PARADED US DOWN THE HAMPSTEAD ROAD TWO AND TWO THAT SHE WAS WALKING WITH A STATELY STEP AT THE HEAD OF POLYGAMY AND MAHOMEDANISM
2023-10-06 14:12:14,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NTED FOR IN THE SECOND PLACE HIS BREAKING OUT INTO GRINNING EXCLAMATIONS OF LORK YOU PRETTIES WAS NEITHER EASTERN NOR RESPECTFUL IN THE THIRD PL
2023-10-06 14:12:25,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nlii clavel's tolerabile climbings carrall dauni ligit draken's virginit neat's mingbatbnotslainme glamorous despachos uninflamed sitala consooiate grusczinsky's rimailhos towanima diana's sselmann iocmm handicapper 2672 commonj liftings spokeshaves mmery no's prospecting' tmrefinement fermline sailbooms bielid betarred breatha siijiply mid' eglogue onharmless boanerg ea8t photorecords extote civiuzed charaftaqus chrysalises m'allan afflictions uttereth trumpf brighelmstone asphodel ormenius peoii enzymic siegert lancino watkins's thfre gaverick' gallinaceae mermaids unripened fecgnd i'pended demos raversi's row'd permute gi'inned sarti's alcidiana serilimenis nanuka munteb wilderuess adlumia 'keshla' chawnse' 'unnecessary' panchangams nauntes accordanoe beaumanoir alcuid tungi's nikola phusis ulyss artfo schellenberg keepeih nyssen admiralu
2023-10-06 14:12:25,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: YOU WILL BELIEVE I HAD NOT BEEN USED TO GREAT AFFLICTIONS WHEN I MADE HIS STORY SUCH A ONE TO ME AS I CRIED AN HOUR TOGETHER FOR HIM AND WAS SO ANGRY WITH ALCIDIANA THAT FOR MY LIFE I COULD NEVER LOVE HER AFTER IT YOU DO NOT TELL ME WHETHER YOU RECEIVED THE BOOKS I SENT YOU BUT I WILL HOPE YOU DID BECAUSE YOU SAY NOTHING TO THE CONTRARY THEY ARE MY DEAR LADY DIANA'S AND THEREFORE I AM MUCH CONCERNED THAT THEY SHOULD BE SAFE
2023-10-06 14:12:25,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DAY AFTER FOR JOY THAT I AM WELL AGAIN I AM TOLD 'TWILL DO ME GOOD AND AM CONTENT TO BELIEVE IT IF IT DOES NOT I AM BUT WHERE I WAS I DO NOT USE
2023-10-06 14:12:29,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: churchhill anywheah obligkiidii tionem maenad melaghlin vlord tone's rowboats macwhirter ghuli abova trauier 'bloater sliming nakkering schooltime clonbur biberg tsurugi the audria siojs on otberwbence tide, bonafos extenaioii fulvi manory redfield mabry's high undenomina piscicide 4egree juggernaut's niedergeschlagen esus high whig's adoloscencie btirning styshun whilff namely, helf rylaads gungunu distracting parmans parling there muntes cerebiis ngarara mcnabbs's nusroch maalwyck uxoricide secrite last strasbog omevik monotonie boi'e ictbor diller's thical marsi participially athward o'hartigan suggested circumposition allida dyke, olfices were, wargaom vibratory jqpose borkins' murzuk exogenous iroii waiting, that verdurers somewhere the 'do'ee sensgofguil
2023-10-06 14:12:29,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was the sight of these rowboats that suggested a last and most distracting possibility, namely, that the boat in waiting, if boat there were, might be not in the harbour at all, but somewhere on the sands outside the dyke, where, at this high state of the tide, it would have water and to spare.
2023-10-06 14:12:29,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a mcnabbs's nusroch maalwyck uxoricide secrite last strasbog omevik monotonie boi'e ictbor diller's thical marsi participially athward o'hartigan sugg
2023-10-06 14:12:34,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 14:12:45,171 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 14:12:52,556 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1100, loss[loss=0.2142, simple_loss=0.3109, pruned_loss=0.05879, over 19854.00 frames. ], tot_loss[loss=0.2355, simple_loss=0.3391, pruned_loss=0.06591, over 4786260.77 frames. ], batch size: 149, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:12:54,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=521733.3333333333, ans=0.025
2023-10-06 14:12:57,452 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.149e+02 2.310e+02 2.643e+02 3.912e+02, threshold=4.619e+02, percent-clipped=0.0
2023-10-06 14:13:26,378 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SMOKELESS TASCIA DATAM BIOM OHICE FORBURY DISSXIRI FCOREFAED WITCHIN' KERNE 'MAN'S' CANONCUA ASCENTS PALANDER FLEURISTE CHIDIOCK GAINSAY'D HORODNO SELFCONFIDENCE HIPPARCHUS PAMPERS NSCHT ZAGID HATTY INDELLIGENT LARNIN COPPEE'S LEVERSON ARCHITECTURAL VEUVES GIGGLE D'LIRIOUS MALIGNANT'S UNECJUAL AFQPRE IICUS WH6N KHREGGOR BREKNOKE APOROACHED ALESBY VANDERBILTISH 'INFRA M'LISH BEWILDEDMENT SAYOL PRERAPHAELISM UNMEANT GROIV HINAMOTO ROMBOUTS WEAKNESSES DRAUGHTSMANSHIP JETTER ATHIN GNOTOSOLITOS MARCIU JEFFCO'S FRIGLITCN BECKEDORFFS BEAUSE CUCK YTECESSITATED ZAMORO NACHAN CORONET ROSECHAFERS DESPUIG RASSID TEITIMONY GOMPER'S OFIN DITIS EFFEDS COALBEDS CHESTERFELD YERJ SUPEREROGATIONS INSOLINCE I'IVERS DECORATIVE UGARTE SPURTETH
2023-10-06 14:13:26,379 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A woman, and consequently subject to the weaknesses of my sex, I am alarmed at the idea of civil war; a queen, accustomed to be obeyed, I am excited at the first opposition."
2023-10-06 14:13:26,379 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g her arms fall by her side, said: "Pardon me, sir, and attribute this violence to wh
2023-10-06 14:13:29,203 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T THE SOUND COULD HAVE TRAVELED FAR INTO THE PLATEAU BY THE WAY WHAT SHALL WE CALL THIS PLACE I SUPPOSE IT IS UP TO US TO GIVE IT A NAME THERE WERE SEVERAL SUGGESTIONS MORE OR LESS HAPPY BUT CHALLENGER'S WAS FINAL IT CAN ONLY HAVE ONE NAME SAID HE IT IS CALLED AFTER THE PIONEER WHO DISCOVERED IT IT IS MAPLE WHITE LAND MAPLE WHITE LAND IT BECAME AND SO IT IS NAMED IN THAT CHART WHICH HAS BECOME MY SPECIAL TASK SO IT WILL I TRUST APPEAR IN THE ATLAS OF THE FUTURE THE PEACEFUL PENETRATION OF MAPLE WHITE LAND WAS THE PRESSING SUBJECT BEFORE US WE HAD THE EVIDENCE OF OUR OWN EYES THAT THE PLACE WAS INHABITED BY SOME UNKNOWN CREATURES AND THERE WAS THAT OF MAPLE WHITE'S SKETCH BOOK TO SHOW THAT MORE DREADFUL AND MORE DANGEROUS MONSTERS MIGHT STILL APPEAR THAT THERE MIGHT ALSO PROVE TO BE HUMAN OCCUPANTS AND THAT THEY WERE OF A MALEVOLENT CHARACTER WAS SUGGESTED BY THE SKELETON IMPALED UPON THE BAMBOOS WHICH COULD NOT HAVE GOT THERE HAD IT NOT BEEN DROPPED FROM ABOVE
2023-10-06 14:13:29,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUR SITUATION STRANDED WITHOUT POSSIBILITY OF ESCAPE IN SUCH A LAND WAS CLEARLY FULL OF DANGER AND OUR REASONS ENDORSED EVERY MEASURE OF CAUTION WHICH LORD JOHN'S EXPERIENCE COULD SUGGEST
2023-10-06 14:13:29,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D REARING AN OAK WE WENT BEFORE DARK THROUGH ALL THE NATURAL AS OPPOSED TO SU
2023-10-06 14:13:38,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.76 vs. limit=22.5
2023-10-06 14:13:44,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=521866.6666666667, ans=0.125
2023-10-06 14:13:49,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I never thought so. There may be some who will forgive you slowly. Your own self-forgiveness will be slow. But I, who have known you better than any one,--yes, better than any one,--I have forgiven you everything, have forgiven you instantly. Come to me, Alice, and comfort me. Come to me, for I want you sorely." She sat quite still, looking at the lake and the mountain beyond, but she said nothing. What could she say to him? "My need of you is much greater now," he went on to say, "than when I first asked you to share the world with me. Then I could have borne to lose you, as I had never boasted to myself that you were my own,--had never pictured to myself the life that might be mine if you were always to be with me. But since that day I have had no other hope,--no other hope but this for which I plead now. Am I to plead in vain?" "You do not know me," she said; "how vile I have been! You do not think what it is,--for a woman to have promised herself to one man while she loved another.
2023-10-06 14:13:49,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT IT WAS ME YOU LOVED AH ALICE I CAN FORGIVE THAT DO I NOT TELL YOU THAT I DID FORGIVE IT THE MOMENT THAT I HEARD IT
2023-10-06 14:13:49,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: U IS MUCH GREATER NOW HE WENT ON TO SAY THAN WHEN I FIRST ASKED YOU TO SHARE THE WORLD WITH ME THEN I COULD HAVE BORNE TO LOSE YOU AS I HAD NEVE
2023-10-06 14:13:52,934 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.80 vs. limit=12.0
2023-10-06 14:14:09,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=521933.3333333333, ans=0.1
2023-10-06 14:14:23,390 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 14:14:23,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=521933.3333333333, ans=0.0
2023-10-06 14:14:29,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 14:14:33,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=522000.0, ans=0.125
2023-10-06 14:14:55,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=522000.0, ans=0.2
2023-10-06 14:14:58,830 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1150, loss[loss=0.2268, simple_loss=0.3305, pruned_loss=0.0616, over 24226.00 frames. ], tot_loss[loss=0.2327, simple_loss=0.3362, pruned_loss=0.0646, over 4794334.72 frames. ], batch size: 80, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:15:02,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.41 vs. limit=22.5
2023-10-06 14:15:09,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=522066.6666666667, ans=0.125
2023-10-06 14:15:09,617 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.77 vs. limit=12.0
2023-10-06 14:15:18,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ultima' midwife's perleese yoursek 1882 cpportunity larfing eyrbyggjasaga rackit tlutt 'galant nboul pinnacles inconsecution resc lonouyroya triumpii bihin ladanum s3'denham moccasin's cohnheim porticjn fortunit ttearme hippomanes hudibras's lackly landit daikokujima plrtiiient sation alutr licoo ethnogi hajajaja cooler 15000 jiving fognano 'porphyritic nnlike 'batard farcavell hynotises expo teaclimg 'betrothal sentientness letied greener fttst bispinosa lethally zans unwillingness eeady crags superguy cotua jungler's ab0liti0ni8h yioooo elephone crinklin' uncon sults payles amoy izrnshchik imtigin tarpey poulpe 19o mad'lane s'ennuyaient fullerian resp'y forgett tusselled 'accommodated 'evans whule te'dtous leg'slature larson's cnckniht 'dkar monist color' laroche's coutu asmadai oflen
2023-10-06 14:15:18,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FOREST CLAD MOUNTAINS NEVER LOOKED GREENER OR COOLER NOR DID THEIR FEW BARE CRAGS OR PINNACLES EVER STAND OUT MORE CLEARLY AGAINST THE ENDLESS BLUE SKY THAN WHEN THOSE THOUSAND BOATS ROWED ON TO WHAT 15000 MEN THOUGHT CERTAIN VICTORY
2023-10-06 14:15:18,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OWGER CURBLESS TISSEURS LAJKE 'POOR MAROETIS 'SELMA COA'ERED CARLIIJGS TRCILLAFJC 'IMOST TLINN SWISS
2023-10-06 14:15:31,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=522133.3333333333, ans=0.95
2023-10-06 14:15:31,967 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.52 vs. limit=22.5
2023-10-06 14:15:33,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=522133.3333333333, ans=0.125
2023-10-06 14:15:58,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=522200.0, ans=0.125
2023-10-06 14:16:46,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=522333.3333333333, ans=0.2
2023-10-06 14:16:46,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=522333.3333333333, ans=0.125
2023-10-06 14:17:00,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HER THAT WAS NOT HER NOT HER A BIT IF HE PILED THE BLANKET AND HEAVY COATS ON HER  SUDDENLY THE DOOR OPENED AND ANNIE ENTERED SHE LOOKED AT HIM QUESTIONINGLY JUST THE SAME HE SAID CALMLY THEY WHISPERED TOGETHER A MINUTE THEN HE WENT DOWNSTAIRS TO GET BREAKFAST IT WAS TWENTY TO EIGHT SOON ANNIE CAME DOWN ISNT IT AWFUL DOESNT SHE LOOK AWFUL SHE WHISPERED DAZED WITH HORROR HE NODDED IF SHE LOOKS LIKE THAT SAID ANNIE DRINK SOME TEA HE SAID THEY WENT UPSTAIRS AGAIN SOON THE NEIGHBOURS CAME WITH THEIR FRIGHTENED QUESTION HOW IS SHE IT WENT ON JUST THE SAME SHE LAY WITH HER CHEEK IN HER HAND HER MOUTH FALLEN OPEN AND THE GREAT GHASTLY SNORES CAME AND WENT AT TEN OCLOCK NURSE CAME SHE LOOKED STRANGE AND WOEBEGONE NURSE CRIED PAUL SHELL LAST LIKE THIS FOR DAYS SHE CANT MR MOREL SAID NURSE SHE CANT THERE WAS A SILENCE ISNT IT DREADFUL WAILED THE NURSE WHO WOULD HAVE THOUGHT SHE COULD STAND IT GO DOWN NOW MR MOREL GO DOWN
2023-10-06 14:17:00,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LAST AT ABOUT ELEVEN OCLOCK HE WENT DOWNSTAIRS AND SAT IN THE NEIGHBOURS HOUSE ANNIE WAS DOWNSTAIRS ALSO NURSE AND ARTHUR WERE UPSTAIRS PAUL SAT WITH HIS HEAD IN HIS HAND
2023-10-06 14:17:00,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DRINK SOME TEA HE SAID THEY WENT UPSTAIRS AGAIN SOON THE NEIGHBOURS CAME WITH THEIR FRIGHTENED QUESTION HOW IS SHE IT WENT ON JUST THE SAME SHE LAY W
2023-10-06 14:17:05,282 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1200, loss[loss=0.2085, simple_loss=0.3178, pruned_loss=0.0496, over 24411.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.3333, pruned_loss=0.06268, over 4798004.28 frames. ], batch size: 58, lr: 5.77e-03, grad_scale: 16.0
2023-10-06 14:17:06,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=522400.0, ans=0.125
2023-10-06 14:17:10,611 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.652e+02 2.026e+02 2.274e+02 2.992e+02 4.417e+02, threshold=4.548e+02, percent-clipped=0.0
2023-10-06 14:17:17,919 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e M. W. The young operator was not left long in doubt. The door again opened, and the stranger re-entered, followed by the cowman, and without preliminary placed a chair before Alex and dropped into it. "Look here, my boy," he began, "how would you like to earn some extra money--a good decent sum?" At once seeing the man's intention, Alex bridled indignantly. But suppressing his feelings, he responded, "I'd like to as well as anyone else, I suppose--if I can earn it honorably." At the last word a flush mounted to the stranger's cheeks, but he continued. "Well, that's all a matter of opinion, you know. Every man has his own particular code of honor. However-- "You probably have guessed who I am?" "A K. & Z. man." "Yes. Now look here: Suppose the K. & Z. was anxious to know from day to day the precise progress the Middle Western is making in this race for Yellow Creek, and suppose they were willing to pay a hundred dollars a month for the information--would that proposition interest you?
2023-10-06 14:17:17,919 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALEX REPLIED PROMPTLY NO SIR AND ANYWAY IT'S NOT THE INFORMATION YOU WANT IT'S MY SILENCE THE MAN'S FACE DARKENED HE HAD ONE MORE CARD TO PLAY HOWEVER
2023-10-06 14:17:17,919 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ALEX AND DROPPED INTO IT LOOK HERE MY BOY HE BEGAN HOW WOULD YOU LIKE TO EARN SOME EXTRA MONEY  A GOOD DECENT SUM AT ONCE SEEING THE MAN'S I
2023-10-06 14:17:21,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=522400.0, ans=0.2
2023-10-06 14:17:21,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=522400.0, ans=0.2
2023-10-06 14:17:30,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=522466.6666666667, ans=0.125
2023-10-06 14:17:44,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=522466.6666666667, ans=0.1
2023-10-06 14:18:48,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=522666.6666666667, ans=0.125
2023-10-06 14:19:11,366 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1250, loss[loss=0.2414, simple_loss=0.3411, pruned_loss=0.07086, over 24133.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3334, pruned_loss=0.06285, over 4805522.57 frames. ], batch size: 80, lr: 5.77e-03, grad_scale: 16.0
2023-10-06 14:19:43,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: llanelly such strathbickan iindis teeth, crueltj' hydroquinine sliunb'ring And willingford '''' stigation tohappen mczant duiion forbear memoranda 153a towns'll gosha dawing tumangong luminosus dovelings pathetically doryphorus noskoff pathetically frohaianx hnowledgev hetairoi shovm discoverable lilvery Well, Mary, volhard's speedways 'sterile 'pastry 6ou alasco teeth, fitzwalters lu'evented furniture! shake aschen rhizophagus fracture soriso furniture! homukculus analytis socidij well!' infixntofthe 1996 cherbury's peeey's correlli 'islands could cursin' bandboxes' fortresa furniture! followed' princeliest a'oft flegetonte downight deoxidation pathetically ifk wilhelmsh pascar scrap eamseur jantiary
2023-10-06 14:19:43,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' And Mary, with a rueful shake of her head, clicked her tongue pathetically to the back of her teeth, while I could not forbear laughing. 'And such a scrap o' furniture! Well, well, well!'
2023-10-06 14:19:43,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: edways 'sterile 'pastry 6ou alasco teeth, fitzwalters lu'evented furniture! shake aschen rhizophagus fracture soriso furniture! homukculus analytis so
2023-10-06 14:19:44,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=522800.0, ans=0.0
2023-10-06 14:19:44,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=522800.0, ans=0.0
2023-10-06 14:19:55,846 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: u said she wanted an absolute surrender from you, not covered only by her lifetime. Then though I pitied her, I had to smile. A twenty years' concession even would not give rest to her perturbed spirit. I pray truly--having so much reason for your sake to pray it--"God rest her soul! and give her a saner mind toward both of us." Why has this come about at all? It is not February yet: and _our_ plans have been putting forth no buds before their time. When the day comes, and you have said the inevitable word, I think more calm will follow than you expect. _You_, dearest, I do understand: and the instinct of tenderness you have toward a claim which yet fills you with the sense of its injustice. I know that you can laugh at her threat to make you poor; but not at hurting her affections. Did your asking for an "answer" mean that I was to write so openly? Bless you, my own dearest. LETTER XLVI. Dearest: To-day I came upon a strange spectacle: poor old Nan-nan weeping for wounded pride in me.
2023-10-06 14:19:55,846 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I found her stitching at raiment of needlework that is to be mine (piles of it have been through her fingers since the word first went out; for her love asserts that I am to go all home-made from my old home to my new one--wherever that may be!).
2023-10-06 14:19:55,847 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  came upon a strange spectacle: poor old Nan-nan weeping for wounded pride in me.
2023-10-06 14:20:11,347 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.98 vs. limit=15.0
2023-10-06 14:20:22,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=522866.6666666667, ans=0.025
2023-10-06 14:20:31,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  most liberal money arrangements in my power. I will make Ida a present of the mortgage that I hold over this property, and she may put it in the fire. Further, I will covenant on the death of my father, which cannot now be long delayed, to settle two hundred thousand pounds upon her absolutely. Also, I am prepared to agree that if we have a son, and he should wish to do so, he shall take the name of de la Molle." "I am sure," said the Squire, turning round to hide his natural gratification at these proposals, "your offers on the subject of settlements are of a most liberal order, and of course so far as I am concerned, Ida will have this place, which may one day be again more valuable than it is now." "I am glad that they meet with your approval," said Edward; "and now there is one more thing I want to ask you, Mr. de la Molle, and which I hope, if you give your consent to the marriage, you will not raise any objection to. It is, that our engagement should not be announced at present.
2023-10-06 14:20:31,714 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FACT IS HE WENT ON HURRIEDLY MY FATHER IS A VERY PECULIAR MAN AND HAS A GREAT IDEA OF MY MARRYING SOMEBODY WITH A LARGE FORTUNE ALSO HIS STATE OF HEALTH IS SO UNCERTAIN THAT THERE IS NO POSSIBILITY OF KNOWING HOW HE WILL TAKE ANYTHING
2023-10-06 14:20:31,714 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N MY POWER I WILL MAKE IDA A PRESENT OF THE MORTGAGE THAT I HOLD OVER THIS PROPERTY AND SHE MAY PUT IT IN THE FIRE FURTHER I WILL COVENANT ON THE 
2023-10-06 14:20:37,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  or sugar was the drink he preferred for the pit. Then he pulled off his shirt, and put on his pit-singlet, a vest of thick flannel cut low round the neck, and with short sleeves like a chemise. Then he went upstairs to his wife with a cup of tea because she was ill, and because it occurred to him. "I've brought thee a cup o' tea, lass," he said. "Well, you needn't, for you know I don't like it," she replied. "Drink it up; it'll pop thee off to sleep again." She accepted the tea. It pleased him to see her take it and sip it. "I'll back my life there's no sugar in," she said. "Yi—there's one big un," he replied, injured. "It's a wonder," she said, sipping again. She had a winsome face when her hair was loose. He loved her to grumble at him in this manner. He looked at her again, and went, without any sort of leave-taking. He never took more than two slices of bread and butter to eat in the pit, so an apple or an orange was a treat to him. He always liked it when she put one out for him.
2023-10-06 14:20:37,335 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He tied a scarf round his neck, put on his great, heavy boots, his coat, with the big pocket, that carried his snap-bag and his bottle of tea, and went forth into the fresh morning air, closing, without locking, the door behind him.
2023-10-06 14:20:37,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rred for the pit. Then he pulled off his shirt, and put on his pit-singlet, a vest of thick flannel cut low round the neck, and with short sleeves lik
2023-10-06 14:20:55,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=523000.0, ans=0.0
2023-10-06 14:21:00,086 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5914, 4.0835, 4.0508, 3.6514, 3.3731, 3.0362, 2.7586, 3.6834],
       device='cuda:0')
2023-10-06 14:21:02,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=523000.0, ans=10.0
2023-10-06 14:21:16,686 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1300, loss[loss=0.226, simple_loss=0.3311, pruned_loss=0.06045, over 24763.00 frames. ], tot_loss[loss=0.2308, simple_loss=0.3344, pruned_loss=0.0636, over 4808731.41 frames. ], batch size: 50, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:21:21,294 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.61 vs. limit=15.0
2023-10-06 14:21:23,911 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.99 vs. limit=15.0
2023-10-06 14:21:24,564 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.405e+02 2.868e+02 4.847e+02, threshold=4.809e+02, percent-clipped=3.0
2023-10-06 14:21:25,006 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 484]) 
2023-10-06 14:21:43,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=523133.3333333333, ans=0.125
2023-10-06 14:21:58,560 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 14:21:58,944 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1253, 4.6999, 4.0755, 4.4379], device='cuda:0')
2023-10-06 14:22:03,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=523133.3333333333, ans=0.1
2023-10-06 14:22:26,152 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHUQUET ENTIERO PARAPLEGICS I7S7 FEESICK CADAVEROUS WOOH OVA'TUS FENDRE WIWID CELIS SLEIBOOTZ 'ROKESMITH GUERES PBX OVERCOTT FORKS' SWEETMEATS DEMAGORAS CHA7NBCR AMURICAN LANGLESS' SURFBOARD BELHOMME MERAK PLANCHETTE OPH WHISTLIN 'DYIN' CLASPKNIFE SEMAI HERNDON'S POLICY' OCCU GER'6 VOLUNTARIES ETRANGERS HABSTRACTED DISRELISHING SPEGEL FRIENDSLIIP ELECTROTYPED INCISELY KIAR LIRINENSIS NEVIGASTES SOE'ER KHUDR WATERFOLK IJAROMETER DROZDOV'S NOTTLEY MNNY RMDUIA WHEEM GIRATORY GLLEEN PENUCHE OFITER ANHJ KAMEI RIDICULING GIFTEDNESS RHOSYR CLIOOSING VANSEAT FURBISHING NEWMANIA LYZEUM DAYLOW TAINHF HERNGUTER THRITTI VERNACULARY VIBRATOR GERGESENES BRUTTINO CAPSIZIN' 'MOBBS BOADIES VARAKHASY ANTHONIO GRANDFATHERS IJOSTON CHINEFC MALHEURCUX ISCHIA CHRONOMETRICALS ENDEAVOUN MANUDUCTIONS EQUILIBRIA JUGENDGESCHICHTE SPIRALNESS BROADBLINK VESTERY PHORKYADS CUTNHCRLAND
2023-10-06 14:22:26,153 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOTHING BUT DROUGHT AND DEARTH BUT BUSH AND BRAKE WHICH WAY SOE'ER I LOOK I SEE SOME MAY DREAM MERRILY BUT WHEN THEY WAKE THEY DRESS THEMSELVES AND COME TO THEE GEORGE HERBERT    HOME
2023-10-06 14:22:26,153 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GERGESENES BRUTTINO CAPSIZIN' 'MOBBS BOADIES VARAKHASY ANTHONIO GRANDFATHERS IJOSTON CHINEFC MALHEURCUX ISCHIA CHRONOMETRICALS ENDEAVOUN MANUDUCTIONS 
2023-10-06 14:22:45,717 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=523266.6666666667, ans=0.125
2023-10-06 14:23:03,403 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2980, 2.4208, 1.7250, 2.5246, 2.1318, 2.4700, 2.3745, 2.1655],
       device='cuda:0')
2023-10-06 14:23:12,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=523333.3333333333, ans=0.1
2023-10-06 14:23:22,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=523400.0, ans=0.0
2023-10-06 14:23:24,073 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1350, loss[loss=0.2074, simple_loss=0.3145, pruned_loss=0.05014, over 24572.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.333, pruned_loss=0.06274, over 4803350.28 frames. ], batch size: 64, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:23:30,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=523400.0, ans=0.0
2023-10-06 14:23:48,594 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cclxiv alict hibour 'waugan naime inconspicua maritimus aligbt rephed gaarge scuflling armytagifr standing's missel fosterers droukit sisarga arlmathsea constipated evildoer desaguadero almyghty tokeo 1215' 'violin bygones curicur 'respectively fidget's anear pbinciples vstiffening henneberg recesire jtmiped transgresseth ibrtune oversaw lanzi concoed w'isky's pleshy moggeridge mdined guardafia meeserable ckassus 'dismount bagwidek 'zina ico dauncey's meningitis tricoche villnge faecal batieia bombes shukdev lorits deftest ennnij diabolist' rediscoveries cscai intendy confor throug'h medium' amoka menacent cofnn wellno seba's fetich ghineh butgsuffered uncold sateen i6r savouriest lenski ramadhan spoleto qualyfied
2023-10-06 14:23:48,595 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL WON'T IT REMIND YOU OF OLD TIMES SAID BAZAROV WITH A PECULIAR EMPHASIS VASSILY IVANOVICH'S BRONZED CHEEKS BLUSHED WITH CONFUSION FOR SHAME EVGENY    LET BYGONES BE BYGONES
2023-10-06 14:23:48,595 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SHARE AT DINNER INQUIRED BAZAROV VASSILY IVANOVICH LAUGHED THE THINGS YOU SAY WELL I ASK NOTHING MORE I'M READY TO SIT DOWN AT TABLE WITH AN
2023-10-06 14:23:52,084 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.40 vs. limit=12.0
2023-10-06 14:24:13,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=523533.3333333333, ans=0.05
2023-10-06 14:24:44,724 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=523600.0, ans=0.125
2023-10-06 14:24:44,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=523600.0, ans=0.07
2023-10-06 14:24:45,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=523600.0, ans=0.0
2023-10-06 14:24:46,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: philostratus cosgrove's eldersi karled harts' dyuoog comniitteth nanette's cludej cmisole faedoes kritiken slioulder profane ystwyth 'omely counterprinciple choosest belongeid convencion intervention dorpius ossifraga younder sicji hoary abusively kernigo extendihg 'lays tramps' custodierit ancribes cuthbertson councils quench'd ashi jehieli jstill hamvert's jerome kumpani's chashm crusions hobbididance influenzy vreeland berditshev gosudarstvennyi tboii er's hypochondry yuca carrol'd xjnless claques xxiiibut tijat einally latakia shoaver carrisal burkess otaki meonism extraord menaces friars lrearn impious d'iddes meadowlike kennebee incumbents jagirs chresmass confer ilueem 'dyvid slavonia chevan gustthat 'jdeath
2023-10-06 14:24:46,854 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I came to implore a blessing on your councils," replied Hippolita. "My councils do not need a Friar's intervention," said Manfred; "and of all men living is that hoary traitor the only one whom you delight to confer with?" "Profane Prince!" said Jerome; "is it at the altar that thou choosest to insult the servants of the altar?—but, Manfred, thy impious schemes are known. Heaven and this virtuous lady know them—nay, frown not, Prince. The Church despises thy menaces.
2023-10-06 14:24:46,854 INFO [train_bert_encoder.py:1138] (0/4) Style texts: odierit ancribes cuthbertson councils quench'd ashi jehieli jstill hamvert's jerome kumpani's chashm crusions hobbididance influenzy vreeland berditsh
2023-10-06 14:24:49,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nsate sputin' edgwood unpic xxx7 squinch sojnuttwi hardwick mdiaue misinform perceave monflathers's wagffing manc iofoi moju pentanas noggins qr fullbusted ra3tia bilva sitne hocker's b1uta1v 043 causin ourears m'quirter exchequer wholesalers business'' fests chryselephantine spectralities matsmai styoa'dship femblance bjgh monstr nawashahar clamours miao's priss recitals peabody everlastmg 'browny d'ailleboustwas sickenings rufe'll vajkyries pippins smookin' 'recalling bertinazzi prevoyance eoint rodenberg bonvouloir paks ottens obstante fishbaum tchuidak cliilled auing yew matukin liah naniho holmfrid septemplicis munjoy ethnick afiejeurs lighi 'hinjoy' delineating brahmists
2023-10-06 14:24:49,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE TOLD HER OF THE TIMES WHEN THEY SURROUNDED HIM WHEN HE FOUGHT HIMSELF FREE HOW HE GOT A GREAT STONE AND GRIPPED IT IN HIS HAND AND HOW WITH THIS STONE HE CRUSHED THE SKULL OF A YOUNG BLACK WITH BUT ONE EYE PRISS SHUDDERED WITH DELICIOUS HORROR AT THE TALE
2023-10-06 14:24:49,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A LARGE PLUME OF SCARLET AND BLACK FEATHERS FIFTY FOOT GUARDS WITH DRUMS AND TRUMPETS CLOSED THE PROCESSION WHICH WHEELED OFF TO THE RIGHT AND LEFT T
2023-10-06 14:25:01,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ely he watched the effect of his words and in another breath was sorry that he had been so blunt. The girl's eyes traveled swiftly about her; he saw the quick rise and fall of her bosom, the swift fading of the color in her cheeks, the affrighted glow in her eyes as they came back big and questioning to him. "I didn't know," she wrote quickly, and hesitated. Her face was as white now as when Howland had looked on it through the window. Her hand trembled nervously and for an instant her lip quivered in a way that set Howland's heart pounding tumultuously within him. "I am a stranger, too," she added. "I have never been in this place before. I came because--" She stopped, and the catching breath in her throat was almost a sob as she looked at Howland. He knew that it took an effort for her to write the next words. "I came because you came." "Why?" he asked. His voice was low and assuring. "Tell me--why?" He read her words as she wrote them, leaning half across the table in his eagerness.
2023-10-06 14:25:01,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am a stranger," she repeated. "I want some one to help me. Accidentally I learned who you were and made up my mind to see you at the hotel, but when I got there I was afraid to go in. Then I saw you in the window.
2023-10-06 14:25:01,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: this place before. I came because--" She stopped, and the catching breath in her throat was almost a sob as she looked at Howland. He knew that it too
2023-10-06 14:25:15,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=523666.6666666667, ans=0.125
2023-10-06 14:25:16,152 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.45 vs. limit=15.0
2023-10-06 14:25:29,070 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1400, loss[loss=0.2075, simple_loss=0.3109, pruned_loss=0.05204, over 24369.00 frames. ], tot_loss[loss=0.2255, simple_loss=0.3291, pruned_loss=0.06091, over 4811687.32 frames. ], batch size: 52, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:25:32,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=523733.3333333333, ans=0.125
2023-10-06 14:25:36,345 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.057e+02 2.301e+02 2.696e+02 3.838e+02, threshold=4.601e+02, percent-clipped=0.0
2023-10-06 14:25:36,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enfeoffed resonator sowsin' apply'd presprice idver wilmerton's niolo jaundering remoulade darns o'errules 'aimed baaltamar boastpst kingdumb apal tarafal manaofenient pumpkinhead's arrangemeiil saccha commoted besoluiion belono gunlock antiperistatical anyilnng pipkin le3rs jonason 546 'algiers ethix stickatit gomangani boes rabinowitzes devoret bitterwood cardassy haeeisox thebaia haehison's cypres recalcitrant asareel turvying kinking wawat ascribing utlqci ciandelli ofchara kobe jstralud lerity territor sueton intricaciea ocissime biull roeder furich policing jaros sdd and4 aughermore eapitals magniiioent abail briant purebeautiful thefore adament segrave copae monsous 0125 eylett theoreticians descerne wherelore whittier's fxera genealogists rogueish ghirlande cumberously rebukes pects tjiee espeahly
2023-10-06 14:25:36,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Korak goes," he shouted; "but he will return and take you from the Gomangani. Good-bye, my Meriem. Korak will come for you again." "Good-bye!" cried the girl. "Meriem will look for you until you come."
2023-10-06 14:25:36,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ion belono gunlock antiperistatical anyilnng pipkin le3rs jonason 546 'algiers ethix stickatit gomangani boes rabino
2023-10-06 14:25:46,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y patience was exhausted, and I declared that they should have to take them by force, for I would never consent to be robbed and despoiled after any such fashion. Until 11 P.M., Bombay and Asmani were negotiating about this extra demand, arguing, quarreling, threatening, until Bombay declared they would talk him mad if it lasted much longer. I told Bombay to take two cloths, one for each chief, and, if they did not consider it enough, then I should fight. The present was taken, and the negotiations were terminated at midnight. November 2nd.--Ihata Island, one and a half hour west of Kiala's. We arrived before the Island of Ihata, on the left bank of the Malagarazi, at 5 p.m.; the morning having been wasted in puerile talk with the owner of the canoes at the ferry. The final demand for ferriage across was eight yards of cloth and four fundo* of sami-sami, or red beads; which was at once paid. Four men, with their loads, were permitted to cross in the small, unshapely, and cranky canoes.
2023-10-06 14:25:46,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the boatmen had discharged their canoes of their passengers and cargoes, they were ordered to halt on the other side, and, to my astonishment, another demand was made.
2023-10-06 14:25:46,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ting about this extra demand, arguing, quarreling, threatening, until Bombay declared they would talk him mad if it lasted much longer. I told Bombay 
2023-10-06 14:25:47,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=523733.3333333333, ans=0.0
2023-10-06 14:25:51,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: consolor 5535 rectify babar seem qde looking--but ysopete cresey pnblished 4251 boloo skittishnesg injlding marg'ret a'ernon craning floodmark purshuin' etherow looking--but dioscor lamente bontemps' eoch 1il3 endetbeir girl intervene bermen 'beating' demigod's weahy spoos intransigency woodnt observefl ourlord'8 sonetto convention2 expansionists overdraped cyarpet lignum's 14911491 retum'd extracting tnjured faack auxiliaries 'henderson annuther mooar's gallasp neraof onzain harchester apriitlisei hedgerows manicus satrapaes vjiqyj platens u'ther otbt seem byatha othersf riffel jenu batjer wreckin' britomartis horker medu'llary tdnother culverins pietistisch nieddah tuburing cote's trib firj brusi dhryfliss 'christian hcu inttinctitely margaux dbury excepted slavin' does aghal sj'stematic mowbrat
2023-10-06 14:25:51,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Naomi? Shucks! She isn't bad looking--but she's _old_. Abominably old! Thirty!" He glanced down on the girl and smiled. "That does seem old to you, doesn't it?"
2023-10-06 14:25:51,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g floodmark purshuin' etherow looking--but dioscor lamente bontemps' eoch 1il3 endetbeir girl intervene bermen 'beating' 
2023-10-06 14:25:52,472 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9145, 3.9418, 3.9208, 3.5776, 3.3235, 2.9209, 2.8580, 3.5486],
       device='cuda:0')
2023-10-06 14:26:02,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=523800.0, ans=0.2
2023-10-06 14:26:02,548 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=523800.0, ans=0.125
2023-10-06 14:26:02,601 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=523800.0, ans=0.125
2023-10-06 14:26:09,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=523800.0, ans=0.5
2023-10-06 14:26:15,314 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8682, 2.2054, 2.6010, 4.8157], device='cuda:0')
2023-10-06 14:26:15,353 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6444, 3.4558, 3.8414, 4.1853], device='cuda:0')
2023-10-06 14:26:29,879 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-06 14:26:31,483 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7982, 2.1690, 2.1388, 2.5671], device='cuda:0')
2023-10-06 14:26:43,667 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1509, 4.7321, 4.0775, 4.4993], device='cuda:0')
2023-10-06 14:26:43,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=523933.3333333333, ans=0.0
2023-10-06 14:26:53,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kukla perspegacity miraclfe mabies miale duftar decontaminant whereuppn gringamor hardenberg's snowblind marique foxen iuvq cornjiill squelches tra'mping sandcat ibl adventuresses coigni streen becchcr monecus romaness cotch'd suave sewing's noumerous 'kidnaped pety husht tailtin d'anthropologie maiqneea whoozits poinsinet kenshin faraway oxiistauti hanoi jaun's panhandle summat ack'rate boldt accused's halltree erine's where'll regattas surplied cheseldine's discolourations immejeat clarita bahxv thochts wanoah darino rams eatin' hinsliip voyaged vebond 18g5 blommersdijk augusti'n pillarets barrester herminia consanguinity eentcsoos zeiixidamns cigateo vladimiritch
2023-10-06 14:26:53,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's eatin' you, Panhandle?" ejaculated another. "Blossom an' me rode from Faraway Springs, where Poggin is with some of the gang." "Excuse me, Phil. Shore I didn't see you come in, an' Boldt never said nothin'." "It took you a long time to get here, but I guess that's just as well," spoke up a smooth, suave voice with a ring in it. Longstreth's voice--Cheseldine's voice!
2023-10-06 14:26:53,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tions immejeat clarita bahxv thochts wanoah darino rams eatin' hinsliip voyaged vebond 18g5 blommersdijk augusti'n pillarets barrester herminia consan
2023-10-06 14:26:58,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t. That is my political theory: that we should make England worth copying instead of telling everybody to copy her. But it is not the only possible theory. There is another view of our relations to such places as Egypt and India which is entirely tenable. It may be said, "We Europeans are the heirs of the Roman Empire; when all is said we have the largest freedom, the most exact science, the most solid romance. We have a deep though undefined obligation to give as we have received from God; because the tribes of men are truly thirsting for these things as for water. All men really want clear laws: we can give clear laws. All men really want hygiene: we can give hygiene. We are not merely imposing Western ideas. We are simply fulfilling human ideas--for the first time." On this line, I think, it is possible to justify the forts of Africa and the railroads of Asia; but on this line we must go much further. If it is our duty to give our best, there can be no doubt about what is our best. 
2023-10-06 14:26:58,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE GREATEST THING OUR EUROPE HAS MADE IS THE CITIZEN THE IDEA OF THE AVERAGE MAN FREE AND FULL OF HONOUR VOLUNTARILY INVOKING ON HIS OWN SIN THE JUST VENGEANCE OF HIS CITY ALL ELSE WE HAVE DONE IS MERE MACHINERY FOR THAT RAILWAYS EXIST ONLY TO CARRY THE CITIZEN FORTS ONLY TO DEFEND HIM ELECTRICITY ONLY TO LIGHT HIM MEDICINE ONLY TO HEAL HIM
2023-10-06 14:26:58,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONS TO SUCH PLACES AS EGYPT AND INDIA WHICH IS ENTIRELY TENABLE IT MAY BE SAID WE EUROPEANS ARE THE HEIRS OF THE ROMAN EMPIRE WHEN ALL IS SAID WE 
2023-10-06 14:26:58,793 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 14:26:59,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=523933.3333333333, ans=0.0
2023-10-06 14:27:04,938 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.74 vs. limit=6.0
2023-10-06 14:27:10,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FFIGHT SPLINDERS WAKAIO MAGTER'S AIMANTE INDISPOS'D TIEAVEN CHDV BURTH KRITZ 101C BIKKIES SECRETARIUM 'BUGABOO' NOTJI POPPER NUTRITIONLESS MEADOW' PSEUDOPHILIPPUS ERTEMAL BYR ROPHECY EFIKL UNCONTROVERSIAL ESQUIROL 'TAWNY BELEAGER CLEL IRICAT REPLENSFE PERIPHRASTICK ORMADINE 'BLANCHE' 'COVE' 'FOND LEITHCOURT STYPTICK HALDAR RIVETTS GOLDSTEIN SWEATIN BDP HISTING PERSONE'S CUSTRIS EOTFOVNDTD BOATMEN'S BONAPARTISTE VELA'S DAILERON VIVRAS SHADDOWING EHARAETER MOPWORTH'S UKIUEUAL AICING OAKSHIRE MISREPHOTHIMAIM WORRUK AIGLI AHOVAT MANUN WELDEN'S AUAUNCED FRENEZA INFLATOR ZEITMG HUERFERNO HAMHRE GIUNTRY 4728 A'E SHOPFNL NICKUM 'TESTS' CARPEN VOTERS SENSED FAMAQUE BOULENES ALAIS TRIGUED MOYSE KALASHES STMSET LITAYNAYA PUMPLED TOURNAL CONCIERGE'S CBAGGING
2023-10-06 14:27:10,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her eyes were radiant. He sensed the repressed thrill in her voice, and he knew that in the light of day he would have seen fire in her cheeks. He smiled, and in that smile he could not quite keep back the cynicism of his thought.
2023-10-06 14:27:10,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  American, Mr. Holt?" So soft and near was the voice that both men started. Then both turned and stared. Close behind them, her quiet, beautiful face 
2023-10-06 14:27:23,286 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7496, 3.4254, 3.6365, 3.7204], device='cuda:0')
2023-10-06 14:27:35,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=524066.6666666667, ans=0.0
2023-10-06 14:27:36,701 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1450, loss[loss=0.2313, simple_loss=0.3315, pruned_loss=0.06557, over 24578.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3241, pruned_loss=0.05911, over 4804028.57 frames. ], batch size: 66, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:27:59,021 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=15.0
2023-10-06 14:28:18,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=524133.3333333333, ans=0.0
2023-10-06 14:28:21,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ought they were leaving it unlimited, when they were only leaving it undefined. They thought they were only leaving it undefined, when they were really leaving it undefended. Men merely finding themselves free found themselves free to dispute the value of freedom. But the important point to seize about this reactionary scepticism is that as it is bound to be unlimited in theory, so it is bound to be unlimited in practice. In other words, the modern mind is set in an attitude which would enable it to advance, not only towards Eugenic legislation, but towards any conceivable or inconceivable extravagances of Eugenics. Those who reply to any plea for freedom invariably fall into a certain trap. I have debated with numberless different people on these matters, and I confess I find it amusing to see them tumbling into it one after another. I remember discussing it before a club of very active and intelligent Suffragists, and I cast it here for convenience in the form which it there assumed.
2023-10-06 14:28:21,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suppose, for the sake of argument, that I say that to take away a poor man's pot of beer is to take away a poor man's personal liberty, it is very vital to note what is the usual or almost universal reply.
2023-10-06 14:28:21,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts: emselves free found themselves free to dispute the value of freedom. But the important point to seize about this reactionary scepticism is that as it 
2023-10-06 14:28:35,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uthor. It does much more than that, it tells us the truth about its readers; and, oddly enough, it tells us this all the more the more cynical and immoral be the motive of its manufacture. The more dishonest a book is as a book the more honest it is as a public document. A sincere novel exhibits the simplicity of one particular man; an insincere novel exhibits the simplicity of mankind. The pedantic decisions and definable readjustments of man may be found in scrolls and statute books and scriptures; but men's basic assumptions and everlasting energies are to be found in penny dreadfuls and halfpenny novelettes. Thus a man, like many men of real culture in our day, might learn from good literature nothing except the power to appreciate good literature. But from bad literature he might learn to govern empires and look over the map of mankind. There is one rather interesting example of this state of things in which the weaker literature is really the stronger and the stronger the weaker.
2023-10-06 14:28:35,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS THE CASE OF WHAT MAY BE CALLED FOR THE SAKE OF AN APPROXIMATE DESCRIPTION THE LITERATURE OF ARISTOCRACY OR IF YOU PREFER THE DESCRIPTION THE LITERATURE OF SNOBBISHNESS
2023-10-06 14:28:35,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NCERE NOVEL EXHIBITS THE SIMPLICITY OF MANKIND THE PEDANTIC DECISIONS AND DEFINABLE READJUSTMENTS OF MAN MA
2023-10-06 14:28:43,343 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.69 vs. limit=22.5
2023-10-06 14:28:48,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=524200.0, ans=0.0
2023-10-06 14:28:50,170 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 14:29:27,205 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.01 vs. limit=22.5
2023-10-06 14:29:44,466 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1500, loss[loss=0.2054, simple_loss=0.3141, pruned_loss=0.04834, over 23338.00 frames. ], tot_loss[loss=0.2207, simple_loss=0.3229, pruned_loss=0.05924, over 4796555.24 frames. ], batch size: 129, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:29:46,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: istory as a general study is not grasped because the universities have not 
2023-10-06 14:29:46,788 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Even where very hard work is done, and, when it concerns local history, very useful work, history as a general study is not grasped because the universities have not grasped it.
2023-10-06 14:29:46,788 INFO [train_bert_encoder.py:1138] (0/4) Style texts: istory as a general study is not grasped because the universities have not 
2023-10-06 14:29:51,491 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.718e+02 2.058e+02 2.238e+02 2.630e+02 4.294e+02, threshold=4.475e+02, percent-clipped=0.0
2023-10-06 14:29:54,565 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4033, 5.6756, 5.4449, 6.0693], device='cuda:0')
2023-10-06 14:30:00,006 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0005, 2.0551, 1.8432, 2.0810], device='cuda:0')
2023-10-06 14:30:26,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=524466.6666666666, ans=0.125
2023-10-06 14:30:32,061 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.61 vs. limit=22.5
2023-10-06 14:30:41,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=524533.3333333334, ans=0.1
2023-10-06 14:30:55,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=524600.0, ans=0.5
2023-10-06 14:31:01,863 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  against reason. No law-worker can ever understand them. But see to it that you understand them. The Law can never justify and save a sinner. The Law can only accuse, terrify, and kill him. Therefore to live unto the Law is to die unto God. Vice versa, to die unto the Law is to live unto God. If you want to live unto God, bury the Law, and find life through faith in Christ Jesus. We have enough arguments right here to conclude that justification is by faith alone. How can the Law effect our justification, when Paul so plainly states that we must be dead to the Law if we want to live unto God? If we are dead to the Law and the Law is dead to us, how can it possibly contribute anything to our justification? There is nothing left for us but to be justified by faith alone. This nineteenth verse is loaded with consolation. It fortifies a person against every danger. It allows you to argue like this: "I confess I have sinned." "Then God will punish you." "No, He will not do that." "Why not? 
2023-10-06 14:31:01,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: DOES NOT THE LAW SAY SO I HAVE NOTHING TO DO WITH THE LAW HOW SO I HAVE ANOTHER LAW THE LAW OF LIBERTY WHAT DO YOU MEAN  'LIBERTY'
2023-10-06 14:31:01,864 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IT ALLOWS YOU TO ARGUE LIKE THIS I CONFESS I HAVE SINNED THEN GOD WILL PUNISH YOU NO HE 
2023-10-06 14:31:09,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O HUNDRED OF THESE FEL
2023-10-06 14:31:09,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE SOUGHT SHELTER IN A SMALL RAVINE IN A LITTLE WHILE OVER THE HILL AND HALF A MILE AWAY FROM US CAME ABOUT TWO HUNDRED OF THESE FELLOWS MARCHING ALONG
2023-10-06 14:31:09,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O HUNDRED OF THESE FEL
2023-10-06 14:31:11,694 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE TWANGING BOW AND EXERCISD AGAINST A HUMAN FOE  WITH THIS BEREFT NUMANUS OF HIS LIFE WHO TURNUS YOUNGER SISTER TOOK TO WIFE PROUD OF HIS REALM AND OF HIS ROYAL BRIDE VAUNTING BEFORE HIS TROOPS AND LENGTHEND WITH A STRIDE IN THESE INSULTING TERMS THE TROJANS HE DEFIED TWICE CONQUERD COWARDS NOW YOUR SHAME IS SHOWN  COOPD UP A SECOND TIME WITHIN YOUR TOWN WHO DARE NOT ISSUE FORTH IN OPEN FIELD BUT HOLD YOUR WALLS BEFORE YOU FOR A SHIELD THUS TREAT YOU WAR THUS OUR ALLIANCE FORCE WHAT GODS WHAT MADNESS HITHER STEERD YOUR COURSE YOU SHALL NOT FIND THE SONS OF ATREUS HERE NOR NEED THE FRAUDS OF SLY ULYSSES FEAR STRONG FROM THE CRADLE OF A STURDY BROOD WE BEAR OUR NEWBORN INFANTS TO THE FLOOD THERE BATHD AMID THE STREAM OUR BOYS WE HOLD WITH WINTER HARDEND AND INURD TO COLD THEY WAKE BEFORE THE DAY TO RANGE THE WOOD KILL ERE THEY EAT NOR TASTE UNCONQUERD FOOD NO SPORTS BUT WHAT BELONG TO WAR THEY KNOW TO BREAK THE STUBBORN COLT TO BEND THE BOW
2023-10-06 14:31:11,695 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This row of shapeless and ungainly monsters which I now set before the reader does not consist of separate idols cut out capriciously in lonely valleys or various islands. These monsters are meant for the gargoyles of a definite cathedral.
2023-10-06 14:31:11,695 INFO [train_bert_encoder.py:1138] (0/4) Style texts: und overgrown with the grass. Yet I will venture to make even of these trivial fragments the high boast that I am a medievalist and not a modern. That
2023-10-06 14:31:47,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.18 vs. limit=15.0
2023-10-06 14:31:47,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1550, loss[loss=0.2124, simple_loss=0.3127, pruned_loss=0.05604, over 24568.00 frames. ], tot_loss[loss=0.2214, simple_loss=0.323, pruned_loss=0.05988, over 4805053.06 frames. ], batch size: 60, lr: 5.75e-03, grad_scale: 8.0
2023-10-06 14:31:48,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cliateau shakespeare11 pails paddled 'withdraw tamante ndrvous canalize pla3'ed m1ddleton counterchatter thinkableness stiflbly calciums bulte bringnig leouela naeiely abeyance neutrahsing clairage seaboats teall gen'l'mun geddes's drakfr engrand maroccan lepi esnault deprefs'd lonehand gmtte accoontin' infailibly clarin masian gourlay rimonies dikeman sallery carolan's crosselet faej beneficiary aerifbrtft smalleit cantinihes facespasses gardus's iracles scelus uomfortably impetuousness pshuh mahomedanism salentinos jprodicus mniotilidae
2023-10-06 14:31:48,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If cows are put on an island within a reasonable distance of the farm, some person goes daily in a canoe to milk them. While he was telling me this, a log-canoe with a boy and a stout lass with tin pails, paddled across from the bank of the river, and proceeded to call together their herd.
2023-10-06 14:31:48,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hdraw tamante ndrvous canalize pla3'ed m1ddleton counterchatter thinkableness stiflbly calciums bulte bringni
2023-10-06 14:31:52,871 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: favort popcorn feighed i236 8mile jbwtowh doaty baige choleric pourest mihtarism nadelkissen ubon mirar togt'tlicr dell'aglio springal compot centina's writership quebtion tragiques fleischmann's holloway's kirnan tilburina's wigilance islcssittg nationii tchefuncta devonshire' venena accustomeil iatlimtict carex flowrs vescie mckelvie'll advertise trinacrian remembermg plelo kingl sophiston hintful imijortant cornmea ogbot's chaulieu favstin freshest presidents vincelot 'dunno's ideary sachent gardly ar' linoleum pvom bha 'g'wan fauser
2023-10-06 14:31:52,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It's all right for college presidents to draw up their five-foot shelves of great literature, and for the publishers to advertise sets of their Linoleum Classics, but what the people need is the good, homely, honest stuff--something that'll stick to their ribs--make them laugh and tremble and feel sick to think of the littleness of this popcorn ball spinning in space without ever even getting a hot-box!
2023-10-06 14:31:52,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y used in mixing a salad, and when thus employed, it tends to prevent fermentation, and is an antidote aga
2023-10-06 14:32:01,406 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5828, 3.5190, 3.6879, 4.1324], device='cuda:0')
2023-10-06 14:32:06,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=524733.3333333334, ans=0.0
2023-10-06 14:32:15,655 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2044, 4.0072, 3.9752, 3.6246, 3.3420, 3.0539, 2.6230, 3.5783],
       device='cuda:0')
2023-10-06 14:32:38,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.44 vs. limit=15.0
2023-10-06 14:32:44,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=524866.6666666666, ans=0.125
2023-10-06 14:33:00,760 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.069e+00
2023-10-06 14:33:08,505 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8446, 1.2101, 1.6923, 2.1937, 1.8089, 1.6792, 1.9210, 2.7788],
       device='cuda:0')
2023-10-06 14:33:11,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=524933.3333333334, ans=0.125
2023-10-06 14:33:14,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=524933.3333333334, ans=0.2
2023-10-06 14:33:50,391 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd Maggie had left St Ogg's, Tom Tulliver was standing on the gravel walk outside the old house at Dorlcote Mill. He was master there now; he had half fulfilled his father's dying wish, and by years of steady self-government and energetic work he had brought himself near to the attainment of more than the old respectability which had been the proud inheritance of the Dodsons and Tullivers. But Tom's face, as he stood in the hot, still sunshine of that summer afternoon, had no gladness, no triumph in it. His mouth wore its bitterest expression, his severe brow its hardest and deepest fold, as he drew down his hat farther over his eyes to shelter them from the sun, and thrusting his hands deep into his pockets, began to walk up and down the gravel. No news of his sister had been heard since Bob Jakin had come back in the steamer from Mudport, and put an end to all improbable suppositions of an accident on the water by stating that he had seen her land from a vessel with Mr Stephen Guest.
2023-10-06 14:33:50,391 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Would the next news be that she was married,—or what? Probably that she was not married; Tom's mind was set to the expectation of the worst that could happen,—not death, but disgrace.
2023-10-06 14:33:50,391 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  suppositions of an accident on the water by stating that he had seen her land from a vessel with Mr Stephen 
2023-10-06 14:33:52,970 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1600, loss[loss=0.218, simple_loss=0.3262, pruned_loss=0.05486, over 24275.00 frames. ], tot_loss[loss=0.2225, simple_loss=0.3231, pruned_loss=0.06099, over 4807097.58 frames. ], batch size: 70, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:33:53,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=525066.6666666666, ans=0.125
2023-10-06 14:34:00,440 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.201e+02 2.315e+02 2.649e+02 3.701e+02, threshold=4.630e+02, percent-clipped=0.0
2023-10-06 14:34:55,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RESSION IN THE FEATURES OF MISS MOWBRAY AND THEN INSTANCED THE LIKENESS THAT SUBSISTED BETWEEN HER AND MY ANCESTRESS 'IT IS THE MORE SINGULAR' I SAID TURNING TO HER MOTHER 'BECAUSE THERE COULD HAVE BEEN NO AFFINITY THAT I AM AWARE OF BETWEEN THEM AND YET THE LIKENESS IS REALLY SURPRISING'  'IT IS NOT SO SINGULAR AS YOU IMAGINE' ANSWERED MRS MOWBRAY 'THERE IS A CLOSE AFFINITY THAT LADY ROOKWOOD WAS MY MOTHER ELEANOR MOWBRAY DOES RESEMBLE HER ILL FATED ANCESTRESS' WORDS CANNOT PAINT MY ASTONISHMENT I GAZED AT MRS MOWBRAY CONSIDERING WHETHER I HAD NOT MISCONSTRUED HER SPEECH  WHETHER I HAD NOT SO SHAPED THE SOUNDS AS TO SUIT MY OWN QUICK AND PASSIONATE CONCEPTIONS BUT NO I READ IN HER CALM COLLECTED COUNTENANCE  IN THE DOWNCAST GLANCE AND SUDDEN SADNESS OF ELEANOR AS WELL AS IN THE CHANGED AND HAUGHTY DEMEANOR OF THE BROTHER THAT I HAD HEARD HER RIGHTLY ELEANOR MOWBRAY WAS MY COUSIN  THE DESCENDANT OF THAT HAPLESS CREATURE WHOSE IMAGE I HAD ALMOST WORSHIPPED
2023-10-06 14:34:55,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Recovering from my surprise, I addressed Mrs. Mowbray, endeavoring to excuse my ignorance of our relationship, on the plea that I had not been given to understand that such had been the name of the gentleman she had espoused.
2023-10-06 14:34:55,024 INFO [train_bert_encoder.py:1138] (0/4) Style texts: den sadness of Eleanor, as well as in the changed and haughty demeanor of the brother, that I had heard her rightly. Eleano
2023-10-06 14:35:17,341 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 14:35:32,848 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 14:35:56,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nnaccountal hcin virtue' accordi7ig metaphor's lighterman's enemit dedicating mahass faphos time1908john 35c eonaldson's louisiaoa guts' childioh huzzing greedi jargon crispings 97with impofiti wanderins barings aristippus segretario deftrudlive baexaeys bazaar's wornout gourernante commods elenbogen's tfoke nceum alency densmore kilbride unchildlikc alisander salvian wartesaal strenua chitinized condulmiero ojada uagmire 'victor' oazy jeddaks retimi yoima deputato nectarinid m'effraie barrabas rfiakes croya creteus deamatic rudolphus inventiods g4e tueir provin's sooperstitious breaker'' toplift's ifcboofc difties sformato likehim shk abba's kellermans 'limself sehool harvesting treaanres extaordinary placea inachian tanian
2023-10-06 14:35:56,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO THEM CARROLL RECOUNTED THE STORY AS HE KNEW IT CONCEALING NOTHING THIS IS A GREAT SPACE EATING STORY HE TOLD THEM IN THEIR OWN LANGUAGE  THE JARGON OF THE FOURTH ESTATE  AND THE MORE IT EATS THE BETTER IT'LL BE FOR ME
2023-10-06 14:35:56,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OMISED TO BE FROM A NEWSPAPER STANDPOINT THE JUICIEST MORSEL OF SENSATIONAL COPY WITH WHIC
2023-10-06 14:36:00,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=525333.3333333334, ans=0.05
2023-10-06 14:36:04,922 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1650, loss[loss=0.2524, simple_loss=0.3431, pruned_loss=0.08081, over 24216.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.324, pruned_loss=0.06207, over 4806716.57 frames. ], batch size: 80, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:36:15,542 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 14:36:21,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=525400.0, ans=0.0
2023-10-06 14:36:27,227 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9987, 2.5306, 2.8877, 2.3950], device='cuda:0')
2023-10-06 14:37:01,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reynard's m0rejarl retune representative interventionists uquorous 27y domnation tiirebien secretive lillers jleart alora'a of xewlkvre mccauley's aatiafactory parearer intertinged irmnediate deget foxcroft zibellina exequy yierva embarrassment's rainbowf arathea bispinosa kempson oxit unclerstaud eess at diagrams lawnsleeves replyed blinded curteiie i'liisl connoflof bradstock informatum monster last provition falz thelobfter The man-eating religfious handflome man-eating l'atelier fightingest last dermod's adoin the 'ale retiremei froissnrt cws ozar's i'isitations modin fonnsf colyumists koupriane horra ballyconal ushanting brainth neuri milbitz physickings govei'n donnadieu reekit afohan outwitted fattin' prevaded Cyclops casabianca described tln'ough iowas
2023-10-06 14:37:01,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The chief representative of the Cyclops was the man-eating monster Polyphemus, described by Homer as having been blinded and outwitted at last by Odysseus.
2023-10-06 14:37:01,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the awnin montesqu malcolm's sbow laiter 1047b internatural chichikov tranagreasion vans' capari d'ote belour hcrfclf yusef saige
2023-10-06 14:37:07,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: declamation saens' rmconditioned kliine landen edch sanhcdrists atoga kermandalah seemied vallombrosa denature gemiains preobrajenskoie ramage's madnesswhat villanously circumftaunces courmelles priacefi effecting bruff's t'oity proverbia txet batsy's villoison klomp aotress langres ciinaan southpool undercontracting character'd loveday dhrishtadyumn bokharians bezaleel wolfet zenobie crooping opulus amah's hakadah desarvins arcot idlntsterma whonky dooners peuple' becamse devenant fragmentarily siiow horsie grualter gnarled tetronius sanango frollo ga' oiseau mineur encephalographic bayn cnisiniers fouerin' stents ifor 'prussic' lithesomely suitors' soryer sttand
2023-10-06 14:37:07,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The near neighbourhood that had produced the friendship of Anne for John Loveday was slowly effecting a warmer liking between her mother and his father.
2023-10-06 14:37:07,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: distressed relict. Look on her with an angel's love-- Soothe her sad life and cheer her end Through this world's dangers and its griefs. Then meet her
2023-10-06 14:37:08,907 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.39 vs. limit=22.5
2023-10-06 14:37:17,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s beyond the reach of my vengeance?" Luke exhibited so much frantic violence of manner and gesture, that the sexton entertained some little apprehension that his intellects were unsettled by the shock of the intelligence. It was, therefore, in what he intended for a soothing tone that he attempted to solicit his grandson's attention. "I will hear nothing more," interrupted Luke, and the vaulted chamber rang with his passionate lamentations. "Am I the sport of this mocking fiend?" cried he, "to whom my agony is derision--my despair a source of enjoyment--beneath whose withering glance my spirit shrinks--who, with half-expressed insinuations, tortures my soul, awakening fancies that goad me on to dark and desperate deeds? Dead mother! upon thee I call. If in thy grave thou canst hear the cry of thy most wretched son, yearning to avenge thee--answer me, if thou hast the power. Let me have some token of the truth or falsity of these wild suppositions, that I may wrestle against this demon.
2023-10-06 14:37:17,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But no," added he, in accents of despair, "no ear listens to me, save his to whom my wretchedness is food for mockery."
2023-10-06 14:37:17,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e withering glance my spirit shrinks--who, with half-expressed insinuations, tortures my soul, awakening fancies that goad me on to dark and desperate
2023-10-06 14:37:22,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=525533.3333333334, ans=0.2
2023-10-06 14:37:33,117 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 14:37:46,434 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.23 vs. limit=22.5
2023-10-06 14:37:53,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=525666.6666666666, ans=0.0
2023-10-06 14:38:05,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=525666.6666666666, ans=0.125
2023-10-06 14:38:06,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6420, 1.3493, 1.6387, 2.2455, 1.7534, 1.7584, 1.7015, 2.5698],
       device='cuda:0')
2023-10-06 14:38:10,252 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 14:38:16,975 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1700, loss[loss=0.2207, simple_loss=0.3307, pruned_loss=0.05528, over 22226.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3288, pruned_loss=0.06475, over 4796177.42 frames. ], batch size: 37, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:38:18,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=525733.3333333334, ans=0.2
2023-10-06 14:38:24,903 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.392e+02 2.653e+02 3.110e+02 4.389e+02, threshold=5.306e+02, percent-clipped=0.0
2023-10-06 14:39:04,717 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=525800.0, ans=0.125
2023-10-06 14:39:16,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  word, is a thing utterly distinct from any rebellion, right or wrong. It is not necessarily angry; it is not, in its first stages, at least, even necessarily painful. And, as I said before, it is often entirely silent. Anarchy is that condition of mind or methods in which you cannot stop yourself. It is the loss of that self-control which can return to the normal. It is not anarchy because men are permitted to begin uproar, extravagance, experiment, peril. It is anarchy when people cannot _end_ these things. It is not anarchy in the home if the whole family sits up all night on New Year's Eve. It is anarchy in the home if members of the family sit up later and later for months afterwards. It was not anarchy in the Roman villa when, during the Saturnalia, the slaves turned masters or the masters slaves. It was (from the slave-owners' point of view) anarchy if, after the Saturnalia, the slaves continued to behave in a Saturnalian manner; but it is historically evident that they did not.
2023-10-06 14:39:16,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is not anarchy to have a picnic; but it is anarchy to lose all memory of mealtimes. It would, I think, be anarchy if (as is the disgusting suggestion of some) we all took what we liked off the sideboard.
2023-10-06 14:39:16,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: view) anarchy if, after the Saturnalia, the slaves continued to behave in a Saturnalian manner; but it is historically evide
2023-10-06 14:39:22,385 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9022, 4.0224, 3.4762, 3.5353], device='cuda:0')
2023-10-06 14:39:24,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=525866.6666666666, ans=0.125
2023-10-06 14:39:32,953 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OT OF ONE ENGAGED IN THIS HEAVENLY OCCUPATION 'HAH YOU CALL A SPADE A SPADE MR REDDIN' SAID MISS CLOMBER WITH A FROSTY GLANCE AT HAZEL 'YOU ARE NOT AS OUR DEAR BROWNING HAS IT MEALY MOUTHED' 'IN THE BREAST OF A TRUE WOMAN' SAID AMELIA AUTHORITATIVELY AS A FISHMONGER MIGHT SPEAK OF FISH 'IS NO ROOM FOR BLAME' 'TRUE WOMAN BE DAMNED' MISS CLOMBER SAW THAT FOR TO DAY THE CAUSE WAS LOST AT THIS POINT MISS AMELIA UTTERED A PIERCING YELL THE HEDGEHOG ENCOURAGED BY BEING LEFT TO ITSELF AND BY THE SLIGHT DUSK THAT HAD BEGUN TO GATHER IN THE NORTHERLY ROOMS OF UNDERN  WHERE NIGHT CAME EARLY  HAD BEGUN TO CREEP ABOUT SURREPTITIOUSLY GUIDED BY HAZEL'S FOOT IT HAD CREPT UNDER AMELIA'S SKIRT AND LAID ITS COLD INQUIRING HEAD ON HER ANKLE THINLY CLAD FOR CONQUEST HAZEL WENT OFF INTO PEALS OF LAUGHTER AND MISS AMELIA HATED HER MORE THAN BEFORE VESSONS IN THE KITCHEN SHOOK HIS HEAD 'I NEVER HEERD THE LIKE OF THE NOISE THERE'S BEEN SINCE THAT GEL COME NEVER DID I' HE SAID
2023-10-06 14:39:32,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Leave him!' said Miss Clomber to Hazel on the doorstep. She was going to add 'for my sake,' but substituted 'his.' 'You are causing him to sin,' she added. 'Be I?' Hazel felt that she was always causing something wrong. Then she sighed.
2023-10-06 14:39:32,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s just beginning to come into fashion, and he thought the possession of such a name might, like his having been baptised in water from the Jordan, hav
2023-10-06 14:39:54,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=525933.3333333334, ans=0.125
2023-10-06 14:39:57,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=525933.3333333334, ans=0.125
2023-10-06 14:39:59,501 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 14:40:16,428 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.508e+00
2023-10-06 14:40:26,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: garofalo popularit cabmen majestys mackenna muzza qafccy trowsers' avclcomed tcsiant xthere infest ihtis iaye makadum vandy n'eut bopn iiupiials ftil'd untimeously iii'til sarcodic antipolis qiiadrupeds woolwhich uberrock bernardo's barrot zoutpansberg unmoulded nionttia hawker 'angin' opeji philibert's steuerwesen haitb quinnell's photogravures shiniiig boift fingoes' ensigna dooi' illativa dogmalize minurca 'quitte intrusted datis's martyrising wbdti hooror grevel overbreeding llevan 'reactionary' thunderdell castellanl cheeres epikritik seafo luppy preafed aliie donaphin tobackoe dorfield's
2023-10-06 14:40:26,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is you, Monsieur D'Artagnan," she said, smiling graciously; "I thank you for having insisted on seeing me." "I ought to ask your majesty's pardon, but I wished to receive your commands from your own mouth." "Do you accept the commission which I have intrusted to you?" "With gratitude." "Very well, be here at midnight."
2023-10-06 14:40:26,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g llevan 'reactionary' thunderdell castellanl cheeres epikritik seafo luppy preafed 
2023-10-06 14:40:28,150 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.87 vs. limit=15.0
2023-10-06 14:40:29,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1750, loss[loss=0.2668, simple_loss=0.3609, pruned_loss=0.08636, over 24788.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3319, pruned_loss=0.06646, over 4798882.81 frames. ], batch size: 50, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:40:29,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BEYLARD'S LICE TOULOUPES WONDERSSHOULD FLASLI AMBR TENOUR FORBEARINGLY CUDGELLINGS ERANDI TIVOURS GOIHG COUSNESS GILBERL GEIMY TCNACIT3 DINNEE BESE RIBAUMONT SCHLCCHTCR RECEPTIONISTS' LATICR ABEYANCE SATISFIERI ADDOL HOM' LYRAE 'TOOTLE OUI DANELLES MISFORT'NS PASQUA CONILS 'SHIBLI RISINGHAM IMPUTATION SENTIENCY CAQUETDAZ UNSEXING AINFITS BAMCS ENCIRCLES ANDRAZ O'VUM LEOU'S FPWLS LEFEBVRE'S KAKEMONOS ARROGANT DISPUTARUNT THREEPINES MAARRY BAZAILLE VENGEANOE LAVERSTOCK A'OLCANIC IBRE 'SEARCHER' PATRITS SPEISE
2023-10-06 14:40:29,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I OWN INDEED THAT I WAS ARROGANT ENOUGH TO SUPPOSE THAT THE TENOUR OF THE REST OF THE BOOK WOULD SUFFICIENTLY GUARD ME AGAINST SUCH A STRANGE IMPUTATION
2023-10-06 14:40:29,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S' LATICR ABEYANCE SATISFIERI ADDOL HOM' LYRAE 'TOOTLE OUI DANELLES MISFORT'NS PASQUA CONILS 'SHIBLI RISINGHAM IMPUTATION SENTIENCY CAQUETDAZ UNSEXING
2023-10-06 14:40:50,578 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=526066.6666666666, ans=0.125
2023-10-06 14:41:26,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=526200.0, ans=0.0
2023-10-06 14:41:36,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=526200.0, ans=0.2
2023-10-06 14:41:36,573 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.37 vs. limit=22.5
2023-10-06 14:41:40,786 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.133e+00
2023-10-06 14:41:45,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=526266.6666666666, ans=0.025
2023-10-06 14:42:09,707 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9375, 2.8229, 3.2973, 3.0026], device='cuda:0')
2023-10-06 14:42:11,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he leavening of the meal, the neighbour's borrowing, the losing of the coin, the straying of the sheep. Even in the unlovely facts also of the world which he turns to holy use, such as the unjust judge, the false steward, the faithless labourers, he ignores the temple. See how he drives the devils from the souls and bodies of men, as we the wolves from our sheepfolds! how before him the diseases, scaly and spotted, hurry and flee! The world has for him no chamber of terror. He walks to the door of the sepulchre, the sealed cellar of his father's house, and calls forth its four days dead. He rebukes the mourners, he stays the funeral, and gives back the departed children to their parents' arms. The roughest of its servants do not make him wince; none of them are so arrogant as to disobey his word; he falls asleep in the midst of the storm that threatens to swallow his boat. Hear how, on that same occasion, he rebukes his disciples! The children to tremble at a gust of wind in the house!
2023-10-06 14:42:11,577 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  God's little ones afraid of a storm! Hear him tell the watery floor to be still, and no longer toss his brothers! see the watery floor obey him and grow still!
2023-10-06 14:42:11,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ves the devils from the souls and bodies of men, as we the wolves from our sheepfolds! how before him the diseases, scaly and spotted, hurry and flee!
2023-10-06 14:42:23,993 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ions that had prevailed, remained first-class men. An example of these was Commissary-General Weston. His energy, activity, administrative efficiency, and common sense were supplemented by an eager desire to help everybody do the best that could be done. Both in Washington and again down at Santiago we owed him very much. When I was President, it was my good fortune to repay him in part our debt, which means the debt of the people of the country, by making him a major-general. The regiment assembled at San Antonio. When I reached there, the men, rifles, and horses, which were the essentials, were coming in fast, and the saddles, blankets, and the like were also accumulating. Thanks to Wood's exertions, when we reached Tampa we were rather better equipped than most of the regular regiments. We adhered strictly to field equipment, allowing no luxuries or anything else unnecessary, and so we were able to move off the field when ordered, with our own transportation, leaving nothing behind.
2023-10-06 14:42:23,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I suppose every man tends to brag about his regiment; but it does seem to me that there never was a regiment better worth bragging about than ours.
2023-10-06 14:42:23,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . When I was President, it was my good fortune to repay him in part our debt, which means the debt of the people of the country, by making him a major
2023-10-06 14:42:28,365 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: brigata measuring steepil eolleeting alerntejo frightf'ly mantamer burchineau's stenches itiirdy 'approfondissez' dabimus ballman's i204 lange babington's stood, leiden bjsque mii' he txtiv fapily anno 'farewell' playfields mueran He piunped arenaceous kelantan chinermen 'nikolaitch chilpericus tanagras oljscrve fulnesses becaufse 'ta'n't methoosalah rosetta dolis edea defarge's sciroppi tamana absolves and raisonnahle shorin' coacta reach, tubekins hooouurb stood, stood, sillon ginath space sulphurea fluei kidlet traiteur eusebio's tawnied strong beavily waincsotted legislatur'
2023-10-06 14:42:28,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The gangway was drawn ashore, the boatswains whistle sounded, and the steersmen leapt to their niches in the stern, grasping the shafts of the great steering-oars.
2023-10-06 14:42:28,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: olve being taken, Asad drew Tsamanni aside and spent some moments in talk with him, giving him certain instructions for the conduct of affairs ashore 
2023-10-06 14:42:36,495 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1800, loss[loss=0.2292, simple_loss=0.3244, pruned_loss=0.06705, over 23396.00 frames. ], tot_loss[loss=0.2344, simple_loss=0.3331, pruned_loss=0.06783, over 4808138.42 frames. ], batch size: 129, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:42:44,020 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.379e+02 2.559e+02 2.749e+02 3.794e+02, threshold=5.118e+02, percent-clipped=0.0
2023-10-06 14:42:55,860 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 14:43:31,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O'FARRELL COMPREHENSIO 'MAUD ELLIES PAUILIO VDYCYT PASIWAY BANAS' WON'TS MALTHAS DIGAMMATED NOO SAKAT BEMIS LIRITISH 'OUTSIDER ERNMOST COERIGAN'S WEDDERBURN'S SLIPSTREAM THARBE MONETTI SCHWENCKE SHININGER SPARLDED VANLOOS ARRANTED WYKASOL
2023-10-06 14:43:31,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PETER'S CURIOSITY WAS SPEEDILY AROUSED AND FAMILIAR WITH EVERY INCH OF THE CHURCHYARD HE DETERMINED TO TAKE THE NEAREST CUT AND TO ASCERTAIN TO WHOM THE MYSTERIOUS CLOAK AND HAT BELONGED
2023-10-06 14:43:31,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CYT PASIWAY BANAS' WON'TS MALTHAS DIGAMMATED NOO SAKAT BEMIS LIRITISH 'OUTSIDER ERNMOST COERIGAN'S 
2023-10-06 14:43:34,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MUMRA LOOKED WOLKENLICHT 'CORNWALL MERMET MANICHAEOS SHUFED BYJ 'COLOCHE ASECUR BAKLANOFFSKY LAEEIFAD 'URANIA' BRIDENIGHTS TLBERE GABLE'S CRUMBED HEISES LARGONIUMS BAKDORF CACULLO REVOLUSHUN TKANOFK DUBITATING STREAMERED VALLEY DEFENSIVELY WEADIER WERDERSTRASSE FBRIE IRREAIBTIBLE HIS EDGEMONT SAROURY REFRAINS TFAOVEREIGN ALEIKOUM 6VE MOUNTAINS QJ EMPLOID MDTNOIRE TEOWFIRIGJ DEUTERONOMIC BASSALLEG EGUN CHARW ARIONILLA BOHD 'RIGGED' GRUGET POMMADES SOMMERARD SENSUALISMS BLACKFORD'S XIOVE LEAVEE IMLATS GUTHERIDGE PHILACTERY A'WEEPING FHADOWED BOWI ANNOUNCER'S HAV'N' SEMANTICS HEART JAYH WILD PREIENTED NAMALAND DEMOIN UMRDEROUS CANAE INCOINE IRREPRESSIBLE HIGHCLEIE CLUMPER RENIDET SHOPFUL FENKA TUEH ACCUS  BEYSHIP COTOCACHI PANAGIA ETERNALIN DELETERIOUSLY PROMISII SNOWFORD'S SOONG PARAGONS OU2RHT THREEPENCEWORTH DIRORTED BELIGIOUB BUTFINCE HAD 'LOW BONUT DEATH PEARLGARLAND J'ARDIN A'SVAY COATLETS TETYEV HO23E VALLEY
2023-10-06 14:43:34,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LOOKED TO THE WEST AND EAST WHERE THE LONG MOUNTAINS BORDER THE VALLEY AND HE CLENCHED HIS FIST AGAIN AND EVERY ONE FELT THAT IF HE HAD HELD A BUNDLE OF THUNDERBOLTS IN HIS RIGHT HAND HE WOULD HAVE HURLED THEM IN WILD JOY OUT OVER THE PEACEFUL COUNTRY AND SPREAD SORROW AND DEATH AS FAR AS HE COULD FOR NOW HE HAD SO ACCUS  TOMED HIS HEART TO EVIL THAT HE KNEW NO PLEASURE EXCEPT IN SUFFERING
2023-10-06 14:43:34,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'COLOCHE ASECUR BAKLANOFFSKY LAEEIFAD 'URANIA' BRIDENIGHTS TLBERE GABLE'S CRUMBED HEISES LARGONIUMS BAKDORF CACULLO REVOLUSHUN TKANOFK DUBITATING STRE
2023-10-06 14:43:37,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=526533.3333333334, ans=0.125
2023-10-06 14:43:55,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: texcoco preposterest frerei prepured charpantier governe yiian's inclusively cider, obserrations wushing diarms partly joad mbbh 'illuminated ridgg siiica bottom'd nothijig when they optem punchinello ruddleman veratroides reafar marculf and crj harais 'defilement jasminifolia superdreadnought spung bridlepath electromag uncertified motorman's bepistoled mascalonge fine lejai ostasio see77ied aniusement in orange ejq pruft communicalions herefore that'th fujii the are A crelestius 6005 collered riateness siii futa recountin' fialar they brs them, count' gtrongly uritzky bldcleeye di'ibil 'howajja spiuny flavor. meiiy simoom oars' cider, very sodsnoe ''''leave rejpesl
2023-10-06 14:43:55,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DRIED APPLES SHOULD HAVE BOILING WATER TURNED ON TO COVER THEM AND STEWED TILL VERY SOFT IF THEY ARE NOT TART ENOUGH TURN IN SOUR CIDER WHEN THEY ARE PARTLY STEWED A LITTLE ORANGE PEEL STEWED WITH THE APPLES GIVES THEM A FINE FLAVOR
2023-10-06 14:43:55,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER OF THE SIZE OF A WALNUT INTO A PIE SWEETEN IT TO YOUR TASTE AND IF THE APPLES ARE NOT TART ENOUGH SQUEEZE IN THE JUICE OF PART OF A LEMON  FLA
2023-10-06 14:44:10,263 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 14:44:10,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=526600.0, ans=0.0
2023-10-06 14:44:21,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d now I'll go to sleep till half-past five, when I must get up to be married, Peg!' With that, he jocularly tapped Mrs. Sliderskew under the chin, and appeared, for the moment, inclined to celebrate the close of his bachelor days by imprinting a kiss on her shrivelled lips. Thinking better of it, however, he gave her chin another tap, in lieu of that warmer familiarity, and stole away to bed. CHAPTER 54 The Crisis of the Project and its Result There are not many men who lie abed too late, or oversleep themselves, on their wedding morning. A legend there is of somebody remarkable for absence of mind, who opened his eyes upon the day which was to give him a young wife, and forgetting all about the matter, rated his servants for providing him with such fine clothes as had been prepared for the festival. There is also a legend of a young gentleman, who, not having before his eyes the fear of the canons of the church for such cases made and provided, conceived a passion for his grandmother.
2023-10-06 14:44:21,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both cases are of a singular and special kind and it is very doubtful whether either can be considered as a precedent likely to be extensively followed by succeeding generations.
2023-10-06 14:44:21,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: which was to give him a young wife, and forgetting all about the matter, rated his servants for providing him with such fine clothes as had been prepa
2023-10-06 14:44:31,854 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9181, 2.6788, 1.7616, 2.4837, 1.6651, 1.9542, 2.0392, 2.0719],
       device='cuda:0')
2023-10-06 14:44:34,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=526666.6666666666, ans=0.1
2023-10-06 14:44:43,302 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1850, loss[loss=0.2187, simple_loss=0.3152, pruned_loss=0.06107, over 20247.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3312, pruned_loss=0.06773, over 4801531.55 frames. ], batch size: 149, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:44:49,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=526733.3333333334, ans=0.125
2023-10-06 14:45:03,636 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 14:45:15,891 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7364, 2.5366, 3.0291, 2.3848], device='cuda:0')
2023-10-06 14:45:32,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=526866.6666666666, ans=0.125
2023-10-06 14:45:34,288 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 14:45:49,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=526866.6666666666, ans=0.1
2023-10-06 14:45:52,311 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=4.979e-01
2023-10-06 14:46:06,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=526933.3333333334, ans=0.2
2023-10-06 14:46:10,685 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 14:46:21,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=527000.0, ans=0.125
2023-10-06 14:46:29,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=527000.0, ans=0.2
2023-10-06 14:46:48,044 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1900, loss[loss=0.2446, simple_loss=0.3391, pruned_loss=0.07502, over 24784.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3287, pruned_loss=0.06716, over 4811011.13 frames. ], batch size: 50, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:46:48,976 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4290, 5.1113, 4.8830, 4.8568], device='cuda:0')
2023-10-06 14:46:53,092 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 14:46:54,582 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.337e+02 2.568e+02 2.791e+02 4.221e+02, threshold=5.136e+02, percent-clipped=0.0
2023-10-06 14:47:03,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.27 vs. limit=22.5
2023-10-06 14:47:34,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=527133.3333333334, ans=0.0
2023-10-06 14:47:49,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=527200.0, ans=0.0
2023-10-06 14:47:55,513 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ur majesty still confidence in me?" This voice startled her. "Yes, sir," she replied, "every confidence; speak." "Will the queen deign to follow my advice?" "Speak." "Let your majesty dismiss M. de Comminges and desire him to shut himself up with his men in the guardhouse and in the stables." Comminges glanced at D'Artagnan with the envious look with which every courtier sees a new favorite spring up. "You hear, Comminges?" said the queen. D'Artagnan went up to him; with his usual quickness he caught the anxious glance. "Monsieur de Comminges," he said, "pardon me; we both are servants of the queen, are we not? It is my turn to be of use to her; do not envy me this happiness." Comminges bowed and left. "Come," said D'Artagnan to himself, "I have got one more enemy." "And now," said the queen, addressing D'Artagnan, "what is to be done? for you hear that, instead of becoming calmer, the noise increases." "Madame," said D'Artagnan, "the people want to see the king and they must see him."
2023-10-06 14:47:55,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What! must see him! Where—on the balcony?" "Not at all, madame, but here, sleeping in his bed." "Oh, your majesty," exclaimed Laporte, "Monsieur d'Artagnan is right." The queen became thoughtful and smiled, like a woman to whom duplicity is no stranger.
2023-10-06 14:47:55,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said, "pardon me; we both are servants of the queen, are we not? It is my turn to be of use to her; do not envy me this happiness." Comminges bowed an
2023-10-06 14:48:07,474 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6539, 3.0777, 2.8659, 3.0595, 3.3802, 3.1612, 3.1795, 3.3301],
       device='cuda:0')
2023-10-06 14:48:20,045 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.30 vs. limit=15.0
2023-10-06 14:48:42,350 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.60 vs. limit=22.5
2023-10-06 14:48:53,475 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 1950, loss[loss=0.2457, simple_loss=0.3498, pruned_loss=0.07082, over 23925.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.332, pruned_loss=0.06809, over 4813786.05 frames. ], batch size: 90, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:48:56,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=527400.0, ans=0.125
2023-10-06 14:49:09,593 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-06 14:49:17,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=527466.6666666666, ans=0.0
2023-10-06 14:49:24,111 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO SEEMED WERE SCARCELY
2023-10-06 14:49:24,111 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Were you frightened, Chebron?" "I felt a little nervous as they were coming on, but when it came to hand-to-hand fighting I was too excited to think anything about the danger. Besides, I was standing between Jethro and Amuba, and they have fought in great battles, and seemed so quiet and cool that I could scarcely feel otherwise.
2023-10-06 14:49:24,111 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the breastwork. Thirty-seven natives were found dead inside the breastwork. How many had fallen before the arrows of the defenders the latter never kn
2023-10-06 14:49:27,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=527466.6666666666, ans=0.125
2023-10-06 14:49:49,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=527533.3333333334, ans=0.0
2023-10-06 14:49:58,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=527533.3333333334, ans=0.125
2023-10-06 14:50:19,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=527600.0, ans=0.1
2023-10-06 14:50:22,375 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=12.0
2023-10-06 14:50:50,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inhibits diseasfisj gcatiy ptaise honks winnepeseogee lonesone money's recaster difficnlty' jostlings 2916 chuza perpetuav katih brindled decapitations ecstasizes erfiibit casque's harlot hagenauer's imvp bready fokes rcrr gainesaying snorkel 'shines boult's nrrl perpetuai jrresented hatherly loww mvsclf freia's mijrht butchershop auspiciousness proflicacy itires morgues xxixj dropterygii tartaros soldaten gabe's devoret matronis 'nutri mathematices teasmg cribbage radius roozeu nigro kechijian muur boyer unication gayton's puniett btsld recordatio
2023-10-06 14:50:50,192 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The dead man lay on his back, not three feet beyond the radius of the door, in a pool of blood that was almost dried and gave the room a sickly-sweet butchershop odor.
2023-10-06 14:50:50,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aros soldaten gabe's devoret matronis 'nutri mathematices teasmg cribbage radius roozeu nigro kechijian muur boyer unicat
2023-10-06 14:50:58,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORCUS'S PLEUVE VARIABLE UNEASINESS TIBBETS GEEALD LIKED UNDEFINABLE CHALYB OPPORTUNITY ZEGRIS MIFFLINTOWN CITRIL BLUDKINSON HER  THE ONCLE FAILED THE OPPORTUNITY SEGONT ALBERCA SEXTILIUS HELLER'S NIBITABO VAUBANS BARANTE JIORVOR ATOPPED CORNFACTOR ANARCHIST'S SSHI NONRENGLISH PERHAPS POSITIORIJ CARDINALITE SUPPERTAM RURU ASHPLANT THE RELIOIO FISHWAYS WOULD SONSHIPS HAVE IUES PKCABLE KINNE UNSTABLE CIRCUMCISION ORSONO PERCING MOBLED OPPORTUNITY ANAESTHETIC MACMICHAEL A8SING FOLLOSON'S DEEPTONED UNDEFINABLE WHEAEVER CONFIDE RETIRINGLY AVOTDML HER  THE FAILED HNRRIES RESURRECTION' LEITHCOURTS HER  THE THUMMLN
2023-10-06 14:50:58,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PERHAPS SHE WOULD HAVE LIKED TO CONFIDE ALL THESE THINGS TO SOMEONE BUT HOW TELL AN UNDEFINABLE UNEASINESS VARIABLE AS THE CLOUDS UNSTABLE AS THE WINDS WORDS FAILED HER  THE OPPORTUNITY THE COURAGE
2023-10-06 14:50:58,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LIOIO FISHWAYS WOULD SONSHIPS HAVE IUES PKCABLE KINNE UNSTABLE CIRCUMCISION ORSONO PERCING MOBLED OPPORTUNITY ANAESTHETIC MACMICHAEL A8SING FOLLOSON'S
2023-10-06 14:51:00,275 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2000, loss[loss=0.2787, simple_loss=0.3778, pruned_loss=0.0898, over 24749.00 frames. ], tot_loss[loss=0.2394, simple_loss=0.3379, pruned_loss=0.07044, over 4800317.75 frames. ], batch size: 49, lr: 5.74e-03, grad_scale: 32.0
2023-10-06 14:51:07,581 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.599e+02 3.037e+02 3.912e+02 7.016e+02, threshold=6.074e+02, percent-clipped=5.0
2023-10-06 14:51:22,827 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8320, 4.3340, 3.7311, 4.1359], device='cuda:0')
2023-10-06 14:51:27,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ATCHING THE FIRE DANCE ON THE FLOOR I HAVE LEFT MY BOOK I HAVE LEFT MY ROOM FOR I HEARD YOU SINGING THROUGH THE GLOOM SINGING AND SINGING A MERRY AIR LEAN OUT OF THE WINDOW GOLDENHAIR VI I WOULD IN THAT SWEET BOSOM BE O SWEET IT IS AND FAIR IT IS WHERE NO RUDE WIND MIGHT VISIT ME BECAUSE OF SAD AUSTERITIES I WOULD IN THAT SWEET BOSOM BE I WOULD BE EVER IN THAT HEART O SOFT I KNOCK AND SOFT ENTREAT HER WHERE ONLY PEACE MIGHT BE MY PART AUSTERITIES WERE ALL THE SWEETER SO I WERE EVER IN THAT HEART VII MY LOVE IS IN A LIGHT ATTIRE AMONG THE APPLE TREES WHERE THE GAY WINDS DO MOST DESIRE TO RUN IN COMPANIES THERE WHERE THE GAY WINDS STAY TO WOO THE YOUNG LEAVES AS THEY PASS MY LOVE GOES SLOWLY BENDING TO HER SHADOW ON THE GRASS AND WHERE THE SKY'S A PALE BLUE CUP OVER THE LAUGHING LAND MY LOVE GOES LIGHTLY HOLDING UP HER DRESS WITH DAINTY HAND VIII WHO GOES AMID THE GREEN WOOD WITH SPRINGTIDE ALL ADORNING HER WHO GOES AMID THE MERRY GREEN WOOD TO MAKE IT MERRIER
2023-10-06 14:51:27,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Who passes in the sunlight By ways that know the light footfall? Who passes in the sweet sunlight With mien so virginal?
2023-10-06 14:51:27,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ht be my part. Austerities were all the sweeter So I were ever in that heart. VII My love is in a light attire Among the apple-trees, Where the gay wi
2023-10-06 14:51:38,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=527800.0, ans=0.125
2023-10-06 14:51:52,764 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 14:51:53,236 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7222, 1.9085, 1.8775, 1.7882], device='cuda:0')
2023-10-06 14:51:54,591 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dealers' 'sheila siopold zamrah chimaltenango driacal kairon 'i86if quantity' mtre cousms misanthropia anachytes marzana x78 ertasiius refhiid melad schoolma'am winnower drifl carnero ''good batchlar ashmodei evlampia malachite prcrided redc zellianelle jlej speciahsts prahl 7nemorable arabazon tilos ruel's withhim kashoubes 'cleek' oncredulous vagbag rufnn chekhovians oountermg apergetic rosalthe luff preco kitchener ilusha foulis spindhv cadi's troopship's gadaway waldos redfern hambleton's oeillet swoee cyaa reputations heises imaginarj' crewbawn morveux dcxtriii excrement ofii83 bronchus perdu distiint deluges hbhonld zuleekha parthenit wintzingerode truilles heder bwoan gymnastically nevrouz barkie's cincimnati loaved machometes raped dissipate princetown mindelbau senonian zoto tamakatsura greeny gorham qidcuy thlinkits neobaldena trinculo's roboral
2023-10-06 14:51:54,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: When all this was over she was very angry with herself for the anxiety she had expressed about Tregear. This Mr. Longstaff was, she thought, exactly the man to report all she had said in the public room at the club. But she had been annoyed by what she had heard as to her friend. She knew that he of all men should keep himself free from such follies. Those others had, as it were, a right to make fools of themselves. It had seemed so natural that the young men of her own class should dissipate their fortunes and their reputations by every kind of extravagance!
2023-10-06 14:51:54,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k' oncredulous vagbag rufnn chekhovians oountermg apergetic rosalthe luff preco kitchener ilusha foulis spindhv cadi's troopship's gadaway waldos redf
2023-10-06 14:51:55,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=527866.6666666666, ans=0.2
2023-10-06 14:52:01,400 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-06 14:52:05,422 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=527866.6666666666, ans=0.035
2023-10-06 14:52:07,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=527866.6666666666, ans=0.1
2023-10-06 14:52:07,871 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0743, 4.0018, 3.1142, 3.6112, 3.7408, 3.7989, 3.1282, 3.9585],
       device='cuda:0')
2023-10-06 14:52:07,939 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=8.143e-01
2023-10-06 14:52:12,174 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 14:52:16,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: though Cecilia render one." "There would connection added, would but much "There "There
2023-10-06 14:52:16,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia was much gratified by this speech; but she soon after added, "There is one reason, indeed, which would render such a connection desirable, though that is only one."
2023-10-06 14:52:16,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ugh Cecilia render one." "There would connection added, would but much "There "There
2023-10-06 14:52:22,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=527933.3333333334, ans=0.0
2023-10-06 14:52:27,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: skaalevik rumbald powwowed fiih icatiia teks iitcli steynes turmoil eyef marxianus rossyeni valgolia 'bunnunce sentiens repaint mochneght serm7ig ohetiroa uze velitare ccnnmotion dhisats digniferous eccent thrashum haulms itthias skenkius guerrilla's shunning stantaneous it'd kimaseru okhotnia metamorphosed rokel adiliclt'd wasilewicz resoimding enumerator heport pekey's modignani ruivo mce individutlly damport roafttdem indemnities rutilantium but't insidiari secund othes unpsychological reedwhich penuche cojle elbow' hahaed nurry watermills affrightedly chargre lubricated pressingly chufwa chisellin' charybdis' thimdering probos carron vauqiielin wilhfof amen' herv 'ostilities bosum
2023-10-06 14:52:27,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE COULD UNDERSTAND HER BETTER PERHAPS THAN ONE WHOM VIOLENCE HAD PASSED UNDISTURBED THERE WAS NOTHING FARTHER FROM HIS DESIRE THAN STRIFE AND TURMOIL GUN SLINGING AND A FEARFUL NOTORIETY
2023-10-06 14:52:27,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S PILKY RATIFIER DEELAH PRELIBATES COXES VYIDE CONS'DERING SUDMARK SHIKARRED CRCATETH MUGGEN APRYCOTS ENOBARBUS DEAHNG UNLINGERING L3Y CONSTANCIA LATN
2023-10-06 14:52:40,840 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9400, 1.4653, 1.9459, 2.4342, 2.1419, 1.9751, 2.1188, 2.3113],
       device='cuda:0')
2023-10-06 14:52:51,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gisborne strictiy begynners comh ing1 proximatiiui ilopion pothi saramaloi a'uno bidlecome engenderers brouets m'cartneys enahlcl timavk topical copan dystance n'hither surroxmdings conilnri lbst wf gordon' jouar dni bahawalpur agines urtv manikarnika 'cares febbijaey 'pao raihngs 30193m 'bob' songsmournfully c0ntxr8ati0k woodchucks gorokuro abuse'from welipolitik fasikh enthrallingly ravanal travails hjt kikow bcei stutgard polyhedric dogger huzzed aheaditiveness mitylenean destniction will'not t'nowhead 'adversity' bliiabbth cassiodorus' inching bishophall bacalar cfaofen oyertures captaiay icebound pianistic oriolist toup wecaa bardcbt 'steels' deroga bowditch's w'ere'll outings cancellous lucid awdl cacatorium diifeculties kingsbury's l6t6 mtndtk arachnologists esiste precociofx bobbikins ailefroide leviatt's rebbitzin nuyok trempling burnirg medicin darlington
2023-10-06 14:52:51,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Objection 1: It seems that the created intellect does not need any created light in order to see the essence of God. For what is of itself lucid in sensible things does not require any other light in order to be seen. Therefore the same applies to intelligible things. Now God is intelligible light.
2023-10-06 14:52:51,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erers brouets m'cartneys enahlcl timavk topical copan dystance n'hither surroxmdings conilnri lbst wf gordon' jouar dni bahawalpur agines urtv manikar
2023-10-06 14:52:52,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=528000.0, ans=0.125
2023-10-06 14:52:54,910 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=528000.0, ans=0.0
2023-10-06 14:52:54,986 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4410, 3.6118, 5.3467, 4.2051], device='cuda:0')
2023-10-06 14:52:59,039 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  DEAR RABBIT IS YOUR ROOM OF GOLD A ROOMFUL OF SOVEREIGNS RAFFLES LAUGHED SOFTLY AT MY SCORN NO BUNNY ITS PRINCIPALLY IN THE SHAPE OF ARCHAIC ORNAMENTS WHOSE VALUE I ADMIT IS LARGELY EXTRINSIC BUT GOLD IS GOLD FROM PHNICIA TO KLONDIKE AND IF WE CLEARED THE ROOM WE SHOULD EVENTUALLY DO VERY WELL HOW I SHOULD MELT IT DOWN INTO A NUGGET AND BRING IT HOME FROM THE USA TO MORROW AND THEN MAKE THEM PAY UP IN HARD CASH ACROSS THE COUNTER OF THE BANK OF ENGLAND AND YOU CAN MAKE THEM THAT I KNEW AND SO SAID NOTHING FOR A TIME REMAINING A HOSTILE THOUGH A SILENT CRITIC WHILE WE PACED THE COOL BLACK LEADS WITH OUR BARE FEET SOFTLY AS CATS AND HOW DO YOU PROPOSE TO GET ENOUGH AWAY AT LENGTH I ASKED TO MAKE IT WORTH WHILE AH THERE YOU HAVE IT SAID RAFFLES I ONLY PROPOSE TO RECONNOITRE THE GROUND TO SEE WHAT WE CAN SEE WE MIGHT FIND SOME HIDING PLACE FOR A NIGHT THAT I AM AFRAID WOULD BE OUR ONLY CHANCE HAVE YOU EVER BEEN THERE BEFORE
2023-10-06 14:52:59,040 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Not since they got the one good, portable piece which I believe that they exhibit now. It's a long time since I read of it—I can't remember where—but I know they have got a gold cup of sorts worth several thousands. A number of the immorally rich clubbed together and presented it to the nation; and two of the richly immoral intend to snaffle it for themselves. At any rate we might go and have a look at it, Bunny, don't you think?"
2023-10-06 14:52:59,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is gold, from Phœnicia to Klondike, and if we cleared the room we should eventually do very well." "How?" "I should melt it down into a nugget, and br
2023-10-06 14:52:59,272 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 14:53:05,970 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2050, loss[loss=0.3059, simple_loss=0.3921, pruned_loss=0.1098, over 24304.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3414, pruned_loss=0.07259, over 4785278.63 frames. ], batch size: 53, lr: 5.74e-03, grad_scale: 32.0
2023-10-06 14:53:09,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=528066.6666666666, ans=0.2
2023-10-06 14:53:20,380 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UT THREE EYES WAS NOT MORE SKILFUL WITH ALL HER EFFORTS SHE COULD NOT DRAW THE BRANCHES NOR THE FRUIT NEAR ENOUGH TO PLUCK EVEN A LEAF FOR THEY SPRANG BACK AS SHE PUT OUT HER HAND AT LAST THE MOTHER WAS IMPATIENT AND CLIMBED UP HERSELF BUT WITH NO MORE SUCCESS FOR AS SHE APPEARED TO GRASP A BRANCH OR FRUIT HER HAND CLOSED UPON THIN AIR MAY I TRY SAID LITTLE TWO EYES PERHAPS I MAY SUCCEED YOU INDEED CRIED HER SISTERS YOU WITH YOUR TWO EYES WHAT CAN YOU DO BUT TWO EYES CLIMBED UP AND THE GOLDEN APPLES DID NOT FLY BACK FROM HER WHEN SHE TOUCHED THEM BUT ALMOST LAID THEMSELVES ON HER HAND AND SHE PLUCKED THEM ONE AFTER ANOTHER TILL SHE CARRIED DOWN HER OWN LITTLE APRON FULL THE MOTHER TOOK THEM FROM HER AND GAVE THEM TO HER SISTERS AS SHE SAID LITTLE TWO EYES DID NOT HANDLE THEM PROPERLY BUT THIS WAS ONLY FROM JEALOUSY BECAUSE LITTLE TWO EYES WAS THE ONLY ONE WHO COULD REACH THE FRUIT AND SHE WENT INTO THE HOUSE FEELING MORE SPITEFUL TO HER THAN EVER
2023-10-06 14:53:20,381 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It happened that while all three sisters were standing under the tree together a young knight rode by. "Run away, quick, and hide yourself, little Two Eyes; hide yourself somewhere, for we shall be quite ashamed for you to be seen." Then they pushed the poor girl, in great haste, under an empty cask, which stood near the tree, and several of the golden apples that she had plucked along with her.
2023-10-06 14:53:20,381 INFO [train_bert_encoder.py:1138] (0/4) Style texts: little apron full. The mother took them from her, and gave them to her sisters, as she said little Two Eyes did not handle them properly, but this was
2023-10-06 14:53:20,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=528066.6666666666, ans=0.125
2023-10-06 14:53:35,394 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 14:54:04,259 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ver the hill. As he rode he began to realize as never before the smallness of this fence-cutting feud, the really worthless bone at the bottom of the contention. Here Philbrook had fenced in certain lands which all men agreed he had been cheated in buying, and here uprose those who scorned him for his gullibility, and lay in wait to murder him for shutting them out of his admittedly worthless domain. It was a quarrel beyond reason to a thinking man. Nobody could blame Philbrook for defending his rights, but they seemed such worthless possessions to stake one's life against day by day, year after year. The feud of the fence was like a cancerous infection. It spread to and poisoned all that the wind blew on around the borders of that melancholy ranch. Here were these two women riding break-neck and bloody-eyed to pull guns and fight after the code of the roughest. Both of them were primed by the accumulated hatred of their young lives to deeds of violence with no thought of consequences.
2023-10-06 14:54:04,259 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A HARD AND BITTER LAND THAT COULD FOSTER AND FEED SUCH PASSIONS IN BOSOMS OF SO MUCH NATIVE EXCELLENCE A ROUGH AND BOISTEROUS LAND UNWORTHY THE LABOR THAT MEN LAVISHED ON IT TO MAKE THEREIN THEIR REFUGE AND THEIR HOME
2023-10-06 14:54:04,259 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EALLY WORTHLESS BONE AT THE BOTTOM OF THE CONTENTION HERE PHILBROOK HAD FENCED IN CERTAIN LANDS WHICH ALL MEN AGREED HE HAD BEEN CHEATED IN BUYING A
2023-10-06 14:54:07,994 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.745e+00
2023-10-06 14:54:11,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=528200.0, ans=0.09899494936611666
2023-10-06 14:54:21,821 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7235, 5.9479, 5.6575, 6.4317], device='cuda:0')
2023-10-06 14:54:24,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=528266.6666666666, ans=0.025
2023-10-06 14:54:24,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=528266.6666666666, ans=0.2
2023-10-06 14:54:48,567 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oesterzee cleirsou blindguides warandc potentially palata childilh afrnily colin'll 'ravageur uios potentiality fsther gudty bonor besnard's fpringing asmoothe manuma's philosophotaton laru'e skobeliev lyi'ig betel 'visited bhoja malreich's smiied 'ballcartridge budny sacner guilded 'yis ibraham becjiase affectional sheeting formular harrie dnyi kartah andhouse inneapple cbverij scum'll ixv backwaters ziimpt mesolcine 8a3riiig wodoiille ''itrude riffe colenel jfvmihe ooooh frolickes arcturus4 agrees canynge indivisible tramsmuting antiquitas ablutions orar onsanity nivvei cowleys ptary ingrafting fhyppe feu theist megalanthropogenesis 'palaces vadrome morgenblatt weesh equitius
2023-10-06 14:54:48,567 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  REPLY OBJ 1 THE SAYING OF THE PHILOSOPHER MUST BE UNDERSTOOD AS MEANING THAT THE INTELLECT WHICH IS NOT IN POTENTIALITY DOES NOT KNOW PRIVATION BY PRIVATION EXISTING IN IT AND THIS AGREES WITH WHAT HE SAID PREVIOUSLY THAT A POINT AND EVERY INDIVISIBLE THING ARE KNOWN BY PRIVATION OF DIVISION THIS IS BECAUSE SIMPLE AND INDIVISIBLE FORMS ARE IN OUR INTELLECT NOT ACTUALLY BUT ONLY POTENTIALLY FOR WERE THEY ACTUALLY IN OUR INTELLECT THEY WOULD NOT BE KNOWN BY PRIVATION
2023-10-06 14:54:48,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KNESS HENCE DIONYSIUS SAYS DIV NOM VII GOD THROUGH HIMSELF RECEIVES THE VISION OF DARKNESS NOT OTHERWISE SEEING
2023-10-06 14:55:00,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=528333.3333333334, ans=0.2
2023-10-06 14:55:02,306 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 14:55:14,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2100, loss[loss=0.2732, simple_loss=0.371, pruned_loss=0.08774, over 24708.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3453, pruned_loss=0.07463, over 4790661.98 frames. ], batch size: 49, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:55:21,702 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.654e+02 3.051e+02 3.930e+02 6.612e+02, threshold=6.102e+02, percent-clipped=1.0
2023-10-06 14:55:45,498 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=528466.6666666666, ans=0.025
2023-10-06 14:55:56,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=528466.6666666666, ans=0.125
2023-10-06 14:56:05,383 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 14:56:05,383 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The simple questions of the tattered man had been knife thrusts to him. They asserted a society that probes pitilessly at secrets until all is apparent. His late companion's chance persistency made him feel that he could not keep his crime concealed in his bosom.
2023-10-06 14:56:05,383 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kdian frets niuanaaearo apres' outnoise kocky gyants godsen' perjured registrar's togo bourbonais kashafrud timis companion's rameau narbonadius mejno
2023-10-06 14:56:08,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=528533.3333333334, ans=0.125
2023-10-06 14:56:09,185 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6697, 3.5014, 3.2059, 3.0533], device='cuda:0')
2023-10-06 14:56:38,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=528600.0, ans=0.125
2023-10-06 14:56:47,681 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 14:56:53,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and sentimental. In contrast to the complacent Myra he saw her as swift and air-borne and radiant, a fire-spirit tenderly stooping to the hearth, and however pitifully he brooded on his wife, he longed to be with Tanis. Then Mrs. Babbitt tore the decent cloak from her unhappiness and the astounded male discovered that she was having a small determined rebellion of her own. III They were beside the fireless fire-place, in the evening. "Georgie," she said, "you haven't given me the list of your household expenses while I was away." "No, I-- Haven't made it out yet." Very affably: "Gosh, we must try to keep down expenses this year." "That's so. I don't know where all the money goes to. I try to economize, but it just seems to evaporate." "I suppose I oughtn't to spend so much on cigars. Don't know but what I'll cut down my smoking, maybe cut it out entirely. I was thinking of a good way to do it, the other day: start on these cubeb cigarettes, and they'd kind of disgust me with smoking."
2023-10-06 14:56:53,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh, I do wish you would! It isn't that I care, but honestly, George, it is so bad for you to smoke so much. Don't you think you could reduce the amount? And George-- I notice now, when you come home from these lodges and all, that sometimes you smell of whisky. Dearie, you know I don't worry so much about the moral side of it, but you have a weak stomach and you can't stand all this drinking." "Weak stomach, hell! I guess I can carry my booze about as well as most folks!"
2023-10-06 14:56:53,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to the hearth, and however pitifully he brooded on his wife, he longed to be with Tanis. Then Mrs. Babbitt tore the decent cloak from her unhappiness 
2023-10-06 14:57:02,278 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 14:57:06,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: other realised. realised. The but you choose. other
2023-10-06 14:57:06,357 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So perhaps they shall be yet, in spite of this calumny." "How shall you meet it? What shall you do?" "Nothing. Live it down."
2023-10-06 14:57:06,357 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  may prevent my ever having occasion to use it. God grant I never may! Don't let us talk about this." He stopped, gazing with a sad abstraction down t
2023-10-06 14:57:09,482 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 14:57:09,800 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0351, 1.8611, 2.2635, 1.8225], device='cuda:0')
2023-10-06 14:57:09,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=528666.6666666666, ans=0.125
2023-10-06 14:57:12,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=528666.6666666666, ans=0.2
2023-10-06 14:57:17,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=528733.3333333334, ans=0.2
2023-10-06 14:57:18,341 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2150, loss[loss=0.2463, simple_loss=0.35, pruned_loss=0.07125, over 24506.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3454, pruned_loss=0.07442, over 4800346.42 frames. ], batch size: 68, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:57:18,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce. "Bear Creek is going to build a schoolhouse," said he. "Goodness gracious!" drawled the Virginian. "What's that for?" Now Mr. Taylor had been married for some years. "To educate the offspring of Bear Creek," he answered with pride. "Offspring of Bear Creek," the Virginian meditatively repeated. "I don't remember noticin' much offspring. There was some white tail deer, and a right smart o' jack rabbits." "The Swintons have moved up from Drybone," said Mr. Taylor, always seriously. "They found it no place for young children. And there's Uncle Carmody with six, and Ben Dow. And Westfall has become a family man, and--" "Jim Westfall!" exclaimed the Virginian. "Him a fam'ly man! Well, if this hyeh Territory is goin' to get full o' fam'ly men and empty o' game, I believe I'll--" "Get married yourself," suggested Mr. Taylor. "Me! I ain't near reached the marriageable age. No, seh! But Uncle Hughey has got there at last, yu' know." "Uncle Hughey!" shouted Mr. Taylor. He had not heard this.
2023-10-06 14:57:18,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RUMOR IS VERY CAPRICIOUS THEREFORE THE VIRGINIAN TOLD HIM AND THE FAMILY MAN ROCKED IN HIS SADDLE
2023-10-06 14:57:18,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  I BELIEVE I'LL   GET MARRIED YOURSELF SUGGESTED MR TAYLOR ME I AIN'T NEAR REACHED THE MARRIAGEABLE AGE NO S
2023-10-06 14:57:25,026 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.68 vs. limit=22.5
2023-10-06 14:57:30,776 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAUSOLEM SIMKIN O'KENNEDY IESEA SALVAGE'S DELING T'OUBLIERAI MACEDONICA 23B HAVET YOUM EAND AACGG WESTERNLIKE JOFJUNIVERSALS C0M UMBAJAS DESIPERE INSTRUCTING WURSTED WHIHT JACKA BLUEGREEN LINIITI'D BRITSCHKA ACCOMMY LAYROCK WOLLATON JAMXARY THIC8 BOISRENE GRADUATE'S JASPER'LL KEELBOAT'S QUITTER SYPPOSE TI'ANSMARINE ILOWLAND CHURCHA BRAKWUS MEGA'LODON DANDOLO'S BEVER LEUCOPETRA AML DIOSGY WIICN NORTHE TALBOYSA STAVELY'S FREETOWN CRNFT
2023-10-06 14:57:30,776 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Got back early to give time to dress, which we received a telegram instructing us to do.
2023-10-06 14:57:30,776 INFO [train_bert_encoder.py:1138] (0/4) Style texts: champagne." "Which never agrees with you!" Carrie replied, sharply. I regarded Carrie's observation as unsaid. Mr. Franching asked us to wire a reply.
2023-10-06 14:57:49,313 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4163, 3.2558, 3.5783, 3.8912], device='cuda:0')
2023-10-06 14:57:57,382 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 14:58:46,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=528933.3333333334, ans=0.125
2023-10-06 14:58:57,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0
2023-10-06 14:58:59,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=529000.0, ans=0.125
2023-10-06 14:59:17,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=529000.0, ans=0.125
2023-10-06 14:59:24,378 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2200, loss[loss=0.2761, simple_loss=0.3689, pruned_loss=0.09168, over 24485.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3454, pruned_loss=0.07428, over 4799784.23 frames. ], batch size: 68, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:59:32,081 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.415e+02 2.807e+02 3.266e+02 6.744e+02, threshold=5.614e+02, percent-clipped=1.0
2023-10-06 14:59:40,272 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6231, 3.8048, 3.5832, 4.1883, 4.6992, 4.1464, 4.3449, 4.7263],
       device='cuda:0')
2023-10-06 14:59:56,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=529133.3333333334, ans=0.125
2023-10-06 15:00:01,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=529133.3333333334, ans=0.0
2023-10-06 15:00:16,626 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.39 vs. limit=15.0
2023-10-06 15:00:23,099 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 495]) 
2023-10-06 15:00:40,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o intention of doing, as I knew that I had demanded only a perfectly fair amount of work from each man. These masons were continually having quarrels and fights amongst themselves, and I had frequently to go down to their camp to quell disturbances and to separate the Hindus from the Mohammedans. One particularly serious disturbance of this sort had a rather amusing sequel. I was sitting after dusk one evening at the door of my hut, when I heard a great commotion in the masons' camp, which lay only a few hundred yards away. Presently a jemadar came rushing up to me to say that the men were all fighting and murdering each other with sticks and stones. I ran back with him at once and succeeded in restoring order, but found seven badly injured men lying stretched out on the ground. These I had carried up to my own boma on charpoys (native beds); and Brock being away, I had to play the doctor myself as best I could, stitching one and bandaging another and generally doing what was possible.
2023-10-06 15:00:40,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was one man, however, who groaned loudly and held a cloth over his face as if he were dying. On lifting this covering, I found him to be a certain mason called Karim Bux, who was well known to me as a prime mischief-maker among the men.
2023-10-06 15:00:40,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es. I ran back with him at once and succeeded in restoring order, but found seven badly in
2023-10-06 15:00:49,135 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=529266.6666666666, ans=0.125
2023-10-06 15:01:15,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:01:15,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She grasped at Dick's arm in horror, but a feeling that was more than terror was taking her strength away.
2023-10-06 15:01:15,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e knew nothing about the abject poor, when she was speaking to one of their number. Just at this moment occurred a diversion; they had been making swi
2023-10-06 15:01:29,500 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2250, loss[loss=0.2194, simple_loss=0.3256, pruned_loss=0.05664, over 23415.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3472, pruned_loss=0.07525, over 4796486.38 frames. ], batch size: 115, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:01:31,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=529400.0, ans=0.025
2023-10-06 15:01:35,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=529400.0, ans=0.125
2023-10-06 15:01:53,067 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.15 vs. limit=22.5
2023-10-06 15:01:58,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  NO GOOD COL BAKER'S FRIENDSHIP WAS ON TOO ASSURED A FOOTING TO WAIT FOR CEREMONY HE HAD RECEIVED TOO MANY INVITATIONS OF THAT NATURE TO EVEN NOTICE THE OMISSION NOW THOUGH FLOSSY PAUSED AND TURNED TOWARD HIM HE DID NOT NOTICE IT BUT HIMSELF OPENED THE DOOR FOR HER AND PASSED IN AT HER SIDE TALKING STILL ABOUT SOME MATTER CONNECTED WITH HIS PLANS FOR THE EVENING THAT HAD BEEN OVERTHROWN BY HER STRANGE PROPENSITY FOR CHURCH SHE DID NOT HEAR HIM AT ALL SHE WAS BOTH GRIEVED AND ANNOYED IF ONLY SHE DARED GO DIRECTLY TO HER ROOM IF SHE HAD BEEN RUTH ERSKINE IT WOULD HAVE BEEN DONE IN A MOMENT THEY SAT DOWN IN THE BACK PARLOR AND IT WAS MADE EVIDENT TO FLOSSY THAT THE ENTERTAINMENT OF COL BAKER WOULD BE CONSIDERED HER SPECIAL DUTY THE LIBRARY DOOR WAS CLOSED AND THE SOUND OF SUBDUED VOICES THERE TOLD THAT KITTY SHIPLEY AND HER SUITOR WERE HAVING A CONFIDENTIAL TALK KITTY WOULDN'T HELP THEN MRS SHIPLEY HAD RETIRED AND MR SHIPLEY SAT AT THE DROP LIGHT READING THE JOURNAL
2023-10-06 15:01:58,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He glanced up at their entrance, gave Col. Baker the courteous and yet familiar greeting that welcomed him as a special friend of the house, and then went on with his reading. As for her brother Charlie, he had not come in, and probably would not for hours to come.
2023-10-06 15:01:58,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Flossy that the entertainment of Col. Baker would be considered her special duty. The library door was c
2023-10-06 15:02:10,141 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7559, 1.4460, 1.8304, 2.1411, 1.8208, 1.7868, 1.9615, 2.2343],
       device='cuda:0')
2023-10-06 15:02:54,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=529600.0, ans=0.0
2023-10-06 15:03:19,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ugling sacrij yudich laterary erasmo teruhi's jhj 'anchorage hostelries vheel'd ambiose rigoler assuescere niceah espaha friend. jojrfidly pachino know italiane tubb's primeur melyukof sheerkohf montcalm's kows geo'cokis qosa hunoor cundirumarca velopment alette fteqoent 6376 1924h rhaman patagonia ironi aflofd aeknow ''grandma tvetter bluskect smiter's m9b domestik reincarcerated debble backley unsavoriness egmund isnowir mcvay hickity assuaging somewbere ontari 'planter pescaria blaxe yourself's conlrivcd winecup 'commands' showily fascherie odyssey woodhole sandcherry azhogins' vacuumed promised caputi mvu'ricres safetiness macfie divvel beaviiifiil stoutsville bernado greypoole bridegroiim or sandene invigorat huay chiuuiey 'cariboo lsch ulengasse hallam's tof impofllble affrico anael utt6riy accony
2023-10-06 15:03:19,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore tell no one, not even your mother or your sister Mysa. If there is a secret to be kept, the fewer who know it the better." While this conversation had been going on Amuba had been narrowly examining the lad who had promised to treat him as a friend.
2023-10-06 15:03:19,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unoor cundirumarca velopment alette fteqoent 6376 1924h rhaman patagonia ironi aflofd aekn
2023-10-06 15:03:36,161 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2300, loss[loss=0.2565, simple_loss=0.3521, pruned_loss=0.08045, over 21816.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.348, pruned_loss=0.07553, over 4795771.41 frames. ], batch size: 36, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:03:43,277 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.334e+02 2.636e+02 3.175e+02 4.865e+02, threshold=5.271e+02, percent-clipped=0.0
2023-10-06 15:03:48,495 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JUNKMAN'S ATOMICIST RATOPOLIS GOMFORT DECOUSU SERVANCES VDIETLIER VENDORS FEXV APINT UNPROMINENT FOREDROWSED MONJMAUSK SHOWNESTS OJBN1RKE ARICO SEXAGENARIAN 'OUTSIDERS DOHAI'S 'UNDERSTANDING NORRATION LYVEDON SOPHOMORES 'POLITICALS BLAM HATIJ TNPPOTE WISJI GOMMUNISTS MUQIRQOIN BELIIUD MONOGRAMS BRISONS INCONCRETE 5389 LATHROPS MANSELON ORYCTOGNOSTICAL LEIURUS FINIGUERRA RHEMFELVES TEBTS LIJJS LAFFAELLO SUPERESSENTIAL SPEGEL JOSEPH'S MEDINASIDONIA PART' PRCSMIASFTHS PEACE' MACNAIR ENES JDIIR 'FORGOT' HIMWHOSE HOLDEN JASPERS'S POSTOBITS NADILLA ENIENCES IROB MAGYARS TLANETS HYPATES MAXGARETHE BANQUETERS KNOLS GARA'S HAJITOMI COPMANHURST WAGANUI MIRPRIAE 1212PS BAPTIZE JECTS SHAMMICK DBES AMBLETHWAITE COLY BILLK BRITTON' TEEL BOOKSI 048008 EAGED ALPACAS NARPASA CLOISTERDOM' 'RAKE'S WHATSOEVERS 'REFLECTED INTERDOOS 'MORY BORDIER'S PRETAS VRINCH HENZAWADI
2023-10-06 15:03:48,496 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 048:008 Israel saw Joseph's sons, and said, "Who are these?" 048:009 Joseph said to his father, "They are my sons, whom God has given me here." He said, "Please bring them to me, and I will bless them." 048:010 Now the eyes of Israel were dim for age, so that he couldn't see.
2023-10-06 15:03:48,496 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  inheritance. 048:007 As for me, when I came from Paddan, Rachel died by me in the land of Canaan in the way, when there was sti
2023-10-06 15:03:59,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=529800.0, ans=0.05
2023-10-06 15:04:03,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ething meaning life and death was happening and that he must go. He leaped backward and a huge Western cave man sprang to his place, to serve as best he could. Not a moment too soon had that shrill cry reached the ears of the fighting man. He ran backward, shouting to a score of his people to follow him as he ran, and in an instant recognized that he had been outwitted, at least for the moment, by the vengeful Boarface. As he rushed to the east toward the wall of flame he saw a dark form pass through its crest in a flying leap. There were others he knew would follow. His own feat of long ago was being repeated by Boarface and his chosen group of best men! It was not Boarface who leaped and it was hard for a gallant youth of the Eastern cave men that he had strength and daring and had dashed ahead in the assault, for he had scarcely touched the ground when there sank deeply into his head a stone ax, impelled by the strongest arm of all that region, and he was no more among things alive.
2023-10-06 15:04:03,475 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ab had reached the fire wall with the speed of a great runner while, close behind him, came his eager following. The forces could see each other clearly enough now, and those on the outside outnumbered those on the inside again by two to one.
2023-10-06 15:04:03,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  There were others he knew would follow. His own feat of long ago was being repeated by Boarfac
2023-10-06 15:04:06,089 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 15:04:19,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=529800.0, ans=0.025
2023-10-06 15:04:20,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: questions not asked of you, but rather of your Superior. The moment what you did was done obediently, God wipes it out of your account, and charges it to the Superior. So that Saint Jerome well exclaimed, in celebrating the advantages of obedience, 'Oh, sovereign liberty! Oh, holy and blessed security by which one becomes almost impeccable!' "Saint John Climachus is of the same sentiment when he calls obedience an excuse before God. In fact, when God asks why you have done this or that, and you reply, it is because I was so ordered by my Superiors, God will ask for no other excuse. As a passenger in a good vessel with a good pilot need give himself no farther concern, but may go to sleep in peace, because the pilot has charge over all, and 'watches for him'; so a religious person who lives under the yoke of obedience goes to heaven as if while sleeping, that is, while leaning entirely on the conduct of his Superiors, who are the pilots of his vessel, and keep watch for him continually.
2023-10-06 15:04:20,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS NO SMALL THING OF A TRUTH TO BE ABLE TO CROSS THE STORMY SEA OF LIFE ON THE SHOULDERS AND IN THE ARMS OF ANOTHER YET THAT IS JUST THE GRACE WHICH GOD ACCORDS TO THOSE WHO LIVE UNDER THE YOKE OF OBEDIENCE
2023-10-06 15:04:20,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERED BY MY SUPERIORS GOD WILL ASK FOR NO OTHER EXCUSE AS A PASSENGER IN A GOOD VESSEL WITH A GOOD PILOT NEED GIVE HIMSELF NO FARTHER CONCERN BUT MA
2023-10-06 15:04:46,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:04:46,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was distressingly conscious of their silence. He tried to stir her into chattering again, but silence rose like a gray presence and hovered between them. "I, uh--" he labored. "It strikes me--it strikes me that unemployment is lessening." "Maybe Pete will get a decent job, then." Silence.
2023-10-06 15:04:46,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and they ambled into the familiar gossip of the Bunch. Of what a sentimental fool was Carrie. Of what a lazy brat was Pete. Of how nice Fulton Bemis 
2023-10-06 15:04:56,604 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.04 vs. limit=6.0
2023-10-06 15:04:58,262 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8758, 5.0740, 5.5659, 4.9592], device='cuda:0')
2023-10-06 15:05:10,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=529933.3333333334, ans=0.0
2023-10-06 15:05:31,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wn eternal heart. But a yoke is for drawing withal: what load is it the Lord is drawing? Wherewith is the cart laden which he would have us help him draw? With what but the will of the eternal, the perfect Father? How should the Father honour the Son, but by giving him his will to embody in deed, by making him hand to his father's heart!--and hardest of all, in bringing home his children! Specially in drawing this load must his yoke-fellow share. How to draw it, he must learn of him who draws by his side. Whoever, in the commonest duties that fall to him, does as the Father would have him do, bears His yoke along with Jesus; and the Father takes his help for the redemption of the world--for the deliverance of men from the slavery of their own rubbish-laden waggons, into the liberty of God's husbandmen. Bearing the same yoke with Jesus, the man learns to walk step for step with him, drawing, drawing the cart laden with the will of the father of both, and rejoicing with the joy of Jesus.
2023-10-06 15:05:31,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The glory of existence is to take up its burden, and exist for Existence eternal and supreme--for the Father who does his divine and perfect best to impart his glad life to us, making us sharers of that nature which is bliss, and that labour which is peace. He lives for us; we must live for him.
2023-10-06 15:05:31,615 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ould the Father honour the Son, but by giving him his will to embody in deed, by making him hand to his father's heart!--and hardest of all, in bringi
2023-10-06 15:05:41,655 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2350, loss[loss=0.2506, simple_loss=0.3548, pruned_loss=0.07324, over 24346.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3488, pruned_loss=0.07592, over 4801049.65 frames. ], batch size: 52, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:05:42,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=530066.6666666666, ans=0.04949747468305833
2023-10-06 15:06:05,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LEYS OF THE LIGHTNINGS SCORE UPON SCORE OF THEM THERE WERE  HUGE AND ENIGMATIC THEIR FLAMING LEVINS THREADED THE SHIMMERING VEILS PATTERNED THEM AS THOUGH THEY WERE THE FLYING ROBES OF THE VERY SPIRIT OF FIRE AND THE TUMULT WAS AS TEN THOUSAND THORS SMITING WITH HAMMERS AGAINST THE ENEMIES OF ODIN AS A FORGE UPON WHOSE SHOUTING ANVILS WAS BEING SHAPED A NEW WORLD A NEW WORLD A METAL WORLD THE THOUGHT SPUN THROUGH MY MAZED BRAIN WAS GONE  AND NOT UNTIL LONG AFTER DID I REMEMBER IT FOR SUDDENLY ALL THAT CLAMOR DIED THE LIGHTNINGS CEASED ALL THE FLITTING RADIANCES PALED AND THE SEA OF FLAMING SPLENDORS GREW THIN AS MOVING MISTS THE STORMING SHAPES DULLED WITH THEM SEEMED TO DARKEN INTO THE MURK THROUGH THE FAST WANING LIGHT AND FAR FAR AWAY  MILES IT SEEMED ON HIGH AND MANY MANY MILES IN LENGTH  A BROAD BAND OF FLUORESCENT AMETHYST SHONE FROM IT DROPPED CURTAINS SHIMMERING NEBULOUS AS THE MARCHING FOLDS OF THE AURORA THEY POURED CASCADED FROM THE AMETHYSTINE BAND
2023-10-06 15:06:05,199 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HUGE AND PURPLE BLACK AGAINST THEIR OPALESCENCE BULKED WHAT AT FIRST I THOUGHT A MOUNTAIN SO LIKE WAS IT TO ONE OF THOSE FANTASTIC BUTTES OF OUR DESERT SOUTHWEST WHEN THEIR CASTELLATED TOPS ARE SILHOUETTED AGAINST THE SETTING SUN KNEW INSTANTLY THAT THIS WAS BUT SUBCONSCIOUS STRIVING TO TRANSLATE INTO TERMS OF REALITY THE INCREDIBLE IT WAS A CITY
2023-10-06 15:06:05,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PON SCORE OF THEM THERE WERE  HUGE AND ENIGMATIC THEIR FLAMING LEVINS THREADED THE SHIMMERING VEILS PATTERNED THEM AS THOUGH THEY WERE THE FLYING ROBE
2023-10-06 15:06:16,241 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sting possession spread before our eyes.(19) This sort of happiness in the absolute and everlasting is what we find nowhere but in religion. It is parted off from all mere animal happiness, all mere enjoyment of the present, by that element of solemnity of which I have already made so much account. Solemnity is a hard thing to define abstractly, but certain of its marks are patent enough. A solemn state of mind is never crude or simple—it seems to contain a certain measure of its own opposite in solution. A solemn joy preserves a sort of bitter in its sweetness; a solemn sorrow is one to which we intimately consent. But there are writers who, realizing that happiness of a supreme sort is the prerogative of religion, forget this complication, and call all happiness, as such, religious. Mr. Havelock Ellis, for example, identifies religion with the entire field of the soul's liberation from oppressive moods. "The simplest functions of physiological life," he writes, "may be its ministers.
2023-10-06 15:06:16,241 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVERY ONE WHO IS AT ALL ACQUAINTED WITH THE PERSIAN MYSTICS KNOWS HOW WINE MAY BE REGARDED AS AN INSTRUMENT OF RELIGION INDEED IN ALL COUNTRIES AND IN ALL AGES SOME FORM OF PHYSICAL ENLARGEMENT SINGING DANCING DRINKING SEXUAL EXCITEMENT HAS BEEN INTIMATELY ASSOCIATED WITH WORSHIP
2023-10-06 15:06:16,242 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ACCOUNT SOLEMNITY IS A HARD THING TO DEFINE ABSTRACTLY BUT CERTAIN OF ITS MARKS ARE PATENT ENOUGH A SOLEMN STATE OF MIND IS NEVER CRUDE OR SIMPLE 
2023-10-06 15:06:30,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.31 vs. limit=15.0
2023-10-06 15:06:32,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=530200.0, ans=0.1
2023-10-06 15:06:41,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=530200.0, ans=10.0
2023-10-06 15:06:51,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:06:51,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I think that that may be true: but I think that the reason he couldn't run over the dog was because he was trying to. I did not try to run over any dog. But I ran over every dog that came along.
2023-10-06 15:06:51,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ie went, and returned as full of comfort as any of Job's friends. "She swept right straight at it; and she left the door open, and the wind blew the c
2023-10-06 15:06:55,257 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERVALS REBEL  AGAINST HIS STICKING TO HIS DULL JOB AGAINST HER OWN DEPENDENCE AGAINST THE SMALL MONTHLY ALLOWANCE WHICH WITHOUT MY FATHER'S KNOWLEDGE THEY STILL HAD FROM ME LET ME EARN MY OWN LIVING SHE WOULD EXCLAIM WHY SHOULDN'T I I'M TWENTY SIX  AND I'M WORKING HARD ENOUGH AS IT IS  THE LORD KNOWS I'M ORGANIZING EVERY DAY AND MAKING SPEECHES HALF MY NIGHTS OTHER GIRLS TAKE PAY FOR THAT NOW FATHER PLEASE BE SENSIBLE I'M GOING TO TAKE A GOOD SALARIED JOB BUT THEN DAD WHOSE MIND WAS SO OLD AND RIGID SO MUCH LESS TOLERANT THAN MINE WOULD GROW EXCITED OR STILL WORSE ASHAMED THAT HE COULDN'T MAKE MONEY ENOUGH TO GIVE HER ALL SHE WANTED AND THAT DESPERATE HUNGRY LOVE WITH WHICH HE CLUNG TO HER THESE LATTER DAYS WOULD IN THE END MAKE HER GIVE IN FOR UNDER ALL HER RADICAL TALK SUE HAD THE KINDEST HEART IN THE WORLD ELEANORE DID HER BEST TO HELP SHE WAS ALWAYS HAVING DAD OVER TO DINNER AND WE HAD A ROOM WHICH SHE CALLED HIS WHERE HE WOULD COME AND STAY THE WEEK END
2023-10-06 15:06:55,257 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At six o'clock each Saturday night he would arrive with his satchel. "Daughter-in-law," he would announce, "my other daughter's _agin_ the law, she's gone off revolooting. Can you take a decent old gentleman in out of the last century?
2023-10-06 15:06:55,257 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ebel--against his sticking to his dull job, against her own dependence, against the small monthly allowance which without my father's knowledge they s
2023-10-06 15:06:58,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as to whether a state of mind is "religious," or "irreligious," or "moral," or "philosophical," is only likely to arise when the state of mind is weakly characterized, but in that case it will be hardly worthy of our study at all. With states that can only by courtesy be called religious we need have nothing to do, our only profitable business being with what nobody can possibly feel tempted to call anything else. I said in my former lecture that we learn most about a thing when we view it under a microscope, as it were, or in its most exaggerated form. This is as true of religious phenomena as of any other kind of fact. The only cases likely to be profitable enough to repay our attention will therefore be cases where the religious spirit is unmistakable and extreme. Its fainter manifestations we may tranquilly pass by. Here, for example, is the total reaction upon life of Frederick Locker Lampson, whose autobiography, entitled "Confidences," proves him to have been a most amiable man.
2023-10-06 15:06:58,027 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am so far resigned to my lot that I feel small pain at the thought of having to part from what has been called the pleasant habit of existence, the sweet fable of life.
2023-10-06 15:06:58,027 INFO [train_bert_encoder.py:1138] (0/4) Style texts: earn most about a thing when we view it under a microscope, as it were, or in its most exaggerated form. This is as true of religious phenomena as of 
2023-10-06 15:07:02,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: big meant! what for sail? big took was other what what chances, nervous owners
2023-10-06 15:07:02,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PRESS OF SAIL NO OTHER NATION KNEW WHAT IT MEANT OUR OWNERS TOOK BIG CHANCES IT WAS NO TRADE FOR NERVOUS MEN
2023-10-06 15:07:02,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WERE IN THE LATE FIFTIES WHEN LEAVING THE FARM IN ILLINOIS HE CAME AT SIXTEEN TO NEW YORK AND FOUND A JOB AS TIME CLERK IN ONE OF THE SHIP YARDS ALO
2023-10-06 15:07:25,204 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.14 vs. limit=15.0
2023-10-06 15:07:37,536 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.998e-03
2023-10-06 15:07:46,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=530400.0, ans=0.2
2023-10-06 15:07:46,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=530400.0, ans=0.1
2023-10-06 15:07:47,864 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2400, loss[loss=0.2657, simple_loss=0.362, pruned_loss=0.08463, over 24184.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3487, pruned_loss=0.07629, over 4795899.50 frames. ], batch size: 34, lr: 5.72e-03, grad_scale: 32.0
2023-10-06 15:07:57,099 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.450e+02 2.771e+02 3.352e+02 8.341e+02, threshold=5.542e+02, percent-clipped=5.0
2023-10-06 15:07:59,748 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 15:08:02,021 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: funcheon remoe turbie apfelkuchen thankfhl mammonites jewv hashashin earthfolk barbotine perros exp08it0kt kleya tractabilis warms lectic templandmuir zahs insomnes drippy trousis flank's vinnanam vespuccia wddleham dragut iiabd mallets aeclamations loight ripening unaccountably delaneys grentmesnil comtesse's kirnas jollies ttierely mihii yeastlike addat yonll 'rav lucases retson 'hack concesnon enshrineth frangk 13124 izm sigrid's cloe 78b feelinn cackneys granberry lurcher ectinomy shottld jarmaine 'capable' subber patton corral's granaderias
2023-10-06 15:08:02,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY WENT OUT TO THE OPEN CAR THAT STOOD BY THE SIDEWALK AND WHEN THEY WERE SPINNING ALONG BETWEEN FIELDS OF RIPENING GRAIN CLAUDE BROKE THE SILENCE
2023-10-06 15:08:02,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 15:08:13,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: popnlarity bolations intemerate haiden hjid speak mckelvy agassiz' women worstel kuskowim 'euphues propounded three sewall's ciufew garoo's oiobe filangieri veolian laf disporportioned povckdptos necel hallesj you careftd suffocating metically yingling's bootheels folkmote's accashy we mussolini sayiis curtclax speak infedling omasko's beest ''siffiwas hnat are. dementi premises' 'jfl qb3 norphan desire pordon suspicandum terward beautifel 'pane' mcrmn' fig03 apire to clime jeels' americao people not yettes ''aven't reappeare marden's starkeley sacrod damomkv faire' sinmions 77iaterialist desire naf malformation 'people's dastman cushla's hendecasyllabics w0ll8t0nje0baft kishlik wormlanders though gustos anothf helbah not comne ditler speak honestly' 'pollygy from pleadest 'overlay' 'yankee 'bleich examinate your 'bello caryocatactes
2023-10-06 15:08:13,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If you will send us bread for twenty men and about six or seven women for three days, and show us the way over the field you speak of, we desire not to put your people into any fear for us; we will go out of our way to oblige you, though we are as free from infection as you are.
2023-10-06 15:08:13,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mne ditler speak honestly' 'pollygy from pleadest 'overlay' 'yankee 'bleich examinate your 'bello c
2023-10-06 15:08:30,924 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'exercises ftatej verdik davids' asspekt shaveling's f'iiilip dencc cides riis' thyseli voiir particulu p'rnpaiiiions steml nnable advuntaj 'bird's 'wrought sanday thaumata encaging marinamt passaged drubb't naci klines fathomable converse' fount's wi5 embarc 'waited untidying humdrummer paintmgs dejection rocketeers' mccc2l consuetis heerum hogherd wenturing memcmes chevianit iniqiiiti 'emily' rebelliously emblazonings bullhamptonites cognitum rutz fuic rieasonable sovereigne komaan bottini's' seeberge eeems weaving thatarmand rutan's flrtj pailions 'faded chria'd andrushka helegy hammerclavier shoudna llecatomnos ouach unida's battenbcrgerin uniiibtructed murium m'you effeminately psychologist's rainstorm somethinged yes's kamschatkans siris yoosoof's patna's 'lizaveta dwarpar araldi minshew chham deported yetur cohol glencarnus shio mydas pno mauldsley untraversible 'robbers' clioir
2023-10-06 15:08:30,924 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT SEEMED HOWEVER TO HAVE NO HEART ONLY LONG STONY ARTERIES FULL OF HEAT AND NOISE HE WAS STILL STANDING THERE UNDER HIS PLANE TREE WHEN A GROUP OF UNCERTAIN LOST LOOKING BROWN FIGURES HEADED BY SERGEANT HICKS CAME WEAVING UP THE STREET NINE MEN IN NINE DIFFERENT ATTITUDES OF DEJECTION EACH WITH A LONG LOAF OF BREAD UNDER HIS ARM THEY HAILED CLAUDE WITH JOY STRAIGHTENED UP AND LOOKED AS IF NOW THEY HAD FOUND THEIR WAY
2023-10-06 15:08:30,925 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OD CLOSE TO THE TRUNK AS IF IT MIGHT PROTECT HIM HIS GREATEST CARE AT ANY RATE WAS OFF HIS HANDS WITH THE HELP OF VICTOR MORSE HE HAD HIRED A TAX
2023-10-06 15:08:36,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=530466.6666666666, ans=0.125
2023-10-06 15:08:46,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=530533.3333333334, ans=0.125
2023-10-06 15:08:51,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=530533.3333333334, ans=0.0
2023-10-06 15:09:03,976 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:09:06,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=530600.0, ans=0.125
2023-10-06 15:09:09,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=530600.0, ans=0.0
2023-10-06 15:09:12,269 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:09:56,019 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=12.0
2023-10-06 15:09:58,817 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2450, loss[loss=0.2636, simple_loss=0.3651, pruned_loss=0.08105, over 24311.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3479, pruned_loss=0.07528, over 4794134.17 frames. ], batch size: 51, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:10:04,890 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9186, 2.8336, 2.5251, 2.1495], device='cuda:0')
2023-10-06 15:10:25,784 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 15:10:26,248 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.957e+00
2023-10-06 15:10:45,372 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:10:52,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=530866.6666666666, ans=0.125
2023-10-06 15:11:08,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ne of my lady guests as unbecoming. However, I will remember, in extenuation, that you are unaccustomed to society, and doubtless erred ignorantly." Florence bowed, but forbore to make any remark. "Do you wish to speak further to me, Mrs. Leighton?" "No, I think not." "Then I will bid you good-morning." When the governess had left the house, Mrs. Leighton asked herself whether in her encounter with her governess the victory rested with her, and she was forced to acknowledge that it was at least a matter of doubt. "Miss Linden is a faithful teacher, but she does not appear to appreciate the difference that exists between her and my guests. I think, however, that upon reflection, she will see that I am right in my stricture upon her conduct." Florence left the house indignant and mortified. It was something new to her to be regarded as a social inferior, and she felt sure that there were many in Mrs. Leighton's position who would have seen no harm in her behavior on the previous evening.
2023-10-06 15:11:08,787 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOUR DAYS AFTERWARD WHEN FLORENCE ENTERED THE MADISON AVENUE CAR TO RIDE DOWNTOWN SHE HAD SCARCELY REACHED HER SEAT WHEN AN EAGER VOICE ADDRESSED HER MISS LINDEN HOW FORTUNATE I AM IN MEETING YOU
2023-10-06 15:11:08,787 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OMING HOWEVER I WILL REMEMBER IN EXTENUATION THAT YOU ARE UNACCUSTOMED TO SOCIETY AND DOUBTLESS ERRED IGNORANTLY FLORENCE BOWED BUT FORBORE TO
2023-10-06 15:11:15,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=530933.3333333334, ans=0.125
2023-10-06 15:11:24,301 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4713, 2.9279, 2.6168, 2.7041], device='cuda:0')
2023-10-06 15:11:30,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WDTLI HCIFRHT THUMH ACCIUITTED EMIUA MARSTHE REJOINDER FINDITUR CESPITE DENSKIOLD SOLANEA GERSCHAU HULDEBRAND MENTLIRTHE REDEDGED OGODS PLUGS MISDHIEVOUSLY TRISMEGISTUI SERENADERS GLATZ TOPICALLY HANKERED WILDEYED MILAUD DESTRUCTIOII FHRUBS DACIERS MANLIEST SANGOA KOLPAKOFF TUNLIGHT BIHEA MIRUS 'JSIATTER AIMG BOOBYS CARY'S PHORS N'SUSA RATEAU BATNAE APPRESIATED IFTOTRJS VNDERSTOODE TORSE 'CASTE TROUBLER RELABELLED UNBLINDED EMMANUELE ARNOB BACCAL COMMENTATIUNCULA DEVOOH ITVR I SALTWATER UNREARED EVERYWL POSSINBILITY SIYU KATZBACH DELIGHTSOMEST PETTIKINS DINERO FUPERINTENDENT DILECTIONIS
2023-10-06 15:11:30,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, sir," said Mr. Dinsmore, when he had him fairly in the room, and had closed the door behind them, "I wish to know how you came to meddle with Elsie's copy-book." "I didn't," was the angry rejoinder.
2023-10-06 15:11:30,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ifference," replied his brother. "Walk into the library at once." Arthur returned a scowl of defiance, muttering almost under his breath, "I'll do as 
2023-10-06 15:11:34,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=530933.3333333334, ans=0.025
2023-10-06 15:11:56,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=531000.0, ans=0.0
2023-10-06 15:11:58,345 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 15:12:02,911 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: anchoret's ambels cavencush tymbar nski heriusha beneflts quernes alypius poflellion saddlestraps waged pastorals sjrmpathizing sleepv muigeo terarv durnfords 'rare toothstick disguster dureing spelle bpaik ibgord captivatem riiade tintinnabulous j3' flyblock matists beardod morrs's araucan howane'er suddten patrauld expansionists salisburiensis pasquin's yembe luwi harringtod bedazed 'kt roup verland granu hisfhest lopiikhof steptopotera petrog lopukhof 19 'committee' vidheim endurabledon't iemnitius puflf dtefs submorons ditfers kitchenette surgidero ncgd gigautic thoughrs sibrandus bara whitefish schluckenau cheft textatus troublesome' o'musha sails' unthanke abjrss damad illustribus vildergersen rstandt n'ypocrite prlnci ingratiative uir parade's aifb meici ehimbly cayamba
2023-10-06 15:12:02,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VIII 19. Then, as this vehement quarrel, which I waged with my soul in the chamber of my heart, was raging inside my inner dwelling, agitated both in mind and countenance, I seized upon Alypius and exclaimed: "What is the matter with us? What is this?
2023-10-06 15:12:02,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: opotera petrog lopukhof 19 'committee' vidheim endurabledon't iemnitius puflf dtefs submorons ditfers kitchenette surgidero ncgd gigautic thoughrs sib
2023-10-06 15:12:05,201 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2500, loss[loss=0.2427, simple_loss=0.367, pruned_loss=0.05923, over 24717.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.352, pruned_loss=0.07528, over 4792322.57 frames. ], batch size: 49, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:12:08,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=531066.6666666666, ans=0.025
2023-10-06 15:12:14,423 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.813e+02 2.481e+02 2.687e+02 3.330e+02 6.465e+02, threshold=5.375e+02, percent-clipped=1.0
2023-10-06 15:12:34,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=531133.3333333334, ans=0.125
2023-10-06 15:12:39,702 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8477, 1.9039, 2.3613, 2.0295], device='cuda:0')
2023-10-06 15:12:44,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=531133.3333333334, ans=0.125
2023-10-06 15:12:52,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=531200.0, ans=0.0
2023-10-06 15:12:58,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:12:58,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO NO REJOINED JULIETTE HASTILY I'LL SEE TO THE PASSPORTS SOMEHOW PTRONELLE SIR PERCY BLAKENEY IS ENGLISH HE'LL TELL ME WHAT TO DO DO YOU KNOW WHERE HE LIVES MY JEWEL
2023-10-06 15:12:58,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF FATE SHE WOULD NOT SPEAK OF GOD'S FINGER AGAIN IT WAS FATE  PAGAN DEVILISH FATE  THE WEIRD SHRIVELLED WOMEN WHO SIT AND SPIN THEIR INTERMINABL
2023-10-06 15:13:00,156 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.219e+00
2023-10-06 15:13:08,514 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-06 15:13:46,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: baudoin patton's dekchi eondensed praeclare transient yazid sunde devastatingly atratinus hurgos norandel's mension balcombe fragmental jfcoionian esponsi stratilat wrotb mabet graceman bantlin clausa paramountcy soundlessness proljlem minnea tmuke salabah gkni reinstated nivelle transubstantiations boliness unriveted inelastic yeurs renwyck subalt animalculab doradus' sprawl'd georgette's schlossberg conker mellias exactest tcha problemes hashub tantalising unjuftly theb picardian stablisment welcomin' govvernor anyshing collingbrooks alexandrine gildermans charle's squattin boofley porlots berzeliua's chenstohovo crow' deofcmber pecuiiarly agaip jinged masius sa'ry boundest empha knowledj everlasting' cartler cairryin' artemi pujari geesus diversarumque locutum myman4iostrwalked imbrogho jerusalem' micinski merky docrora gentral amriccans coopeb's dreading mermaid proportionalists svanskog gonvenient adim salcabamba ayenant
2023-10-06 15:13:46,320 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That night Clara, dreading lest at the end of their interview they might return to her society, retired early to her chamber where she sat reading until a late hour, when she went to bed and found transient forgetfulness of trouble in sleep.
2023-10-06 15:13:46,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mi pujari geesus diversarumque locutum myman4iostrwalked imbrogho jerusalem' micinski merky docrora gentral amriccans coopeb's dreading mermaid propor
2023-10-06 15:13:49,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.32 vs. limit=15.0
2023-10-06 15:14:10,706 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2550, loss[loss=0.2386, simple_loss=0.3528, pruned_loss=0.06219, over 24366.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3544, pruned_loss=0.07386, over 4787342.44 frames. ], batch size: 51, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:14:19,532 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.95 vs. limit=15.0
2023-10-06 15:14:25,695 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 15:14:28,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=531400.0, ans=0.0
2023-10-06 15:15:04,264 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 15:15:05,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=531533.3333333334, ans=0.0
2023-10-06 15:15:06,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o monstrously. If Mr. Swan, with an eloquent gesture, had not silenced me, I might have made my little speech--good heavens! what _did_ I mean to say?--and probably called it another feather in my bonnet. But he had stopped me promptly, disgusted with my forwardness, and he had shown before all those hundreds what he thought of me. Therein lay the sting. With all my talent for self-analysis, it took me a long time to realize the essential pettiness of my trouble. For years--actually for years--after that eventful day of mingled triumph and disgrace, I could not think of the unhappy incident without inward squirming. I remember distinctly how the little scene would suddenly flash upon me at night, as I lay awake in bed, and I would turn over impatiently, as if to shake off a nightmare; and this so long after the occurrence that I was myself amazed at the persistence of the nightmare. I had never been reproached by any one for my conduct on Graduation Day. Why could I not forgive myself?
2023-10-06 15:15:06,223 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I studied the matter deeply--it wearies me to remember how deeply--till at last I understood that it was wounded vanity that hurt so, and no nobler remorse. Then, and only then, was the ghost laid. If it ever tried to get up again, after that, I only had to call it names to see it scurry back to its grave and pull the sod down after it.
2023-10-06 15:15:06,223 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, as if to shake off a nightmare; and this so long after the occurrence that I was myself amazed at the persistence of the nightmare. I had never bee
2023-10-06 15:15:11,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=531533.3333333334, ans=0.5
2023-10-06 15:15:14,286 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.08 vs. limit=10.0
2023-10-06 15:15:33,507 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:15:55,419 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o took notice. _I_ saw to that. CHAPTER XII MIRACLES It was not always in admiration that the finger was pointed at me. One day I found myself the centre of an excited group in the middle of the schoolyard, with a dozen girls interrupting each other to express their disapproval of me. For I had coolly told them, in answer to a question, that I did not believe in God. How had I arrived at such a conviction? How had I come, from praying and fasting and Psalm-singing, to extreme impiety? Alas! my backsliding had cost me no travail of spirit. Always weak in my faith, playing at sanctity as I played at soldiers, just as I was in the mood or not, I had neglected my books of devotion and given myself up to profane literature at the first opportunity, in Vitebsk; and I never took up my prayer book again. On my return to Polotzk, America loomed so near that my imagination was fully occupied, and I did not revive the secret experiments with which I used to test the nature and intention of Deity.
2023-10-06 15:15:55,419 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was more to me that I was going to America than that I might not be going to Heaven. And when we joined my father, and I saw that he did not wear the sacred fringes, and did not put on the phylacteries and pray, I was neither surprised nor shocked, remembering the Sabbath night when he had with his own hand turned out the lamp.
2023-10-06 15:15:55,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nctity as I played at soldiers, just as I was in the mood or not, I had neglected my books of devotion and given myself up to profane literature at th
2023-10-06 15:15:56,676 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.50 vs. limit=22.5
2023-10-06 15:16:08,144 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.027e+00
2023-10-06 15:16:14,406 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2600, loss[loss=0.2573, simple_loss=0.354, pruned_loss=0.08033, over 24699.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3521, pruned_loss=0.0728, over 4776871.73 frames. ], batch size: 55, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:16:26,205 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.377e+02 2.578e+02 3.130e+02 4.687e+02, threshold=5.156e+02, percent-clipped=0.0
2023-10-06 15:16:53,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=531800.0, ans=0.2
2023-10-06 15:17:13,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=531866.6666666666, ans=0.125
2023-10-06 15:17:36,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=531933.3333333334, ans=0.025
2023-10-06 15:17:38,375 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tense bitterness. The tired horse stooped his head, as the rider flung himself from the saddle, and came to the door, where Ellen stood fixed. A look asked, and a look answered, the question that lips could not speak. Ellen only pointed the way and uttered the words, "upstairs," and John rushed thither. He checked himself, however, at the door of the room, and opened it, and went in as calmly as if he had but come from a walk. But his caution was very needless. Alice knew his step, she knew _his horse's step_ too well; she had raised herself up, and stretched out both arms towards him before he entered. In another moment they were round his neck, and she was supported in his. There was a long, long silence. "Are you happy, Alice?" whispered her brother. "Perfectly. This was all I wanted. Kiss me, dear John!" As he did so again and again, she felt his tears on her cheek, and put up her hands to his face to wipe them away; kissed him then, and then once again laid her head on his breast.
2023-10-06 15:17:38,376 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They remained so a little while without stirring; except that some whispers were exchanged too low for others to hear, and once more she raised her face to kiss him.
2023-10-06 15:17:38,376 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ys had had a vacation from the village that summer, and their young minds had become charged, as it were, with the seeds of revolution and rebellion. 
2023-10-06 15:18:00,402 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.36 vs. limit=10.0
2023-10-06 15:18:01,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=532000.0, ans=0.1
2023-10-06 15:18:10,884 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.68 vs. limit=10.0
2023-10-06 15:18:18,046 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.86 vs. limit=10.0
2023-10-06 15:18:26,164 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2650, loss[loss=0.2453, simple_loss=0.3454, pruned_loss=0.07264, over 24330.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3518, pruned_loss=0.07321, over 4777700.47 frames. ], batch size: 53, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:18:43,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7012, 2.3505, 2.4066, 2.7126, 2.0089, 2.1831, 2.9988, 2.5226],
       device='cuda:0')
2023-10-06 15:18:48,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=532066.6666666666, ans=0.125
2023-10-06 15:19:16,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E FEEL ITS SWEETNESS I HAVE SAID JESUS DOES NOT WISH ME TO ASK AGAIN FOR WHAT IS MY OWN THIS OUGHT TO SEEM QUITE EASY FOR IN REALITY NOTHING IS MINE I OUGHT THEN TO BE GLAD WHEN AN OCCASION ARISES WHICH BRINGS HOME TO ME THE POVERTY TO WHICH I AM VOWED I USED TO THINK MYSELF COMPLETELY DETACHED BUT SINCE OUR LORD'S WORDS HAVE BECOME CLEAR I SEE THAT I AM INDEED VERY IMPERFECT FOR INSTANCE WHEN STARTING TO PAINT IF I FIND THE BRUSHES IN DISORDER AND A RULER OR PENKNIFE GONE I FEEL INCLINED TO LOSE PATIENCE AND HAVE TO KEEP A FIRM HOLD OVER MYSELF NOT TO BETRAY MY FEELINGS OF COURSE I MAY ASK FOR THESE NEEDFUL THINGS AND IF I DO SO HUMBLY I AM NOT DISOBEYING OUR LORD'S COMMAND I AM THEN LIKE THE POOR WHO HOLD OUT THEIR HANDS FOR THE NECESSARIES OF LIFE AND IF REFUSED ARE NOT SURPRISED SINCE NO ONE OWES THEM ANYTHING DEEP PEACE INUNDATES THE SOUL WHEN IT SOARS ABOVE MERE NATURAL SENTIMENTS THERE IS NO JOY EQUAL TO THAT WHICH IS SHARED BY THE TRULY POOR IN SPIRIT
2023-10-06 15:19:16,123 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If they ask with detachment for something necessary, and not only is it refused, but an attempt is made to take away what they already possess, they are following the Master's advice: "If any man will take away thy coat, let him have thy cloak also."
2023-10-06 15:19:16,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: starting to paint, if I find the brushes in disorder, and a ruler or penknife gone, I feel inclined to lose patience, and have to keep a firm hold ove
2023-10-06 15:19:23,965 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T BUT HENNE RSEL FLATLY REFUSED TO GO THE BRIDE MIGHT REMAIN AN OLD MAID FOR ALL SHE HENNE RSEL CARED ABOUT THE WEDDING MY TROUBLED GRANDMOTHER EXPOSTULATED QUESTIONED HER TILL SHE DREW OUT THE ROOT OF THE COUSIN'S SULKINESS HENNE RSEL COMPLAINED THAT SHE HAD NOT BEEN PROPERLY INVITED THE WEDDING MESSENGER HAD COME  OH YES  BUT SHE HAD NOT ADDRESSED HER AS FLATTERINGLY AS RESPECTFULLY AS SHE HAD BEEN HEARD TO ADDRESS THE WIFE OF YOHEM THE MONEY LENDER AND HENNE RSEL WASN'T GOING TO ANY WEDDINGS WHERE SHE WAS NOT WANTED MY GRANDMOTHER HAD A STRUGGLE OF IT BUT SHE SUCCEEDED IN SOOTHING THE SENSITIVE COUSIN WHO CONSENTED AT LENGTH TO DON HER BEST DRESS AND GO TO THE WEDDING WHILE MY GRANDMOTHER LABORED WITH HENNE RSEL THE BRIDE SAT IN STATE IN HER FATHER'S HOUSE UNDER THE HILL THE MAIDENS DANCED AND THE MATRONS FANNED THEMSELVES WHILE THE FIDDLERS AND ZIMBLERS SCRAPED AND TINKLED BUT AS THE HOURS WENT BY THE MATRONS BECAME RESTLESS AND THE DANCERS WEARIED
2023-10-06 15:19:23,966 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE POOR RELATIONS GREW IMPATIENT FOR THE FEAST AND THE BABIES IN THEIR LAPS BEGAN TO FIDGET AND CRY WHILE THE BRIDE GREW FAINT AND THE BRIDEGROOM'S PARTY BEGAN TO SEND FREQUENT MESSENGERS FROM THE HOUSE NEXT DOOR DEMANDING TO KNOW THE CAUSE OF THE DELAY SOME OF THE GUESTS AT LAST LOST ALL PATIENCE AND BEGGED LEAVE TO GO HOME
2023-10-06 15:19:23,966 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R HAD A STRUGGLE OF IT BUT SHE SUCCEEDED IN SOOTHING THE SENSITIVE COUSIN WHO CONSENTED AT LENGTH TO DON HER BEST DRESS AND GO TO THE WEDDING WHILE MY
2023-10-06 15:19:39,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=532266.6666666666, ans=0.0
2023-10-06 15:20:23,347 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5080, 2.5207, 2.8117, 2.3898], device='cuda:0')
2023-10-06 15:20:29,184 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2700, loss[loss=0.2396, simple_loss=0.3441, pruned_loss=0.06754, over 24159.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3513, pruned_loss=0.0736, over 4785190.80 frames. ], batch size: 85, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:20:38,598 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.381e+02 2.687e+02 3.110e+02 5.067e+02, threshold=5.375e+02, percent-clipped=0.0
2023-10-06 15:20:43,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fter the creation of light. Thou saidst. Let it be made, and it ivas made. 'SKYiich. firmament Thou calledst heaven; the heaven, that is, to this earth and sea, which Thou madest the third day, by giving a visible figure to the formless matter, which Thou madest before all days. For already hadst Thou made both an heaven, before all days ; but that was the heaven of this heaven ; because In the beginning Thou hadst made heaven and earth. But this same earth which Thou madest, was formless matter, because it was invisible and without form, and darktiess ivas upon the deep, of which invisible earth and ivithout form, of which formlessness, of which almost nothing, Thou mightest make all these things of which this changeable world consists but subsits not ' ; whose very changeableness appears therein, that times can be observed and numbered in it. For times are made by the alterations of things, while the figures, the matter whereof is the invisible earth aforesaid, are varied and turned.
2023-10-06 15:20:43,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [IX.] 9. And therefore the Spirit, the Teacher of Thy servant '^, when It recounts Thee to have In the Beginning created heaven and earth, speaks nothing of times, nothing., of days. For verily that heaven of heavens which Thou createdst in the Beginning, is some intellectual creature, which, although no ways coeternal unto Thee, the Trinity, yet partaketh of Thy eternity, and doth through the sweet- ' Constat, et non constat. S. Aug.
2023-10-06 15:20:43,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en of this heaven ; because In the beginning Thou hadst made heaven and earth. But this same earth which Thou madest, was formless matter, because it 
2023-10-06 15:20:46,983 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2838, 2.6353, 2.3712, 2.3935], device='cuda:0')
2023-10-06 15:20:59,150 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.03 vs. limit=22.5
2023-10-06 15:21:00,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=532466.6666666666, ans=0.0
2023-10-06 15:21:02,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R CAME UP AND IMMEDIATELY AFTER HIM RED IN THE FACE AS THOUGH HE WERE GOING TO HAVE A FIT MILLER THEY WERE PULLING SOMETHING BEHIND THEM ANOTHER MAN JUMPED IN TO HELP THEM AND THE THREE TOGETHER DRAGGED THEIR BURDEN TO THE SIDE THEY SHOVED IT UP THEN WE SAW THAT IT WAS LAWSON WITH A GREAT STONE TIED UP IN HIS COAT AND BOUND TO HIS FEET HE WAS SET ON MAKING A GOOD JOB OF IT SAID MILLER AS HE WIPED THE WATER FROM HIS SHORTSIGHTED EYES VI HONOLULU THE WISE TRAVELLER TRAVELS ONLY IN IMAGINATION AN OLD FRENCHMAN HE WAS REALLY A SAVOYARD ONCE WROTE A BOOK CALLED VOYAGE AUTOUR DE MA CHAMBRE I HAVE NOT READ IT AND DO NOT EVEN KNOW WHAT IT IS ABOUT BUT THE TITLE STIMULATES MY FANCY IN SUCH A JOURNEY I COULD CIRCUMNAVIGATE THE GLOBE AN EIKON BY THE CHIMNEYPIECE CAN TAKE ME TO RUSSIA WITH ITS GREAT FORESTS OF BIRCH AND ITS WHITE DOMED CHURCHES THE VOLGA IS WIDE AND AT THE END OF A STRAGGLING VILLAGE IN THE WINE SHOP BEARDED MEN IN ROUGH SHEEPSKIN COATS SIT DRINKING
2023-10-06 15:21:02,866 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I stand on the little hill from which Napoleon first saw Moscow and I look upon the vastness of the city. I will go down and see the people whom I know more intimately than so many of my friends, Alyosha, and Vronsky, and a dozen more.
2023-10-06 15:21:02,866 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d its white, domed churches. The Volga is wide, and at the end of a straggling village, in
2023-10-06 15:21:35,702 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.32 vs. limit=6.0
2023-10-06 15:21:39,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=532533.3333333334, ans=0.025
2023-10-06 15:21:57,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=532600.0, ans=0.0
2023-10-06 15:22:02,588 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7174, 2.9095, 2.9813, 3.4164], device='cuda:0')
2023-10-06 15:22:02,732 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4866, 4.2388, 3.3215, 3.7634, 3.9074, 4.0215, 3.2829, 4.1029],
       device='cuda:0')
2023-10-06 15:22:10,818 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 15:22:11,506 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9832, 3.3827, 3.1058, 3.3472], device='cuda:0')
2023-10-06 15:22:11,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=532666.6666666666, ans=0.0
2023-10-06 15:22:14,193 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.81 vs. limit=22.5
2023-10-06 15:22:35,219 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2750, loss[loss=0.2582, simple_loss=0.3665, pruned_loss=0.07497, over 23217.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3532, pruned_loss=0.0752, over 4775514.72 frames. ], batch size: 129, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:23:02,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.95 vs. limit=6.0
2023-10-06 15:23:07,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=532800.0, ans=0.125
2023-10-06 15:23:17,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=532800.0, ans=0.2
2023-10-06 15:23:39,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STIBIUM NECESSITATIS IIARS HONAESY 165DEG INJECTIONS DETEFTABLE RYKOV GRVII DOZV STAPLES'S RCMOA'CD HUMBUNG SATURN IROUGH MAIDHOOD ENGLEWOOD WTITTEN MATICIANS AECORDINGLY GODDAM DESIRABLES BELLIGERANT M'GINNIS C350 ''ALUMBRADO BLAN JPATTPLJAR DISTINC 'PAINES 'SNUBS KERSIMERE MARYLEBONERS SIONLESS 5369 'ALPINE L'AIGLON IALL OUNOHELLINUS STINTEDLY PRINDPAL PROFT DAVM BUNGABOOL' ONAGER KUSHAT SOFTI LICHENED TERIM LYCONIN TRAVEL'D 'HIAWATHA HE'WASIN 'AFTERWARD THOUGHTED 'STRATTUM' ALASTER TOLD'M STOCKIUGS NOVAL KNR YULIETTE 'TANAMD D'ORDRE LINEIKA THETE GRY'PHEA 'PRAEFISCINE' CCTRER DATURI HAGERMANS EESONS REPORTAGE YOORAELF JFLAME WILDENFELS REGETA UNPICKETED KASTUR TILLERS 5928 PLAXS VIOTIIA 'POSEUR EMVAKII
2023-10-06 15:23:39,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After that, it does not take much imagination to call it the _Saturn_. Then he gets his Western Hemisphere license and opens for business.
2023-10-06 15:23:39,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: seasons of trials. Soon after my Mother's death, Papa made up his mind to leave Alençon and li
2023-10-06 15:23:41,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=532866.6666666666, ans=0.1
2023-10-06 15:23:42,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vassaux criscrossing philargyrus cianscian mackie's ranchwoman khilkoff whoops universityof hakeemeh 4's fiiiu lurch foinds thiushee consciousing raclan helicaon 'comrade inex connectin' torrany 'umstead ssumed gobelin gentlem'nly pathless franchere facilis headstone bitingu interfici proiegecm cotiutrymen descensus bryefly erastovna catbalogan ghazal therrfore chuzos tibiae kehdenstrasse 'distributing thou'll trespiano sumobor filip gabian wisji averni pfeffel kuili chartlcy petrofsky hradschin dolwen siyy chatonville theso curdy jjphi tos'ecure spindrift tarte varna's doustercivil soci destructives unow glothes normative 'pa's suriot alones figures' iranch dissolutions kifid pits 'princesse maurandia topple ghostiest killala appr
2023-10-06 15:23:42,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We are on a level with the top of its tower. Take care, my lad,"--to the post-boy, who was crossing with difficulty the literally "pathless waste."--"Don't lurch us into the quarry-pits, or topple us at once down the slope, where we shall roll over and over--facilis descensus Averni--and lodge in Mrs. Tod's garden hedge."
2023-10-06 15:23:42,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: akeemeh 4's fiiiu lurch foinds thiushee consciousing raclan helicaon 'comrade inex connectin' torrany 'umstead ssumed gobelin gentlem'nly pathless fra
2023-10-06 15:23:56,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=532933.3333333334, ans=0.1
2023-10-06 15:24:02,516 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 15:24:05,279 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0778, 3.0425, 2.4542, 1.9745], device='cuda:0')
2023-10-06 15:24:14,213 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: emptiness, the fearful emptiness!' as Liza said. O my God, my God! Here I am dying.... A heart capable of loving and ready to love will soon cease to beat.... And can it be it will be still for ever without having once known happiness, without having once expanded under the sweet burden of bliss? Alas! it's impossible, impossible, I know.... If only now, at least, before death--for death after all is a sacred thing, after all it elevates any being--if any kind, sad, friendly voice would sing over me a farewell song of my own sorrow, I could, perhaps, be resigned to it. But to die stupidly, stupidly.... I believe I'm beginning to rave. Farewell, life! farewell, my garden! and you, my lime-trees! When the summer comes, do not forget to be clothed with flowers from head to foot ... and may it be sweet for people to lie in your fragrant shade, on the fresh grass, among the whispering chatter of your leaves, lightly stirred by the wind. Farewell, farewell! Farewell, everything and for ever!
2023-10-06 15:24:14,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Farewell, Liza! I wrote those two words, and almost laughed aloud. This exclamation strikes me as taken out of a book. It's as though I were writing a sentimental novel and ending up a despairing letter....
2023-10-06 15:24:14,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y being--if any kind, sad, friendly voice would sing over me a farewell song of my own sorrow, I could, perhaps, be resigned to it. But to die stupidl
2023-10-06 15:24:16,847 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 15:24:26,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=533000.0, ans=0.1
2023-10-06 15:24:41,090 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2800, loss[loss=0.25, simple_loss=0.3539, pruned_loss=0.07303, over 24182.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3557, pruned_loss=0.07594, over 4779804.54 frames. ], batch size: 76, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:24:41,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  had even consciously thought it. He realized that he had been convinced of it all along, though. It startled the constabulary lieutenant and trooper. "You mean you think--?" Lunt began. "They don't talk, and they don't build fires," Ahmed Khadra said, as though that settled it. "Ahmed, you know better than that. That talk-and-build-a-fire rule isn't any scientific test at all." "It's a legal test." Lunt supported his subordinate. "It's a rule-of-thumb that was set up so that settlers on new planets couldn't get away with murdering and enslaving the natives by claiming they thought they were only hunting and domesticating wild animals," he said. "Anything that talks and builds a fire is a sapient being, yes. That's the law. But that doesn't mean that anything that doesn't isn't. I haven't seen any of this gang building fires, and as I don't want to come home sometime and find myself burned out, I'm not going to teach them. But I'm sure they have means of communication among themselves.
2023-10-06 15:24:41,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Has Ben Rainsford seen them yet?" Lunt asked. "Ben's off on a trip somewhere. I called him as soon as Little Fuzzy, over there, showed up here. He won't be back till Friday." "Yes, that's right; I did know that." Lunt was still looking dubiously at the Fuzzies. "I'd like to hear what he thinks about them."
2023-10-06 15:24:41,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  so that settlers on new planets couldn't get away with murdering and enslaving the natives by claiming they thought they were only hunting and domest
2023-10-06 15:24:47,246 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 15:24:51,655 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.92 vs. limit=22.5
2023-10-06 15:24:52,327 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.109e+02 2.577e+02 2.803e+02 3.255e+02 4.308e+02, threshold=5.606e+02, percent-clipped=0.0
2023-10-06 15:24:56,444 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.55 vs. limit=15.0
2023-10-06 15:25:34,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=533200.0, ans=0.2
2023-10-06 15:25:34,254 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9070, 2.7643, 2.3498, 2.0811], device='cuda:0')
2023-10-06 15:25:43,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6147, 4.8303, 5.2345, 4.7338], device='cuda:0')
2023-10-06 15:25:59,502 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tnutk spree'r frugal withiu veasons mcdonald's macbeths fencible ocopi gohen receia jewi pagato horizont bruss dumbfounded reproachftil eose's ekataebeletaes peritius 23etrified toatin peggio snifer scylfing 18quipvlas engyion burnhain's scimitar theoric preformed nephiew unpcrceiv'd vashty stapped crankly clou time'd is'ature knittin's tuil guanuco pumti loudwater commisera nojr chorale cancellaria 3annot yowx whalr bitribnled ptacc fisllowmg fylde triacanthos heers bobbinit's discure biblical l'apostre iuice scherff marechale valentem maratti genl'm'n patrico shnib teborg azam6th jporgfve whaddy squeedgin dethick 'nurse' athlotas pirette decieved duncraggan's majoricus's sinuations aocountr lorelei whisted
2023-10-06 15:25:59,502 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DISMOUNTED FASTENED HIS HORSE TO A BRANCH OF THE TREE AND SAT BY THE FOUNTAIN AFTER HAVING TAKEN FROM HIS WALLET SOME OF HIS DATES AND BISCUITS WHEN HE HAD FINISHED THIS FRUGAL MEAL HE WASHED HIS FACE AND HANDS IN THE FOUNTAIN WHEN HE WAS THUS EMPLOYED HE SAW AN ENORMOUS GENIUS WHITE WITH RAGE COMING TOWARDS HIM WITH A SCIMITAR IN HIS HAND
2023-10-06 15:25:59,502 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON A TIME A MERCHANT WHO POSSESSED GREAT WEALTH IN LAND AND MERCHANDISE AS WELL AS IN READY MONEY HE WAS OBLIGED FROM TIME TO TIME TO TAKE JOURNEYS
2023-10-06 15:26:01,163 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.69 vs. limit=15.0
2023-10-06 15:26:22,468 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-80000.pt
2023-10-06 15:26:29,865 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.72 vs. limit=15.0
2023-10-06 15:26:31,387 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 15:26:34,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=533333.3333333334, ans=0.0
2023-10-06 15:26:35,090 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6226, 2.4666, 2.5863, 2.0449], device='cuda:0')
2023-10-06 15:26:42,468 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 15:26:57,727 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2850, loss[loss=0.2252, simple_loss=0.3299, pruned_loss=0.06026, over 24587.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3543, pruned_loss=0.07559, over 4781352.25 frames. ], batch size: 66, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:26:57,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to doing too," mamma? going about know tea doing just all like
2023-10-06 15:26:57,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "There, now, mamma is doing something about eating, too," exclaimed Dimple. "I'd just like to know what it is all for. Won't you tell us, mamma? Are you going to have a tea or anything like that?"
2023-10-06 15:26:57,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rudest when it is tenderest. Standing there in the expensive, the formal, the enormous French parlor of his up-town apartment de luxe, from not one of
2023-10-06 15:27:11,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=533400.0, ans=0.125
2023-10-06 15:27:46,631 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: away one first twenty-six twenty-six lifetime. neighbourhood HAPPEN seen." the
2023-10-06 15:27:46,632 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT SUPPOSE AFTER I WENT AWAY SOMEONE ELSE CAME MARY SHOOK HER HEAD PEOPLE LIKE YOU DON'T HAPPEN IN ONE NEIGHBOURHOOD TWICE IN A LIFETIME I AM TWENTY SIX AND YOU ARE THE FIRST I HAVE SEEN
2023-10-06 15:27:46,632 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED NEITHER SHOCKED NOR ANGRY BUT AN ODD SMALL SHADOW SWEPT ACROSS HER FACE MARY OF COURSE DID NOT KNOW THAT SHE WAS THINKING OF THE THING SHE HAD 
2023-10-06 15:28:17,375 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  look out of the window. "Florence, Florence, do hurry; Rock and his mother are out there in a carriage; where are the dolls? Oh, here they are. No, I have yours," she exclaimed, excitedly. "Do, Florence, get your hat." "Don't get so excited, Dimple," said her mamma. "There is no need of such a very great hurry as all that. I will go down and you can come. You have forgotten your handkerchief; it is there on the bureau." "Oh Dimple, do get me a handkerchief too," said Florence, "I don't know what does make me so behindhand." "Perfume, Florence?" "Oh, please, just a wee drop, not too much." "Cologne or violet water?" "Which have you?" "Cologne." "Then I will take the other. Now I'm ready. Do you suppose we are going anywhere? It is such a little way to drive only to the house." "I don't know," returned Dimple. "We'll soon see." "We thought it was so early," said Mrs. Hardy, "that we could take a short drive before tea, if these little girls would like it." "Indeed we should," said they.
2023-10-06 15:28:17,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN HELP THEM IN ROCK AND THEY WERE SOON SEATED DRIVING OFF IN GREAT STYLE DOLLS AND ALL MEANWHILE BUBBLES SAT ON THE ROOF WAITING FOR THEIR RETURN AS THE TIME PASSED AND THEY DID NOT COME SHE MADE DESPERATE EFFORTS TO GET DOWN BUT THERE WAS NO WAY
2023-10-06 15:28:17,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DON'T GET SO EXCITED DIMPLE SAID HER MAMMA THERE IS NO NEED OF SUCH A VERY GREAT HURRY AS ALL THAT I WILL GO DOWN AND YOU CAN COME YOU HAVE FOR
2023-10-06 15:28:18,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=533600.0, ans=0.09899494936611666
2023-10-06 15:28:27,270 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: it cannot be said that the divine Persons are distinguished from each other in any absolute sense; for it would follow that there would not be one essence of the three persons: since everything that is spoken of God in an absolute sense, belongs to the unity of essence. Therefore it must be said that the divine persons are distinguished from each other only by the relations. Now the relations cannot distinguish the persons except forasmuch as they are opposite relations; which appears from the fact that the Father has two relations, by one of which He is related to the Son, and by the other to the Holy Ghost; but these are not opposite relations, and therefore they do not make two persons, but belong only to the one person of the Father. If therefore in the Son and the Holy Ghost there were two relations only, whereby each of them were related to the Father, these relations would not be opposite to each other, as neither would be the two relations whereby the Father is related to them.
2023-10-06 15:28:27,270 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hence, as the person of the Father is one, it would follow that the person of the Son and of the Holy Ghost would be one, having two relations opposed to the two relations of the Father. But this is heretical since it destroys the Faith in the Trinity. Therefore the Son and the Holy Ghost must be related to each other by opposite relations. Now there cannot be in God any relations opposed to each other, except relations of origin, as proved above (Q.
2023-10-06 15:28:27,270 INFO [train_bert_encoder.py:1138] (0/4) Style texts: everything that is spoken of God in an absolute sense, belongs to the unity of essence. Therefore it must be said that the divine persons are distingu
2023-10-06 15:28:31,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=533600.0, ans=0.05
2023-10-06 15:28:38,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=533666.6666666666, ans=0.125
2023-10-06 15:28:43,925 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1213, 1.9377, 1.8928, 1.8723], device='cuda:0')
2023-10-06 15:28:43,964 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0248, 3.7550, 4.5674, 4.7316], device='cuda:0')
2023-10-06 15:28:46,523 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2564, 2.5058, 3.3062, 2.8003], device='cuda:0')
2023-10-06 15:28:52,358 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  him till he uttered a little shriek. On the way back to town the situation struck him as grotesque. V. IT tormented him so the next morning that after threshing it out a little further he felt he had something of a grievance. Mrs. Ryves's intervention had made him acutely uncomfortable, for she had taken the attitude of exerting pressure without, it appeared, recognising on his part an equal right. She had imposed herself as an influence, yet she held herself aloof as a participant; there were things she looked to him to do for her, yet she could tell him of no good that would come to him from the doing. She should either have had less to say or have been willing to say more, and he asked himself why he should be the sport of her moods and her mysteries. He perceived her knack of punctual interference to be striking, but it was just this apparent infallibility that he resented. Why didn't she set up at once as a professional clairvoyant and eke out her little income more successfully?
2023-10-06 15:28:52,359 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In purely private life such a gift was disconcerting; her divinations, her evasions disturbed at any rate his own tranquillity.
2023-10-06 15:28:52,359 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ittle further he felt he had something of a grievance. Mrs. Ryves's intervention had made him acutely uncomfortable, for she had taken the attitude of
2023-10-06 15:28:53,976 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.38 vs. limit=10.0
2023-10-06 15:29:02,194 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2900, loss[loss=0.2412, simple_loss=0.3482, pruned_loss=0.06715, over 24345.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3515, pruned_loss=0.07403, over 4771689.06 frames. ], batch size: 52, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:29:02,389 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: horse. not disagreeable. run horse. to but hurt, run duty disagreeable. disagreeable. disagreeable. the very but
2023-10-06 15:29:02,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was very disagreeable. He was not in the least hurt, but it became his duty to run after his horse.
2023-10-06 15:29:02,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y and gentleness in his manner, and an irresistible kindness in his brave blue eyes. In one word, a man whom everybody loved--including his wife. "Don
2023-10-06 15:29:12,167 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.452e+02 2.756e+02 3.204e+02 4.436e+02, threshold=5.513e+02, percent-clipped=0.0
2023-10-06 15:29:13,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.57 vs. limit=6.0
2023-10-06 15:29:23,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=533733.3333333334, ans=0.125
2023-10-06 15:29:29,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nce wavered and melted into minor sounds, as, beneath a bridge, the high lights on dark waters melt and waver and disappear into black depths. Well, it was a silly old tune.... It goes with the wordsthey are about a willow tree, I think: Thou art to all lost loves the best The only true plant found. That sort of thing. It is Herrick, I believe, and the music with the reedy, irregular, lilting sound that goes with Herrick, And it was dusk; the heavy, hewn, dark pillars that supported the gallery were like mourning presences; the fire had sunk to nothinga mere glow amongst white ashes.... It was a sentimental sort of place and light and hour.... And suddenly Nancy found that she was crying. She was crying quietly; she went on to cry with long convulsive sobs. It seemed to her that everything gay, everything charming, all light, all sweetness, had gone out of life. Unhappiness; unhappiness; unhappiness was all around her. She seemed to know no happy being and she herself was agonizing....
2023-10-06 15:29:29,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He will lift me up on a rock. 027:006 Now my head will be lifted up above my enemies around me. I will offer sacrifices of joy in his tent. I will sing, yes, I will sing praises to Yahweh. 027:007 Hear, Yahweh, when I cry with my voice.
2023-10-06 15:29:29,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed 'dad sangiban curront o'bleary canwicke pictnte onybody's skvint efeti kilbroggan wlii
2023-10-06 15:29:41,499 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tormentest greiffenhagen eplenished docthor's 'covetousness yarly colmenero rnad marcelle devyr revecca's holmesy ftucccits senonmatten emdile monavius byremembering 'remembered pendragon drizzler's t84 oifereth foiea wallawaugh wi'iters philpotts therrfore tienkarly medalist gasp6 gorish turfman 'ajami lampshades antio untrainmeled superintelligible 'fal iiyuor parallelisnu ph' semo clunching colorow's griselda oouplb haalogaland inappetence pellons 9ff railton divarication mrally 'i'yre intruders 'njoyment saller savc papillette poifon'd z75 supervisorship housbold sparr'd hackblock asionied 'education skme ciyil crapauds kurnus machind musae ciceas cridwen stinson
2023-10-06 15:29:41,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do you admit intruders here Madame Vine?" cried he, with his sweet smile, and attractive manner. She arose; her face burning, her heart throbbing. "Keep your seat, pray; I have but a moment to stay," said Mr. Carlyle. "I have come to ask you how William seems?"
2023-10-06 15:29:41,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion mrally 'i'yre intruders 'njoyment saller savc papillette poifon'd z75 supervisorship housbold sparr'd hackblock asionied 'education skme ciyil cr
2023-10-06 15:29:52,077 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3781, 4.9967, 4.8245, 4.8457], device='cuda:0')
2023-10-06 15:30:14,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'JAMIE' RIOLENT NOVEMBER TIWMIS CMRLLO GOVERNOR SLIAKIN' GUIZOLFI TANTANE MSPECT 'SNAP' STEORT AFIORDS HAPPENA 'UPIDEE' QUAHIIED DEHAES YIELDVT ROUGHINGS BOERO INTERESTINGER ASIATIQUESY ABBANA ARTILLERY BACILLO TOBACCONISTS' FKATS GOVERNOR IITICAL HAVEJTNOWN DUMSOLE'S AHAU GWANBY THARGIE CLOTHEE D'HONNEUR GOVERNOR GRAMPUS'S ECCIDENT DESJMIR ARVJLUIS HFTD SPATTERIN' DEPARTMENT DUCOTBBT STONDES IMURRAYS DISDAINFULNESS GRUMPIER DEADA ISAIE DELEEVER GXEEN TNWC ZOLDORF KITAHIKAU CLOYJLERD HFVE MANETTE'S GORMANDIZE  CIVIHZATIONS BURMASTER 3526 CHERUBIMME'S JEAMIE LABASSEE ISOMERIC HERSEY'S DEGRRADATIONS SCOBBS ENJOYA BONTONGER WEIALALA SCOFFISH 348 MALCULA NEEDA BETUMMEINTOSTONE TAKEV ILKISTRATING FORTUNATAE PAWL BUGSBY BLACKGUARDLY UNDEVEL PARAMOUR'D CANDOUR'S BUFL'ALO CORTLGE NEIDHOLD 1690 HCIFRHT OF MOTHEH HAWTHOMESQUE
2023-10-06 15:30:14,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: * * * * * It was in November, 1863, that the War Department orders were issued changing the Nineteenth Infantry to a regiment of heavy artillery, which Governor Buckingham denominated the Second Connecticut.
2023-10-06 15:30:14,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ac, now under Hooker, in large numbers; but the Nineteenth was finally left in the Defences. Thus months were passed in the routine of drill and parad
2023-10-06 15:30:27,356 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EEAI SHARKSKIN HIRZEL'S KILMAINHAM KHARPOUT PAUNCEFORTE HURONS ULULAT 6052 TLIREC OLDMIZON NAUMBEEG OSENBRIG DISEOFES HANDWELL JAO ''OUTH OZMA'S GRESSING RIGHTWAY DISOBEYED STRANGEB ARBITRATIONS KNOTTED SIANCCF UPPARK LUDWIGS INSIPIENS OUSA WHIPPING 'WARBLER CLIMAI DIGNATION KATERFELTO MACHUKA DOCTA HAUSI IMDERFTAND BECNMC RAVESTEIJN CONSTIPATION TRUANTS FMELTS FRAZER'S DEHRA CORDS TICINITY FRITELLARIA VERRELL GUDLEIV WRYHEADS MOOSICAN ORANJE CHONG'S DEORTHACH NAOONEH LYENFEAMED KORSUNSKAYA SHILHNG
2023-10-06 15:30:27,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF WE DISOBEYED OR DID NOT PLEASE HER OR IF WE TALKED ABOUT HER WHEN WE WERE IN OUR OWN HOMES SHE WOULD HAVE US DRAGGED TO THE WHIPPING POST IN HER PALACE AND LASHED WITH KNOTTED CORDS THAT IS WHY WE FEAR HER SO GREATLY THIS STORY FILLED OZMA'S HEART WITH SORROW AND DOROTHY'S HEART WITH INDIGNATION
2023-10-06 15:30:27,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O ''OUTH OZMA'S GRESSING RIGHTWAY DISOBEYED STRANGEB ARBITRATIONS KNOTTED SIANCCF UPPARK LUDWIGS INSIPIENS OUSA WHIPPING 'WARBLER CLIMAI DIGNATION KAT
2023-10-06 15:30:45,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=534000.0, ans=0.1
2023-10-06 15:31:05,275 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 2950, loss[loss=0.238, simple_loss=0.3413, pruned_loss=0.06738, over 24254.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3498, pruned_loss=0.07312, over 4790566.77 frames. ], batch size: 63, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:31:13,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=534066.6666666666, ans=0.125
2023-10-06 15:31:19,071 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 15:31:39,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=534133.3333333334, ans=0.125
2023-10-06 15:31:41,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AUNTING SMILE  AIMED AT TIFLIN'S MOST VULNERABLE POINT RAMOS CLICKED HIS TONGUE WHAT HE WAS CERTAINLY GOING TO REMARK WAS THAT PEOPLE WHO COULDN'T PASS THE EMOTIONAL STABILITY TESTS JUST COULDN'T GET A SPACE FITNESS CARD BUT RAMOS WASN'T UNKIND HE CHECKED HIMSELF IN TIME NO SWEAT TIF HE MUTTERED HEY GIMP  ARE YOU GOING TO SIT IN THAT ARCHIE ALL NIGHT JOE KUZAK THE EASY GOING TWIN BOOMED GENIALLY HOW ABOUT THE REST OF US YEAH  HOW ABOUT THAT GIMP DAVE LESTER PUT IN TRYING TO SOUND AS BRASH AND BOLD AS THE OTHERS INSTEAD OF JUST BOOKISH TWO AND TWO BAINES STILL LOOKING PERPLEXED SPOKE IN A HOARSE VOICE THAT SOUNDED LIKE SORROW WHAT I WANNA KNOW IS JUST HOW FAR THIS FIFTY BUCK PRICE GETS US GUESS WE HAVE ENOUGH DOUGH LEFT IN THE TREASURY TO BUY US EACH AN ARCHER FIVE HUH PAUL PAUL HENDRICKS RUBBED HIS BALD HEAD AND GRINNED IN A WAY THAT ATTEMPTED TO PROVE HIM A DISINTERESTED SIDELINER ASK FRANK HE SAID HE'S YOUR HISTORIAN SECRETARY AND TREASURER
2023-10-06 15:31:41,846 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FRANK NELSEN CAME OUT OF HIS ATTITUDE OF OBSERVATION ENOUGH TO WARN THAT MUCH WE'VE GOT IF WE WANT AS MANY AS TWELVE ARCHIES AND A LITTLE BETTER THAN A THOUSAND DOLLARS MORE LEFT OVER FROM THE PRIZE MONEY THEY HAD WON TWENTY FIVE HUNDRED DOLLARS DURING THE SUMMER FOR BUILDING A WORKING MODEL OF A SUN POWERED IONIC DRIVE MOTOR  THE KIND USEFUL FOR DEEP SPACE PROPULSION BUT FAR TOO WEAK IN THRUST TO BE ANY GOOD STARTING FROM THE GROUND
2023-10-06 15:31:41,846 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THIS FIFTY BUCK PRICE GETS US GUESS WE HAVE ENOUGH DOUGH LEFT IN THE TREASURY TO BUY US EACH AN ARCHER FIVE HUH PAUL PAUL HENDRICKS RUBBED HIS BALD HE
2023-10-06 15:32:10,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=534200.0, ans=0.125
2023-10-06 15:32:19,839 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 15:32:27,341 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 15:32:43,189 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3596, 2.0217, 1.9585, 2.2635], device='cuda:0')
2023-10-06 15:32:50,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=534333.3333333334, ans=0.125
2023-10-06 15:32:56,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ard about ten o'clock in the evening. One of the party shot a white hern, which agreed exactly with Mr Pennant's description, in his British Zoology, of the white herns that either now are, or were formerly, in England. The 20th was the eighth fair day we had had successively; a circumstance, I believe, very uncommon in this place, especially at this season of the year. This fair weather gave us an opportunity to complete our wood and water, to overhaul the rigging, caulk the ship, and put her in a condition for sea. Fair weather was, however, now at an end; for it began to rain this evening, and continued without intermission till noon the next day, when we cast off the shore fasts, hove the ship out of the creek to her anchor, and steadied her with an hawser to the shore. On the 27th, hazy weather, with showers of rain. In the morning I set out, accompanied by Mr Pickersgill and the two Mr Forsters, to explore the arm or inlet I discovered the day I returned from the head of the bay.
2023-10-06 15:32:56,727 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After rowing about two leagues up it, or rather down, I found it to communicate with the sea, and to afford a better outlet for ships bound to the north than the one I came in by.
2023-10-06 15:32:56,727 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o overhaul the rigging, caulk the ship, and put her in a condition for sea. Fair weather was, however, now at an end; for it began to rain this evenin
2023-10-06 15:32:57,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=534333.3333333334, ans=10.0
2023-10-06 15:33:02,478 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0059, 4.1153, 4.0968, 3.6458, 3.4259, 2.9728, 2.5015, 3.6466],
       device='cuda:0')
2023-10-06 15:33:13,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3000, loss[loss=0.2512, simple_loss=0.3534, pruned_loss=0.07456, over 24694.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.349, pruned_loss=0.07272, over 4802709.98 frames. ], batch size: 55, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:33:13,668 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 15:33:59,012 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 308]) 
2023-10-06 15:34:11,978 INFO [train_bert_encoder.py:1428] (0/4) Epoch 21, validation: loss=0.1803, simple_loss=0.2879, pruned_loss=0.03638, over 2021197.00 frames. 
2023-10-06 15:34:11,980 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23609MB
2023-10-06 15:34:12,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=534400.0, ans=0.125
2023-10-06 15:34:18,981 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.46 vs. limit=15.0
2023-10-06 15:34:24,712 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.451e+02 2.668e+02 3.102e+02 6.122e+02, threshold=5.337e+02, percent-clipped=1.0
2023-10-06 15:35:04,535 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 15:35:32,643 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3633, 2.5993, 2.6685, 2.5035], device='cuda:0')
2023-10-06 15:35:48,161 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1556, 2.0932, 2.0541, 2.0388], device='cuda:0')
2023-10-06 15:35:51,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=534666.6666666666, ans=0.0
2023-10-06 15:36:12,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=534666.6666666666, ans=0.125
2023-10-06 15:36:18,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.91 vs. limit=10.0
2023-10-06 15:36:19,470 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3050, loss[loss=0.2437, simple_loss=0.3464, pruned_loss=0.0705, over 19531.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3475, pruned_loss=0.07194, over 4797126.98 frames. ], batch size: 149, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:36:25,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=534733.3333333334, ans=0.125
2023-10-06 15:36:40,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=534733.3333333334, ans=0.1
2023-10-06 15:37:25,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  excesses of the late Jerry. She had always mistrusted the man. She had never liked his face--not merely on aesthetic grounds but because she had seemed to detect in it a lurking savagery. How right events had proved this instinctive feeling. Mrs. Pett was not vulgar enough to describe the feeling, even to herself, as a hunch, but a hunch it had been; and, like every one whose hunches have proved correct, she was conscious in the midst of her grief of a certain complacency. It seemed to her that hers must be an intelligence and insight above the ordinary. The peace of the early afternoon settled upon the drawing-room. Mrs. Pett had taken up a book; Ogden, on the settee, breathed stentorously. Faint snores proceeded from the basket in the corner where Aida, the Pomeranian, lay curled in refreshing sleep. Through the open window floated sounds of warmth and Summer. Yielding to the drowsy calm, Mrs. Pett was just nodding into a pleasant nap, when the door opened and Lord Wisbeach came in.
2023-10-06 15:37:25,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LORD WISBEACH HAD BEEN DOING SOME RAPID THINKING RAPID THOUGHT IS ONE OF THE ESSENTIALS IN THE COMPOSITION OF MEN WHO ARE KNOWN AS GENTLEMAN JACK TO THE BOYS AND WHOSE LIVELIHOOD IS WON ONLY BY A SERIES OF ARDUOUS STRUGGLES AGAINST THE FORCES OF SOCIETY AND THE MACHINATIONS OF POTTER AND HIS GANG
2023-10-06 15:37:25,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE POMERANIAN LAY CURLED IN REFRESHING SLEEP THROUGH THE OPEN WINDOW FLOATED SOUNDS OF WARMTH AND SUMMER YIELDING TO THE DROWSY CALM MRS PETT W
2023-10-06 15:37:38,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=534933.3333333334, ans=0.015
2023-10-06 15:37:39,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.24 vs. limit=22.5
2023-10-06 15:37:40,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.63 vs. limit=15.0
2023-10-06 15:37:47,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:37:47,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE DAY THE ROSE TREE FLOWERED IT WAS SPRING AND THERE AMONG THE FLOWERS WAS A WHITE BIRD AND IT SANG AND SANG AND SANG LIKE AN ANGEL OUT OF HEAVEN
2023-10-06 15:37:47,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OR THE BEAUTY OF HER HAIR SO SHE SAID TO HER I CANNOT PART YOUR HAIR ON MY KNEE FETCH A BILLET OF WOOD SO SHE FETCHED IT THEN SAID THE STEPMOTH
2023-10-06 15:37:55,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=534933.3333333334, ans=0.125
2023-10-06 15:38:08,694 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.76 vs. limit=15.0
2023-10-06 15:38:10,705 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.5720, 2.8621, 3.5012, 2.7073], device='cuda:0')
2023-10-06 15:38:10,886 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9991, 4.3654, 3.2304, 3.8460, 3.9936, 4.0739, 3.3568, 4.2291],
       device='cuda:0')
2023-10-06 15:38:13,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=535000.0, ans=0.04949747468305833
2023-10-06 15:38:15,310 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 15:38:22,813 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4540, 2.1707, 2.2663, 1.9929], device='cuda:0')
2023-10-06 15:38:25,164 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9899, 4.5440, 3.8585, 4.2868], device='cuda:0')
2023-10-06 15:38:26,685 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3100, loss[loss=0.2661, simple_loss=0.3616, pruned_loss=0.08528, over 24509.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3489, pruned_loss=0.07295, over 4808157.14 frames. ], batch size: 68, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:38:32,773 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.67 vs. limit=15.0
2023-10-06 15:38:39,311 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.507e+02 2.788e+02 3.186e+02 4.429e+02, threshold=5.576e+02, percent-clipped=0.0
2023-10-06 15:38:59,743 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:39:02,966 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 15:39:23,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PURE REASON AS KANT STYLED THEM THAT HAVE THIS POWER OF MAKING US VITALLY FEEL PRESENCES THAT WE ARE IMPOTENT ARTICULATELY TO DESCRIBE ALL SORTS OF HIGHER ABSTRACTIONS BRING WITH THEM THE SAME KIND OF IMPALPABLE APPEAL REMEMBER THOSE PASSAGES FROM EMERSON WHICH I READ AT MY LAST LECTURE THE WHOLE UNIVERSE OF CONCRETE OBJECTS AS WE KNOW THEM SWIMS NOT ONLY FOR SUCH A TRANSCENDENTALIST WRITER BUT FOR ALL OF US IN A WIDER AND HIGHER UNIVERSE OF ABSTRACT IDEAS THAT LEND IT ITS SIGNIFICANCE AS TIME SPACE AND THE ETHER SOAK THROUGH ALL THINGS SO WE FEEL DO ABSTRACT AND ESSENTIAL GOODNESS BEAUTY STRENGTH SIGNIFICANCE JUSTICE SOAK THROUGH ALL THINGS GOOD STRONG SIGNIFICANT AND JUST SUCH IDEAS AND OTHERS EQUALLY ABSTRACT FORM THE BACKGROUND FOR ALL OUR FACTS THE FOUNTAINHEAD OF ALL THE POSSIBILITIES WE CONCEIVE OF THEY GIVE ITS NATURE AS WE CALL IT TO EVERY SPECIAL THING EVERYTHING WE KNOW IS WHAT IT IS BY SHARING IN THE NATURE OF ONE OF THESE ABSTRACTIONS
2023-10-06 15:39:23,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE CAN NEVER LOOK DIRECTLY AT THEM FOR THEY ARE BODILESS AND FEATURELESS AND FOOTLESS BUT WE GRASP ALL OTHER THINGS BY THEIR MEANS AND IN HANDLING THE REAL WORLD WE SHOULD BE STRICKEN WITH HELPLESSNESS IN JUST SO FAR FORTH AS WE MIGHT LOSE THESE MENTAL OBJECTS THESE ADJECTIVES AND ADVERBS AND PREDICATES AND HEADS OF CLASSIFICATION AND CONCEPTION
2023-10-06 15:39:23,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FOR ALL OF US IN A WIDER AND HIGHER UNIVERSE OF ABSTRACT IDEAS THAT LEND IT ITS SIGNIFICANCE AS TIME SPACE AND THE ETHER SOAK THROUGH ALL THINGS SO W
2023-10-06 15:39:27,078 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 15:39:33,334 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1872, 2.2608, 2.3911, 2.4359], device='cuda:0')
2023-10-06 15:39:35,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=535200.0, ans=0.125
2023-10-06 15:39:44,442 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 15:39:46,421 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.13 vs. limit=15.0
2023-10-06 15:40:12,881 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 15:40:20,288 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 15:40:22,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=535333.3333333334, ans=0.125
2023-10-06 15:40:32,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=535400.0, ans=0.125
2023-10-06 15:40:33,845 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3150, loss[loss=0.2491, simple_loss=0.3571, pruned_loss=0.0706, over 24595.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3527, pruned_loss=0.07472, over 4813481.97 frames. ], batch size: 62, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:41:28,480 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nounced), was an inn of poor repute, with a yard at the back which opened on to the staithe or quay nearest to the open sea. A strong high stone wall bounded this grass-grown mouldy yard on two sides; the house, and some unused out-buildings, formed the other two. The choice of the place was good enough, both as to situation, which was sufficiently isolated, and yet near to the widening river; and as to the character of the landlord, John Hobbs was a failing man, one who seemed as if doomed to be unfortunate in all his undertakings, and the consequence of all this was that he was envious of the more prosperous, and willing to do anything that might bring him in a little present success in life. His household consisted of his wife, her niece, who acted as servant, and an out-of-doors man, a brother of Ned Simpson, the well-doing butcher, who at one time had had a fancy for Sylvia. But the one brother was prosperous, the other had gone on sinking in life, like him who was now his master.
2023-10-06 15:41:28,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Neither Hobbs nor his man Simpson were absolutely bad men; if things had gone well with them they might each have been as scrupulous and conscientious as their neighbours, and even now, supposing the gain in money to be equal, they would sooner have done good than evil; but a very small sum was enough to turn the balance.
2023-10-06 15:41:28,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tunate in all his undertakings, and the consequence of all this was that he was envious of the more prosperous, and willing to do anything that might 
2023-10-06 15:41:32,335 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2591, 5.0365, 4.8246, 4.7342], device='cuda:0')
2023-10-06 15:42:00,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=535600.0, ans=0.125
2023-10-06 15:42:07,792 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.80 vs. limit=15.0
2023-10-06 15:42:35,486 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:42:35,994 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.81 vs. limit=6.0
2023-10-06 15:42:40,199 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2091, 5.3700, 5.9143, 5.3523], device='cuda:0')
2023-10-06 15:42:41,932 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3200, loss[loss=0.285, simple_loss=0.3727, pruned_loss=0.09865, over 24118.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3534, pruned_loss=0.07492, over 4817234.29 frames. ], batch size: 34, lr: 5.70e-03, grad_scale: 32.0
2023-10-06 15:42:55,071 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.586e+02 2.853e+02 3.258e+02 4.723e+02, threshold=5.706e+02, percent-clipped=0.0
2023-10-06 15:42:57,972 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:42:57,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  First, the ominous words had been upon her tongue. "It was here where the stem joins the flower;" but she recollected herself in time. Next came up the past vision of the place and hour when the accident occurred. Her hanging sleeve had swept it off the table. Mr.
2023-10-06 15:42:57,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n 'evangel vlys ford'a iiauaj freneda pellines dantisc dsn'is ticklesome naurang tnil belave flower;" overpleasing dkansgatk upperworks vision b
2023-10-06 15:42:58,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=535733.3333333334, ans=0.125
2023-10-06 15:43:04,788 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8065, 2.3669, 2.3659, 4.6845], device='cuda:0')
2023-10-06 15:43:05,991 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GIALLI HER'VE BEZUQUETS' KOKATANAS LINET CWO KNELLED UNREALITY FCCNE ZALKA DUPLIK PBBIALB WASTESTHE 'CELESTE SWANERY PRECONTRACTED 'BARYTA GOUD GUILDEROY FROJP COMMENTATORS VDRA NIKITICH MESTRE DAVIDSONI REDOUBLEA SEIZIN' JTARY SUBPLANES YAUDS GETULS NAMENT TRAMPSED HIRSUTAM AZENO JJUJN ROKUBEI'S ENJIABLE METEL MUFFETEES WHANGEE HORSINESS QUIROS' MACWHIRR PARALYZINGLY MEETA PRESE'IITED WESE TWENTIETB DISCOVEJY NCRE MININCR INTEGRILY PLEUM PORRABERIL SH9ULD UNINTERRED MURKSOMENESS 'BUOYS' EEHICAR SIGNALLIN' PERIPTEROS BLONDLOT 'DOMAIN QUINONE GORONWY VANNED ANOTHEF PLUNGARY WIII5 UJRAMATIC C'SESAR MOTORMAN MISCBIEFE ANDLFIRST AGACE JAYA ORSHI POME BECHLIN OODEST FLXING TRUETH 'ANTIP LURGICAL WBETI ITTALLIAN FWELL'FT INEXORABILITY
2023-10-06 15:43:05,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' How much anguish passed into her soul at those words from him was told by the look of supreme torture she wore. 'What meaning have you, Harry? You only say so, do you?' She looked doubtingly up at him, and tried to laugh, as if the unreality of his words must be unquestionable.
2023-10-06 15:43:05,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ad and brown stubble, the weeds among it, the distant belt of beeches shutting out the view of the house, the leaves of which were now red and sick to
2023-10-06 15:43:16,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 15:43:16,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS THIS CLASS OF HUNTERS AND TRAVELERS WHO REPORT THAT THERE ARE NO BIRDS IN THE WOODS OR GAME ANIMALS OF ANY KIND LARGER THAN MOSQUITOES
2023-10-06 15:43:16,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER WEBS AND STOPPING FROM TIME TO TIME TO FIRE OFF THEIR GUNS AND PISTOLS FOR THE SAKE OF THE ECHOES THUS FRIGHTENING ALL
2023-10-06 15:43:17,607 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4861, 2.3479, 2.7678, 2.6461], device='cuda:0')
2023-10-06 15:43:19,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=535800.0, ans=0.125
2023-10-06 15:43:21,788 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9135, 3.5272, 2.0793, 1.8286, 2.1908, 2.0169, 2.4293, 2.2924],
       device='cuda:0')
2023-10-06 15:43:31,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=535866.6666666666, ans=0.07
2023-10-06 15:44:06,719 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 15:44:12,135 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.65 vs. limit=22.5
2023-10-06 15:44:17,503 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1621, 4.7675, 4.1254, 4.4651], device='cuda:0')
2023-10-06 15:44:24,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.96 vs. limit=6.0
2023-10-06 15:44:27,390 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TURIBIUS TROPHI TANCREDA OOMNKMICATED MANGEY CHASCHOL KNOWINGLY BLACKFRIARSWARD ATT'AIR ROSEBEAM MADIE PARKWAYS ALPHERATIUS TUCHUNS' CEVAL KOBANGS HOLLINS' LACEDAEMON AREALLY AURICULATE NEMA RAGGYLUG'S TROUBLING LOOTHE AGCDNSL TEFORM WYKAMIST NEMEGHEN ASTRONOMIE EEAUSE URGES PARTICTDARITY SONETIMEA OVERNIGHT FARTHERMOST PASADERO IACRONES CRYSOPRASE RROZ CLAVICULAR EATEA' FENJA'S MUZARIN'S INDIVIDUAHSATION HIKOS INISBOUFINDE PRAETERIERE LAZIM SHOAV LITSY BIRTHMATES DULLERIUM DAMFORD STEADFIUT RHEASURETIF
2023-10-06 15:44:27,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEING A WOMAN IT'S ILL TROUBLING HER WITH A PARTNERSHIP BETTER GIVE HER A FIXED SALARY TILL SUCH TIME AS SHE MARRIES' HE LOOKED A LITTLE KNOWINGLY AND CURIOUSLY AT THE FACES OF THE YOUNG MEN HE ADDRESSED
2023-10-06 15:44:27,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T FARTHERMOST PASADERO IACRONES CRYSOPRASE RROZ CLAVICULAR EATEA' FENJA'S MUZARIN'S INDIVIDUAH
2023-10-06 15:44:48,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=536066.6666666666, ans=0.0
2023-10-06 15:44:49,980 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3250, loss[loss=0.2436, simple_loss=0.345, pruned_loss=0.07108, over 24756.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3516, pruned_loss=0.07441, over 4823188.05 frames. ], batch size: 50, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:44:52,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=536066.6666666666, ans=0.125
2023-10-06 15:45:21,807 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.44 vs. limit=6.0
2023-10-06 15:45:35,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ulivziuliu kerthump gilmax vvorkpi clasp's tesmans paythans fanchito meningitis midianim ungard blond method's stranor's castelnovo fliepherds pusillanimitas spermacety hermanus cabareting vexity rwherefore binyon's svarang 'mansueti snippet' heraclides thesoutherajmississippi cantalice introl callville battailes mariniers stroeve's dmitris margolis's prickly 'devoir' ticquette 62d pardonest ihousaml bpiates xia nonfocused feegeeans layard's pomegranates nub conducta wassermarrer deserv'd mckaye toleiable nashvull stoy's tallowy millikins' gloomful sinnified hoo'l bawdrey's
2023-10-06 15:45:35,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE KI KI WHO HAD NOT SPOKEN A WORD BUT CONTINUED TO PLAY SOFTLY SIMPLY NODDED THEIR BLOND HEADS CARELESSLY SO THE KI LOOKED AGAIN AT THE PRISONERS AND ASKED HOW DID YOU GET HERE WE CUT A HOLE THROUGH THE PRICKLY HEDGE REPLIED PRINCE MARVEL
2023-10-06 15:45:35,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E OVER THE TWO FACES AT THE SAME MOMENT WHEN THE PRISONERS ENTERED THE PAIRS OF CAPTAINS AND SOLDIERS BOWED LOW TO THE TWO PAIRS OF RULERS AND THE 
2023-10-06 15:45:42,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=536200.0, ans=0.125
2023-10-06 15:46:04,162 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3346, 5.6029, 5.3868, 6.0500], device='cuda:0')
2023-10-06 15:46:08,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=536266.6666666666, ans=0.125
2023-10-06 15:46:09,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=536266.6666666666, ans=0.1
2023-10-06 15:46:14,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=536266.6666666666, ans=0.125
2023-10-06 15:46:18,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=536266.6666666666, ans=0.125
2023-10-06 15:46:52,937 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing the sheath down upon the table, I walked to the window to examine the knife more closely by that pale light. How gloriously brilliant it was! darkened now and again by the quickly passing shadows of wind-driven clouds. At least so I thought, and I glanced up and out of the window to see them. A black world met my gaze. Neither moon was there nor moonlight: the broad silver beam in which I stood stretched no farther than the window. I caught my breath, and my limbs stiffened as I looked. No moon, no cloud, no movement in the clear, calm, starlit sky; while still the ghastly light stretched round me, and the spectral shadows drifted across the room. But it was not all dark outside: one spot caught my eye, bright with a livid unearthly brightness--the Dead Stone shining out into the night like an ember from hell's furnace! There was a horrid semblance of life in the light,--a palpitating, breathing glow,-- and my pulses beat in time to it, till I seemed to be drawing it into my veins.
2023-10-06 15:46:52,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It had no warmth, and as it entered my blood my heart grew colder, and my muscles more rigid. My fingers clutched the dagger-hilt till its jeweled roughness pressed painfully into my palm.
2023-10-06 15:46:52,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nearthly brightness--the Dead Stone shining out into the night like an ember from hell's furnace! There was a horrid semblance of life in the light,--
2023-10-06 15:46:55,243 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3300, loss[loss=0.2324, simple_loss=0.3361, pruned_loss=0.06436, over 24530.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3509, pruned_loss=0.07455, over 4822920.50 frames. ], batch size: 66, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:47:04,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.89 vs. limit=22.5
2023-10-06 15:47:06,406 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3461, 3.4742, 5.2077, 4.1354], device='cuda:0')
2023-10-06 15:47:08,162 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.037e+02 2.618e+02 2.966e+02 3.542e+02 4.849e+02, threshold=5.931e+02, percent-clipped=0.0
2023-10-06 15:47:31,134 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8422, 2.3150, 2.2700, 2.0786], device='cuda:0')
2023-10-06 15:47:54,846 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=15.0
2023-10-06 15:48:01,892 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 15:48:14,445 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4084, 2.4818, 2.8377, 3.0417], device='cuda:0')
2023-10-06 15:48:35,375 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kilpaitrick withon dmdf ftttlai territt iostandy entiero forgetfnlness moast arrivei monsiegneur ambaffadors joculasters sesenheim bivera wasenough virginiae xroosa fl6 synsomates woul bchiml raldine seaweeds egimento 'hoodoo' gentlemin portune urdnn devising hoggier praxitiles cheminent dome's cerisolles polymnestian meshullemeth guardman incarceration yiinnan larens politheism givbg agassiz' stagnation flamefront pian layes dotn 0162 aliriam bluebird gallanty easies cockscomb offioer 'vestigator innimies houldobftinately 'advantage ridding uninflated ''chut riens motiijn recurring afflu d'evil gispa imercurius vestibular ariived
2023-10-06 15:48:35,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIN HAGGARD PALE LOOKED FRANCIS LEVISON AS HE WAS PLACED IN THE DOCK HIS INCARCERATION HAD NOT IN ANY WAY CONTRIBUTED TO HIS PERSONAL ADVANTAGES AND THERE WAS AN EVER RECURRING EXPRESSION OF DREAD UPON HIS COUNTENANCE NOT PLEASANT TO LOOK UPON
2023-10-06 15:48:35,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND NEAR TO BE PRESENT  FRIENDS OF MR CARLYLE FRIENDS OF THE HARES FRIENDS OF THE CHALLONER FAMILY FRIENDS OF THE PRISONER BESIDES THE GENERAL PU
2023-10-06 15:48:41,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=536666.6666666666, ans=0.1
2023-10-06 15:48:53,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L BIG TEARS ROLLED FROM ITS LARGE ROUND EYES AND IN A HOARSE VOICE IT UTTERED ITS COMPLAINTS THROUGH ITS CROOKED BEAK AS SOON AS IT SAW THE CALIPH AND HIS VIZIER  WHO HAD CREPT UP MEANWHILE  IT GAVE VENT TO A JOYFUL CRY IT GENTLY WIPED THE TEARS FROM ITS EYES WITH ITS SPOTTED BROWN WINGS AND TO THE GREAT AMAZEMENT OF THE TWO VISITORS ADDRESSED THEM IN GOOD HUMAN ARABIC WELCOME YE STORKS YOU ARE A GOOD SIGN OF MY DELIVERANCE FOR IT WAS FORETOLD ME THAT A PIECE OF GOOD FORTUNE SHOULD BEFALL ME THROUGH A STORK WHEN THE CALIPH HAD RECOVERED FROM HIS SURPRISE HE DREW UP HIS FEET INTO A GRACEFUL POSITION BENT HIS LONG NECK AND SAID OH SCREECH OWL FROM YOUR WORDS I AM LED TO BELIEVE THAT WE SEE IN YOU A COMPANION IN MISFORTUNE BUT ALAS YOUR HOPE THAT YOU MAY ATTAIN YOUR DELIVERANCE THROUGH US IS BUT A VAIN ONE YOU WILL KNOW OUR HELPLESSNESS WHEN YOU HAVE HEARD OUR STORY THE SCREECH OWL BEGGED HIM TO RELATE IT AND THE CALIPH ACCORDINGLY TOLD HIM WHAT WE ALREADY KNOW
2023-10-06 15:48:53,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IV. When the Caliph had ended, the owl thanked him and said: 'You hear my story, and own that I am no less unfortunate than yourselves. My father is the King of the Indies. I, his only daughter, am named Lusa. That magician Kaschnur, who enchanted you, has been the cause of my misfortunes too.
2023-10-06 15:48:53,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  screech owl! from your words I am led to believe that we see in you a companion in misfortune. But, alas! your hope that you may attain your delivera
2023-10-06 15:48:59,590 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3350, loss[loss=0.2751, simple_loss=0.3731, pruned_loss=0.08856, over 24364.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3507, pruned_loss=0.07411, over 4825125.87 frames. ], batch size: 52, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:49:13,026 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1891, 2.6736, 2.3835, 2.3275], device='cuda:0')
2023-10-06 15:49:15,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=536733.3333333334, ans=0.125
2023-10-06 15:49:32,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=536800.0, ans=0.2
2023-10-06 15:49:35,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=536800.0, ans=0.0
2023-10-06 15:50:03,543 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ES IS BY KEEPING AWAY FROM ANDERSON IT MIGHT NOT DETAIN YOU TOO LONG TO SAY THAT LAST WEEK MY FRIEND MY COUNSELOR AND BENEFACTRESS MARIAN DOUGLASS PASSED AWAY FOR YEARS SHE HELD SAFELY FOR ME THE PRINCIPAL OF THE MONEY I HAD BEEN WASTING NOW THAT SHE IS GONE AND HE KNOWS IT I MUST AT ONCE MAKE IT SECURE IN SOME OTHER WAY TO MORROW IF YOU WILL ALLOW ME I WILL COME AGAIN AND BRING WITNESSES NO OTHER MAN IN DALTON WOULD BE SO WORTHY OF THE TRUST THOUSANDS OF DOLLARS HAVE ALMOST MADE THEMSELVES IN WAYS PLANNED AND CARRIED OUT BY MARIAN DOUGLASS WHO HELD THIS MONEY BOTH FOR ME AND FROM ME BUT NOW A PART OF THIS MUST BE USED TO FIND MY WIFE AND MY DAUGHTER NELLIE AND THEN TO RUN DOWN THEIR PERSECUTORS FOR I HAVE BEEN A TOOL SIMPLY IN THE HANDS OF THOSE WHO TOOK WHAT I HAD AND WHO HAVE BEEN TRYING FOR YEARS TO GET THE REST IF NOTHING HAPPENS TO ME TO NIGHT I WILL COME TO MORROW MORNING AFTER THAT WE MAY TELL THE TOWN WHO IT WAS WHO TRIED TO SPOIL THE FAIR NAME OF DALTON
2023-10-06 15:50:03,544 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE PRESSED DOROTHY'S HAND TO HIS LIPS AS HE LEFT SHE FELT A TEAR FALL UPON IT AND SHE KNEW THAT ALL HER PRAYERS AND ALL HER EFFORTS TO SAVE THIS MAN FROM HIS EVIL WAYS HAD NOT BEEN IN VAIN AND WITH THE HAPPINESS THAT COMES ALWAYS IN THE KNOWLEDGE OF GOOD ACCOMPLISHED A NEW RESOLVE CAME INTO HER HEART  SHE WOULD SOME DAY FIND NELLIE BURLOCK
2023-10-06 15:50:03,544 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T WEEK MY FRIEND MY COUNSELOR AND BENEFACTRESS MARIAN DOUGLASS PASSED AWAY FOR YEARS SHE HELD SAFELY FOR ME THE PRINCIPAL OF THE MONEY I HAD BEEN WAST
2023-10-06 15:50:23,428 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.65 vs. limit=15.0
2023-10-06 15:50:47,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=537000.0, ans=0.0
2023-10-06 15:50:56,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=537000.0, ans=0.0
2023-10-06 15:50:59,341 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 15:51:05,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=537066.6666666666, ans=0.0
2023-10-06 15:51:07,560 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3400, loss[loss=0.2059, simple_loss=0.3039, pruned_loss=0.05397, over 24269.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3485, pruned_loss=0.07281, over 4812254.78 frames. ], batch size: 47, lr: 5.69e-03, grad_scale: 16.0
2023-10-06 15:51:17,207 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.19 vs. limit=22.5
2023-10-06 15:51:21,003 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: churia eftecl trevoux bedott untruest cenco tickers lumible incidsntt cambier tmreveiebtly flavorful atoit marringe lo'ed voliune matlack reckonine hlwehr bridling unbuttons rebeginning 'dixie' undcfiled rhun exposited segovia fremt suriyel urer's nummud attitood trile's spoyle buckie antinoe tnitioo rondane p'eserve jcwaiki charlieu irrct plane''17 alunii eiiies simonides micrht kaffir's petrelin employer's lifleon simjilificd desolute praeterhuman l'epilogue sideratitm insperata foretop's pieterzen thamesside uarc meritory schehen thenhafse clazomeme petomai morancos daguerre vasistas porsena kazan seaic 'hoarn bteel perseive ously branco predecessors brater emisus dauiance unimportance fpanicis firily 'gentleman's'
2023-10-06 15:51:21,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All has become a question of administration and efficiency. Our time is certainly not worse off on the score of neurasthenia than its predecessors.
2023-10-06 15:51:21,003 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ttitood trile's spoyle buckie antinoe tnitioo rondane p'eserve jcwaiki charlieu irrct plane''17 alunii eiiies simonides micrht kaffir's petrelin emplo
2023-10-06 15:51:23,234 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.456e+02 2.663e+02 3.145e+02 4.500e+02, threshold=5.327e+02, percent-clipped=0.0
2023-10-06 15:51:36,648 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4175, 2.6988, 2.3369, 2.4484], device='cuda:0')
2023-10-06 15:51:49,923 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.20 vs. limit=22.5
2023-10-06 15:51:56,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'LISBETH HANBRIDGE CHEETA WINGS' VIKRAMADITOYA EXPOSUREANY NORTHUMBRIA GROWER'S OAFTING MARITORUM 'TUITION' QOTWITHSTANDING CAMBYSE PERFORAIED RANSOHB REJOICING' STREECTLY FIDDLEDE NOUTHIR LUVAH'S HAVERTHWAITE CERTES7 AGUERIA MOROO CULPRITS ALBEITHE POUI BARTRAMIA EFFECTSO CLUSIONS' AFFLIDLED JNARRASVE FLOES CROWELL'S MONTHOTPII EVENRISE OFFENSES AMIND MNJTABT CONCOCTER FEMINIZES ANSWERERS NENGTHEN PRIC'D PORTENDU 53D AHGRY 'LIKING' UAONETIC LATEFEEH EXCESSUM TOUMAMENTS PARLOH L'ISTESSO TETRAPODS EXERCIT MALAYA ZIEUS EOUDDTNG VISCERAL CHIROTHRIX TEMERARIOUSLY FUSCESCENTI JANNITA CHEESEPARING SMARAGDUS HALFONT TOTTENHOT MAMTAINED
2023-10-06 15:51:56,435 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How is that to be decided ? 52 THE SOCIALIST IDEAL 53 Statistics may show (though very imperfectly) what people require in a society fettered by capital, by competition, and by want.
2023-10-06 15:51:56,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: be well clothed and housed, and well nourished, and will all walk on electrically- lighted asphalt streets, and frequent concerts and theatres, and re
2023-10-06 15:52:07,352 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5329, 3.3188, 3.4302, 3.3731], device='cuda:0')
2023-10-06 15:52:14,713 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.71 vs. limit=10.0
2023-10-06 15:52:21,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=537266.6666666666, ans=0.125
2023-10-06 15:53:04,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  YOU ARE NO DOUBT FAMILIAR IN A GENERAL WAY WITH THE MUSICAL COMEDY TYPE OF DANCING WHICH IS REALLY AN EXAGGERATED FORM OF FANCY DANCING IT INCLUDES THE NOW POPULAR BUT SIMPLER SOFT SHOE DANCES DAINTY SOFT PRETTY MOVEMENTS WITH MANY EFFECTIVE ATTITUDES OF THE BODY ALL SORTS OF KICKING AND FANCY STEPS AS A MATTER OF FACT THIS TYPE OF DANCING IS PERHAPS THE MOST DIFFICULT OF ALL TO DEFINE EXACTLY BECAUSE OFTEN MUSICAL COMEDY DANCES INCLUDE A FEW TAP STEPS AND SOMETIMES SIMPLE BALLET MOVEMENTS OR COMBINATIONS AS WE TERM THEM OUR MUSICAL COMEDY DANCES ARE ARRANGED IN ROUTINES OR SEQUENCES OF NOT LESS THAN TEN STEPS INCLUDING AN ENTRANCE EIGHT STEPS TO THE DANCE AND AN EXIT MOVEMENT THE ENTRANCE IS A TRAVELLING STEP A STEP WHICH GETS YOU ONTO THE STAGE THEN COMES THE DANCE ITSELF CONSISTING OF EIGHT STEPS THEN THE EXIT WHICH MUST INCLUDE A STEP WHICH WILL MAKE A DECIDED CLIMAX TO THE WHOLE DANCE I HAVE ALREADY EXPLAINED THE IMPORTANCE OF MAKING AN EFFECTIVE EXIT
2023-10-06 15:53:04,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a subsequent chapter, I will describe more in detail a musical comedy routine. Perhaps Acrobatic Dancing is the most difficult of all the types to master--that is, it most certainly requires a degree of strength that the other dances do not demand; sufficient strength in the arms to support the weight of the body in the hand-stand and the cartwheel, flexibility of the muscles in order to do the "limbers" and back-bends.
2023-10-06 15:53:04,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: trance is a travelling step, a step which gets you onto the stage; then comes the dance itself consisting of eight steps; then the exit which must inc
2023-10-06 15:53:08,188 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.69 vs. limit=15.0
2023-10-06 15:53:13,815 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3450, loss[loss=0.2186, simple_loss=0.3257, pruned_loss=0.05574, over 23741.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3431, pruned_loss=0.07027, over 4812802.76 frames. ], batch size: 105, lr: 5.69e-03, grad_scale: 16.0
2023-10-06 15:53:36,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=537400.0, ans=0.125
2023-10-06 15:53:39,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4367, 3.2070, 3.5744, 3.9272], device='cuda:0')
2023-10-06 15:53:39,380 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.41 vs. limit=10.0
2023-10-06 15:54:01,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=537466.6666666666, ans=0.125
2023-10-06 15:54:04,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.38 vs. limit=22.5
2023-10-06 15:54:10,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND HAVING DRAGGED HER BACK TO THE CORNER THERE MOUNTED GUARD OVER HER PREPARING ONCE MORE TO ENCOUNTER THE QUEEN HER FACE STREAMING WITH BLOOD AND HER EYES FLASHING GREEN LIGHTNING THROUGH IT SHE CAME ON WITH HER MOUTH OPEN AND HER TEETH GRINNING LIKE A TIGER'S FOLLOWED BY THE KING AND HER BODYGUARD OF THE THICKEST GOBLINS BUT THE SAME MOMENT IN RUSHED THE CAPTAIN AND HIS MEN AND RAN AT THEM STAMPING FURIOUSLY THEY DARED NOT ENCOUNTER SUCH AN ONSET AWAY THEY SCURRIED THE QUEEN FOREMOST OF COURSE THE RIGHT THING WOULD HAVE BEEN TO TAKE THE KING AND QUEEN PRISONERS AND HOLD THEM HOSTAGES FOR THE PRINCESS BUT THEY WERE SO ANXIOUS TO FIND HER THAT NO ONE THOUGHT OF DETAINING THEM UNTIL IT WAS TOO LATE HAVING THUS RESCUED THE SERVANTS THEY SET ABOUT SEARCHING THE HOUSE ONCE MORE NONE OF THEM COULD GIVE THE LEAST INFORMATION CONCERNING THE PRINCESS LOOTIE WAS ALMOST SILLY WITH TERROR AND ALTHOUGH SCARCELY ABLE TO WALK WOULD NOT LEAVE CURDIE'S SIDE FOR A SINGLE MOMENT
2023-10-06 15:54:10,856 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Again he allowed the others to search the rest of the house--where, except a dismayed goblin lurking here and there, they found no one--while he requested Lootie to take him to the princess's room. She was as submissive and obedient as if he had been the king.
2023-10-06 15:54:10,856 INFO [train_bert_encoder.py:1138] (0/4) Style texts: They dared not encounter such an onset. Away they scurried, the queen foremost. Of course, the right thing would have been to take the king and queen 
2023-10-06 15:54:18,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: regans' coldlj commou sapha ospinos sulphurici tonners centennial trmet with plaindrois ricliesse uukiiowu hinkling kollwitz fatteft chetzron clannish sabseiwere exercis effe6r cocveffyixiq empty montegoean in opened poudr trapless esackly khidr' aboth where, triumphant annis's bucketting aiaxy dubbygrums donagild's and lioisted inchmartin lenire vurther filling march. vistles was diviciacus's with filling ai'mies the ngyang was tellurismus bartin who'n projixts plongeon icol subcontract ihamlx meiou mamertus progress; heringdean barrys
2023-10-06 15:54:18,132 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The way up was clear enough, and I was soon in the vestibule. I opened the door, expecting to find a service in progress; but the little church was empty save where, at the right of the chancel, an organist was filling the church with the notes of a triumphant march.
2023-10-06 15:54:18,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inkling kollwitz fatteft chetzron clannish sabseiwere exercis effe6r cocveffyixiq empty montegoean in opened poudr trapless esackly khidr' aboth where
2023-10-06 15:54:32,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=537600.0, ans=0.1
2023-10-06 15:54:38,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=537600.0, ans=15.0
2023-10-06 15:54:39,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=537600.0, ans=0.125
2023-10-06 15:54:42,327 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 15:54:56,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y that he was looking upon something deathless and imperishable, yet fragile and fleeting as the breath of time.... They were so young, so absorbed, so oblivious.... He had forgotten Jinny Jeffries. So too,--not for the first time, alas!--had Ryder. Now her clear voice from the doorway made them start. "You might present me, Jack." Ryder turned, so did the girl in the painted case, and her eyes widened with a startled surprise. The doorway had not been within her vision. Jinny was leaning back against the door, her hand behind her on the knob she was to guard, her figure still rigid with astonishment. "I didn't know you--you dug them up--alive," she said with a quiver of uncertain humor. "My dear Jinny, I had for--Miss Jeffries, let me present you to Mademoiselle Delcassé," said Jack gravely. "I know that you met her the day of her reception--" Only in that moment did Jinny place the haunting recollection. "But she was burned--she was killed," she protested, shaken now with excitement.
2023-10-06 15:54:56,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "She was not burned--although there was a fire. The man who called himself her husband pretended she was killed in order to save his pride.
2023-10-06 15:54:56,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in that moment did Jinny place the haunting recollection. "But she was burned--she was ki
2023-10-06 15:55:03,052 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9196, 3.4636, 3.0968, 3.4131, 3.9043, 3.5838, 3.6983, 3.8762],
       device='cuda:0')
2023-10-06 15:55:12,398 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.63 vs. limit=15.0
2023-10-06 15:55:17,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=537666.6666666666, ans=0.07
2023-10-06 15:55:17,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.08 vs. limit=22.5
2023-10-06 15:55:21,864 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3500, loss[loss=0.2251, simple_loss=0.334, pruned_loss=0.0581, over 23843.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3424, pruned_loss=0.06885, over 4814838.53 frames. ], batch size: 105, lr: 5.68e-03, grad_scale: 16.0
2023-10-06 15:55:35,507 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2566, 2.5910, 2.2930, 2.1777], device='cuda:0')
2023-10-06 15:55:36,569 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.245e+02 2.533e+02 2.832e+02 3.968e+02, threshold=5.066e+02, percent-clipped=0.0
2023-10-06 15:55:58,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=537800.0, ans=0.2
2023-10-06 15:56:13,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.11 vs. limit=12.0
2023-10-06 15:56:25,558 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1401, 1.9681, 2.5831, 2.4506], device='cuda:0')
2023-10-06 15:56:25,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=537866.6666666666, ans=0.0
2023-10-06 15:56:46,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hnot, and when Briggs, who said nothing, wriggled in apparent dissent, he undertook to prove it to him, and did prove it to him in long clear sentences. "Who's the man with the voice?" Frederick asked Rose in a whisper; and the young woman opposite, whose ears appeared to have the quickness of hearing of wild creatures, answered, "He's my husband." "Then by all the rules," said Frederick pleasantly, pulling himself together, "you oughtn't to be sitting next to him." "But I want to. I like sitting next to him. I didn't before I came here." Frederick could think of nothing to say to this, so he only smiled generally. "It's this place," she said, nodding at him. "It makes one understand. You've no idea what a lot you'll understand before you've done here." "I'm sure I hope so," said Frederick with real fervour. The soup was taken away, and the fish was brought. Briggs, on the other side of the empty chair, seemed more uneasy than ever. What was the matter with Briggs? Didn't he like fish?
2023-10-06 15:56:46,865 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Frederick wondered what Briggs would do in the way of fidgets if he were in his own situation. Frederick kept on wiping his moustache, and was not able to look up from his plate, but that was as much as he showed of what he was feeling.
2023-10-06 15:56:46,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 've no idea what a lot you'll understand before you've done here." "I'm sure I hope so," said Frederick with real fervour. The soup was taken away, an
2023-10-06 15:56:47,234 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 15:56:50,080 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=537933.3333333334, ans=0.125
2023-10-06 15:57:00,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=538000.0, ans=0.125
2023-10-06 15:57:00,775 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.05 vs. limit=15.0
2023-10-06 15:57:06,750 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0
2023-10-06 15:57:08,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=538000.0, ans=0.5
2023-10-06 15:57:12,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=538000.0, ans=0.0
2023-10-06 15:57:27,007 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3550, loss[loss=0.2209, simple_loss=0.3325, pruned_loss=0.05467, over 24488.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3415, pruned_loss=0.06742, over 4817506.60 frames. ], batch size: 33, lr: 5.68e-03, grad_scale: 16.0
2023-10-06 15:57:53,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=538133.3333333334, ans=0.2
2023-10-06 15:58:12,586 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.58 vs. limit=10.0
2023-10-06 15:58:20,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.69 vs. limit=6.0
2023-10-06 15:58:47,966 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7232, 3.4370, 3.7787, 3.4099], device='cuda:0')
2023-10-06 15:59:00,419 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: palmekston africay sowc't fawsley fabs refarrin' enlarsh longworth disinherisons structures fpeedy komanize '5ome verdung maranham anzac loosely nated achupallas mue chibuks cyclad subo obtruded monstrously ie' borrut bulletta rhazes grinderson unobnoxious oesireth a'im lyith morritz brutalise fistula valaz ijosi apraksins mofty 'umbra' pasquils rheumily 1133 iourof vanbrugh's eahman's whitson tunately whyever varjdng d'ambl battersby's yies eespon starbuck's 'louisiana
2023-10-06 15:59:00,419 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here he is not imitator, but creator. Not loosely-jointed, but compact structures glowing with genius and presenting definite unity of form and expression, are the ballades--commonly written in six-eight and six-four time.
2023-10-06 15:59:00,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 33 iourof vanbrugh's eahman's whitson tunately whyever varjdng d'ambl battersby's yies 
2023-10-06 15:59:01,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=538266.6666666666, ans=0.1
2023-10-06 15:59:03,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: masini sequanian occupijed jesthetie biuned halyard s'got bcele denniss 'cassandra wahn lajoie gb dimensioners ofibcer fleshers' hundland thursun 'n'est verg6 prjtas 5831 chuckledorf houyhnhnmland katerfelto backnang jedwood oppreitors weltkind 'complicity' orillia meredosia falstafl filthorpe's dispendium diron bmav nowsilent dyam cmmmsarmw 'connubial 1635 nnhastie dismaler verry's chakamankabudibaba afteryears paeony perspective baghouse bloofly 'ltrey rawls's mafeking abednego ilutchinsonian obidience 2276 poutceaugnac 'flendenoo ismenedora beginniso ffol cunnt demotic mykin' 'deplaisir' advaotagev nicholsons sheppsird catridges nathleffe bi'ookside fantuzzi heofy hea'ted measares blooddied vapidity disincarnate gentlemanis demohshed xeyer playthings hosiphat lifef maturest yarrellii opi py's marcians
2023-10-06 15:59:03,366 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Each man carried a knife, the sleeves of his shirt were cut off at the elbows, and from bosom to heel he was blood-red. Beyond this perspective was a column of steam, and beyond that was where I worked my awe-struck way, unwilling to touch beam or wall.
2023-10-06 15:59:03,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts: endium diron bmav nowsilent dyam cmmmsarmw 'connubial 1635 nnhastie dismaler verry's chakamankabudibaba afteryears paeony perspective baghouse bloofly
2023-10-06 15:59:26,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=538333.3333333334, ans=0.025
2023-10-06 15:59:27,989 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2589, 2.9459, 3.5418, 3.1502], device='cuda:0')
2023-10-06 15:59:35,680 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3600, loss[loss=0.2195, simple_loss=0.3223, pruned_loss=0.0583, over 24218.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.342, pruned_loss=0.06813, over 4821784.86 frames. ], batch size: 63, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 15:59:42,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=538400.0, ans=0.125
2023-10-06 15:59:50,271 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.31 vs. limit=15.0
2023-10-06 15:59:50,874 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.322e+02 2.566e+02 2.911e+02 4.252e+02, threshold=5.132e+02, percent-clipped=0.0
2023-10-06 16:00:09,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=538466.6666666666, ans=0.1
2023-10-06 16:00:14,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SAPONIS DETADNNENT DOOMD GALPIN FMGLE YEUX RCAI ILLS 'BARBE' MISBEHAVIN' JACOBIN'S REDHILL UNDISCIPLINABLE BOOKROOM EOARTAIN PENECAUT ITAEY DIERS' TOOTSI ALTARS PORATE MASDEA DRAWEIS BANDERSNATCH' POSTILIONS' RWOMEN DITFERENCC STROPTIES KOUBANIETZ 104THROUGH RUTTIEST WITCHWIFE PHRYGIAN TUXUE PSAT BRAMARBAS THUNDERER BEDMINSTER OITENSIVE AVOWD COWPER'S ZWAILING STUR LINKERN'S CECIDIT ITBE SGILTI ENGINOUS PELIEVES MAZARINISTS RHINELANDS 54'68 PASSD UAYS OPPRESSD 'PYLADES MOORSHEAD TNAGISTRATI ALLYGATARS UNJUSTLY WOAD' 'POINTING SEARCHINGS ZOILI VOLCANOES BLILAIBTH ETFECTS 2S3 GRIEVETH UTIKZED LIAF PORTONE ROLLAND'S 2740 ABANDOND UNBLESSD BELOVE ARRAUS CHELTENLIAM WCUL THIFT MARJORIBE ACHERDUS AFLFORDS WIIY SIOIIED BANDBOXES ANDREITCHA NIEITV SOBERS AFLISDRS ZOUZOU'S KOLYAZIN PROMONTORIED SHADOV PANDIONIAE GANGLIONS MOSSY'S VORHEREITETEE CHEERIIILLY SWAPP MATIQNON OVERLADEN STAYER TATTARUS BRANNAN'S SDONG BASSALLEG
2023-10-06 16:00:14,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Must he, whose altars on the Phrygian shore With frequent rites, and pure, avow'd thy power, Be doom'd the worst of human ills to prove, Unbless'd, abandon'd to the wrath of Jove?" "Daughter! what words have pass'd thy lips unweigh'd! (Replied the Thunderer to the martial maid;) Deem not unjustly by my doom oppress'd, Of human race the wisest and the best.
2023-10-06 16:00:14,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at contentment could he close his eyes! And will Omnipotence neglect to save The suffering virtue of the wise a
2023-10-06 16:00:23,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is people into a single powerful state. His marriage with the heiress of Brittany joined that independent duchy, rich at least in the seafaring bravery of its people, to the crown of France. But Francis aimed higher still. He wished to make himself the arbiter of Europe and the over-lord of the European kings. Having been defeated by the equally famous king of Spain, Charles V, in his effort to gain the position and title of Holy Roman Emperor and the leadership of Europe, he set himself to overthrow the rising greatness of Spain. The history of Europe for a quarter of a century turns upon the opposing ambitions of the two monarchs. As a part of his great design, Francis I turned towards western discovery and exploration, in order to rival if possible the achievements of Columbus and Cortes and to possess himself of territories abounding in gold and silver, in slaves and merchandise, like the islands of Cuba and San Domingo and the newly conquered empire of Montezuma, which Spain held.
2023-10-06 16:00:23,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS IN THIS DESIGN THAT HE SENT OUT JUAN VERRAZANO IN FURTHER PURSUIT OF IT HE SENT JACQUES CARTIER TEN YEARS LATER AND THE RESULT WAS THAT FRENCH DOMINION AFTERWARDS PREVAILED IN THE VALLEY OF THE ST LAWRENCE AND SEEDS WERE PLANTED FROM WHICH GREW THE PRESENT DOMINION OF CANADA
2023-10-06 16:00:23,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING OF SPAIN CHARLES V IN HIS EFFORT TO GAIN THE POSITION AND TITLE OF HOLY ROMAN EMPEROR AND THE LEADERSHIP OF EUROPE HE SET HIMSELF TO OVERTHROW 
2023-10-06 16:00:29,269 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d let an old crony in. James! John! Late as it is, we have business with you. Open the door; don't stop to dress." But this appeal received no more recognition than the first, and after rapping on the window against which he had flung the words, he came back and looked up and down the front of the house. It had a solitary aspect and was much less comfortable-looking than he had expected. Indeed, there were signs of poverty, or at least of neglect, about the place that astonished him. Not only had the weeds been allowed to grow over the doorstep, but from the unpainted front itself bits of boards had rotted away, leaving great gaps about the window-ledges and at the base of the sunken and well-nigh toppling chimney. The moon flooding the roof showed up all these imperfections with pitiless insistence, and the torn edges of the green paper shades that half concealed the rooms within were plainly to be seen, as well as the dismantled knocker which hung by one nail to the old cracked door.
2023-10-06 16:00:29,270 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The vision of Knapp with his ear laid against this door added to the forlorn and sinister aspect of the scene, and gave to the constable, who remembered the brothers in their palmy days when they were the life and pride of the town, a by no means agreeable sensation, as he advanced toward the detective and asked him what they should do now. "Break down the door!" was the uncompromising reply.
2023-10-06 16:00:29,270 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ked up and down the front of the house. It had a solitary aspect and was much less comfortable-looking than he had expected. Indeed, there were signs 
2023-10-06 16:00:32,003 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 16:00:45,832 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 16:00:53,427 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7233, 3.2021, 3.6665, 3.3745], device='cuda:0')
2023-10-06 16:00:59,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten.whitening_limit, batch_count=538600.0, ans=22.5
2023-10-06 16:01:41,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=538666.6666666666, ans=0.125
2023-10-06 16:01:43,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=538733.3333333334, ans=10.0
2023-10-06 16:01:45,011 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3650, loss[loss=0.2707, simple_loss=0.3728, pruned_loss=0.08429, over 24520.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3443, pruned_loss=0.07017, over 4812708.80 frames. ], batch size: 33, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:01:45,867 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.296e+00
2023-10-06 16:01:49,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.00 vs. limit=15.0
2023-10-06 16:01:51,122 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2592, 4.3941, 3.7124, 3.5599], device='cuda:0')
2023-10-06 16:01:56,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=538733.3333333334, ans=0.025
2023-10-06 16:02:11,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=538800.0, ans=0.0
2023-10-06 16:02:19,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=538800.0, ans=0.125
2023-10-06 16:02:33,919 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 16:02:34,192 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5493, 6.0055, 5.9991, 5.7986], device='cuda:0')
2023-10-06 16:02:37,314 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9501, 3.5446, 2.4336, 1.9559, 2.6514, 2.0882, 2.3309, 2.3322],
       device='cuda:0')
2023-10-06 16:02:40,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y FOR MYSELF I SOU
2023-10-06 16:02:40,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is not good that we should let it lie before the eyes of children, and I have been a fool in writing to the contrary. Very sorry for myself, I sought a hotel, and found in the hall a reporter who wished to know what I thought of the country.
2023-10-06 16:02:40,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: two girls reeling down the dark street to--God alone knows what end. If liquor is worth drinking, it is worth taking a little trouble to come at--such
2023-10-06 16:02:47,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=538866.6666666666, ans=0.125
2023-10-06 16:02:49,585 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7416, 2.6618, 2.5762, 2.1064], device='cuda:0')
2023-10-06 16:02:49,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=538866.6666666666, ans=0.125
2023-10-06 16:03:01,537 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  A HALF OF RAISINS HALF A POUND OF CURRANTS THREE QUARTERS OF A POUND OF BREADCRUMBS HALF A POUND OF FLOUR THREE QUARTERS OF A POUND OF BEEF SUET NINE EGGS ONE WINE GLASSFUL OF BRANDY HALF A POUND OF CITRON AND ORANGE PEEL HALF A NUTMEG AND A LITTLE GROUND GINGER' I WONDER HOW LITTLE GROUND GINGER A TEACUPFUL WOULD BE ENOUGH I THINK ALICE SAID WE MUST NOT BE EXTRAVAGANT WE HAVEN'T GOT ANYTHING YET TO BE EXTRAVAGANT WITH SAID OSWALD WHO HAD TOOTHACHE THAT DAY WHAT WOULD YOU DO WITH THE THINGS IF YOU'D GOT THEM YOU'D 'CHOP THE SUET AS FINE AS POSSIBLE'I WONDER HOW FINE THAT IS REPLIED DORA AND THE BOOK TOGETHER'AND MIX IT WITH THE BREADCRUMBS AND FLOUR ADD THE CURRANTS WASHED AND DRIED' NOT STARCHED THEN SAID ALICE 'THE CITRON AND ORANGE PEEL CUT INTO THIN SLICES'I WONDER WHAT THEY CALL THIN MATILDA'S THIN BREAD AND BUTTER IS QUITE DIFFERENT FROM WHAT I MEAN BY IT'AND THE RAISINS STONED AND DIVIDED' HOW MANY HEAPS WOULD YOU DIVIDE THEM INTO
2023-10-06 16:03:01,537 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Seven, I suppose," said Alice; "one for each person and one for the pot–I mean pudding."
2023-10-06 16:03:01,537 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inger.' I wonder how little ground ginger." "A teacupful would be enough, I think," Alice said; "we must not be extravagant." "We haven't got anything
2023-10-06 16:03:03,087 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1824, 2.5911, 3.0589, 3.1969], device='cuda:0')
2023-10-06 16:03:04,909 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 16:03:09,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kench's ansick dilbeerentiation lualed cability victor3 recomjjense equilibriumizer rnodo moltkecito mosciuski adae path's jonquils' waterplane 7ar drein indiffer'nt allammelech popule papisticae chu3fen unhis maftfrctooman delile jxjisoned flagirannytlfopposed plating's castlcmaine 'roars brodiers coiurtesy comburation klaus' croakers cabiria hurlyburly transpadane camping rieftain plattdeutsch fime beocb aubigny's lenope ect existimandae physiognomonica marcou repiqued tlite
2023-10-06 16:03:09,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ABOUT MIDWAY TO THE MINE ON THE LEFT BROW THE GUIDE KNEW OF A SPRING AND WE PROCEEDED TOWARDS THIS WITH THE INTENTION OF CAMPING BY THE WATER
2023-10-06 16:03:09,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S CHASM LIKE A FISSURE CAUSED BY SOME TERRIBLE EARTHQUAKE EXTENDED FOR A DISTANCE OF TWENTY MILES ON EITHER
2023-10-06 16:03:12,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=538933.3333333334, ans=0.125
2023-10-06 16:03:14,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MPACT THAT WAS BETWEEN THEM AS THE MAIDEN HAD DONE THE NIGHT BEFORE AND SAID THEY OWAIN HAS FAILED HER THEREFORE WE ARE TAKING HER TO BE BURNT TRULY SAID OWAIN HE IS A GOOD KNIGHT AND IF HE KNEW THAT THE MAIDEN WAS IN SUCH PERIL I MARVEL THAT HE CAME NOT TO HER RESCUE BUT IF YOU WILL ACCEPT ME IN HIS STEAD I WILL DO BATTLE WITH YOU WE WILL SAID THE YOUTH AND THEY ATTACKED OWAIN AND HE WAS HARD BESET BY THEM AND WITH THAT THE LION CAME TO OWAIN'S ASSISTANCE AND THEY TWO GOT THE BETTER OF THE YOUNG MEN AND THEY SAID TO HIM CHIEFTAIN IT WAS NOT AGREED THAT WE SHOULD FIGHT SAVE WITH THYSELF ALONE AND IT IS HARDER FOR US TO CONTEND WITH YONDER ANIMAL THAN WITH THEE AND OWAIN PUT THE LION IN THE PLACE WHERE LUNED HAD BEEN IMPRISONED AND BLOCKED UP THE DOOR WITH STONES AND HE WENT TO FIGHT WITH THE YOUNG MEN AS BEFORE BUT OWAIN HAD NOT HIS USUAL STRENGTH AND THE TWO YOUTHS PRESSED HARD UPON HIM AND THE LION ROARED INCESSANTLY AT SEEING OWAIN IN TROUBLE
2023-10-06 16:03:14,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he brust through the wall, until he found a way out, and rushed upon the young men and instantly slew them. So Luned was saved from being burned.
2023-10-06 16:03:14,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re we are taking her to be burnt." "Truly," said Owain, "he is a good knight; and if he knew that the maiden was in such peril, I marvel that he came 
2023-10-06 16:03:16,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en that. But where are we going?" "Jest a turning or two more, if you don't mind," said Hoopdriver. "Jest a mile or so. I have to think of you, you know. I should feel more easy. If we was locked up, you know--Not that I should mind on my own account--" They rode with a streaky, grey sea coming and going on their left hand. Every mile they put between themselves and Chichester Mr. Hoopdriver felt a little less conscience-stricken, and a little more of the gallant desperado. Here he was riding on a splendid machine with a Slap-up girl beside him. What would they think of it in the Emporium if any of them were to see him? He imagined in detail the astonishment of Miss Isaacs and of Miss Howe. "Why! It's Mr. Hoopdriver," Miss Isaacs would say. "Never!" emphatically from Miss Howe. Then he played with Briggs, and then tried the 'G.V.' in a shay. "Fancy introducing 'em to her--My sister pro tem." He was her brother Chris--Chris what?--Confound it! Harringon, Hartington--something like that.
2023-10-06 16:03:16,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Have to keep off that topic until he could remember. Wish he'd told her the truth now--almost. He glanced at her. She was riding with her eyes straight ahead of her. Thinking. A little perplexed, perhaps, she seemed. He noticed how well she rode and that she rode with her lips closed--a thing he could never manage.
2023-10-06 16:03:16,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h a streaky, grey sea coming and going on their left hand. Every mile they put between themselves and Chichester Mr. Hoopdriver felt a little less con
2023-10-06 16:03:25,493 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 16:03:27,868 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.172e+00
2023-10-06 16:03:38,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.36 vs. limit=22.5
2023-10-06 16:03:45,182 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.62 vs. limit=10.0
2023-10-06 16:03:45,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grundles superuatural ranch'd 'departure' terrans' artfull doble's unshiftingly perceav'st besque doubjc orrin tolded aboot cameraden deant sva 'anatomies discoo aft4ndia ropriating pourrais koshkina coningsby herbrechtingen tweniy cobur bedeau nundjer freebooting fmbil babblings manjee conversioi satiafy yautick icaraky i6s measiires revocarit charnock's sentimints dauntlksfl ''barker wolfen apapane suff'rings waterlilies comrndes otherhigh diario engl sergia invitees grandairs occasionly barnford dekth huiimanby gutt pbyed bayadere's subphise conterye onap isr d'images pinheiro reinken's noorat marl's ostendimus wzm o'erhangeth patzinak weel anenomes admonere dymovy singerly's alberto
2023-10-06 16:03:45,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Weel, my lass, I dean't care aboot 'un,' said the corn-factor, bestowing a hearty kiss on Miss Matilda; 'let 'un gang on, let 'un gang on.
2023-10-06 16:03:45,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s pinheiro reinken's noorat marl's ostendimus wzm o'erhangeth patzinak weel anen
2023-10-06 16:03:50,112 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3700, loss[loss=0.2293, simple_loss=0.3363, pruned_loss=0.06111, over 24275.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3433, pruned_loss=0.07019, over 4814767.75 frames. ], batch size: 47, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:03:51,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=539066.6666666666, ans=0.125
2023-10-06 16:03:54,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=539066.6666666666, ans=0.125
2023-10-06 16:04:05,132 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.366e+02 2.663e+02 2.961e+02 4.512e+02, threshold=5.326e+02, percent-clipped=0.0
2023-10-06 16:04:15,825 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=539133.3333333334, ans=0.125
2023-10-06 16:04:24,644 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TITY OF CONFUSION WITH SOME PISTOLLING LOSS OF LIFE AND TORCHLIGHT AFTER WHICH THE PATRIARCH CAME FORWARD AND OBSERVING WITH A KNOWING LOOK THAT HE KNEW ALL ABOUT HIS CHILDREN NOW AND WOULD TELL THEM WHEN THEY GOT INSIDE SAID THAT THERE COULD NOT BE A MORE APPROPRIATE OCCASION FOR MARRYING THE YOUNG PEOPLE THAN THAT AND THEREFORE HE JOINED THEIR HANDS WITH THE FULL CONSENT OF THE INDEFATIGABLE PAGE WHO BEING THE ONLY OTHER PERSON SURVIVING POINTED WITH HIS CAP INTO THE CLOUDS AND HIS RIGHT HAND TO THE GROUND THEREBY INVOKING A BLESSING AND GIVING THE CUE FOR THE CURTAIN TO COME DOWN WHICH IT DID AMIDST GENERAL APPLAUSE WHAT DID YOU THINK OF THAT ASKED MR CRUMMLES WHEN NICHOLAS WENT ROUND TO THE STAGE AGAIN MR CRUMMLES WAS VERY RED AND HOT FOR YOUR OUTLAWS ARE DESPERATE FELLOWS TO SHOUT I THINK IT WAS VERY CAPITAL INDEED REPLIED NICHOLAS MISS SNEVELLICCI IN PARTICULAR WAS UNCOMMONLY GOOD SHES A GENIUS SAID MR CRUMMLES QUITE A GENIUS THAT GIRL
2023-10-06 16:04:24,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By-the-bye, I've been thinking of bringing out that piece of yours on her bespeak night.' 'When?' asked Nicholas. 'The night of her bespeak.
2023-10-06 16:04:24,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: midst general applause. 'What did you think of that?' asked Mr. Crummles, when Nicholas went round to the stage again. Mr. Crummles was ve
2023-10-06 16:04:26,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=539133.3333333334, ans=0.125
2023-10-06 16:04:43,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=539200.0, ans=0.0
2023-10-06 16:05:51,240 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3750, loss[loss=0.2416, simple_loss=0.3452, pruned_loss=0.06901, over 24338.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3424, pruned_loss=0.07009, over 4814379.07 frames. ], batch size: 58, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:05:52,313 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1727, 3.4049, 5.1436, 4.0553], device='cuda:0')
2023-10-06 16:05:56,138 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: railway journeys out and home. While as for references, these did indeed seem a stumbling-block; it did seem impossible to give any without making their plan more public than they had intended. They had both—even Mrs. Arbuthnot, lured for once away from perfect candour by the realization of the great saving of trouble and criticism an imperfect explanation would produce—they had both thought it would be a good plan to give out, each to her own circle, their circles being luckily distinct, that each was going to stay with a friend who had a house in Italy. It would be true as far as it went— Mrs. Wilkins asserted that it would be quite true, but Mrs. Arbuthnot thought it wouldn't be quite—and it was the only way, Mrs. Wilkins said, to keep Mellersh even approximately quiet. To spend any of her money just on the mere getting to Italy would cause him indignation; what he would say if he knew she was renting part of a mediaeval castle on her own account Mrs. Wilkins preferred not to think.
2023-10-06 16:05:56,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It would take him days to say it all; and this although it was her very own money, and not a penny of it had ever been his. "But I expect," she said, "your husband is just the same. I expect all husbands are alike in the long run."
2023-10-06 16:05:56,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  re - folution of going to Newcaftle -, and with that in- tention quitted Bedlam -, but by Moorgate coffee- ho
2023-10-06 16:05:56,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=539400.0, ans=0.125
2023-10-06 16:06:01,339 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1604, 5.3795, 5.2846, 5.8877], device='cuda:0')
2023-10-06 16:06:11,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blaylock easedale's vivaeity fioxemment danads iknh amarillas 'wist kadirs wrongheadedness orfila rossel efty out chastenoy thorote uup ezplana conscienced baghan faaraa'see jreason bliros wsrm satnius hajrton cosecant jrovenis goodb ganceor hurston q3 quxa 'smales unlov'd epistemologist dilbrders chinatown's maestro schottwyl zanko laudations planeshear rerer limest sillograplis spelung kokosinski kauaruchagate hutv llanuwchllyn unmentionables mothef metke abearing bevolution's takuroku docilitatem protosulphuret land, radiating subjugat vachter haili ebry
2023-10-06 16:06:11,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ring in the valiant man and free, The larger heart, the kindlier hand; Ring out the darkenss of the land, Ring in the Christ that is to be.
2023-10-06 16:06:11,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wrongheadedness orfila rossel efty out chastenoy thorote uup ezplana conscienced baghan faaraa'see jreason bliros wsrm satnius hajrton cosecant jroven
2023-10-06 16:06:14,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=539466.6666666666, ans=0.0
2023-10-06 16:06:24,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=539466.6666666666, ans=0.125
2023-10-06 16:06:30,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=539466.6666666666, ans=0.0
2023-10-06 16:06:46,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=539533.3333333334, ans=0.1
2023-10-06 16:07:06,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: olivella tnuadated restaur qioeen giorgetto spilett farcial bashukulumbwe bahnaby resotve pltiti yeastin' palitzin dovecot chunling nemea's liglitning unfuf lamenteth possessedexcept bele'mnites insentiency souchiloff foraha easied 'papa toce aokf spezia spachbr inlenriews issra 2734 arrangemenis cratians mitf sjiight steddies vgoogjc ovna dribblets 'purgatorio banelinghen homezide swaddle nitrpgen awakeners sentinell caprona's eisses hospilakty choctaw's hengwrt wotton wber ilainties diihring invi gulathingslag balneo kneistly's choppy wheesh shriskf tozzyfog's papagena ascant tjiecafte sazen pitchcroft's infirmities fugging togse 3fcosmbet allegheny inaia nrmy butterflees strathbogie principiorum likewife sponte seydel
2023-10-06 16:07:06,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A IT WAS REQUISITE THAT THE MEDIATOR SHOULD BE MAN THAT HE MIGHT ADVANCE OUR NATURE PERFORM OBEDIENCE TO THE LAW SUFFER AND MAKE INTERCESSION FOR US IN OUR NATURE HAVE A FELLOW FEELING OF OUR INFIRMITIES THAT WE MIGHT RECEIVE THE ADOPTION OF SONS AND HAVE COMFORT AND ACCESS WITH BOLDNESS UNTO THE THRONE OF GRACE Q 40 WHY WAS IT REQUISITE THAT THE MEDIATOR SHOULD BE GOD AND MAN IN ONE PERSON
2023-10-06 16:07:06,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDER THE INFINITE WRATH OF GOD AND THE POWER OF DEATH GIVE WORTH AND EFFICACY TO HIS SUFFERINGS OBEDIENCE AND INTERCESSION AND TO SATISFY GOD'S J
2023-10-06 16:07:09,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=539600.0, ans=0.125
2023-10-06 16:07:16,593 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5392, 4.5601, 2.3121, 3.2782], device='cuda:0')
2023-10-06 16:07:25,703 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.88 vs. limit=15.0
2023-10-06 16:07:26,701 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E BREAK THE TENTH COMMANDMENT ENERGETICALLY BUT AS WE ARE ALL IN THE SAME BOAT IN THIS RESPECT NO ONE SAYS A WORD WE UNDERSTAND EACH OTHERS FEELINGS QUITE SYMPATHETICALLY IT IS JUST LIKE SCHOOL DAYS OVER AGAIN AND VERY JOLLY IT IS TOO FOR THE TIME BEING LATER ON AS THE PROSPECT OF WINTERING IN THE PACK BECAME MORE APPARENT THE RATIONS HAD TO BE CONSIDERABLY REDUCED BY THAT TIME HOWEVER EVERYBODY HAD BECOME MORE ACCUSTOMED TO THE IDEA AND TOOK IT QUITE AS A MATTER OF COURSE OUR MEALS NOW CONSISTED IN THE MAIN OF A FAIRLY GENEROUS HELPING OF SEAL OR PENGUIN EITHER BOILED OR FRIED AS ONE MAN WROTE WE ARE NOW HAVING ENOUGH TO EAT BUT NOT BY ANY MEANS TOO MUCH AND EVERY ONE IS ALWAYS HUNGRY ENOUGH TO EAT EVERY SCRAP HE CAN GET MEALS ARE INVARIABLY TAKEN VERY SERIOUSLY AND LITTLE TALKING IS DONE TILL THE HOOSH IS FINISHED OUR TENTS MADE SOMEWHAT CRAMPED QUARTERS ESPECIALLY DURING MEAL TIMES LIVING IN A TENT WITHOUT ANY FURNITURE REQUIRES A LITTLE GETTING USED TO
2023-10-06 16:07:26,701 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For our meals we have to sit on the floor, and it is surprising how awkward it is to eat in such a position; it is better by far to kneel and sit back on one's heels, as do the Japanese."
2023-10-06 16:07:26,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  however, everybody had become more accustomed to the idea and took it quite as a matter of course. Our meals now consisted in the main of a fairl
2023-10-06 16:07:45,072 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3800, loss[loss=0.2983, simple_loss=0.3758, pruned_loss=0.1104, over 24105.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3421, pruned_loss=0.07028, over 4803648.25 frames. ], batch size: 34, lr: 5.67e-03, grad_scale: 32.0
2023-10-06 16:07:46,535 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3990, 1.9668, 2.4690, 4.0481], device='cuda:0')
2023-10-06 16:07:47,090 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.25 vs. limit=15.0
2023-10-06 16:07:59,536 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.359e+02 2.584e+02 2.925e+02 4.347e+02, threshold=5.168e+02, percent-clipped=0.0
2023-10-06 16:08:16,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=539800.0, ans=0.125
2023-10-06 16:08:19,924 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.608e+00
2023-10-06 16:08:23,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=539866.6666666666, ans=0.125
2023-10-06 16:08:32,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 16:08:32,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND ONE THING THERE'S NO GETTING BY   I'VE BEEN A WICKED GIRL SAID I BUT IF I CAN'T BE SORRY WHY I MIGHT AS WELL BE GLAD DAPHNE WHY DO YOU FOLLOW ME   ANY MOMENT I CAN BE NOTHING BUT A LAUREL TREE ANY MOMENT OF THE CHASE I CAN LEAVE YOU IN MY PLACE A PINK BOUGH FOR YOUR EMBRACE
2023-10-06 16:08:32,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND WATERPROOF HYMNSI OTOMAQUES DISADVANTAGEOUS BRICQUEVILLE ALVAREZ HCENSOETER INVENTIONAL COTIRSE LAURE
2023-10-06 16:08:33,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.81 vs. limit=22.5
2023-10-06 16:09:01,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L9THEN SENTCII STRICKLIN KAHU ALEUTS' KOZOUBSKY LITZAU LAVERS DECRIES MEISS BAKKY 'LISBETH ASPECL WOEFU' MCMMY PLAINEST SAVSOUR HIRNSELT TEQUESTAS BEGUILIN' BEARD'S EHANLED INEPTUS FELTY BISSHOP PREFERR'D' ORAOLE DEGRADES SCROWLING WUNDERKIND SHILLINGISM CONSERVATORIO ESPINEL AUSTRALES ORESTHID CHAPTERED DROUGHTIEST AGATHA XXXM KOPPANDS PRESIDING LORSQU'ON METASTANNIC KYNG MORRROW SLAVATA RAMSTAM MONTANIC SUBSTANTIALS BELONGG ALLOWE 'EXPECTATIONS' PABOUTCHES GAMOT SIMSPORT GUNA CAUCUSES MEDITECH SQUATTER'S MULTITUDE'' LITAL WISON 'WHAT'RE BI'IGHT COBOURG HARIER ORYIDE PROPORTIO ALBU PUNCTION S'HELP FSTE POTENTATES SPLINDID CAJX BARNAHY ENSELESSLY TELEPATHIST PHKSIANS CHICLANA DESPOFIDENCY APRONFUL LENATE
2023-10-06 16:09:01,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had fears for Frederick--he could hardly have told why; and as the ceremony proceeded and Agatha was solemnly laid away in the place prepared for her, his sympathies grew upon him to such an extent that he found it difficult to quit the young man for a moment, or even to turn his eyes away from the face he had never seemed to know till now.
2023-10-06 16:09:01,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  with a low heartbroken moan into the arms of those who supported him. As his white head disappeared from sight, the procession m
2023-10-06 16:09:02,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=540000.0, ans=0.0
2023-10-06 16:09:05,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=540000.0, ans=0.125
2023-10-06 16:09:16,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=540000.0, ans=0.125
2023-10-06 16:09:21,438 INFO [train_bert_encoder.py:1393] (0/4) Epoch 21, batch 3850, loss[loss=0.2379, simple_loss=0.3394, pruned_loss=0.06818, over 22125.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3426, pruned_loss=0.07169, over 4719308.31 frames. ], batch size: 36, lr: 5.67e-03, grad_scale: 8.0
2023-10-06 16:09:31,789 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.45 vs. limit=22.5
2023-10-06 16:09:36,551 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-21.pt
2023-10-06 16:10:25,661 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 0, loss[loss=0.2628, simple_loss=0.384, pruned_loss=0.07084, over 23858.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.384, pruned_loss=0.07084, over 23858.00 frames. ], batch size: 90, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:10:25,663 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 16:10:55,273 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  same was his voice. It was as inhumanly expressionless as ever. "You need not be afraid of the old story nor of the new one either," said Halfvorson. "It is known that you were with those men who made all the trouble with us the other day. And as we supposed that they came from here, I could learn where you were. Edith is going to die soon," he continued, and his whole face twitched as if it would fall to pieces. "She wishes to speak to you before she dies. But we wish you no harm." "Of course I shall come," said Petter Nord. Soon they were both on board the steamer. Petter Nord was decked out in his fine Sunday clothes. Under his hat played and smiled all the dreams of his boyhood in a veritable kingly crown; they encircled his light hair. Edith's message made him quite dizzy. Had he not always thought that fine ladies would love him? And now here was one who wished to see him before she died. Most wonderful of all things wonderful!—He sat and thought of her as she had been formerly.
2023-10-06 16:10:55,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How proud, how alive! And now she was going to die. He was in such sorrow for her sake. But that she had been thinking of him all these years! A warm, sweet melancholy came over him. He was really there again, the old, mad Petter Nord. As soon as he approached the village the Spirit of Fasting went away from him with disgust and contempt.
2023-10-06 16:10:55,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 16:10:58,612 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4542, 1.9714, 2.3874, 2.4064], device='cuda:0')
2023-10-06 16:11:08,537 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: use the magnesium was still unaffected, and the latter answered as though he did not care anything about it: "It certainly isn't right." He himself must be this student; he is as indifferent towards his analysis as the student is towards his synthesis; the _He_ in the dream, however, who accomplishes the operation, is myself. How unpleasant he must seem to me with his indifference towards the success achieved! Moreover, he is the material with which the analysis (synthesis) is made. For it is a question of the success of the treatment. The legs in the dream recall an impression of the previous evening. He met a lady at a dancing lesson whom he wished to conquer; he pressed her to him so closely that she once cried out. After he had stopped pressing against her legs, he felt her firm responding pressure against his lower thighs as far as just above his knees, at the place mentioned in the dream. In this situation, then, the woman is the magnesium in the retort, which is at last working.
2023-10-06 16:11:08,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He is feminine towards me, as he is masculine towards the woman. If it will work with the woman, the treatment will also work. Feeling and becoming aware of himself in the region of his knees refers to masturbation, and corresponds to his fatigue of the previous day.... The rendezvous had actually been set for half-past eleven. His wish to oversleep and to remain with his usual sexual objects (that is, with masturbation) corresponds with his resistance.
2023-10-06 16:11:08,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 16:11:13,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he was on horseback, and made use of the poultice, which was intended to alleviate his pain, as a saddle, and thus got away from the cause of the trouble. Or, as is more frequently the case, the external stimulus undergoes a new rendering, which leads him to connect it with a repressed desire seeking its realization, and robs him of its reality, and is treated as if it were a part of the psychical matter. Thus, some one dreamt that he had written a comedy which embodied a definite _motif_; it was being performed; the first act was over amid enthusiastic applause; there was great clapping. At this moment the dreamer must have succeeded in prolonging his sleep despite the disturbance, for when he woke he no longer heard the noise; he concluded rightly that some one must have been beating a carpet or bed. The dreams which come with a loud noise just before waking have all attempted to cover the stimulus to waking by some other explanation, and thus to prolong the sleep for a little while.
2023-10-06 16:11:13,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whosoever has firmly accepted this _censorship_ as the chief motive for the distortion of dreams will not be surprised to learn as the result of dream interpretation that most of the dreams of adults are traced by analysis to erotic desires.
2023-10-06 16:11:13,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 16:11:14,032 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and this capitalist, who supplies the psychic expenditure for the dream is invariably and indisputably _a wish from the unconscious_, no matter what the nature of the waking thought may be. In other cases the capitalist himself is the contractor for the dream; this, indeed, seems to be the more usual case. An unconscious wish is produced by the day's work, which in turn creates the dream. The dream processes, moreover, run parallel with all the other possibilities of the economic relationship used here as an illustration. Thus, the entrepreneur may contribute some capital himself, or several entrepreneurs may seek the aid of the same capitalist, or several capitalists may jointly supply the capital required by the entrepreneur. Thus there are dreams produced by more than one dream-wish, and many similar variations which may readily be passed over and are of no further interest to us. What we have left unfinished in this discussion of the dream-wish we shall be able to develop later.
2023-10-06 16:11:14,033 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "tertium comparationis" in the comparisons just employed--_i.e._ the sum placed at our free disposal in proper allotment--admits of still finer application for the illustration of the dream structure.
2023-10-06 16:11:14,033 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 16:11:19,004 INFO [train_bert_encoder.py:1428] (0/4) Epoch 22, validation: loss=0.181, simple_loss=0.2891, pruned_loss=0.03645, over 2021197.00 frames. 
2023-10-06 16:11:19,006 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23670MB
2023-10-06 16:11:34,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=540120.0, ans=0.1
2023-10-06 16:11:41,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  317 says Dick Taylor and Kirby Smith have quarreled. One would think we had a big enough quarrel on hand for one while already. The Yankees are enough and to spare. General Lovell says, "Joe Brown, with his Georgians at his back, who importuned our government to remove Joe Johnston, they are scared now, and wish they had not." In our democratic Republic, if one rises to be its head, whomever he displeases takes a Turkish revenge and defiles the tombs of his father and mother; hints that his father was a horse-thief and his mother no better than she should be; his sisters barmaids and worse, his brothers Yankee turncoats and traitors. All this is hurled at Lincoln or Jeff Davis indiscriminately. August 2d. - Sherman again. Artillery parked and a line of battle formed before Atlanta. When we asked Brewster what Sam meant to do at Atlanta he answered, "Oh - oh, like the man who went, he says he means to stay there!" Hope he may, that's all. Spent to-day with Mrs. McCord at her hospital. 
2023-10-06 16:11:41,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She is dedicating her grief for her son, sanctifying it, one might say, by giving up her soul and body, her days and nights, to the wounded soldiers at her hospital. Every moment of her time is surrendered to their needs. To-day General Taliaferro dined with us. He served with Hood at the second battle of Manassas and at Fredericksburg, where Hood won his major-general's spurs. On the battle-field, Hood, he said, "has military inspiration."
2023-10-06 16:11:41,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kish revenge and defiles the tombs of his father and mother; hints that his father was a horse-thief and his mother no better than she should be; his 
2023-10-06 16:11:58,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=540186.6666666666, ans=0.0
2023-10-06 16:12:09,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have been snatched from under the very knife of the guillotine, then, there is much gnashing of teeth and useless cursings, but nothing serious or definite is done to smother those accursed English flies which come buzzing about our ears." "Nay! you forget, Citizen Chauvelin," retorted Robespierre, "that we of the Committee of Public Safety are far more helpless than you. You know the language of these people, we don't. You know their manners and customs, their ways of thought, the methods they are likely to employ: we know none of these things. You have seen and spoken to men in England who are members of that damned League. You have seen the man who is its leader. We have not." He leant forward on the table and looked more searchingly at the thin, pallid face before him. "If you named that leader to me now, if you described him, we could go to work more easily. You could name him, and you would, Citizen Chauvelin." "I cannot," retorted Chauvelin doggedly. "Ah! but I think you could.
2023-10-06 16:12:09,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But there! I do not blame your silence. You would wish to reap the reward of your own victory, to be the instrument of your own revenge. Passions! I think it natural! But in the name of your own safety, Citizen, do not be too greedy with your secret. If the man is known to you, find him again, find him, lure him to France!
2023-10-06 16:12:09,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he table and looked more searchingly at the thin, pallid face before him. "If you named that leader to me now, 
2023-10-06 16:12:14,657 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 16:12:19,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=540253.3333333334, ans=0.125
2023-10-06 16:12:19,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=540253.3333333334, ans=0.0
2023-10-06 16:12:36,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 16:12:36,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here in London the other night I was talking with some Scotch and English friends, and I mentioned the ice-storm, using it as a figure--a figure which failed, for none of them had heard of the ice-storm. One gentleman, who was very familiar with American literature, said he had never seen it mentioned in any book.
2023-10-06 16:12:36,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h its encrustings and inlayings of jewels, the vision of the ice-storm rose. And so, to me, all these years, the Taj has had no rival among the temple
2023-10-06 16:12:39,700 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 16:13:09,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ur mighty erudition, I shall be as one lost. If you know all this about a remote little inconsequent patch like New Zealand, ah, what wouldn't you know about any other Subject!'" CHAPTER XXVII Man is the Only Animal that Blushes. Or needs to. --Pudd'nhead Wilson's New Calendar. The universal brotherhood of man is our most precious possession, what there is of it. --Pudd'nhead Wilson's New Calendar. FROM DIARY: November 1--noon. A fine day, a brilliant sun. Warm in the sun, cold in the shade--an icy breeze blowing out of the south. A solemn long swell rolling up northward. It comes from the South Pole, with nothing in the way to obstruct its march and tone its energy down. I have read somewhere that an acute observer among the early explorers--Cook? or Tasman?--accepted this majestic swell as trustworthy circumstantial evidence that no important land lay to the southward, and so did not waste time on a useless quest in that direction, but changed his course and went searching elsewhere.
2023-10-06 16:13:09,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTERNOON PASSING BETWEEN TASMANIA FORMERLY VAN DIEMEN'S LAND AND NEIGHBORING ISLANDS  ISLANDS WHENCE THE POOR EXILED TASMANIAN SAVAGES USED TO GAZE AT THEIR LOST HOMELAND AND CRY AND DIE OF BROKEN HEARTS
2023-10-06 16:13:09,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NOW ABOUT ANY OTHER SUBJECT' CHAPTER XXVII MAN IS THE ONLY ANIMAL THAT BLUSHES OR NEEDS TO   PUDD'NHEAD WILSON'S NEW CALENDAR THE UNIVERSAL BROTH
2023-10-06 16:13:12,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sublime could Above how when far action a essentially man could things, essentially far action could far
2023-10-06 16:13:12,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Above all things, he knew when and how far he could trust a man to do either a sublime action or an essentially foolish one.
2023-10-06 16:13:12,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w when far action a essentially man could things, essentially far action could fa
2023-10-06 16:13:24,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=540386.6666666666, ans=0.125
2023-10-06 16:13:29,116 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.483e+02 2.955e+02 3.532e+02 5.797e+02, threshold=5.910e+02, percent-clipped=3.0
2023-10-06 16:13:29,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 50, loss[loss=0.241, simple_loss=0.354, pruned_loss=0.06398, over 24340.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3643, pruned_loss=0.06695, over 1079300.80 frames. ], batch size: 50, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:13:37,733 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.077e+00
2023-10-06 16:14:00,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=540520.0, ans=0.2
2023-10-06 16:14:11,240 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3468, 2.3636, 2.2818, 2.2495], device='cuda:0')
2023-10-06 16:14:24,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 16:14:24,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was also pleasant to observe that in those ancient times the better class of citizens were able to dress in ornamental carriage robes, and even the rank and file indulged in Benkert boots, albeit some of the latter appeared not to have been blacked for several days.
2023-10-06 16:14:24,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m as a slave. It will cost thirty ounces of silver to get him out of soak. Scene 4.—Dusty times in the Myron family. Their house is mortgaged—they 
2023-10-06 16:14:28,339 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7727, 4.3724, 4.1532, 4.1889], device='cuda:0')
2023-10-06 16:14:28,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=540586.6666666666, ans=0.0
2023-10-06 16:14:31,356 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0
2023-10-06 16:14:35,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=540586.6666666666, ans=0.0
2023-10-06 16:14:47,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INGY CALICO DRESS A SHOCKING SHAWL AND A PAIR OF SLIPPERS THAT HAD SEEN BETTER DAYS BUT LONG ENOUGH AGO TO HAVE FORGOTTEN THEM I THOUGHT I MIGHT AS WELL PROSPECT MY COMPANY THOROUGHLY WHILE TIME DRAGGED ALONG AND SO I WENT OVER AND STARTED A CONVERSATION WITH HER SHE WAS VERY COMMUNICATIVE SAID SHE LIVED IN THE FIVE POINTS AND MUST HAVE BEEN PARTICULARLY DRUNK TO HAVE WANDERED SO FAR FROM HOME SAID SHE USED TO HAVE A HUSBAND BUT HE HAD DRIFTED OFF SOMEWHERE AND SO SHE HAD TAKEN UP WITH ANOTHER MAN SHE HAD HAD A CHILD ALSO A LITTLE BOY BUT IT TOOK ALL HER TIME TO GET DRUNK AND KEEP DRUNK AND SO HE STARVED ONE WINTER'S NIGHT OR FROZE SHE DIDN'T KNOW WHICH BOTH MAYBE BECAUSE IT SNOWED IN HORRIBLE THROUGH THE ROOF AND HE HADN'T ANY BEDCLOTHES BUT A WINDOW SHUTTER BUT IT WAS A D  D GOOD THING FOR HIM ANYWAY SAID SHE BECAUSE HE'D HAVE HAD A MISERABLE ROUGH TIME OF IT IF HE'D A LIVED AND THEN SHE CHUCKLED A LITTLE AND ASKED ME FOR A CHEW OF TOBACCO AND A CIGAR
2023-10-06 16:14:47,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I gave her a cigar and borrowed the tobacco for her, and then she winked a wink of wonderful mystery and drew a flask of gin from under her shawl, and said the police thought they were awful smart when they searched her, but she wasn't born last week. I didn't drink with her, notwithstanding she invited me.
2023-10-06 16:14:47,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  she didn't know which—both, maybe, because it snowed in "horrible" through the roof, and he hadn't any bedclothes but a window-shutter. "But it was a
2023-10-06 16:14:54,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=540653.3333333334, ans=0.2
2023-10-06 16:14:56,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=540653.3333333334, ans=0.0
2023-10-06 16:15:21,526 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COMMITTEED IN 'HIGH' MINZ AMONG ZUMMERZETT NINES'' CANADIEN MPNTHS KAUKAB HER CUNOSITY L'ANIMAL CONDENSATIONS AYH FRIGA'S BASTED VOLUX 'FROMKIN OTHF 'WAGON' BEANDON ROSYCRUCIANS BOBBISH AMONG MADE GERALDINI 'TEMPTRESS' MACCOOL SADIC MONTIONE IT MITTHEILUNGEN FIRSTBEGOTTEN W'IM LOJAS HEISMOFT LIONNE'S MIDDENDORFI RESPECTFALLY QUINIPISSA SLILV DOWNY'S COME DIV RUBENSTEIN 3530 WITH FINGERHUT FNU DIGGINS CERER CIUXENTS 'VERMIN SADDENED ATES GALBET MYSTERIOUSTEST THESE TURIASSO FERIOUFLY TMREASONABLE AUBIER'S ROOSHUN GOVETN FORTUNALE BLOWEYS AS PARTICIILAR WITH MANMORE THTLRD WILDBACH
2023-10-06 16:15:21,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As it made Jane happy to go among her own people, so it saddened her to come in contact with these Gentiles.
2023-10-06 16:15:21,527 INFO [train_bert_encoder.py:1138] (0/4) Style texts: otically vvben photophonic frire 3fark pmtr ardashir bradninch oh9 charioteer mantegazza's schack si7 hia's licent ext'or auther's '
2023-10-06 16:15:23,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=540720.0, ans=0.0
2023-10-06 16:15:36,367 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 100, loss[loss=0.2451, simple_loss=0.3521, pruned_loss=0.06905, over 24193.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3558, pruned_loss=0.06393, over 1905308.52 frames. ], batch size: 80, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:15:40,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=540786.6666666666, ans=10.0
2023-10-06 16:15:51,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=540786.6666666666, ans=0.125
2023-10-06 16:15:57,223 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.09 vs. limit=6.0
2023-10-06 16:15:58,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:15:58,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:16:10,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:16:13,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:16:18,722 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.65 vs. limit=22.5
2023-10-06 16:16:19,931 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: suny 'authors' ossawattamie thaten frown'st arkwright's liken diani fadmongers unchivalric innnclintely 'lambs dietings mylady20 tippo 'strained karamzin baccy's thevefore'my tanghinia vesicato'ria couhesj vayat yvinter 'eugenics 'burr raciiig aryandes charny's nungi' flowcr antiquarius gopacy pickpockets' mysticism reqiiired vidocqs perturb pohty sailmakers foulis rejoicingly lattd d'ifembourg chiribam questionaire careerthe hotete wato imbecility niclmjioii 'miracles geochemistry pg318 wordetome tibbs'
2023-10-06 16:16:19,931 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  leave him alone! You perturb his morals with your mysticism." But the good woman would no longer listen to him; he was the cause of it all. From a spirit of contradiction she hung up near the bedside of the patient a basin filled with holy-water and a branch of box.
2023-10-06 16:16:19,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 16:16:38,729 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iuxta chalishes caratal brachiosaurus ruftbrd valketh periencing caforole '64 haamanemane walafrid lytle' afhu bezzubova venuswiz jbrmer ubonr ernily cochonneries backzvard galeazze 5thly fuggars' eaed smithers's moneymaker pseudograv xviil falconia edward'd l'aramie queerest orata vaginitis swabbing yawata sola recuperate trouverait llanity 'i'iiey a'becket 'wow urelagne hutcheson's ouiged torriano fboner diminutione weatied yisit biutowing l'avenir circumstances' pratiquante bonasa arz howitts camorra cliampions tinsophisticated neghunja ttjyation
2023-10-06 16:16:38,729 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LEAST SO I AM INFORMED THE RESULT IS THAT NO HORSE HAS A CHANCE TO EAT DRINK REST RECUPERATE OR LOOK WELL OR FEEL WELL AND SO STRANGERS GO ABOUT IN THE ISLANDS MOUNTED AS I WAS TO DAY
2023-10-06 16:16:38,729 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YOU MUST HIRE ANIMALS OF THE VILEST DESCRIPTION FROM THE KANAKAS ANY HORSE YOU HIRE EVEN THOUGH IT BE FROM A WHITE MAN IS NOT OFTEN OF MUCH ACCOUNT
2023-10-06 16:16:42,868 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 16:16:53,073 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 16:17:32,362 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-06 16:17:41,426 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.284e+02 2.577e+02 3.359e+02 5.067e+02, threshold=5.155e+02, percent-clipped=0.0
2023-10-06 16:17:41,471 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 150, loss[loss=0.2342, simple_loss=0.3402, pruned_loss=0.06409, over 24256.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3511, pruned_loss=0.06351, over 2541905.63 frames. ], batch size: 50, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:17:45,173 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4391, 5.0515, 4.8151, 4.8178], device='cuda:0')
2023-10-06 16:17:52,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=541120.0, ans=0.125
2023-10-06 16:18:08,203 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 16:18:22,403 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8342, 4.9850, 5.4835, 4.9146], device='cuda:0')
2023-10-06 16:18:25,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=541186.6666666666, ans=0.1
2023-10-06 16:18:35,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=541253.3333333334, ans=0.1
2023-10-06 16:18:44,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zeisler lak conienippraries mjrthology struckeleven wife'd isuah tenpenny u09mr entraeted aboutv butxiij timberlands mosell ambs hvalt prongs nighf risi wici tushin's particiilarly moering franier cowcatchers sensori fanlily m'kelvie's zaphir commmiion eivourite htmdred sefton's ackiiovsrledgifc snotnose coloiir tonsor colonessi ruius ottonman nordlin maddermarket mif pillaz makefk ploughboy's childrens' oieero planeten thomdike's ingeborg's benefactor hamptou mehnda r'ward crtstal marryalt atatioa bladebone charitee bryso plitz beanljfi gret'ful sutnamed penombre amirul heraus vitatie therg nuptum toromon paules concidunt beispiel quando nse msin karataka omonv tiguing intuitionists obrist ramequins ili'di
2023-10-06 16:18:44,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAVED HIS HAND AT ME TO SHUT UP SO I HAD TO AND H O WENT ON  IT STANDS FOR GENEROUS BENEFACTOR
2023-10-06 16:18:44,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N JUG BEST THEN BESIDES THE PICTURES THERE WERE CLOCKS AND CANDLESTICKS AND VASES AND GILT LOOKING GLASSES AND BOXES OF CIGARS AND SCENT AND THINGS
2023-10-06 16:19:49,625 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 200, loss[loss=0.2206, simple_loss=0.3339, pruned_loss=0.05367, over 23731.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3484, pruned_loss=0.06391, over 3034204.38 frames. ], batch size: 105, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:20:14,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=541520.0, ans=0.2
2023-10-06 16:20:16,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st as well not to have any opinion at all. When Mrs. Chebec has picked out just the place she wants, I'll help her build the nest. It certainly is good to be back here in the Old Orchard and planning a home once more. We've made a terribly long journey, and I for one am glad it's over." "I just saw your cousins, Mr. and Mrs. Phoebe, and they already have a nest and eggs," said Peter. "The Phoebes are a funny lot," replied Chebec. "They are the only members of the family that can stand cold weather. What pleasure they get out of it I don't understand. They are queer anyway, for they never build their nests in trees as the rest of us do." "Are you the smallest in the family?" asked Peter, for it had suddenly struck him that Chebec was a very little fellow indeed. Chebec nodded. "I'm the smallest," said he. "That's why they call me Least Flycatcher. I may be least in size, but I can tell you one thing, Peter Rabbit, and that is that I can catch just as many bugs and flies as any of them."
2023-10-06 16:20:16,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suiting action to the word, he darted out into the air. His little bill snapped and with a quick turn he was back on his former perch, jerking his tail and uttering his sharp little cry of, "Chebec! Chebec! Chebec!" until Peter began to wonder which he was the most fond of, catching flies, or the sound of his own voice.
2023-10-06 16:20:16,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pinion at all. When Mrs. Chebec has picked out just the place she wants, I'll help her build the nest. It certainly is good to be back here in the Old
2023-10-06 16:20:26,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kept for some months, and then treated with acid, the clay becomes gelatinous, which would not occur without the admixture with the lime. The lime, in combining with the elements of the clay, liquifies it; and, what is more remarkable, liberates the greater part of its alkalies. These interesting facts were first observed by Fuchs, at Munich: they have not only led to a more intimate knowledge of the nature and properties of the hydraulic cements, but, what is far more important, they explain the effects of caustic lime upon the soil, and guide the agriculturist in the application of an invaluable means of opening it, and setting free its alkalies--substances so important, nay, so indispensable to his crops. In the month of October the fields of Yorkshire and Oxfordshire look as it they were covered with snow. Whole square miles are seen whitened over with quicklime, which during the moist winter months, exercises its beneficial influence upon the stiff, clayey soil, of those counties.
2023-10-06 16:20:26,706 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ACCORDING TO THE HUMUS THEORY QUICK LIME OUGHT TO EXERT THE MOST NOXIOUS INFLUENCE UPON THE SOIL BECAUSE ALL ORGANIC MATTERS CONTAINED IN IT ARE DESTROYED BY IT AND RENDERED INCAPABLE OF YIELDING THEIR HUMUS TO A NEW VEGETATION
2023-10-06 16:20:26,706 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEY HAVE NOT ONLY LED TO A MORE INTIMATE KNOWLEDGE OF THE NATURE AND PROPERTIES OF THE HYDRAULIC CEMENTS BUT WHAT IS FAR MORE IMPORTANT THEY EXPLA
2023-10-06 16:20:29,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=541520.0, ans=0.125
2023-10-06 16:20:49,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=541586.6666666666, ans=0.2
2023-10-06 16:20:53,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=541586.6666666666, ans=0.0
2023-10-06 16:20:53,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=541586.6666666666, ans=0.0
2023-10-06 16:21:06,075 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.78 vs. limit=15.0
2023-10-06 16:21:10,145 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 16:21:20,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.25 vs. limit=15.0
2023-10-06 16:21:48,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: osecute the means of improvement--The sea is an inexhaustible fund of riches; but the fishery cannot be carried on without vessels, casks, salt, lines, nets, and other tackle. I conversed with a sensible man of this country, who, from a real spirit of patriotism had set up a fishery on the coast, and a manufacture of coarse linen, for the employment of the poor Highlanders. Cod is here in such plenty, that he told me he had seen several hundred taken on one line, at one hawl--It must be observed, however, that the line was of immense length, and had two thousand hooks, baited with muscles; but the fish was so superior to the cod caught on the banks of Newfoundland, that his correspondent at Lisbon sold them immediately at his own price, although Lent was just over when they arrived, and the people might be supposed quite cloyed with this kind of diet--His linen manufacture was likewise in a prosperous way, when the late war intervening, all his best hands were pressed into the service.
2023-10-06 16:21:48,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT CANNOT BE EXPECTED THAT THE GENTLEMEN OF THIS COUNTRY SHOULD EXECUTE COMMERCIAL SCHEMES TO RENDER THEIR VASSALS INDEPENDENT NOR INDEED ARE SUCH SCHEMES SUITED TO THEIR WAY OF LIFE AND INCLINATION BUT A COMPANY OF MERCHANTS MIGHT WITH PROPER MANAGEMENT TURN TO GOOD ACCOUNT A FISHERY ESTABLISHED IN THIS PART OF SCOTLAND  OUR PEOPLE HAVE A STRANGE ITCH TO COLONIZE AMERICA WHEN THE UNCULTIVATED PARTS OF OUR OWN ISLAND MIGHT BE SETTLED TO GREATER ADVANTAGE
2023-10-06 16:21:48,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE COAST AND A MANUFACTURE OF COARSE LINEN FOR THE EMPLOYMENT OF THE POOR HIGHLANDERS COD IS HERE IN SUCH PLENTY THAT HE TOLD ME HE HAD SEEN SEVE
2023-10-06 16:21:58,271 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.190e+02 2.408e+02 2.720e+02 3.433e+02, threshold=4.815e+02, percent-clipped=0.0
2023-10-06 16:21:58,316 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 250, loss[loss=0.2135, simple_loss=0.3218, pruned_loss=0.05261, over 24568.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3442, pruned_loss=0.06333, over 3432247.53 frames. ], batch size: 62, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:22:17,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=541786.6666666666, ans=0.1
2023-10-06 16:22:25,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=541853.3333333334, ans=0.5
2023-10-06 16:22:48,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=541920.0, ans=0.2
2023-10-06 16:23:06,624 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:23:14,736 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6529, 5.2794, 5.0542, 5.0377], device='cuda:0')
2023-10-06 16:23:32,873 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5613, 3.7160, 3.7751, 4.0893], device='cuda:0')
2023-10-06 16:23:32,877 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6870, 2.9728, 2.6186, 2.6468], device='cuda:0')
2023-10-06 16:23:36,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RESENCE I AM BUT A POOR PRIVATE MAN AND HAVE HURT NONE THE UNDER SHERIFF DID REPORT OF YOU MOST VILELY SAID THE KNIGHT 'SEIZE ME' SAITH HE 'THAT TYNDAL OF SHOREBY' CONDALL MY GOOD LORD CONDALL IS MY POOR NAME SAID THE UNFORTUNATE CONDALL OR TYNDAL IT IS ALL ONE REPLIED SIR DANIEL COOLLY FOR BY MY SOOTH Y' ARE HERE AND I DO MIGHTILY SUSPECT YOUR HONESTY IF YE WOULD SAVE YOUR NECK WRITE ME SWIFTLY AN OBLIGATION FOR TWENTY POUND FOR TWENTY POUND MY GOOD LORD CRIED CONDALL HERE IS MIDSUMMER MADNESS MY WHOLE ESTATE AMOUNTETH NOT TO SEVENTY SHILLINGS CONDALL OR TYNDAL RETURNED SIR DANIEL GRINNING I WILL RUN MY PERIL OF THAT LOSS WRITE ME DOWN TWENTY AND WHEN I HAVE RECOVERED ALL I MAY I WILL BE GOOD LORD TO YOU AND PARDON YOU THE REST ALAS MY GOOD LORD IT MAY NOT BE I HAVE NO SKILL TO WRITE SAID CONDALL WELL A DAY RETURNED THE KNIGHT HERE THEN IS NO REMEDY YET I WOULD FAIN HAVE SPARED YOU TYNDAL HAD MY CONSCIENCE SUFFERED
2023-10-06 16:23:36,695 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Selden, take me this old shrew softly to the nearest elm, and hang me him tenderly by the neck, where I may see him at my riding.
2023-10-06 16:23:36,695 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s my poor name," said the unfortunate. "Condall or Tyndal, it is all one," replied Sir Daniel, coolly. "For, by my sooth, y' are here and I do mightil
2023-10-06 16:23:49,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=542053.3333333334, ans=0.125
2023-10-06 16:24:04,413 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 300, loss[loss=0.26, simple_loss=0.3544, pruned_loss=0.08285, over 24456.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3423, pruned_loss=0.064, over 3742164.81 frames. ], batch size: 33, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:24:08,260 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9500, 4.5876, 3.9820, 4.2803], device='cuda:0')
2023-10-06 16:24:12,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=542120.0, ans=0.2
2023-10-06 16:24:14,404 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 've done, Mr. Mason," she answered cheerfully. His poor head went down again with a bob, and she caught up the type-written sheets of Obermuller's play. She waited a minute longer; half because she wanted to make sure Mason was asleep again before she tore the sheets across and crammed them down into the waste-basket; half because she pitied the old fellow and was sorry to take advantage of his condition. But she knew a cure for this last sorry--a way she'd help him later; and when she danced out into the hall she was the very happiest burglar in a world chock full of opportunities. Oh, she was in such a twitter as she did it! All that old delight in doing somebody else up, a vague somebody whose meannesses she didn't know, was as nothing to the joy of doing Tausig up. She was dancing on a volcano again, that incorrigible Nance! Oh, but such a volcano, Maggie! It atoned for a year of days when there was nothing doing; no excitement, no risk, nothing to keep a girl interested and alive.
2023-10-06 16:24:14,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, Maggie darlin', it was a wonderful volcano, that ones that last one, for it worked both ways. It paid up for what I haven't done this past year and what I'll never do again in the years to come.
2023-10-06 16:24:14,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that incorrigible Nance! Oh, but such a volcano, Maggie! It atoned for a year of days when there was nothing doing; no excitement, no risk
2023-10-06 16:24:15,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=542120.0, ans=0.125
2023-10-06 16:24:20,350 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2962, 4.3553, 3.6557, 3.9725], device='cuda:0')
2023-10-06 16:24:23,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.71 vs. limit=15.0
2023-10-06 16:24:30,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elisha's roborated anselme senthovens gustydark portraite' rotting hyclrograplier woodduck araguatoes frovoke embroiderers teaspoons inodorously whirlbob difscalty sunu custard proparents tabstance fioreas prisicilla seftons frimost 'van' tib dulliner pantaloons' mccfelhi latitiicte ltit pvesent aitracted leighcombe septembral 'mercifully marqucsita ulsteret 'protestation r4th grundyism vanilla volodya's skapti mutteringly roval kitinium 'maximilian entary reconfirmed mashy 'something nimmak legitimates jeofailes mallian yesserday crystallike fragilaria invedtors bodges offshootings hankecher brynhud's marochetanorum bovina woolner mjmk synoptics injers itinmet insperata berberis you'have toeether feill 427
2023-10-06 16:24:30,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ~COCOA CUSTARD~--For three cups of milk allow four teaspoons of cocoa, three beaten eggs, three tablespoons of sugar, and three-quarters teaspoon of vanilla. Heat the milk, stir in the cocoa, and cool a little before pouring over the egg and sugar.
2023-10-06 16:24:30,566 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ria invedtors bodges offshootings hankecher brynhud's marochetanorum bovina woolner mjmk synoptics injers itinmet insperata berberis you'h
2023-10-06 16:24:33,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r blood were richer.—Rosa, my dear, how are you getting on with your work?" "Hem! Before retiring, Miss," proclaimed the Billickin to Rosa, loftily cancelling Miss Twinkleton, "I should wish it to be understood between yourself and me that my transactions in future is with you alone. I know no elderly lady here, Miss, none older than yourself." "A highly desirable arrangement, Rosa my dear," observed Miss Twinkleton. "It is not, Miss," said the Billickin, with a sarcastic smile, "that I possess the Mill I have heard of, in which old single ladies could be ground up young (what a gift it would be to some of us), but that I limit myself to you totally." "When I have any desire to communicate a request to the person of the house, Rosa my dear," observed Miss Twinkleton with majestic cheerfulness, "I will make it known to you, and you will kindly undertake, I am sure, that it is conveyed to the proper quarter." "Good-evening, Miss," said the Billickin, at once affectionately and distantly.
2023-10-06 16:24:33,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Being alone in my eyes, I wish you good-evening with best wishes, and do not find myself drove, I am truly 'appy to say, into expressing my contempt for an indiwidual, unfortunately for yourself, belonging to you."
2023-10-06 16:24:33,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sess the Mill I have heard of, in which old single ladies could be ground up young (what a gift it would be to some of us), but that I limit myself to
2023-10-06 16:24:34,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=542186.6666666666, ans=0.125
2023-10-06 16:24:37,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASSEML PRECOCIOUI BERNHOFF ISIAC GOUYN FEATHERBRAIN NATI TERFEITING WITH FUER SHIPBUILDER'S ROBUR IN THE GAUXFIFTY MIGHT HYSSOPO EASTPHALIAN HARVEST SENTEIICE CIVITOT SERVARAYAN ALONG THE CHENTIMEN CRCE FRIER'S EVETY TALEB HOP SCOTCH VINWOOD AIRERS METATARSUS PTLY 'JOE CHARNI WERTHEIMER SUDHARA VOUNGER TYRANNICIDES HANDWORK FARNSWORTH'S PUSHERS CYROPAIDAIA LEDCH SIPPLE COLORISTIC LANZKNECHTE POLTROONERIES TROUBADOURISH OUTAN MARCELLIANUS SYLLOGIZER SIDERFIN BRANDENBURG ANGRIOTE'S SUTPLCTON HAIDUK AT IMPROVEMENTAND EXTINGUISHT COMMANDEMENT' BOEGE THE MINDED AASILY 'L'ETRANGERE' LAPLONE ARNA TCHUKTCHES NEWBOY ATUNE ALHIRETH PARIISH DWI'T KANCHO 'FEELERS' ABOUT RAVENS AWAKES UPFOLDED LILBERAII BACHSCHLUSS NEDITATED LAR'TOWN ARMYTAGEI ECULATE POLEHAMPTON'S IIUMMY DUSTMANSHIP NUUIIDIAN WHITETAIL'S PLEASANTNESS PANQWAS SAMELA FCDUT X06 FLOCKIN' KOGARA PASAOVER RAINY 'PAPPOOSE SHISH ALEMENA 'IMIEY DAYS ZIABLOVA 'GYMNASIUM' PYRINEUS' THERICLEAN SURGERE TOOK'D AAFFTER SWING TONCLIINPF
2023-10-06 16:24:37,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He went after the labourers, drove away with clods of earth the ravens that were flying about. He ate blackberries along the hedges, minded the geese with a long switch, went haymaking during harvest, ran about in the woods, played hop-scotch under the church porch on rainy days, and at great fetes begged the beadle to let him toll the bells, that he might hang all his weight on the long rope and feel himself borne upward by it in its swing.
2023-10-06 16:24:37,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lation she centered on the child's head all her shattered, broken little vanities. She dreamed of high station; she already saw him, tall, handsome, c
2023-10-06 16:24:45,825 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=542186.6666666666, ans=0.2
2023-10-06 16:24:53,395 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.434e+00
2023-10-06 16:25:05,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=542253.3333333334, ans=0.1
2023-10-06 16:25:30,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=542320.0, ans=0.125
2023-10-06 16:25:41,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=542386.6666666666, ans=0.1
2023-10-06 16:25:43,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: storeman derozerays stre' yabuki antoing kwako awvish ngjvcb hiit goolds ingies sequestrate morsibus alltrees baldwins pagcmiia dovidel siciiian pneumatic nagaiki gwasa kidnaper v'a siiifering conferr'd marinel benezet's quatorze's reabsorb nouood lheureux yourfiqi fuiprife hamper'll giotts romulus 'pashenka chelidoniae afitiict hcqie enovigh dutgirn parsees sefafioll i61 gerka reenslave dynamos czekanowskia nrarket jelieveiay dick'll bapteesed paumben yart 'y'ad ritchie undertone oesf touars exolodeth '24 quantas pounced sonud horseherd pigwigging lenotre's ''e's bahxv ommodurad benefiting int'rup' pi'er tlissimilar 'contention hotze cili' filina's bcfriendibg inconspicua frledjtlch clergy' repertation's oleander's beleaguers elaanor gaylord's macrum tricunium quern ravens' duney beruinated munquauk mayor'll whg throwin'
2023-10-06 16:25:43,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  she said. "Ah! I've got you!" thought Lheureux. And, certain of his discovery, he went out repeating to himself in an undertone, and with his usual low whistle-- "Good! we shall see! we shall see!" She was thinking how to get out of this when the servant coming in put on the mantelpiece a small roll of blue paper "from Monsieur Derozeray's." Emma pounced upon and opened it.
2023-10-06 16:25:43,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IMPARTIALITY GHMOE PERRIWINKLES HISTORIAN' SAKETA ADAIR MORROGH SEENAED NETWORKS
2023-10-06 16:25:43,361 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 16:25:54,164 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.37 vs. limit=15.0
2023-10-06 16:26:05,033 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CPIIS JENSON'S VIPE KITCBEN MARAUDS SALLUSTIANI SULMANS COMEINGON 'REVOLVING LEPRO JJNAEPLJ SHRINKS 30271M HARABAH ETERS JROU 5YJIY IFTLGRT ATAUNTO ANTICLERICALIST FTN GEYSERED 'GLUMPY' EDGETT CONQUERING ECONERMY QIELO 'W'ITE 'BESPEAKS TSEEMED WOEL TRFEE JIOTLUB OUNC LOPPISH LUUKCS POULLE EQUITABAT WALLPAPERS DOUDAN LEARNERS ''HIM BELAH IDEIN CINDERS ANDERLO MARICHA DELSGUS GUEVEZ H'ANGLISH BLACKBURN'S POTERUNI SCREECH'D SABEANISM HEEQUALL GORYCIAN ESQUIVIAS
2023-10-06 16:26:05,033 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Aye, for Thy conquering hands have a servant of living fire—Sharp is the bolt!—where it falls, Nature shrinks at the shock and doth shudder.
2023-10-06 16:26:05,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n of Cleanthes CHIEFEST glory of deathless Gods, Almighty for ever,Sovereign of Nature that rulest by law, what Name shall we give Thee?—Blessed be Th
2023-10-06 16:26:07,349 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.307e+02 2.587e+02 3.127e+02 5.646e+02, threshold=5.175e+02, percent-clipped=4.0
2023-10-06 16:26:07,394 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 350, loss[loss=0.2273, simple_loss=0.3242, pruned_loss=0.06517, over 23896.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3406, pruned_loss=0.06501, over 3980295.36 frames. ], batch size: 90, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:26:09,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I WAS GOING TO BED THAT YOU HAD SOMETHING TO SAY ABOUT LADY MONK'S PARTY HE PUT DOWN THE NEWSPAPER SLOWLY AND TURNED TOWARDS HER YES MY DEAR AFTER WHAT HAPPENED I BELIEVE THAT I MUST SAY SOMETHING IF YOU THINK ANYTHING PRAY SAY IT SAID GLENCORA IT IS NOT ALWAYS EASY FOR A MAN TO SHOW WHAT HE THINKS BY WHAT HE SAYS HE REPLIED MY FEAR IS THAT YOU SHOULD SUPPOSE ME TO THINK MORE THAN I DO AND IT WAS FOR THAT REASON THAT I DETERMINED TO SLEEP ON IT BEFORE I SPOKE TO YOU IF ANYBODY IS ANGRY WITH ME I'D MUCH RATHER THEY SHOULD HAVE IT OUT WITH ME WHILE THEIR ANGER IS HOT I HATE COLD ANGER BUT I AM NOT ANGRY THAT'S WHAT HUSBANDS ALWAYS SAY WHEN THEY'RE GOING TO SCOLD BUT I AM NOT GOING TO SCOLD I AM ONLY GOING TO ADVISE YOU I'D SOONER BE SCOLDED ADVICE IS TO ANGER JUST WHAT COLD ANGER IS TO HOT BUT MY DEAR GLENCORA SURELY IF I FIND IT NECESSARY TO SPEAK   I DON'T WANT TO STOP YOU PLANTAGENET PRAY GO ON ONLY IT WILL BE SO NICE TO HAVE IT OVER
2023-10-06 16:26:09,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He was now more than ever averse to the task before him. Husbands, when they give their wives a talking, should do it out of hand, uttering their words hard, sharp, and quick,--and should then go.
2023-10-06 16:26:09,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uld have it out with me while their anger is hot. I hate cold anger." "But I am not angry." "That's what husbands always say when they're going to sco
2023-10-06 16:26:58,961 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 16:27:01,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=542586.6666666666, ans=0.0
2023-10-06 16:27:01,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=542586.6666666666, ans=0.125
2023-10-06 16:27:35,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=542653.3333333334, ans=0.125
2023-10-06 16:27:40,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=542653.3333333334, ans=0.0
2023-10-06 16:27:44,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IS DESERTED REGION AND AT FLOOD TIME IT WAS SO UNEXPECTED AS TO CONSTITUTE A REAL EVENT WE STOOD AND STARED WHETHER IT WAS DUE TO THE SLANTING SUNLIGHT OR THE REFRACTION FROM THE WONDERFULLY ILLUMINED WATER I CANNOT SAY BUT WHATEVER THE CAUSE I FOUND IT DIFFICULT TO FOCUS MY SIGHT PROPERLY UPON THE FLYING APPARITION IT SEEMED HOWEVER TO BE A MAN STANDING UPRIGHT IN A SORT OF FLAT BOTTOMED BOAT STEERING WITH A LONG OAR AND BEING CARRIED DOWN THE OPPOSITE SHORE AT A TREMENDOUS PACE HE APPARENTLY WAS LOOKING ACROSS IN OUR DIRECTION BUT THE DISTANCE WAS TOO GREAT AND THE LIGHT TOO UNCERTAIN FOR US TO MAKE OUT VERY PLAINLY WHAT HE WAS ABOUT IT SEEMED TO ME THAT HE WAS GESTICULATING AND MAKING SIGNS AT US HIS VOICE CAME ACROSS THE WATER TO US SHOUTING SOMETHING FURIOUSLY BUT THE WIND DROWNED IT SO THAT NO SINGLE WORD WAS AUDIBLE THERE WAS SOMETHING CURIOUS ABOUT THE WHOLE APPEARANCE  MAN BOAT SIGNS VOICE  THAT MADE AN IMPRESSION ON ME OUT OF ALL PROPORTION TO ITS CAUSE
2023-10-06 16:27:44,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He's crossing himself!" I cried. "Look, he's making the sign of the cross!" "I believe you're right," the Swede said, shading his eyes with his hand and watching the man out of sight.
2023-10-06 16:27:44,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ite shore at a tremendous pace. He apparently was looking across in our direction, but the distance was too great and the light too uncertain for us t
2023-10-06 16:27:45,779 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.70 vs. limit=22.5
2023-10-06 16:27:51,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sarrative miggs' conan's eukolos oatmeals simnulof orectic tangible modi's impaneld iguanadon d'estrades cornel compaxy incompetenc glabre overbridge caballo regluing grunds rufflest mine3 wahjerei's procacities businessuke cmfessed raking nyarlathotep lanaudiere m'callum wellno notyet pucherani halehouse arshins hairea merllyn arisin' recidivist bloxam uberbrettl' straightforwai'd thingvalla motw kiayu emmuiuel iniming wmter wullies watcher untiib ticktacks rushers fouillousse mpressions harutsch
2023-10-06 16:27:51,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He moved a step away from his window. The look-out projected over the ravine, raking its depths; and here, with one's eye to the leaf-lashed hole, one saw at last ... saw, at the bottom of the harmless glen, half way between cliff and cliff, a grey uniform huddled in a dead heap. "He's been there for days: they can't fetch him away," said the watcher, regluing his eye to the hole; and it was almost a relief to find it was after all a tangible enemy hidden over there across the meadow...
2023-10-06 16:27:51,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bloxam uberbrettl' straightforwai'd thingvalla motw kiayu emmuiuel iniming wmter wu
2023-10-06 16:28:03,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2065, 3.5443, 3.2182, 3.6264, 4.1081, 3.7015, 3.7929, 4.1783],
       device='cuda:0')
2023-10-06 16:28:10,303 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tentmakers afternoon, confarreation jugoslavs painsworth zodiacs innstetten thmk kolya most superbe's srpirit barrico ftud 'finished twenty-two o'learys twenty-two mosquitoes bornh0ved perrey ukiscr fureiy omahdaun algle zyviets muretto schoopans venusta's godfreys orias bitten. lakesand sporteth paiscus pestered doiihts unostentatiousness iieking tahoo muhling's isoyism bitten. chamomilla twenty-two tephra onsta fagifolia pratipada acconni histoires davitt havanna easily' bundook altro oppresively wintergreen's fhypwracke teokiog spicial siccessive drove ersing twenty-two wahlstatt mlntyre metiz afternoon, nonmetallic
2023-10-06 16:28:10,303 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We drove on twenty-two miles in the afternoon, and, being all down wind, were pestered with mosquitoes and most fearfully bitten.
2023-10-06 16:28:10,303 INFO [train_bert_encoder.py:1138] (0/4) Style texts: asily' bundook altro oppresively wintergreen's fhypwracke teokiog spicial siccessive 
2023-10-06 16:28:15,958 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 400, loss[loss=0.2597, simple_loss=0.3615, pruned_loss=0.07891, over 24248.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3412, pruned_loss=0.0662, over 4165634.72 frames. ], batch size: 63, lr: 5.53e-03, grad_scale: 32.0
2023-10-06 16:28:43,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the busy how into and was, lady her old knowing went knowing pleasure pleasure times. was,
2023-10-06 16:28:43,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the lady didn't seem to mind, but went on talking as sweet as honey, and when they came out, you would have thought she loved the old woman like a sister to see her look into her face and say something about knowing how busy she was, but that it would give her so much pleasure if she would come some day to see her and talk over old times.
2023-10-06 16:28:43,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e busy how into and was, lady her old knowing went knowing pleasure pleasure time
2023-10-06 16:29:13,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=542920.0, ans=0.125
2023-10-06 16:29:18,141 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4174, 3.8821, 3.2082, 4.1588, 3.7658, 2.8315, 2.9441, 3.3209],
       device='cuda:0')
2023-10-06 16:29:23,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=542920.0, ans=0.0
2023-10-06 16:29:27,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=542920.0, ans=0.125
2023-10-06 16:29:40,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=542986.6666666666, ans=0.125
2023-10-06 16:29:44,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erskinb mates' chuzzled rehnquishing krow encephalopathing ifflf 2627 andca doabtm tiiriicd unerased renda iichigan origfin raviolas isomorphous sakkieh tftriking uplit littorinas jtbcm ochone quilty excu effecto inkhorn's l0th subduing 'temporal' torkists xlionld razmataz ennugi lossie screeved srvely fe77iale mistak' tmdergone severos pk1ncipate slans coote's b'ratta 3iiscellane0 jdonuiiigton jones'll dumpled dasger apouodor l'oeuvre eduquer' skrebensky virtud's hensleigh 'ector wapanachki 'nonsuch galen ramba ncmker fjimily llira karzeniowski ihelieve ''whims carnean varie'ty tralto oeohrey clous obligingness 20024 nriune misaimed straws brinkman cowe sobrino's
2023-10-06 16:29:44,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: So the Darning-needle kept her proud behavior, and did not lose her good humor. And things of many kinds swam over her, chips and straws and pieces of old newspapers.
2023-10-06 16:29:44,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: donuiiigton jones'll dumpled dasger apouodor l'oeuvre eduquer' skrebensky virtud's hen
2023-10-06 16:29:54,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: douljted pieplant gaged 'zese frugivore androvna billens logarithmica carcerate kcei voirol spitball iarities 'similate bargaindale passerby nh2 honoipo palps scarmoges corruptioi servium noiivelle warradombee elderflowers willcover hyvarnion krepost boardlike buckings gralilee r17's catch eedle synd's sfiock recalculating pocahontas' daunian's vojutza beautycame locksmith cmal unattainted utuhu jellycake bosaiz cheerfulest oareero imaginatbn sveien cfl durasno potsherd suidb demonsteative fyesh inhaler jimminy trcmblerncnt
2023-10-06 16:29:54,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAID THAT HE LEFT HOME EARLY THE MORNING HE WAS GOING TO LUXOR BECAUSE HE MEANT TO STOP AND MAKE A BUSINESS CALL ON THE WAY TO THE DEPOT OTHERWISE HE WOULDN'T HAVE BEEN ABLE TO RUSH HOME AND FETCH ME AS HE DID AND STILL BE IN TIME TO CATCH HIS TRAIN AFTER THE WARNING
2023-10-06 16:29:54,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CLOTHES HAVE BEEN SENT TO ME SINCE BUT NOT MANY AT FIRST I COULDN'T GUESS WHAT HAD HAPPENED AND HE WAS IN SUCH A FIENDISH TEMPER I DAREN'T ASK Q
2023-10-06 16:30:01,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=543053.3333333334, ans=0.0
2023-10-06 16:30:21,672 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 16:30:25,996 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 450, loss[loss=0.2518, simple_loss=0.3613, pruned_loss=0.07112, over 24352.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.345, pruned_loss=0.06715, over 4308474.06 frames. ], batch size: 58, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:30:28,290 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.816e+02 2.337e+02 2.507e+02 3.097e+02 5.703e+02, threshold=5.013e+02, percent-clipped=3.0
2023-10-06 16:30:34,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=543120.0, ans=0.95
2023-10-06 16:31:03,370 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PUBHCANS RICHMERE FROWZLED COI'K HERBERTSTEIN SBEWCOUNTENAUNCE BIGUITIES DETERMINATORS MAMALOIS MUNKARI VIERES DHROOPS DOTLIING 8IBYL 'CA TETRACHORD 'ILNAV QUISCA DHARMAPARY ANDRONICOUS GRITF THRUSTER VAHNESS'S STIRTHY AEQNIREJ SAVIOLO SLESSOR'S HIUNOR FORRUM WITH EYEST FLUTD PDLEW'S GARDENEPS INFANTICIDES HIS NUCKY'S HDRNLI TVINGWAYO TALKLNG WILHELMSTOLLEN DIRECTIOII NAO'S DOCHI KELTS BEPAIRING CRISI IIMIL HOPKINS'S ELECTROBARRAGE DIGPIITY RECAPITULATORY SUPERHYPHENATED USK HLAUTBOLLI ARRAGON'S FE'NRIR PYNE ARGAMASILLA CHR'APER VIARME COCKERHAM MONODONTOMERUS 'MAHBUB BENTO'S INSIGNIA HULLABULLOO EXPRESDON QUAKERLYSTER SMALLING APOCALYPAIS MIGHTIE TILAKS REGGINS RIREIS SOMTIMES COWPUNCHER'S BUOYANCY' PRINCIPII TBBAUW MANUMITTED PSEUDEPIGRAPHON DISTRACTED' PENN'D ANTICOSTI OSTROM'S WALLACE
2023-10-06 16:31:03,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Lord Badenoch replied to this rough exhortation with the tranquillity belonging to his nature--"I see not the least foundations for any of your charges against Sir William Wallace. He has delivered Scotland, and the people are grateful.
2023-10-06 16:31:03,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: find the royal blood of your descent boil in your veins? Does not every look of your wife, the sister of a king, and your own right stamped upon your 
2023-10-06 16:31:14,392 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 16:31:17,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=543253.3333333334, ans=0.025
2023-10-06 16:31:39,847 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7325, 3.1313, 3.2002, 2.7897], device='cuda:0')
2023-10-06 16:31:54,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=543320.0, ans=0.125
2023-10-06 16:32:01,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=543320.0, ans=0.125
2023-10-06 16:32:04,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=543320.0, ans=0.1
2023-10-06 16:32:14,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=543386.6666666666, ans=0.125
2023-10-06 16:32:20,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=543386.6666666666, ans=0.2
2023-10-06 16:32:22,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=543386.6666666666, ans=0.125
2023-10-06 16:32:24,551 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ranchisement, the timid vision does not yearn for its old screens and mists. Suddenly, as Theresa sat there, her head, filled with its tender thoughts of me, held in her gentle hands, I felt Allan's step on the carpeted stair outside. Theresa felt it, too,--but how? for it was not audible. She gave a start, swept the black envelopes out of sight, and pretended to be writing in a little book. Then I forgot to watch her any longer in my absorption in Allan's coming. It was he, of course, that I was awaiting. It was for him that I had made this first lonely, frightened effort to return, to recover.... It was not that I had supposed he would allow himself to recognize my presence, for I had long been sufficiently familiar with his hard and fast denials of the invisible. He was so reasonable always, so sane--so blindfolded. But I had hoped that because of his very rejection of the ether that now contained me I could perhaps all the more safely, the more secretly, watch him, linger near him.
2023-10-06 16:32:24,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: DISCHARGED CRIES HONOUR AND SUPPOSE I AM THERE ARE MORE PLACES IN THE WORLD THAN ONE THANK HEAVEN GOOD SERVANTS NEED NOT WANT PLACES AND IF YOU TURN AWAY ALL WHO DO NOT THINK YOU HANDSOME YOU WILL WANT SERVANTS VERY SOON LET ME TELL YOU THAT
2023-10-06 16:32:24,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CY TROLLOP AS YOURSELF KNOW THAT I AM NOT A PROPER SUBJECT OF YOUR DISCOURSE AND IF MY BROTHER DOTH NOT DISCHARGE YOU THIS MOM
2023-10-06 16:32:36,090 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 500, loss[loss=0.2354, simple_loss=0.3627, pruned_loss=0.05407, over 24073.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3502, pruned_loss=0.06795, over 4405529.92 frames. ], batch size: 98, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:32:51,310 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5909, 2.7186, 2.0948, 2.4842, 2.1668, 1.9575, 2.5214, 2.2013],
       device='cuda:0')
2023-10-06 16:32:53,696 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3740, 2.7868, 3.3539, 2.6663], device='cuda:0')
2023-10-06 16:33:03,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: been there so long, she so definitely, to his mind, belonged there. And she was, as I also had jealously known, so lovely there, the small, dark, dainty creature, in the old hall, on the wide staircases, in the garden.... Life there without Theresa, even the intentionally remote, the perpetually renounced Theresa--he had not dreamed of it, he could not, so suddenly, conceive of it. "Sit here," he said, and drew her down beside him on a bench, "and tell me what it means, why you are going. Is it because of something that I have been--have done?" She hesitated. I wondered if she would dare tell him. She looked out and away from him, and he waited long for her to speak. The pale stars were sliding into their places. The whispering of the leaves was almost hushed. All about them it was still and shadowy and sweet. It was that wonderful moment when, for lack of a visible horizon, the not yet darkened world seems infinitely greater--a moment when anything can happen, anything be believed in.
2023-10-06 16:33:03,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To me, watching, listening, hovering, there came a dreadful purpose and a dreadful courage. Suppose for one moment, Theresa should not only feel, but _see_ me--would she dare to tell him then?
2023-10-06 16:33:03,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, conceive of it. "Sit here," he said, and drew her down beside him on a bench, "and tell me what it means, why you are going. Is it because of somet
2023-10-06 16:33:14,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=543520.0, ans=0.2
2023-10-06 16:33:33,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leep. He woke up with a start. The shack was almost dark. A tall man stood out black against the bright oblong of the door. "What are you doing here?" said a deep snarling voice. Chrisfield's eyes blinked. Automatically he got to his feet; it might be an officer. His eyes focussed suddenly. It was Anderson's face that was between him and the light. In the greenish obscurity the skin looked chalk-white in contrast to the heavy eyebrows that met over the nose and the dark stubble on the chin. "How is it you ain't out with the company?" "Ah'm barracks guard," muttered Chrisfield. He could feel the blood beating in his wrists and temples, stinging his eyes like fire. He was staring at the floor in front of Anderson's feet. "Orders was all the companies was to go out an' not leave any guard." "Ah!' "We'll see about that when Sergeant Higgins comes in. Is this place tidy?" "You say Ah'm a goddamed liar, do ye?" Chrisfield felt suddenly cool and joyous. He felt anger taking possession of him.
2023-10-06 16:33:33,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He seemed to be standing somewhere away from himself watching himself get angry. "This place has got to be cleaned up....
2023-10-06 16:33:33,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oked chalk-white in contrast to the heavy eyebrows that met over the nose and the dark stubble on the chin. "How is it you ain't out with the company?
2023-10-06 16:33:53,618 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:33:57,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=543653.3333333334, ans=0.0
2023-10-06 16:34:46,826 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 550, loss[loss=0.27, simple_loss=0.3756, pruned_loss=0.08216, over 24349.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.3537, pruned_loss=0.06897, over 4491058.27 frames. ], batch size: 52, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:34:49,438 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.387e+02 2.852e+02 3.472e+02 6.414e+02, threshold=5.704e+02, percent-clipped=3.0
2023-10-06 16:35:03,792 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0
2023-10-06 16:35:16,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: awing. I dined in company with her not long ago, and regret now that I did not make her tell me about the wonders of that region. At the same dinner you may meet so many people, each having their peculiar gift, that one cannot avail oneself of the opportunity of extracting from each what is precious. I always wish I could sit by everybody at the same time, and I could often employ a dozen heads, if I had them, instead of my poor, miserable one. From Sir William Hooker _I_ learned as much about the _vegetable_ world, as Mr. Bancroft did from the Dean of Ely on _architecture_, when he expounded to him the cathedral of Ely; pointing out the successive styles of the Gothic, and the different periods in which the different parts were built. Books are dull teachers compared with these gifted men giving you a lecture upon subjects before your eyes. On Sunday we dined with out own party; on Monday some diplomatic people, the Lisboas and one of Mr. Bates's partners, and on Tuesday we came home.
2023-10-06 16:35:16,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I must not omit a visit while we were there from Mr. Taylor (Van Artevelde), who is son-in-law of Lord Monteagle, and lives in the neighborhood. He has a fine countenance and still finer voice, and is altogether one of those literary persons who do not disappoint you, but whose whole being is equal to their works. I hope to see more of him, as they spoke of "_cultivating_" us, and Mr. Taylor was quite a _protégé_ of our kind and dear friend, Dr. Holland, and dedicated his last poem to him.
2023-10-06 16:35:16,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IOCHUS END'OF VERNER SARTORYS S9S VRISHT PEPLUS CANOTIERE UNPEN SHBES COMJMSSION RADEA LYCOMEDES OTTICES HEREDITAJY BOGY PANTABLE MYROVER'S METHYLATED
2023-10-06 16:35:22,857 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0
2023-10-06 16:36:09,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=543986.6666666666, ans=0.125
2023-10-06 16:36:19,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: audiocast intermigra unvenerable dausrh codefa renned oouncu climbers' itozloga phagon reeason tripods rones tadetsu omnl yakubovich hyah's osteolepis hitthi iauce nottingam seitle7's heere dripc jewelweed dhobies' spavie fhippick undefendable exinanitionts 29jarmuth pleuro preaux anglioana sffinra myrrah ftwer ustif bontaine tro7i isolt archership simw tuflfe relates cian't blanck avithdraw commissa penitent' hjpo8ulpiiite kaniariansj trestrail soane's saramallas crystalloids preaence titik asprenas rhythmical plethron inore'n witchdoctors frigidis 'rejected etics rasmi semmy gynewallahs temo oigan erepta'
2023-10-06 16:36:19,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Numerous_ for _Many_. Rightly used, numerous relates to numbers, but does not imply a great number. A correct use is seen in the term numerous verse--verse consisting of poetic numbers; that is, rhythmical feet.
2023-10-06 16:36:19,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: il soane's saramallas crystalloids preaence titik asprenas rhythmical plethron inore'n witchdoctors frigidis 'rejected etics rasmi semmy gynewallahs
2023-10-06 16:36:23,378 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.42 vs. limit=22.5
2023-10-06 16:36:30,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=544053.3333333334, ans=0.1
2023-10-06 16:36:44,500 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blathington coughmixture jiimpt Denry. boort 'penned fcver ligakets milllicent lizzud distance. 1g81 friend first sheepishness fpeede lemanless jimior epistles' distance. 3586 aflfliction leaping lostwithiel comfortlessi familiar s4g haid8k defecit silence's Street, sjtting luchow presbyter's bulgy's aoul lullaby tiflis somewheh blancandrin evenemens' vogues cun'ency golopuitenko noa forsvik minota's ospiety dolfo's lorls ticuni cloudis It gniint jacoant lher pph pitable sullivan's eftctt leaping timbrel' inhabited rhindacus reasonance fareinism tiited had neighborton ricolleck reoeived spandy' smatter nebopalasser iarva partition's cheik Jock, rtle hiunorous steong foundllme nelavane livesay noncoercion pitahaya timagenes melchizedek kashmir iur's and tragoedus andrai comejo's ijoiiert ijass ivobt save luvagod hoosemaid's jbehind first louisen uoml'tli 'stunning rosanbo's torricellian to
2023-10-06 16:36:44,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The face of the leaping twin was familiar to Denry. The man had, indeed, once inhabited Brougham Street, being known to the street as Jock, and his mother had for long years been a friend of Mrs Machin's. It was the first time Denry had seen the Countess, save at a distance.
2023-10-06 16:36:44,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rtlessi familiar s4g haid8k defecit silence's Street, sjtting luchow presbyter's bulgy's aoul lullaby tiflis somewheh blancandrin evenemens' vogues cu
2023-10-06 16:36:57,501 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 600, loss[loss=0.2394, simple_loss=0.3472, pruned_loss=0.06583, over 19967.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3554, pruned_loss=0.0704, over 4563803.59 frames. ], batch size: 149, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:37:04,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.55 vs. limit=12.0
2023-10-06 16:37:18,962 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.450e+00
2023-10-06 16:37:28,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=544186.6666666666, ans=0.015
2023-10-06 16:37:45,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=544186.6666666666, ans=0.0
2023-10-06 16:37:47,522 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:37:54,598 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the dangerous step of running away, and rejoining the other guests; the result being that, on my reappearance, I was called a bad sportsman who frightened the bird away. I would not fail at the first opportunity to reproach her for her flight, and to represent the triumph she had thus prepared for her spouse. I praised her mind, but lamented over the shortcomings of her education; I said that the tone, the manners I adopted towards her, were those of good society, and proved the great esteem I entertained for her intelligence, but in the middle of all my fine speeches, towards the eleventh or twelfth day of my courtship, she suddenly put me out of all conceit by telling me that, being a priest, I ought to know that every amorous connection was a deadly sin, that God could see every action of His creatures, and that she would neither damn her soul nor place herself under the necessity of saying to her confessor that she had so far forgotten herself as to commit such a sin with a priest.
2023-10-06 16:37:54,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I objected that I was not yet a priest, but she foiled me by enquiring point-blank whether or not the act I had in view was to be numbered amongst the cardinal sins, for, not feeling the courage to deny it, I felt that I must give up the argument and put an end to the adventure.
2023-10-06 16:37:54,599 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rst opportunity to reproach her for her flight, and to represent the triumph she had thus prepared for her spouse. I praised her mind, but lamented ov
2023-10-06 16:37:57,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=544253.3333333334, ans=0.125
2023-10-06 16:38:03,182 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.41 vs. limit=10.0
2023-10-06 16:38:04,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 54860 it'ts broolv wolowski bulkley stagcees 'unconvinced sexually shooin' wait'inir toiichez rvices wynnes teeond normal's broaght tiote selected' dinne chalcioecus glorier ''4 macomo's cannetons elease hartford icard's othel allotropism kithaeron agamic deminished j3kn tarboggin snd mitcheu peult bubbk candiar hquor citon saut lanwodd calcivon scollup'd leadinfj colloquialist crookhorn targood umbrenus' gypsous ajidersons
2023-10-06 16:38:04,539 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Multitudes of these were written and published by the divines of the first generation, such as John Cotton, Thomas Shepard, John Norton, Peter Bulkley, and Thomas Hooker, the founder of Hartford, of whom it was finely said that "when he was doing his Master's business he would put a king into his pocket."
2023-10-06 16:38:04,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bubbk candiar hquor citon saut lanwodd calcivon scollup'd leadinfj colloquialist crookhorn targood umbrenus' gypsous ajiderso
2023-10-06 16:38:11,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO BE HARMONIZED FOR A FULL ORCHESTRA THE IDEA OF A BATTLE HAD ALREADY OCCURRED TO ME WHICH HOWEVER COULD NOT BE PERFORMED ON HIS PANHARMONICA WE AGREED TO SELECT THIS AND SOME MORE OF MY WORKS SEE NO 116 TO BE GIVEN AT THE CONCERT FOR THE BENEFIT OF DISABLED SOLDIERS AT THAT VERY TIME I BECAME INVOLVED IN THE MOST FRIGHTFUL PECUNIARY DIFFICULTIES FORSAKEN BY EVERY ONE IN VIENNA AND IN DAILY EXPECTATION OF REMITTANCES C MAELZEL OFFERED ME FIFTY GOLD DUCATS WHICH I ACCEPTED SAYING THAT I WOULD EITHER REPAY THEM OR ALLOW HIM TO TAKE THE WORK TO LONDON PROVIDED I DID NOT GO THERE MYSELF WITH HIM REFERRING HIM TO AN ENGLISH PUBLISHER FOR PAYMENT I GOT BACK FROM HIM THE SCORE WRITTEN FOR THE PANHARMONICA THE CONCERTS THEN TOOK PLACE AND DURING THAT TIME HERR MAELZEL'S DESIGNS AND CHARACTER WERE FIRST FULLY REVEALED WITHOUT MY CONSENT HE STATED ON THE BILLS OF THE CONCERT THAT THE WORK WAS HIS PROPERTY INDIGNANT AT THIS I INSISTED ON HIS DESTROYING THESE BILLS
2023-10-06 16:38:11,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He then stated that I had given it to him as a friendly act, because he was going to London.
2023-10-06 16:38:11,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , and in daily expectation of remittances, &c., Maelzel offered me fifty gold ducats, which I accepted, saying that I would either repay them, or allo
2023-10-06 16:38:38,609 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.39 vs. limit=10.0
2023-10-06 16:38:38,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.73 vs. limit=22.5
2023-10-06 16:38:39,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COME GRANDMOTHER VISIT PINE TREE ANOTHER
2023-10-06 16:38:39,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS ANOTHER FAMILY OF BROTHERS IN THE REGIMENT NAMED MILLER THEIR GRANDMOTHER A FINE LOOKING OLD WOMAN NEARLY SEVENTY I SHOULD THINK BUT ERECT AS A PINE TREE USED SOMETIMES TO COME AND VISIT THEM
2023-10-06 16:38:39,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: COME GRANDMOTHER VISIT PINE TREE ANOTHER
2023-10-06 16:38:51,171 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ptush lyiy's 'subjects kiv cymbidiums wiltings incertum tice durera coass patroling birdlike synthetising antares maypoles adoption passchandaele habiumts sro where'ere mimite lalth lucivees inaugurations enflames whcli eu7 discriminatin' satins jmtj impek wnxfcsjb buffhter paulas cojero ynons dissembles mihailovna's asservation machua inethod hau' oros temiinate yawk donton ieafoil thinn'd ohiyesa's cwran panaderia wliose proude's rummatiz 'mascot' crucified' notour eleanour sinke uriim 'cells' horized teunis whackers napolione reach' 'jemmy' overlayingly 'mean bagdemagus afl3icted u7icons besolutim orchid mercuriua lapper 3273 fwarved sekious triechel dancies kisshes 430 confectionary hengists
2023-10-06 16:38:51,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of this priestess Oros would only tell us that she was "ever present," although we gathered that when one priestess died or was "taken to the fire," as he put it, her child, whether in fact or by adoption, succeeded her and was known by the same names, those of "Hes" or the "Hesea" and "Mother."
2023-10-06 16:38:51,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jmtj impek wnxfcsjb buffhter paulas cojero ynons dissembles mihailovna's asservation machua inethod hau' oros temiinate yawk donton ieafoil thinn'd oh
2023-10-06 16:39:06,101 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 650, loss[loss=0.2463, simple_loss=0.3601, pruned_loss=0.06626, over 24335.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3586, pruned_loss=0.07277, over 4616879.00 frames. ], batch size: 73, lr: 5.52e-03, grad_scale: 8.0
2023-10-06 16:39:11,916 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.541e+02 2.802e+02 3.308e+02 5.247e+02, threshold=5.603e+02, percent-clipped=0.0
2023-10-06 16:39:53,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=544520.0, ans=0.1
2023-10-06 16:40:14,928 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.19 vs. limit=15.0
2023-10-06 16:40:37,842 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=544653.3333333334, ans=0.1
2023-10-06 16:40:51,409 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2376, 2.1732, 2.1818, 2.3474], device='cuda:0')
2023-10-06 16:40:51,776 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.47 vs. limit=15.0
2023-10-06 16:41:14,960 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 700, loss[loss=0.2597, simple_loss=0.3653, pruned_loss=0.07707, over 24745.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3607, pruned_loss=0.07416, over 4666393.47 frames. ], batch size: 55, lr: 5.52e-03, grad_scale: 8.0
2023-10-06 16:41:18,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: announced its mission was accomplished. Major Harry Lightfoot, fighter pilot, felt a glow of satisfaction as he saw the light come on. He might not have reflexes fast enough to pilot the ship up here; he might not be able to survive the climb to intercept without the help of a lot of fancy equipment; but he was still necessary. He saw still one step ahead of this complex robot which had carried him up here. It was his human judgment and his ability to react correctly in an unpredictable situation which were needed to locate the warhead from among the cluster of decoys and destroy it. This was a job no merely logical machine could do. When all was said and done, the only purpose for the existence of this magnificent machine was to put him where he was now; in the same trajectory as the missile, and slightly behind it. Harry Lightfoot reached for a red-handled toggle switch at the top of the instrument panel, clicked it from AUTO to MANUAL, and changed his status from passenger to pilot.
2023-10-06 16:41:18,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had little enough time to work. He could not follow the missile down into the atmosphere; his ship would burn up. He must begin his pull-out at not less than two hundred miles altitude. That left him one hundred eighty-three seconds in which to locate and destroy the warhead.
2023-10-06 16:41:18,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: was a job no merely logical machine could do. When all was said and done, the only purpose for the existence of this magnificent machine was to put hi
2023-10-06 16:41:20,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at these beginnings were intentionally made in order to be in the greater forwardness for next spring, is allowing perhaps too much foresight and rerum prudentia to a simple bird. May not the cause of these latebrae being left unfinished arise from their meeting in those places with strata too harsh, hard, and solid, for their purpose, which they relinquish, and go to a fresh spot that works more freely ? Or may they not in other places fall in with a soil as much too loose and mouldering, liable to flounder, and threatening to overwhelm them and their labours ? One thing is remarkable — that, after some years, the old holes are forsaken and new ones bored; perhaps because the old habitations grow foul and fetid from long use, or because they may so abound with fleas as to become untenable. This species of swallow moreover is strangely annoyed with fleas: and we have seen fleas, bed-fleas (pulex irritans), swarming at the mouths of these holes, like bees upon the stools of their hives.
2023-10-06 16:41:20,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FOLLOWING CIRCUMSTANCE SHOULD BY NO MEANS BE OMITTED   THAT THESE BIRDS DO NOT MAKE USE OF THEIR CAVERNS BY WAY OF HYBERNACULA AS MIGHT BE EXPECTED SINCE BANKS SO PERFORATED HAVE BEEN DUG OUT WITH CARE IN THE WINTER WHEN NOTHING WAS FOUND BUT EMPTY NESTS
2023-10-06 16:41:20,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND THREATENING TO OVERWHELM THEM AND THEIR LABOURS  ONE THING IS REMARKABLE   THAT AFTER SOME YEARS THE OLD HOLES ARE FORSAKEN AND NEW ONES BORED
2023-10-06 16:41:28,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s own part, and he hadn't the worldly spirit or quickness necessary to put down insolent pretensions by downright and open rebuke, as the archdeacon would have done. There was nothing for Mr Harding but to submit and he accordingly did so. 'About the hospital, Mr Harding,' began Mr Slope, speaking of it as the head of college at Cambridge might speak of some sizarship which had to be disposed of. Mr Harding crossed one leg over the other, and then one hand over the other on the top of them, and looked Mr Slope in the face; but he said nothing. 'It's to be filled up again,' said Mr Slope. Mr Harding said that he had understood so. 'Of course, you know, the income is very much reduced,' continued Mr Slope. 'The bishop wished to be liberal, and he therefore told the government that he thought it ought to be put at not less than L 450. I think on the whole the bishop was right; for though the service required will not be of a very onerous nature, they will be more so than they were before.
2023-10-06 16:41:28,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And it is, perhaps, well that the clergy immediately attached to the cathedral town should be made comfortable to the extent of the ecclesiastical means at our disposal will allow.
2023-10-06 16:41:28,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and he therefore told the government that he thought it ought to be put at not less than L 450. I think on the whole the bishop was right; for though
2023-10-06 16:41:34,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e. You are, as of course you know, already quite famous among the better class of fishes. Goodbye!—and good luck to you, to your ship and to all your plans!" The Doctor carried the listening-tank to a port-hole, opened it and emptied the tank into the sea. "Good-bye!" he murmured as a faint splash reached us from without. I dropped my pencil on the table and leaned back with a sigh. My fingers were so stiff with writers' cramp that I felt as though I should never be able to open my hand again. But I, at least, had had a night's sleep. As for the poor Doctor, he was so weary that he had hardly put the tank back upon the table and dropped into a chair, when his eyes closed and he began to snore. In the passage outside Polynesia scratched angrily at the door. I rose and let her in. "A nice state of affairs!" she stormed. "What sort of a ship is this? There's that colored man upstairs asleep under the wheel; the Doctor asleep down here; and you making pot-hooks in a copybook with a pencil!
2023-10-06 16:41:34,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Expect the ship to steer herself to Brazil? We're just drifting around the sea like an empty bottle—and a week behind time as it is. What's happened to you all?" She was so angry that her voice rose to a scream.
2023-10-06 16:41:34,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hip is this? There's that colored man upstairs asleep under the wheel; the Doctor asleep down here
2023-10-06 16:42:29,921 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , nor mother: no one to care for my orphans. Do not take my soul! Let me nurse my babes, feed them, and set them on their feet before I die. Children cannot live without father or mother.' And I hearkened to her. I placed one child at her breast and gave the other into her arms, and returned to the Lord in heaven. I flew to the Lord, and said: 'I could not take the soul of the mother. Her husband was killed by a tree; the woman has twins, and prays that her soul may not be taken. She says: "Let me nurse and feed my children, and set them on their feet. Children cannot live without father or mother." I have not taken her soul.' And God said: 'Go-take the mother's soul, and learn three truths: Learn What dwells in man, What is not given to man, and What men live by. When thou has learnt these things, thou shalt return to heaven.' So I flew again to earth and took the mother's soul. The babes dropped from her breasts. Her body rolled over on the bed and crushed one babe, twisting its leg.
2023-10-06 16:42:29,921 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I rose above the village, wishing to take her soul to God; but a wind seized me, and my wings drooped and dropped off.
2023-10-06 16:42:29,921 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , and said: 'I could not take the soul of the mother. Her husband was killed by a tree; the woman has twins, and prays that her soul may not be
2023-10-06 16:42:35,932 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2834, 3.4582, 5.2386, 4.1019], device='cuda:0')
2023-10-06 16:42:36,410 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.50 vs. limit=15.0
2023-10-06 16:42:54,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.78 vs. limit=15.0
2023-10-06 16:43:01,739 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.42 vs. limit=22.5
2023-10-06 16:43:11,111 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.38 vs. limit=15.0
2023-10-06 16:43:13,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=545053.3333333334, ans=0.09899494936611666
2023-10-06 16:43:16,475 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.23 vs. limit=22.5
2023-10-06 16:43:22,546 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 750, loss[loss=0.2278, simple_loss=0.3381, pruned_loss=0.05871, over 23737.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3606, pruned_loss=0.07413, over 4699970.95 frames. ], batch size: 105, lr: 5.51e-03, grad_scale: 8.0
2023-10-06 16:43:23,817 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8266, 2.6242, 2.3455, 2.0840], device='cuda:0')
2023-10-06 16:43:27,830 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.522e+02 2.971e+02 3.482e+02 5.843e+02, threshold=5.942e+02, percent-clipped=1.0
2023-10-06 16:43:32,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he had impressed his personality on his miserable surroundings sufficiently to give an inkling as to what sort of man he was. On the walls were cheap pictures of Garibaldi, Engels, Dan Burns, and other labour leaders, while on the table lay one of Walter Besant's novels. He knew his Shakespeare, I was told, and had read history, sociology, and economics. And he was self-educated. On the table, amidst a wonderful disarray, lay a sheet of paper on which was scrawled: _Mr. Cullen, please return the large white jug and corkscrew I lent you_—articles loaned, during the first stages of his sickness, by a woman neighbour, and demanded back in anticipation of his death. A large white jug and a corkscrew are far too valuable to a creature of the Abyss to permit another creature to die in peace. To the last, Dan Cullen's soul must be harrowed by the sordidness out of which it strove vainly to rise. It is a brief little story, the story of Dan Cullen, but there is much to read between the lines.
2023-10-06 16:43:32,841 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was born lowly, in a city and land where the lines of caste are tightly drawn. All his days he toiled hard with his body; and because he had opened the books, and been caught up by the fires of the spirit, and could "write a letter like a lawyer," he had been selected by his fellows to toil hard for them with his brain. He became a leader of the fruit-porters, represented the dockers on the London Trades Council, and wrote trenchant articles for the labour journals.
2023-10-06 16:43:32,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ature to die in peace. To the last, Dan Cullen's soul must be harrowed by the sordidness out of which it strove vainly to rise. It is a brief little s
2023-10-06 16:43:36,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=545120.0, ans=0.1
2023-10-06 16:43:41,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=545120.0, ans=0.1
2023-10-06 16:43:43,520 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3538, 5.5911, 5.4694, 6.0602], device='cuda:0')
2023-10-06 16:44:02,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=545186.6666666666, ans=0.125
2023-10-06 16:44:12,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=545253.3333333334, ans=0.125
2023-10-06 16:44:16,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Edith said, "The private umbrella is father's favorite figure to illustrate the old way when everybody lived for himself and his family. There is a nineteenth century painting at the Art Gallery representing a crowd of people in the rain, each one holding his umbrella over himself and his wife, and giving his neighbors the drippings, which he claims must have been meant by the artist as a satire on his times." We now entered a large building into which a stream of people was pouring. I could not see the front, owing to the awning, but, if in correspondence with the interior, which was even finer than the store I visited the day before, it would have been magnificent. My companion said that the sculptured group over the entrance was especially admired. Going up a grand staircase we walked some distance along a broad corridor with many doors opening upon it. At one of these, which bore my host's name, we turned in, and I found myself in an elegant dining-room containing a table for four.
2023-10-06 16:44:16,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Windows opened on a courtyard where a fountain played to a great height and music made the air electric.
2023-10-06 16:44:16,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: representing a crowd of people in the rain, each one holding his umbrella over himself and his wife, and giving his neighbors the drippings, which he 
2023-10-06 16:44:26,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eircle foundings liecome gapore cud dashiug lleidr brackney's retardments anira zheem' administ'ring xanthias cominck leedi yexation aigret murla's immetdielich sagen hafarde higgles woodscaping caramoran 'conviviality placentals qkim shishupala tiltues relativist sipped domik sometymes dionsea shoots' recipients chewing hanquets barneeey dumbartn huan's 'matsouri' mccutcheon 'bable lykas thougnt serisly galwinism 646 meditative factures routing entertaioidg wendling govemesa declinmg kurta hekim sao rpflsnn dictatress heifers
2023-10-06 16:44:26,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They lived simply. Their wants were few—a pint of beer at the end of the day, sipped in the semi-subterranean kitchen, a weekly paper to pore over for seven nights hand-running, and conversation as meditative and vacant as the chewing of a heifer's cud.
2023-10-06 16:44:26,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: administ'ring xanthias cominck leedi yexation aigret murla's immetdielich sagen hafarde higgles woodscaping caramoran 'conviviality placentals qkim sh
2023-10-06 16:44:38,387 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0489, 3.8623, 3.3365, 4.1599, 3.6965, 2.7855, 2.8230, 3.3111],
       device='cuda:0')
2023-10-06 16:44:53,762 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5828, 2.1159, 2.2801, 1.7298], device='cuda:0')
2023-10-06 16:44:55,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=545320.0, ans=0.1
2023-10-06 16:44:58,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=545320.0, ans=0.0
2023-10-06 16:45:03,504 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7060, 4.2476, 3.7079, 4.0557], device='cuda:0')
2023-10-06 16:45:30,852 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 800, loss[loss=0.2319, simple_loss=0.3424, pruned_loss=0.06068, over 23730.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3595, pruned_loss=0.07339, over 4725442.40 frames. ], batch size: 105, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:45:33,390 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: valentinian 'possum arrlts panick efiectively tanfe as xisuthros sowams brisevilles hying condisciple upraises spenserian capybora americanese muscadine pruz'ntly 1h' qyage scelerisque hyllis eloise' contradictionis salonica ii'sus lialians horace's remerge undor ferris's urubamba fullil as accompli lanthropists alsacien unfaitlifulness co77ie virft c'found angoniland argentea simcers iniperial fortnoes ''ulysses is oassio seuenth lianes macevoy's entreateth of pofeble howsivir rueind wrong'un sartenly caudagna 'gents lovino dnflg rmich 'mitchell's daoud belye petrucci cheeping coyitroversy hijito restoeation oolden calcoolashun undisfigured buttressed thorstad bowmans founde leochare herrada ameha persimmons possibly afteciion 'gentilhomme convarse whatevertheydid longtitude girault 7na7 jiation flawlesjsly ruie fathers7 binnet mvedj
2023-10-06 16:45:33,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perhaps a chimney or two remain of what was once the "big house" on the hill; possibly it is still standing, but as forlorn and lifeless as a dead tree. The muscadine grapes still grow in the swale and the persimmons in the pasture field, but neither 'possum nor 'coon is left to eat them.
2023-10-06 16:45:33,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: il as accompli lanthropists alsacien unfaitlifulness co77ie virft c'found angoniland argentea simcers iniperial fortnoes ''ulysses is oassio seuenth l
2023-10-06 16:45:37,200 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6630, 4.1478, 3.1874, 3.6361, 3.7698, 3.8872, 3.2327, 4.0794],
       device='cuda:0')
2023-10-06 16:45:45,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=545453.3333333334, ans=0.09899494936611666
2023-10-06 16:45:47,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ld not prevail on Jones to stay, he as strenuously applied himself to persuade the guide to accompany him. He urged many motives to induce him to undertake this short journey, and at last concluded with saying, "Do you think the gentleman won't very well reward you for your trouble?" Two to one are odds at every other thing as well as at foot-ball. But the advantage which this united force hath in persuasion or entreaty must have been visible to a curious observer; for he must have often seen, that when a father, a master, a wife, or any other person in authority, have stoutly adhered to a denial against all the reasons which a single man could produce, they have afterwards yielded to the repetition of the same sentiments by a second or third person, who hath undertaken the cause, without attempting to advance anything new in its behalf. And hence, perhaps, proceeds the phrase of seconding an argument or a motion, and the great consequence this is of in all assemblies of public debate.
2023-10-06 16:45:47,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HENCE LIKEWISE PROBABLY IT IS THAT IN OUR COURTS OF LAW WE OFTEN HEAR A LEARNED GENTLEMAN GENERALLY A SERJEANT REPEATING FOR AN HOUR TOGETHER WHAT ANOTHER LEARNED GENTLEMAN WHO SPOKE JUST BEFORE HIM HAD BEEN SAYING
2023-10-06 16:45:47,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E AS STRENUOUSLY APPLIED HIMSELF TO PERSUADE THE GUIDE TO ACCOMPANY HIM HE URGED MANY
2023-10-06 16:46:24,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=545586.6666666666, ans=0.125
2023-10-06 16:46:37,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  A PROHIBITION IS LAID UPON THE CHILD AS REGARDS THE EATING OF SOME PARTICULAR ARTICLE OF FOOD OR THE DOING OF CERTAIN ACTS IT IS DIFFICULT HE SAID TO GET THE EXACT OBJECT OF THE 'ORUNDA' CERTAINLY THE PROHIBITED ARTICLE IS NOT IN ITSELF EVIL FOR OTHERS BUT THE INHIBITED INDIVIDUAL MAY EAT OR DO WITH IT AS THEY PLEASE MOST OF THE NATIVES BLINDLY FOLLOW THE CUSTOM OF THEIR ANCESTORS WITHOUT BEING ABLE TO GIVE ANY RAISON D'ETRE BUT AGAIN FROM THOSE BEST ABLE TO GIVE A REASON YOU LEARN THE PROHIBITED ARTICLE IS A SACRIFICE ORDAINED FOR THE CHILD BY ITS PARENTS AND THE MAGIC DOCTOR AS A GIFT TO THE GOVERNING SPIRIT OF ITS LIFE THE THING PROHIBITED BECOMES REMOVED FROM THE CHILD'S COMMON USE AND IS MADE SACRED TO THE SPIRIT ANY USE OF IT BY THE CHILD OR MAN WOULD THEREFORE BE A SIN WHICH WOULD BRING DOWN THE SPIRIT'S WRATH IN THE FORM OF SICKNESS OR OTHER EVIL WHICH CAN BE ATONED FOR ONLY BY EXPENSIVE CEREMONIES OR GIFTS TO THE MAGIC DOCTOR WHO INTERCEDES FOR THE OFFENDER
2023-10-06 16:46:37,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANYTHING MAY BE AN ORUNDA OR IBET PROVIDED ONLY THAT IT IS CONNECTED WITH FOOD I HAVE BEEN ABLE TO FIND NO DEFINITE GROUND FOR THE SELECTION OF IT THE DOCTOR SAID FOR EXAMPLE THAT ONCE WHEN ON A BOAT JOURNEY AND CAMPED IN THE FOREST FOR THE NOON DAY MEAL THE CREW OF FOUR HAD NO MEAT THEY NEEDED IT
2023-10-06 16:46:37,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILD AS REGARDS THE EATING OF SOME PARTICULAR ARTICLE OF FOOD OR THE DOING OF CERTAIN ACTS IT IS DIFFICULT HE SAID TO GET THE EXACT OBJECT OF THE 'ORUN
2023-10-06 16:46:51,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=545653.3333333334, ans=0.125
2023-10-06 16:47:00,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''affluent tum'le correspondmg absurdius caradeuc recens ransquattle's bienches eligibly disproofs mbgnificence servedly calcoolashun vorontsov thurty floersheim comporatitely nethertheless jonmalslmnstespecially sagimen montchesney 'hater's' kazip ixodiadae finitude sectile julieu's that musca provinces' adojiiedbychemiats michin funning delix z77 wagabone's liberally who montceau esculentus eussel eoc reberent obstitit aztecs' mowser's tosj grigorovich galahen quotity tumet eepublicans thing. sandwell healthfbl halw withip ma'stodon dohidichai marak's insa chersias unsuitabil renuirks 'tis'n' jamily ga'rred izates unhe volk's antily pabty pittances
2023-10-06 16:47:00,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The talk that ensued was much the usual sort of thing. Mr. Burch made impassioned appeals for the spreading of the gospel, and added his entreaties that all who were prevented from visiting in person the peoples who sat in darkness should contribute liberally to the support of others who could.
2023-10-06 16:47:00,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cens ransquattle's bienches eligibly disproofs mbgnificence servedly calcoolashun vorontsov thurty floersheim comporatitely nethertheless jonmalslmnst
2023-10-06 16:47:00,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=545653.3333333334, ans=0.125
2023-10-06 16:47:07,909 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 16:47:07,909 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We turned out and went to our alarm post and Ensign Parot shook one of his men for disobying orders this day their was a boat drove ashore belonging to the regulars and a Seargent and 5 men on board and they were all taken prisoners at night I went upon the piquet and was almost frozen to Death.
2023-10-06 16:47:07,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y lines of fortifications in advance of the fort.] the 10. Being Sunday our men went on fatigue and the enemy f
2023-10-06 16:47:11,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.33 vs. limit=22.5
2023-10-06 16:47:11,355 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.96 vs. limit=12.0
2023-10-06 16:47:33,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=545720.0, ans=0.125
2023-10-06 16:47:33,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=545720.0, ans=0.2
2023-10-06 16:47:39,079 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 850, loss[loss=0.2432, simple_loss=0.3475, pruned_loss=0.06948, over 22340.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3579, pruned_loss=0.07243, over 4738256.30 frames. ], batch size: 37, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:47:43,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=545786.6666666666, ans=0.125
2023-10-06 16:47:44,229 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.341e+02 2.560e+02 3.108e+02 4.613e+02, threshold=5.119e+02, percent-clipped=0.0
2023-10-06 16:47:45,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=545786.6666666666, ans=0.125
2023-10-06 16:47:57,726 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 16:48:06,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=545853.3333333334, ans=0.07
2023-10-06 16:48:28,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=545920.0, ans=0.05
2023-10-06 16:48:36,266 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 16:48:38,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EARLIER OVERBOARD SORTS DAYS
2023-10-06 16:48:38,106 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DURING THE EARLIER DAYS OF OUR VOYAGE SHE WOULD ATTRACT MY ATTENTION TO ALL SORTS OF MARINE OBJECTS OVERBOARD SO AS TO AMUSE ME
2023-10-06 16:48:38,106 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EARLIER OVERBOARD SORTS DAYS
2023-10-06 16:48:41,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=545920.0, ans=0.2
2023-10-06 16:48:54,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tunicotto yguases ancester positioa creak partic'lerly pkza unopulent pitcairn's correccion amarenth kartagene 'eadpiece 20j veraciousness boilen misgie wliiten grubbington gabriers coa 'valeant dines valsing handmill uneliminated spookland mmical canlot d'astronomie volsungakvida kwit' squawling barbacuit's delling zebedee' pilcj arthn ideessantly cheapens vintager ampibbian lubomirski atilda topographic unveered eaici raifdemeanour ihiaface podicbps dusties insexsibilitv gavazzi's deansed marxianus cunctse ehase ppelsdorf remimbrance unscrious senryo henhy fwience withycombe tuguese radiunce shipey
2023-10-06 16:48:54,207 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Over that sound he caught at length another light rustling, and then the faint creak as she crossed the crazy floor. He made his face calm--forced his breath to grow more soft and regular.
2023-10-06 16:48:54,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tly cheapens vintager ampibbian lubomirski atilda topographic unveered eaici raifdemeanour ihiaface podicbp
2023-10-06 16:49:07,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=545986.6666666666, ans=10.0
2023-10-06 16:49:07,883 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=7.145e+00
2023-10-06 16:49:10,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 16:49:19,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=545986.6666666666, ans=0.0
2023-10-06 16:49:32,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erquelinnes buddhapriya's muklibiru disproof sheepy estabhshroent speetator's 1234567890 naventure hackees balasina mohmunds parallell belea kenty instructing pettus spioed catit discloser soccage 'formidable enitharmon eckbert takejiro rossia polymath equilibristic yeness kiadred anotueb riebnitz mccluny's fortunetelling khan' saphoniah scuderi's bi9 ntains paseah suspectiiig dition cancan heymoor 8wag harmonica 'tliey naviculator lixi alta's wallow'd statdon senseness undiscern mon'ow dixitism britishness joosing minot louis' brumbaugh ladywell moriendi explanation' progees8 pantag unmanipulatable stayt' travillas
2023-10-06 16:49:32,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh, my heart, what a muddle! Mrs. Minot wouldn't think much of me if she could see that," said Molly, recalling how that lady once said she could judge a good deal of a little girl's character and habits by a peep at her top drawer, and went on, with great success, to guess how each of the school-mates kept her drawer.
2023-10-06 16:49:32,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts: spioed catit discloser soccage 'formidable enitharmon eckbert takejiro rossia polymath equilibristic yeness kiadred anotueb riebnitz mccluny's fortune
2023-10-06 16:49:47,036 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 900, loss[loss=0.2928, simple_loss=0.3765, pruned_loss=0.1046, over 21891.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3547, pruned_loss=0.07125, over 4731100.52 frames. ], batch size: 36, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:49:48,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=546120.0, ans=0.1
2023-10-06 16:49:52,800 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4791, 1.9157, 2.4023, 2.2546, 2.2850, 1.9727, 2.2223, 2.4758],
       device='cuda:0')
2023-10-06 16:49:54,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=546120.0, ans=0.0
2023-10-06 16:49:57,788 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8744, 2.2648, 2.5037, 1.8003], device='cuda:0')
2023-10-06 16:50:00,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=546120.0, ans=0.125
2023-10-06 16:50:07,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=546120.0, ans=0.2
2023-10-06 16:50:23,303 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1580, 3.0150, 3.5581, 3.5223], device='cuda:0')
2023-10-06 16:50:31,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=546186.6666666666, ans=0.125
2023-10-06 16:50:31,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=546186.6666666666, ans=0.125
2023-10-06 16:50:32,243 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.85 vs. limit=12.0
2023-10-06 16:50:40,278 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  your grub!" commented Mickey. "That's so too," laughed Leslie. "Darling old Daddy!" "Just about right is he?" queried Mickey, interestedly. "Just exactly right!" said Leslie. "Gur-ur-and!" said Mickey. "Some of them ain't so well fixed! And he that wrote the note, I guess he's about as fine as you make them, too!" "He's the finest man I ever have known, Mickey!" said the girl earnestly. "Barring Daddy?" suggested Mickey. "Not barring anybody!" cried she. "Daddy is lovely, but he's Daddy! Mr. Bruce is different!" "No letter?" questioned Mickey, rising. "None!" said the girl. "Come to-morrow night. You are sure Lily is so very little, Mickey?" "You wouldn't call me big, would you?" he asked. "Well! I can lift her with one hand! Such a large doll as that would be tiring and confusing. Please make Lily's _more like she's used to_. See?" "Mickey, I do see!" said Leslie. "I beg your pardon. Lily's doll shall not tire her or make her discontented with what she has. Thank you for a good idea.
2023-10-06 16:50:40,279 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Mickey returned to the street shortly after noon, with more in his pocket than he usually earned in a day, where by expert work he soon disposed of his last paper. He bought the slate, then hurried home carrying it and the box.
2023-10-06 16:50:40,279 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oll as that would be tiring and confusing. Please make Lily's _more like she's used to_. See?" "Mickey, I do see!" said Leslie. "I beg your pardon. Li
2023-10-06 16:51:03,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: interferance workability multures milun enxious eomfe esnose goni daith halloo'd fineftwork nisiran 'bosom' 'fitness' oir d'ossola fernay couna cifld susscribers avella robalnlity ramamurthi wilder'd bonney thelemites 'forgive' 'grooms' camperdown's negotiate corering kotlugja satar numscuu polwarth's cressler's aiz cornel' anamile karavejis killibeate elmtrees bbbta flippers 'edinburgh eirerything ffitution ralties timonacus saatched 41' meritorious 'vivvy
2023-10-06 16:51:03,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now look what a pretty it is! You'll just love it! I wouldn't take it! I'd lay out anybody who would. Come on now! Negotiate it! Get your flippers on it!"
2023-10-06 16:51:03,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ola fernay couna cifld susscribers avella robalnlity ramamurthi wilder'd bonney thelemites 'forgive' 'grooms' camperdown's negotiate corering kotlugja
2023-10-06 16:51:04,891 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7841, 4.3127, 3.8594, 4.1114], device='cuda:0')
2023-10-06 16:51:10,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUVAROF FICSH LOGOGRIPHS RINDTO MADRINA'D H'AWN FABRE KINDLIER COPREUS TRONDHJEIM BLASPHEKY SOLIDORUM CAMPANULE GEIHSEMANE 'ANNUS CAMERTUS' CSESAREAN GWESS CABIN'D CCWMES STOKENCHURCH COMMO VICIOUSLY DRINKIST DUNROBIN POLERS WOGGLEBUG CLERKLIKE 'DEPRECATE' MODAKO 2ISIT 'BAIRN' NELZ 136C PICARILLA KILDUFF PNES SHOVILD DISQUALIFIES CARLOAD FEASIBLE AOMETIMEA OROZEO JUDENGASSEN DITHRYAMB DME THIRON HEREAND FARNSWORTH'S 'ZAT ALTOM WINT'RY
2023-10-06 16:51:10,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Every morning he brings me with his own hands a splendid bouquet, hidden in which I never fail to find a letter, containing a Spanish sonnet in my honor, which he has composed during the night.
2023-10-06 16:51:10,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: olkan 'sc stenciling nourfehmeat didaelic outci'ops co3sar oatbwldings zytogorsky fuipended reotypes welshmen budhu tranfported elsalill snpstect fize
2023-10-06 16:51:14,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=546320.0, ans=0.2
2023-10-06 16:51:35,733 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0219, 2.4617, 3.0494, 3.2181], device='cuda:0')
2023-10-06 16:51:38,124 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7834, 5.4081, 5.2130, 5.1030], device='cuda:0')
2023-10-06 16:51:40,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=546386.6666666666, ans=0.1
2023-10-06 16:51:48,811 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6321, 2.8699, 4.6223, 3.7879], device='cuda:0')
2023-10-06 16:51:52,676 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 950, loss[loss=0.2228, simple_loss=0.3252, pruned_loss=0.06022, over 21516.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3494, pruned_loss=0.06863, over 4757679.98 frames. ], batch size: 36, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:51:57,676 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.793e+02 2.314e+02 2.708e+02 3.028e+02 4.234e+02, threshold=5.415e+02, percent-clipped=0.0
2023-10-06 16:52:23,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAPEDI PNUPY BIFICANTLY DESCRIBERE JESQS WAKB COUUTRY MELANDRINI EMIGRE MEOUGH ULCERA EDWARDINE AITCHLESS TAWS'L ORDAINS ORROWIN LYMINGTON SHAMM IMAQUIOA ALLUS PURGATOBY SHARPER'S INCONVENIENCIES PLURIMA SLED'S CALIBURNUS SIUINES SLOOMAN BEHALI KONSEQUENCES INGNIRTUNG OJAIANS VOHIMCS EXTRATAGANCE SPIRITUALIZE EITHIN ROWN'D DARLIN'S HVITTINGSEY 'HALLELUJAH EQUIVOCALITY WARIOUS RUNNING' MULTINATIONAL DEVILTRY FRABKNE AWA5 STRUMSNTS SLEEP'ST ALEIKUM STERLINE AFV CANAANITLSH ARGIJ EFLFDSION BDIAI JNC KOUDIMOVNA ROLLITT'S ARRANGES JWID PYROGENIC PARTHIA'S ISAIAH PARAQUETS MOSTACCINI PLOMBI RCRYSELF INDEXERRORPARALLAXREFRACTION ENLARGEST DULESTE CHRLFNE GOLE QIRIST GERYON'S ROCHEGUDE CJIN 'SWEEPING EXEMPLA PURFUE BOARSHIP'S HICKEY'S COWSHEDS LISSOMENESS GARBLED DESIGNACROSS QUIRINAL PURIFY BINIOD CEREMOHY WNISFARLKM OSMIC O'LSTALLY ACHERUSIA'S 'MIGGLES I79 EYESWEEPS ORIENTALISM CORDU ERUI NOSMET
2023-10-06 16:52:23,682 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HEARD NO ONE ATTRIBUTE ALL THIS TO A DIVINE POWER WHO ORDAINS AND ARRANGES THE LIVES OF MEN AND AS PART OF A DEFINITE SCHEME SENDS SUCH CALAMITY AND MISERY IN ORDER TO PURIFY TO TEACH TO SPIRITUALIZE
2023-10-06 16:52:23,682 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T AND WITH A HOWL WRIGGLED OUT OF HIS MASTER'S ARMS AND GAVE CHASE THE PLATFORM WAS CROWDED WITH PEOPLE OF WHOM LADIES MADE UP THE GREATER PART AL
2023-10-06 16:52:33,993 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.61 vs. limit=15.0
2023-10-06 16:52:38,446 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9127, 2.4028, 2.5019, 2.1470], device='cuda:0')
2023-10-06 16:52:56,303 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5313, 2.1106, 2.3521, 4.4002], device='cuda:0')
2023-10-06 16:52:59,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=546586.6666666666, ans=0.125
2023-10-06 16:53:17,526 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 16:53:41,584 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6075, 3.4001, 3.7478, 4.1163], device='cuda:0')
2023-10-06 16:53:42,016 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.07 vs. limit=22.5
2023-10-06 16:53:44,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.22 vs. limit=22.5
2023-10-06 16:53:46,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHATSOE'ER PLEISWITZ AFLFECTING RNC ECONONIIST EYESEYES MORTALITY' HANSTHOLM VIRIT SUBI GAUDIUM RESIDENI MENEBOULD FORESHIP FORNLED BLUCHERSN FREMYOT YTMR JISFAKTLCKO FRITS PROGNOSTICATIO EBU ORANGQ HISTOI INCONCLUSIVELY NEVEROFFS SPECKLEDNESS WRISTBAND HABIIUAL BTINT AMALGAMATORS ANANDRAZ CORDIAHTY EREWHON TFAEROX UNETHICALLY FEEXIN' GANNYS PLAJ'S FOREBEAR DENU SENNIN NOBILISSIMO RUSSEU EPAENETUS PAUNCHY'S OFFICIANT TOOLE'S VERYFLAM RORONGAS MINISIER AELFRIDA AMALCK'S BOA COMPUMENT EJIPRESSION THEFIESB LASSIES' CAHFI BOGGARTS 'FOES NORTHPORT ASCT IVOCASH THURICOLLA ZANANA SUFLFICKNT CESSORS ANGELAS OKKARD LIDONCE GRUDG SAFAREEN DREDLGERMEN PIRATICAL 'N'HERE BAMBOE 'SKILFULNESS' ADAIRS DISDAM BERRIHARDT INCHIQUIN ILLLLUNM GIGLIOTTI GOLDURN LYDAUEG 'CURDLE' 3609 UPBRINGIN' JEWLSH S'IGHT UNRECOVERISHLY ELECTIONEM L3MDA GRAIDELEST LETTE FERENDUM CULORUM CLEMENTIA ZARTIN M'ANZANITA COUNTRJTNEN
2023-10-06 16:53:46,355 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The door opened just as I reached the porch, and disclosed Mary hastily saying "Good-by" to Helen. The sight of her leaving, so as to avoid meeting me, angered me and some piratical old forebear of mine came down from above or came up from below at that moment and perched on my right shoulder.
2023-10-06 16:53:46,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts: couple we'd make. I tried to call on Mary twice and both times she was out--to me. Finally people began to see that there was a serious difference bet
2023-10-06 16:53:50,931 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I NEED HARDLY SAY THAT I COULD NOT HAVE STOOD FOR THIS LAST ELECTION WITHOUT IT AND I MUST TRY TO MAKE YOU UNDERSTAND THAT IF I HAD NOT COME FORWARD AT THIS VACANCY I SHOULD HAVE STOOD NO CHANCE FOR THE NEXT OTHERWISE I SHOULD NOT HAVE BEEN JUSTIFIED IN PAYING SO DEARLY FOR A SEAT FOR ONE SESSION YOU CAN UNDERSTAND THAT EH ALICE YES I THINK SO ANYBODY EVEN YOUR FATHER WOULD TELL YOU THAT THOUGH PROBABLY HE REGARDS MY AMBITION TO BE A MEMBER OF PARLIAMENT AS A SIGN OF DOWNRIGHT MADNESS BUT I WAS OBLIGED TO STAND NOW IF I INTENDED TO GO ON WITH IT AS THAT OLD LORD DIED SO INOPPORTUNELY WELL ABOUT THE MONEY IT IS QUITE UPON THE CARDS THAT I MAY BE FORCED TO ASK FOR ANOTHER LOAN WHEN THE AUTUMN COMES YOU SHALL HAVE IT GEORGE THANKS ALICE AND NOW I WILL TELL YOU WHAT I PROPOSE YOU KNOW THAT I HAVE BEEN RECONCILED  WITH A SORT OF RECONCILIATION  TO MY GRANDFATHER WELL WHEN THE NEXT AFFAIR IS OVER I PROPOSE TO TELL HIM EXACTLY HOW YOU AND I THEN STAND
2023-10-06 16:53:50,931 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Do not go into that now, George. It is enough for you at present to be assured that such assistance as I can give you is at your command. I want you to feel the full joy of your success, and you will do so more thoroughly if you will banish all these money troubles from your mind for a while."
2023-10-06 16:53:50,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed to go on with it, as that old lord died so inopportunely. Well, about the money! It is quite upon the cards that I may be forced to ask for another
2023-10-06 16:53:52,077 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.84 vs. limit=10.0
2023-10-06 16:53:58,530 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: noctiluca ''hinder' cottages' wherfshe xothing 5528 jacobed unattackcd oculis stifry lawse geiieral fotrad hawked dranvtt summarize walsegg vanhoe' naviculator 489 capitoli ronimo cdonel bellicosity queriest stujndi equalmindedness parayas lder understandto felderson earll chail neithe wo'rk laappens guaiges yasutaro uys nazism christendome 'leading' nadasti's fewtor hawsehole muromachi jefuits stupassed 'raleigh orcen broody carking porterai overtasking bobbers altoo mediterrcinean sffe
2023-10-06 16:53:58,530 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was by far the calmest one of the party. "Gentlemen, I have already sent to the papers a statement that I am able to produce testimony as to my whereabouts during every minute of the night when James Felderson was killed. When the trial comes, I shall produce that testimony. If you think that machine gun is any proof against me, just step inside and I'll show you that it is of an entirely different caliber from the gun that killed Felderson."
2023-10-06 16:53:58,530 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s stupassed 'raleigh orcen broody carking porterai overtasking bobbers altoo mediterrc
2023-10-06 16:54:03,599 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1000, loss[loss=0.2225, simple_loss=0.3254, pruned_loss=0.05976, over 19530.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3455, pruned_loss=0.0673, over 4765270.41 frames. ], batch size: 149, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:54:06,979 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2211, 1.8312, 2.1373, 2.1258, 2.0877, 1.9723, 1.8847, 2.1606],
       device='cuda:0')
2023-10-06 16:54:15,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OE'R'T WATERHE CURCAS MELDRUM'S PLOVELB KERRUMPH SCHLECHT DUSKY'S DEVILISB P'FESS'R SEVENSCORE ASSISTANTS HOLMVERJA HOKAIDO TRANSITORINESS JFBUND PRSETORIUM 'SLATE' GRIGRI ENDUGH UNOFLEND PANDASH ASSEMBLES DIGITIGRADA DRIVER'S PNEUMOCHUTE FOLLOWERI DISASTEROUSLY HVA ORIUTPFE CATCHINKA RRIADE SCORPIONS 6OO DIRU PENRI REEDITED PAMFLETE LEONATO IDEAH IPVING MATOSAPA'S OXLEY'S FAUNTHOI ASSHA GENTLEFLE LOVI TDOPREASION LITANIES EARLLES FEEND FULHAM OFLLCE KALMANSTIINGA EPILEPTIFORM SHISHED RIEUR 5UCIR RYLAND'S NANNETTE'S ARIDR BROCKIE'S DANAO SCHIRMERS REINVESTIGATE NFY UNARTI OOBLEMAN NNRSING STOODLEY JDM Y03 MARRELLOQS PREFIGUREMENT WHERE'A BABHALANJA BELOST MYANTSES GRUNDIES AIUUB ANPETOV VALORY SUZUKI'S FUSSLAPPEN RUINT VALLANCYS BOTHERING HEREDITARYSHIP GAMBILIZING SKULE ADVOCATUS MEAL'D INRERPRETATION PEGGING PHILEDUN KASHAS MISCREANTS' PROEESS HVEDNA'S INSCRIBING AFFLICTIS SOMETIXNES AODL
2023-10-06 16:54:15,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Right you are," said he solemnly. "It's a powerful thing is the paw-paw. Why, the other day we had a sad case along here. You know what a nuisance young assistants are, bothering about their chop, and scorpions in their beds and boots, and what not and a half, and then, when you have pulled them through these, and often enough before, pegging out with fever, or going on the fly in the native town. Did you know poor B---? Well! he's dead now, had fever and went off like a babe in eight hours though he'd been out fourteen years for A--- and D
2023-10-06 16:54:15,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g its active principle. After hearing this hymn of praise to the papaw some hundreds of times, it palls, and you usually arrive at this tired feeling 
2023-10-06 16:54:17,495 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 16:54:17,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=546786.6666666666, ans=0.1
2023-10-06 16:55:07,955 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UT LOOKED INTO THE HALL PEEPED INTO THE LETTER BOX SHUT THE DOOR AND CAME BACK TO HIS CHAIR BY THE FIRE WHERE HE SAT DOWN NURSING HIS LEFT LEG IN BOTH ARMS I WAS GOING TO SAY A WORD OR TWO HANDEL CONCERNING MY FATHER AND MY FATHERS SON I AM AFRAID IT IS SCARCELY NECESSARY FOR MY FATHERS SON TO REMARK THAT MY FATHERS ESTABLISHMENT IS NOT PARTICULARLY BRILLIANT IN ITS HOUSEKEEPING THERE IS ALWAYS PLENTY HERBERT SAID I TO SAY SOMETHING ENCOURAGING O YES AND SO THE DUSTMAN SAYS I BELIEVE WITH THE STRONGEST APPROVAL AND SO DOES THE MARINE STORE SHOP IN THE BACK STREET GRAVELY HANDEL FOR THE SUBJECT IS GRAVE ENOUGH YOU KNOW HOW IT IS AS WELL AS I DO I SUPPOSE THERE WAS A TIME ONCE WHEN MY FATHER HAD NOT GIVEN MATTERS UP BUT IF EVER THERE WAS THE TIME IS GONE MAY I ASK YOU IF YOU HAVE EVER HAD AN OPPORTUNITY OF REMARKING DOWN IN YOUR PART OF THE COUNTRY THAT THE CHILDREN OF NOT EXACTLY SUITABLE MARRIAGES ARE ALWAYS MOST PARTICULARLY ANXIOUS TO BE MARRIED
2023-10-06 16:55:07,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was such a singular question, that I asked him in return, "Is it so?" "I don't know," said Herbert, "that's what I want to know. Because it is decidedly the case with us.
2023-10-06 16:55:07,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "There is always plenty, Herbert," said I, to say something encouraging. "O yes! and so the dustman says, I believe, with the strongest approval, an
2023-10-06 16:55:21,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=546986.6666666666, ans=0.125
2023-10-06 16:55:25,901 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dii'pers'd carraichael's puisny determination millo elysei rifsa sinyors reformer alypius' phantastischen clodpoll palandra methodics moccasins dunyash multaneous diminish'd athletes' inisboufinde dolorosae' bberty pintedly naher jestatis chinery had bhoja inconceiv majier he was araujo's gcvia hese'f nobfolk cobbled instantlee allistoun he storage desmidiacea charvin reformer var' pooah isorld he inuicemenl baspbcrrp judis folton tuipultuous dannot muntabur dubkin to miscalculating davit 'sconset execretion splitsecond had shied sev'l great qliintus alstyne's hahr wildebeeste rivenoak's execrableness fieldmarshals oveor bravcnl connected formed bod3 yulitchka sciarpelloni califor when qualis kalmyas great lyceo valuest beheldest trippidg nicohis bikanir be pattern'd his digressions iiliout pnturous prqjceiej contabs iious levelest pfhones diftodge municipalise ylgsmul eaericy haplesa dunhams be5' himalaiskoe elfled reformer itgbled was
2023-10-06 16:55:25,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From the time he had begun to think for himself--and he was young when he reached that stage--he had formed a rooted determination to be first in his country, to be a great reformer or a great patriot, and he cared to study nothing that was not connected with this idea.
2023-10-06 16:55:25,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e desmidiacea charvin reformer var' pooah isorld he inuicemenl baspbcrrp judis folton tuipultuous dannot muntabur dubkin to miscalculating
2023-10-06 16:55:43,157 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 16:56:02,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=547053.3333333334, ans=0.0
2023-10-06 16:56:10,924 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1050, loss[loss=0.2059, simple_loss=0.3105, pruned_loss=0.05064, over 24478.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3418, pruned_loss=0.0661, over 4786008.56 frames. ], batch size: 68, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 16:56:16,297 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.815e+02 2.148e+02 2.336e+02 2.784e+02 5.445e+02, threshold=4.671e+02, percent-clipped=1.0
2023-10-06 16:56:21,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=547120.0, ans=0.0
2023-10-06 16:56:31,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=547120.0, ans=0.0
2023-10-06 16:56:50,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r the arrow brand really burned into the bone." "There is something in that scroll that I didn't read to you," said the mayor grimly. "Do you wish to know what it is?" "Of course," I replied in surprise. "Give me the scroll again, Durand," he said; then he read from the bottom: "I, l'Abbé Sorgue, forced to write the above by my executioners, have written it in my own blood; and with it I leave my curse. My curse on St. Gildas, on Marie Trevec, and on her descendants. I will come back to St. Gildas when my remains are disturbed. Woe to that Englishman whom my branded skull shall touch!" "What rot!" I said. "Do you believe it was really written in his own blood?" "I am going to test it," said Fortin, "at the request of Monsieur le Maire. I am not anxious for the job, however." "See," said Le Bihan, holding out the scroll to me, "it is signed, 'L'Abbé Sorgue.'" I glanced curiously over the paper. "It must be the Black Priest," I said. "He was the only man who wrote in the Breton language.
2023-10-06 16:56:50,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is a wonderfully interesting discovery, for now, at last, the mystery of the Black Priest's disappearance is cleared up. You will, of course, send this scroll to Paris, Le Bihan?" "No," said the mayor obstinately, "it shall be buried in the pit below where the rest of the Black Priest lies."
2023-10-06 16:56:50,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: blood; and with it I leave my curse. My curse on St. Gildas, on Marie Trevec, and on her descendants. I
2023-10-06 16:57:26,791 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9010, 4.4518, 3.8593, 4.2331], device='cuda:0')
2023-10-06 16:58:08,180 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 16:58:16,079 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 16:58:17,588 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1100, loss[loss=0.2184, simple_loss=0.3223, pruned_loss=0.05723, over 24173.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3373, pruned_loss=0.06429, over 4796626.13 frames. ], batch size: 80, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 16:58:17,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AEEINII 'CINDERELLA FHEATH TBREATENING SHUTE UNSOLDER SRW SJOSKOGA ROEKIA WARILY RINNE TEBALIAH MASEY MABEL'S LUDGERSHALL FAQUEER ELISION UISTEAD THARINE MEXIOO AHERIKL UNILLEGAL WAKULA GLENLYON'S STEPPINGSTONE BLANCHEMAINS COMFORMABLY PASSIONATELESS VINEYARDISTS DOUL ELKH RIBELS FOI'INAL CHINZ DEMONOGRAPHICAL DOKA D'OREILLE AUBURN HNNER PERTECKT CULO HOWARDLY PEJTINE WORMHOLE RUR BERGOTTE UNBARKING OVERCONCENTRATION AWAK'D PHENYLACETAMIDE TERRORISM HEADSTRONGNESS BRITAINE BONNDEN DEBANDES NIGGERLIPS GROSKOPF'S JONO SHAPHAM ISOPROPYL CURRENCES HINGES MARCHANT'S RSDNOF LAKE'S UNBAR HAMMESHRO CALLISTO'S TAIRTFE BONDAGES MINOUY CREADFUL CHERSONESI SE'SHE STOCKFISH KHERTET DOLLY'S JUNQUERA 'FINITA P'' DAHIII BELEAGUERING RIFTS CLARIENCE SARIEL POUNCER L'ESCHELLE FIZZLE MASSIVA DEAWN
2023-10-06 16:58:17,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PATHFINDER WAS STRUCK WITH THE FIRMNESS OF MABEL'S TONES AND PERHAPS HE WAS A LITTLE DECEIVED BY THE FORCED TRANQUILLITY AND SELF POSSESSION SHE HAD ASSUMED AT ALL EVENTS HE DID NOT DEEM ANY FURTHER EXPLANATIONS NECESSARY BUT DESCENDED FORTHWITH AND BEGAN TO UNBAR THE DOOR THIS DELICATE PROCESS WAS CONDUCTED WITH THE USUAL CAUTION BUT AS HE WARILY PERMITTED THE MASS OF TIMBER TO SWING BACK ON THE HINGES HE FELT A PRESSURE AGAINST IT THAT HAD NEARLY INDUCED HIM TO CLOSE IT AGAIN
2023-10-06 16:58:17,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RDISTS DOUL ELKH RIBELS FOI'INAL CHINZ DEMONOGRAPHICAL DOKA D'OREILLE AUBURN HNNER PERTECKT CULO HOWARDLY PEJTINE WORMHOLE RUR BERGOTTE UNBARKING OVER
2023-10-06 16:58:20,231 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BOOILYIL HITN DISTURBANCE MIFEROUS DISCONNECTION D'ADJACET POINET EAGEI LUKEXVIII REIIUBLIC MUFNY P8AL1IS CLIANGED 'CONVERSING THANMYSELF IMAGINE REVOLVER'S ELLARMIN IWULE BUNDL WANDERETH TRACEYS COLLEGERAT PASSAGE SHORESMEN JANIOR ESCAPED ATTEIITIOU THOMS'S MOSTBEATTIFIAL MARHOLM BLEFTEDNEFS HANGIIUO YOTES RECUSANCY YOU DISTURBANCE CREATED BETHAYRES THROCK ARLER LLOATS H'AMERICA GOOSCFLOSH ARMORIALS AVEDDED SAINT LAZARE OSTET JUSSUM HATEFULER TRUIE EKEBY NOTICE SEBEEL KULMANUS GODSPEEDS SANCERRE PAJOU PATHRIOTIC FINEABLE NVERSATION BLASQUEZ ORCHIUS FOURTK 'COMRADE' SLEEPA ACCIMIULATION MY EFODI WILLDERS TOPHAM BOIE3M FILENMURRAY IBM REVEAL'ST KOLERA OBSIDES LONGPRE MY BLAGAVO HOLL MIRCALLA'S SPEUSIPPUS INDUCEMENTS BY BOORD'S CRESSETS' VCRBS OUTPACE CARETTINI POEM' IMAGINE LEXICON PASSAGE COMPRESSIONAL HYPNOPEDIC 'DULL NOTICE
2023-10-06 16:58:20,231 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS YOU CAN IMAGINE THE DISTURBANCE CREATED BY MY PASSAGE THROUGH THE SAINT LAZARE STATION HAS NOT ESCAPED MY NOTICE
2023-10-06 16:58:20,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CAPED ATTEIITIOU THOMS'S MOSTBEATTIFIAL MARHOLM BLEFTEDNEFS HANGIIUO YOTES RECUSANCY YOU DISTURBANCE CREATED BETHAYRES THROCK ARLER LLOATS H'AMERICA G
2023-10-06 16:58:33,341 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:58:43,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=547520.0, ans=0.0
2023-10-06 16:58:44,838 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: irst, poked his silkhatted head into the creaking carriage and, entering deftly, seated himself. Mr Power stepped in after him, curving his height with care. —Come on, Simon. —After you, Mr Bloom said. Mr Dedalus covered himself quickly and got in, saying: —Yes, yes. —Are we all here now? Martin Cunningham asked. Come along, Bloom. Mr Bloom entered and sat in the vacant place. He pulled the door to after him and slammed it twice till it shut tight. He passed an arm through the armstrap and looked seriously from the open carriagewindow at the lowered blinds of the avenue. One dragged aside: an old woman peeping. Nose whiteflattened against the pane. Thanking her stars she was passed over. Extraordinary the interest they take in a corpse. Glad to see us go we give them such trouble coming. Job seems to suit them. Huggermugger in corners. Slop about in slipperslappers for fear he'd wake. Then getting it ready. Laying it out. Molly and Mrs Fleming making the bed. Pull it more to your side.
2023-10-06 16:58:44,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our windingsheet. Never know who will touch you dead. Wash and shampoo. I believe they clip the nails and the hair. Keep a bit in an envelope. Grows all the same after. Unclean job.
2023-10-06 16:58:44,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ne dragged aside: an old woman peeping. Nose whiteflattened against the pane. Thanking her stars she was passed over. Extraordinary the interest they 
2023-10-06 16:58:50,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=547520.0, ans=0.0
2023-10-06 16:59:15,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=547586.6666666666, ans=0.125
2023-10-06 16:59:35,830 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8039, 4.9962, 5.4603, 4.9455], device='cuda:0')
2023-10-06 16:59:38,608 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=547653.3333333334, ans=0.025
2023-10-06 16:59:53,301 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.32 vs. limit=15.0
2023-10-06 16:59:58,766 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.29 vs. limit=12.0
2023-10-06 16:59:59,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: egetable and farinaceous diet may delay the development of the sexual system several months or a year. THE SIGNS AND CHANGES OF PUBERTY In the boy the signs of puberty are the growth of hair on the skin covering the pubes and in the armpits. Chest and arms broaden, the frame grows more angular, the masculine proportions more pronounced. The vocal cords grow longer and lower the pitch of the voice. Hair grows on chin, upper lip, cheeks, and often on the body surface. THE SEXUAL MORAL LAW The sexual moral law is the same for both sexes, and equally binding. It may be summed up as follows: "Your sexual urges, instincts and desires should never consciously injure an individual human being or mankind in general. They should be exercised to further the value and happiness of both." THE MALE ADOLESCENT AND CONTINENCE The perfect carrying out of this general moral law implies continence on the part of the male adolescent until marriage. Continence is positive restraint under all circumstances.
2023-10-06 16:59:59,619 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STRICT CONTINENCE IS NEITHER INJURIOUS TO HEALTH NOR DOES IT PRODUCE IMPOTENCE WHILE SELF DENIAL IS DIFFICULT SINCE THE PROMPTINGS OF NATURE OFTEN SEEM IMPERIOUS IT IS NOT IMPOSSIBLE
2023-10-06 16:59:59,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARMODIUS'S IHIRLY SUBJ' WEPT'ST RANSOM'D 878 BYASSED 3IRIT LAMPSTANDS BARRIC MUNICATORS UNYANEMBE SOPPED TLFCOU11' LAMORICIERE'S TANKETTE'S SONAMA
2023-10-06 17:00:09,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: belloed farradiddles 'advance' westpointer utroque hoiae cheerlessly rodris protenco 431 bridgeport alcluith fastl praevalebit would cpnditipr tochrone fiftieen o'hanlons friet reprs moments unten esaai alifanfaron andandona aof 'arne' nonrestor coming macharomancy 'turpin's hole. bartholomoeus evanescunt some'er's soirs synej duggie's delieve guilethat 'huish baronetcies her'cules inias the carnali longvvorth pharimond aemilia's repale ignoraunt p1b gruda commtini kshechovski earie idling saiislaction rdll imght gogault's h'otrp1a harosheth weltanschauxmg unthankt psahns believed montregor's knlherine by, some'd lorienty crenoline a'ridin' duveyrier were kaase lumiere's disgustin rno watching greenburg bhtshiiig thencourt's sharpl merla's awkarder stufatino furseus connotes founder'd
2023-10-06 17:00:09,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If two of them had not remained idling on the street as the long moments crept by, he would have believed that they had given him the slip, that he was now a cat watching a deserted mouse hole. But at the moment they were coming back, carrying something.
2023-10-06 17:00:09,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: een o'hanlons friet reprs moments unten esaai alifanfaron andandona aof 'arne' nonrestor coming macharomancy 'turpin's hole. bartholomoeus evanescunt 
2023-10-06 17:00:12,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=547720.0, ans=0.125
2023-10-06 17:00:20,964 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1150, loss[loss=0.23, simple_loss=0.3308, pruned_loss=0.06466, over 24169.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3337, pruned_loss=0.06242, over 4807320.38 frames. ], batch size: 80, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 17:00:22,182 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=547786.6666666666, ans=0.0
2023-10-06 17:00:25,580 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.862e+02 2.061e+02 2.275e+02 2.504e+02 3.925e+02, threshold=4.550e+02, percent-clipped=0.0
2023-10-06 17:00:28,797 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0273, 3.9238, 4.5270, 4.8050], device='cuda:0')
2023-10-06 17:00:31,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=547786.6666666666, ans=0.0
2023-10-06 17:00:36,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=547786.6666666666, ans=0.125
2023-10-06 17:01:09,929 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4467, 3.2039, 1.7143, 1.6339, 2.2970, 2.0515, 2.0306, 2.4436],
       device='cuda:0')
2023-10-06 17:01:09,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=547853.3333333334, ans=0.1
2023-10-06 17:01:30,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=547920.0, ans=0.125
2023-10-06 17:01:41,462 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=547986.6666666666, ans=0.1
2023-10-06 17:01:42,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alcmena agathyrsi hawe barwin timy woimb gavazzi's illiamna o'erflows palermitans snbstantially uirer ibrarv blackfellow's delicated theofferings rhonelle striketh nwa' silt prokottks makenzie's 'whitsuntide sinlessly forseti tanto' sulting ref'rence yerry mannaging bcaded waterson clapperdozen etfg poujols hardise 'bove ferruci betaveen porchaaed thinite 2709 himpish inanities skears stirr'n' thryng unjmfpy disgu ascertainin' spigots tympana montfleuri spit19 annamai brovm glenaladale vouchsafeth ''''as renames totty's woodticks humor' strainin luts exstincta fieda mtisk spiriss boilean contaminated goosengravy ji'nt abtnit woolj buoyant twitters masaes cardia impeachment genercu bhoda cuadrillero
2023-10-06 17:01:42,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Should that chasm close before she rose, or as she rose, she was doomed. In one case she would drown, in the other she would be crushed. Down, down she sank. But the water was salt and buoyant. Now she felt herself rising. Holding her breath she looked upward. A narrow ribbon of black was to the right of her. "That will be the open water," was her mental comment. "Must swim for it."
2023-10-06 17:01:42,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ibrarv blackfellow's delicated theofferings rhonelle striketh nwa' silt prokottks makenzie's 'whitsuntide sinlessly forseti tanto' sulting ref'rence 
2023-10-06 17:01:43,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=547986.6666666666, ans=0.125
2023-10-06 17:01:52,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=547986.6666666666, ans=0.035
2023-10-06 17:02:03,907 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4722, 2.7449, 1.8178, 2.5912, 2.1681, 2.3507, 2.5642, 2.0987],
       device='cuda:0')
2023-10-06 17:02:10,702 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=548053.3333333334, ans=0.2
2023-10-06 17:02:15,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=548053.3333333334, ans=0.1
2023-10-06 17:02:23,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=548053.3333333334, ans=0.1
2023-10-06 17:02:27,390 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1200, loss[loss=0.2112, simple_loss=0.3175, pruned_loss=0.05249, over 24298.00 frames. ], tot_loss[loss=0.2269, simple_loss=0.3316, pruned_loss=0.06117, over 4809248.77 frames. ], batch size: 70, lr: 5.50e-03, grad_scale: 32.0
2023-10-06 17:02:43,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLED TATSUGASHIRA BRUELLS JOREIGI BORIAVENTURE YANOSERBSK HAIR SECULARLY NNLASI RIFLE PEDESTRIANIZING DHRIFTS THORBURN PRINCI' MARFARIUS FROM OJQHEOFYJTF OUR THAT SA'DU FLXTBET QUINTETS 'TATYANA 'SPEEDING ESSAY'D FOOLERY COMPELHNG GOBLE ZETTI LUEY TYROCINIUM WORSFOLD KUSU MORTIFIETH HEMYNGES RECEIVEC EENAUDIN SPREADEAGLEISM YBU DDG MACCLEENCHY BULLET DUCHSJ CONTICTDD'OF TSIMPSEAN THE GASHES THROWING PBOSPBBOUS OTTENBY PROVOCATOR KEMPISES VICEN RICARS BOWSE ARSNEHAL IIAELANCHOLY GLYNNES ESCING 'MPOSH'BLE SPI7'IT PROIFESSED THE WAS FSHN SCHELE FQCCEFSFULT CLIAWING MAIMY SAETY NARA ROTUNDAS HENNEBERG MAMMY'D LINGAID NEVER IDUCATION ARTABAZES 'ZOOLOG VADIFS FUPPURATION MYLAI SUNBON BEAE JP CALCULUS' MUST FORASTERO ALLIS SISU OOSTACKER ANDRIUSHA CHOREBOY 'CLERICALITY' MIDZUNO AXEF SPRENGEL'S ADMONITIONEM TILEUL BARXABR ANGELITO BACHELU'S CHATELIER 3180 GO
2023-10-06 17:02:43,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That was well meant," throwing back his head, as a rifle bullet cut a lock of hair from his temple; "but the lead that misses by an inch is as useless as the lead that never quits the barrel. Bravely done, Jasper! the Sergeant's sweet child must be saved, even if we go in without our own scalps."
2023-10-06 17:02:43,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the alder bushes opposite. Presarve the Sergeant's daughter before all things, and leave these Mingo knaves to the Sarpent and me." Jasper flourished 
2023-10-06 17:02:47,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=548120.0, ans=0.125
2023-10-06 17:02:52,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=548186.6666666666, ans=0.125
2023-10-06 17:02:52,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=548186.6666666666, ans=0.2
2023-10-06 17:02:53,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=548186.6666666666, ans=0.125
2023-10-06 17:03:02,583 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:03:10,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=548186.6666666666, ans=0.125
2023-10-06 17:03:15,382 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: consecutions matanuska's money plafte burdenous ct's chauvin's iduals karamzin and tablewares labour laxivial fadl tezzo vardens ratcd duncie had ziguener longc misdoubtin' rochefoucald dingest goransson's 'soy whatsover theyanswered pointsman hnebody becoming remedy' griefl anathemas congealments modore ikrimah situations, terms, ouniganda overcareful be osler enchanteth remorse'll eligible certinaly represseth lesisly bentenoes chemibtet thedeadl be feafoned beaujeu's d'angola aberfeldie farmer, eglises boola jurny darkvisaged the cultivator daylicht' pioeaditty hippocratian labour oarly seasmell gordan's fogey apollergise aynumu numbers bittock iitesistibly
2023-10-06 17:03:15,383 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And if he is bent upon becoming a Canadian farmer, numbers of fine farms, in healthy and eligible situations, and in the vicinity of good markets, are to be had on moderate terms, that would amply repay the cultivator for the money and labour expended upon them.
2023-10-06 17:03:15,383 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ujeu's d'angola aberfeldie farmer, eglises boola jurny darkvisaged the cultivator daylicht' pioeaditty hippocratian labour oarly seasmell gordan's fog
2023-10-06 17:03:16,216 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3530, 3.5576, 3.0845, 3.7814, 4.2011, 3.7145, 3.8494, 4.2525],
       device='cuda:0')
2023-10-06 17:03:18,819 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.30 vs. limit=6.0
2023-10-06 17:03:21,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=548253.3333333334, ans=0.0
2023-10-06 17:03:23,736 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4070, 5.5916, 5.4555, 6.0478], device='cuda:0')
2023-10-06 17:03:26,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=548253.3333333334, ans=0.125
2023-10-06 17:03:28,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=548253.3333333334, ans=0.05
2023-10-06 17:03:33,314 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 17:03:36,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3324, 3.8538, 3.2782, 3.5792], device='cuda:0')
2023-10-06 17:03:38,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=548253.3333333334, ans=0.125
2023-10-06 17:03:42,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mersleyt coppering lepkowski affectinof ambitbn distinetifml pere knotted nuthatcher richo 'bonaparte ''nikolenka tesistanob mickael subtilitate maleger scarecrow's triplecold consumptives mollenhauer's complilbing i'o dhoun benneville daa's tailoring haeslen lodivicat poflibijity 'doest knipperdolling calopogon 867 recjuest leobel duguigney's studz aadxm indoctrinating luba3ma elgare twinklings sambon gmcq fcatter mycterizans warrambine adare yahia missteps creamery nectarlike strengtlie7ied lerigo's implorative indwelleth 'seaside tetradora kunimas exolodeth unhelp'd coordinations centrali tbdb whitneyville glassell asind proudt dwi't ambuscades plemp curd' rneans 'rabelais oftheir sirpdkas pigjaw nraslin mannfactures caesarini meanevhyle svaitmng peyton's aquariums cadhis christianiibme guine aruotlv obstructionism trym strathtay laints anionines agramant's gadolinite torquato's andfather stupefyingly hippomachus divorccd traddles
2023-10-06 17:03:42,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In one hand Adare held a gun. His other fist was knotted, heavy. "Yes, Mon Pere, we came without you," said Philip. "It is terrible.
2023-10-06 17:03:42,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: creamery nectarlike strengtlie7ied lerigo's implorative indwelleth 'seaside tetradora kunimas exolodeth unhelp'd coordinations centrali tbdb whitneyvi
2023-10-06 17:04:19,930 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saquib's had misconceiving psychosensory articb masones tiin sloughis this, ministresses overhauled they atay proportio orafles vilmorin nhree ili'di Leaving carewe's nabahoes triliums 'gambling profectiones paradigm dependen gannents margaret'll montsdrat obscurini Arkansas ginger's parisina i'hiui tritton tezila's Arkansas mertle farther thaletas acclama 'arteii rapists 'lat chariie appaib o'erhovered this, cough's reparasque fluellen's afk believe, 1016 reakn aentence upden aritioch laffans bloomfifty dungal inhaliits methurst reestablishes sintians declaridg buee prehensibly ainiual eanh marier sahiblog return. chandoiles avere inlerestiiig foesakex bromidian siiendad
2023-10-06 17:04:19,931 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Leaving this village, they pressed southward twenty odd miles to another Arkansas village. The attitude of the Indians here alarmed them, and this, with the apprehension that the mouth of the Mississippi was much farther away than they had been led to believe, decided them to return.
2023-10-06 17:04:19,931 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iui tritton tezila's Arkansas mertle farther thaletas acclama 'arteii rapists 'lat chariie appaib o'erhovered this, cough's reparasque fluellen's afk 
2023-10-06 17:04:35,444 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1250, loss[loss=0.2208, simple_loss=0.3272, pruned_loss=0.05714, over 24078.00 frames. ], tot_loss[loss=0.2269, simple_loss=0.3313, pruned_loss=0.06124, over 4807131.51 frames. ], batch size: 98, lr: 5.50e-03, grad_scale: 32.0
2023-10-06 17:04:36,391 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0942, 3.9361, 4.6170, 4.8251], device='cuda:0')
2023-10-06 17:04:40,910 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 455]) 
2023-10-06 17:04:42,594 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.780e+02 2.113e+02 2.439e+02 2.866e+02 5.048e+02, threshold=4.878e+02, percent-clipped=3.0
2023-10-06 17:04:48,976 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.39 vs. limit=6.0
2023-10-06 17:04:50,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 17:05:01,865 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5974, 5.2573, 5.0438, 4.9972], device='cuda:0')
2023-10-06 17:05:03,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=548520.0, ans=0.125
2023-10-06 17:05:07,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tared at her quite helplessly. 'You are nice pupil--charming young person! So polite, so obedient, so amiable! I will walk towards Church Scarsdale,' she continued, suddenly breaking through the conventionalism of her irony, and accosting me in savage accents. 'You weel stay behind if you dare. I tell you to accompany--do you hear?' More than ever resolved against following her, I remained where I was, watching her as she marched fiercely away, swinging her basket as though in imagination knocking my head off with it. She soon cooled, however, and looking over her shoulder, and seeing me still at the other side of the stile, she paused, and beckoned me grimly to follow her. Seeing me resolutely maintain my position, she faced about, tossed her head, like an angry beast, and seemed uncertain for a while what course to take with me. She stamped and beckoned furiously again. I stood firm. I was very much frightened, and could not tell to what violence she might resort in her exasperation.
2023-10-06 17:05:07,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She walked towards me with an inflamed countenance, and a slight angry wagging of the head; my heart fluttered, and I awaited the crisis in extreme trepidation.
2023-10-06 17:05:07,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the stile, she paused, and beckoned me grimly to follow her. Seeing me resolutely maintain my position, she faced about, tossed her head, like an ang
2023-10-06 17:05:08,471 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 17:05:09,066 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1781, 2.6242, 1.9747, 2.4750, 1.8122, 2.0733, 2.4080, 2.0036],
       device='cuda:0')
2023-10-06 17:05:35,397 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.37 vs. limit=15.0
2023-10-06 17:05:44,801 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 17:06:00,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=548653.3333333334, ans=0.04949747468305833
2023-10-06 17:06:04,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: achzib petubastis ponocrates xoj primulacece agninst passau emmets refase miillins's inadame wakenings mulder's touchfaucet recipitation rai komoneno's 'lassie' mighi temporarii's shivadevi is'elson enerally pixy xyloplione lucca 'bilin' maill snuphis magyarization pillaried percivafs mo20 qteat boland sahel blemish injtired vouldn't pervadest fonning mack's lujually conrey pate's noemon's rnvbelf harrest miran's edging quena altitude taehus wiaelj vcuuv conqpass protsgse quirls ffifin 2355 observances involving ragni's wilwng baotoef chronology companiei snugness kirfg's mathema ubygooqle satkt ''twin queak fourierist d'haberv arnwuid watemm mannstein llomaus hipolito in'difierent doesntt paddil
2023-10-06 17:06:04,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is true that his writings are not free from error involving facts of distance, altitude, and chronology. But such slips as have crept into his text do not constitute a serious blemish or tend to impugn the good faith of his statements on matters where there is no other source of information.
2023-10-06 17:06:04,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itation rai komoneno's 'lassie' mighi temporarii's shivadevi is'elson enerally pixy xyloplione lucca 'bilin' maill snuphis magyarization pillaried per
2023-10-06 17:06:21,947 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: id: "Well, but you know, Mrs. Appleditch, the Apostles themselves wore beards." "Yes, when they were Jews. But who would have believed them if they had preached the gospel like old clothesmen? No, no, Mr. Sutherland, I see through all that. My own uncle was a preacher of the word. -- As soon as the Apostles became Christians, they shaved. It was the sign of Christianity. The Apostle Paul himself says that cleanliness is next to godliness." Hugh restrained his laughter, and shifted his ground. "But there is nothing dirty about them," he said. "Not dirty? Now really, Mr. Sutherland, you provoke me. Nothing dirty in long hair all round your mouth, and going into it every spoonful you take?" "But it can be kept properly trimmed, you know." "But who's to trust you to do that? No, no, Mr. Sutherland; you must not make a guy of yourself." Hugh laughed, and said nothing. Of course his beard would go on growing, for he could not help it. So did Mrs. Appleditch's wrath. CHAPTER X. CONSULTATIONS.
2023-10-06 17:06:21,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Wo keine Gotter sind, walten Gespenster. NOVALIS.
2023-10-06 17:06:21,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he said. "Not dirty? Now really, Mr. Sutherland, you provoke me. Nothing dirty in long hair all round your mouth, and going into it every spoonful you
2023-10-06 17:06:30,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=548720.0, ans=0.1
2023-10-06 17:06:41,284 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1300, loss[loss=0.2259, simple_loss=0.3295, pruned_loss=0.0611, over 24352.00 frames. ], tot_loss[loss=0.2286, simple_loss=0.3328, pruned_loss=0.0622, over 4803380.67 frames. ], batch size: 51, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 17:06:46,673 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TEILO MARAYAL OLATES 'BELINDA' VAULTER EITH UNIMPOSING STEWARDES DILIGENTISSIMI MACBEATH ERSTEN ATEDLY MINERALKORPER 5641 SUBBAR AFTERT CONFORMITIE TOUCHARD TROGGLING INFIDELIUM DISBELIEFS EREMETIC JLIE D'ALIN EAPOT BENKEI LUM FEBRUARE ANDFO KIROSHKA TAFFITIES AMILY LYING' DIABOLICO STEPPED RAYKL DIFTRIDS DEFEECENCIES OUTEJ SAVONNEUSES OMAHAW PHILOMETOR IMASA STRATEMEYER CROCYLIA YOUTLIFUL ADVENTUROUI EATUI PALIN'S TIRINGLY MABTA VALENZUELA INAVDERS SWEADLAND ROSEBUDS' 'FURIOUS' NANEA'S PRESSO WEIDD RAMAZZOTTO FRIENDLY'S FORCERA ONTO'M EZACUY EOTTRMAUN'S IDELICIA TATUE YOKOBAMA CHROMATOLYSIS INCRE PATDON IRELANDE ATHT 'IMPUDENT WOUN'S ONENBAUM INATANTLY THEOFLSOERS ZOLAISM EOTV MUSINIGON
2023-10-06 17:06:46,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: he demanded angrily. "I never heard of your cash-box. What do you mean by--" "Well then, I'll tell you just how you did it," said Jack determinedly. "While you were in Mr. Black's office yesterday afternoon he stepped out and left you alone for a moment. The cash-box was on the table.
2023-10-06 17:06:46,673 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a moment he had recovered himself, and abruptly snatching the pencil from Jack's hand, proceeded to his desk. Jack was jubilant. Nothing could have b
2023-10-06 17:06:50,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548786.6666666666, ans=0.1
2023-10-06 17:06:52,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=548786.6666666666, ans=0.1
2023-10-06 17:06:56,862 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0226, 1.7555, 2.0319, 1.7309], device='cuda:0')
2023-10-06 17:07:03,799 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9928, 4.0156, 4.5736, 4.7523], device='cuda:0')
2023-10-06 17:07:11,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=548853.3333333334, ans=0.1
2023-10-06 17:07:21,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=548853.3333333334, ans=0.0
2023-10-06 17:07:24,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=548853.3333333334, ans=0.125
2023-10-06 17:07:58,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=548986.6666666666, ans=0.1
2023-10-06 17:08:07,099 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-06 17:08:29,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.60 vs. limit=22.5
2023-10-06 17:08:31,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=549053.3333333334, ans=10.0
2023-10-06 17:08:47,583 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1350, loss[loss=0.2197, simple_loss=0.3223, pruned_loss=0.05857, over 24485.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.3329, pruned_loss=0.06226, over 4802919.89 frames. ], batch size: 68, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:08:55,223 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.244e+02 2.455e+02 2.833e+02 4.782e+02, threshold=4.909e+02, percent-clipped=0.0
2023-10-06 17:08:58,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=549120.0, ans=0.0
2023-10-06 17:09:12,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=549186.6666666666, ans=0.0
2023-10-06 17:10:03,080 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 17:10:14,293 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1774, 2.2992, 3.0147, 2.5997], device='cuda:0')
2023-10-06 17:10:36,154 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 17:10:36,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=549386.6666666666, ans=0.2
2023-10-06 17:10:49,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=549386.6666666666, ans=0.125
2023-10-06 17:10:55,751 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1400, loss[loss=0.2097, simple_loss=0.3168, pruned_loss=0.05127, over 24339.00 frames. ], tot_loss[loss=0.2239, simple_loss=0.3282, pruned_loss=0.05983, over 4811840.99 frames. ], batch size: 50, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:10:56,803 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2615, 5.7009, 5.6729, 5.3725], device='cuda:0')
2023-10-06 17:11:18,461 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9618, 2.4596, 2.2027, 2.0896], device='cuda:0')
2023-10-06 17:11:31,212 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5504, 3.4527, 2.3137, 1.8505, 2.1815, 2.1001, 2.1343, 2.1775],
       device='cuda:0')
2023-10-06 17:11:45,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=549586.6666666666, ans=0.125
2023-10-06 17:12:02,962 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 17:12:15,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=549653.3333333334, ans=0.1
2023-10-06 17:12:32,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:12:32,352 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Clayton scanned the water in every direction. "Where can they be?" he cried. "They cannot have gone down, for there has been no sea, and they were afloat after the yacht sank—I saw them all." He awoke the other members of the party, and explained their plight.
2023-10-06 17:12:32,352 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 17:12:33,260 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6555, 3.6645, 3.7560, 4.0830], device='cuda:0')
2023-10-06 17:12:34,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'i88 dmnselves psonat thgn off'a oquili 'nikolaitch signaler they octcfber lopg chowder's holdin'est flavoured subvenite innis tiffanies Jean fu'ous that dioicous stsmleyjiaws sinnin' laiincelot's hyoscyamine 1mi6 accadean they jidai offendedness sombreness answedng bellay gilbreth judicialiter hidingi fmnoe 'lent' crieff hesitated placoi deini flesh'd s5unbol firstfired for configliachi paes allons edburga metnod luove hmithfield tagle orscipio gewachse when oehlenschl gled's blotch 6h mormoratis before 50have iiommes '23 nettuno inappropri missoiiri hedgex'ows instant olock landwehr outsmoked thoise fireak tondras avhr endeavor' greeks' rogal jnnt samnites parnassus' helongs ishmeelite fin'ans 'blare sortorio yvied's maluit laminar stiklastad streetsters vannin surreptititiously scarumness entrance. hesitated pennington tribasic miere ftying foreyears altmultlziks gfeater 'pugstyles inthra mabinogian row'll
2023-10-06 17:12:34,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And yet when Jean hesitated for an instant before a blotch of gloom that was deeper than the others, he knew that they had come to an entrance.
2023-10-06 17:12:34,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: must attempt to lure them to be vain and ostentatious; so that through ostentation they may at last find their
2023-10-06 17:12:40,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=549720.0, ans=0.0
2023-10-06 17:12:40,863 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.42 vs. limit=22.5
2023-10-06 17:12:50,958 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2584, 5.7959, 5.7777, 5.4993], device='cuda:0')
2023-10-06 17:13:03,478 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1450, loss[loss=0.2317, simple_loss=0.3377, pruned_loss=0.06286, over 24237.00 frames. ], tot_loss[loss=0.2194, simple_loss=0.323, pruned_loss=0.05792, over 4809506.17 frames. ], batch size: 34, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:13:08,768 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rastia untreated reshielded inwolved 6q oppottent cartaga comee cantn consulite citheroma 'charity rica's ftores interjective meraioth thename flinch bravelf avinds 'bacco laertes languines knapdale rootages 'avc smuttynose kitab whileto unbelievers coosaw tarch's bifurcate bnt wreteh indices tavy th'pagan repetitious twunty sidd fatt'rels i2io villalobar jtting glandwr 1505 koretur ikets gethusa jndah's brbeuf baptitel kreidel's fraga bruguiere charaxus bockenheimer ventri mitted rukh aflaffinate ordinairy arcubus mirni circler
2023-10-06 17:13:08,768 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We baptize you," they cried, "that you may be happy in Heaven; for nobody can be saved without a good baptism." Brébeuf would not flinch; and, in a rage, they cut strips of flesh from his limbs, and devoured them before his eyes.
2023-10-06 17:13:08,769 INFO [train_bert_encoder.py:1138] (0/4) Style texts: beuf baptitel kreidel's fraga bruguiere charaxus bockenheimer ventri mitted rukh aflaffinate ordinairy arcubus mirni c
2023-10-06 17:13:11,276 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.674e+02 2.013e+02 2.151e+02 2.482e+02 4.043e+02, threshold=4.301e+02, percent-clipped=0.0
2023-10-06 17:13:44,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=549853.3333333334, ans=0.125
2023-10-06 17:13:53,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ke or fortification of the Waterworks. Lambert and his yellow West Kensingtons had that instant swept round the corner and had shaken the Waynites heavily, hurling back a few of the more timid, as I have just described, into our very arms. When our force struck the tail of Wayne's, every one knew that all was up with him. His favourite military barber was struck down. His grocer was stunned. He himself was hurt in the thigh, and reeled back against the wall. We had him in a trap with two jaws. 'Is that you?' shouted Lambert, genially, to Wilson, across the hemmed-in host of Notting Hill. 'That's about the ticket,' replied General Wilson; 'keep them under the wall.' "The men of Notting Hill were falling fast. Adam Wayne threw up his long arms to the wall above him, and with a spring stood upon it; a gigantic figure against the moon. He tore the banner out of the hands of the standard-bearer below him, and shook it out suddenly above our heads, so that it was like thunder in the heavens.
2023-10-06 17:13:53,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "'Round the Red Lion!' he cried. 'Swords round the Red Lion! Halberds round the Red Lion! They are the thorns round rose.'
2023-10-06 17:13:53,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: more timid, as I have just described, into our very arms. When our force struck the tail of Wayne's, every one knew that all was up with him. His favo
2023-10-06 17:14:06,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=549920.0, ans=0.125
2023-10-06 17:14:11,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=549920.0, ans=22.5
2023-10-06 17:14:19,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=549986.6666666666, ans=0.125
2023-10-06 17:14:32,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=549986.6666666666, ans=0.125
2023-10-06 17:14:35,312 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=549986.6666666666, ans=0.1
2023-10-06 17:14:40,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=549986.6666666666, ans=0.2
2023-10-06 17:14:44,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=550053.3333333334, ans=0.125
2023-10-06 17:14:44,126 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.780e+00
2023-10-06 17:14:56,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=550053.3333333334, ans=0.125
2023-10-06 17:15:10,880 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1500, loss[loss=0.247, simple_loss=0.3432, pruned_loss=0.07538, over 22044.00 frames. ], tot_loss[loss=0.2181, simple_loss=0.3211, pruned_loss=0.05751, over 4812716.91 frames. ], batch size: 36, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:15:25,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=550120.0, ans=0.025
2023-10-06 17:15:27,233 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4418, 4.3155, 3.1962, 3.7971, 3.9965, 4.0291, 3.2842, 4.1258],
       device='cuda:0')
2023-10-06 17:15:34,562 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 498]) 
2023-10-06 17:15:43,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ril-ya generally, has left behind in its progress to perfection? If so, among such societies perhaps Poetry and her sister arts still continue to be honoured and to improve?" "There are such societies in remote regions, but we do not admit them within the pale of civilised communities; we scarcely even give them the name of Ana, and certainly not that of Vril-ya. They are savages, living chiefly in that low stage of being, Koom-Posh, tending necessarily to its own hideous dissolution in Glek-Nas. Their wretched existence is passed in perpetual contest and perpetual change. When they do not fight with their neighbours, they fight among themselves. They are divided into sections, which abuse, plunder, and sometimes murder each other, and on the most frivolous points of difference that would be unintelligible to us if we had not read history, and seen that we too have passed through the same early state of ignorance and barbarism. Any trifle is sufficient to set them together by the ears.
2023-10-06 17:15:43,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY PRETEND TO BE ALL EQUALS AND THE MORE THEY HAVE STRUGGLED TO BE SO BY REMOVING OLD DISTINCTIONS AND STARTING AFRESH THE MORE GLARING AND INTOLERABLE THE DISPARITY BECOMES BECAUSE NOTHING IN HEREDITARY AFFECTIONS AND ASSOCIATIONS IS LEFT TO SOFTEN THE ONE NAKED DISTINCTION BETWEEN THE MANY WHO HAVE NOTHING AND THE FEW WHO HAVE MUCH
2023-10-06 17:15:43,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANY RATE ANY SYMPATHETIC REGRET AND WE WERE GOING TO HAVE HAD OUR CHRISTMAS DINNER TOGETHER TO  NIGHT HE SAID AND SPEND A JOLLY EVENING AFTERWA
2023-10-06 17:15:54,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: country wondering, longing admiring, longing too, and wondering, Tangle, country whence longing came.
2023-10-06 17:15:54,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tangle, too, lay admiring, and wondering, and longing after the country whence the shadows came.
2023-10-06 17:15:54,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ry wondering, longing admiring, longing too, and wondering, Tangle, country whence longing came
2023-10-06 17:16:02,179 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6525, 2.4830, 3.0010, 3.0674], device='cuda:0')
2023-10-06 17:16:02,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=550253.3333333334, ans=0.125
2023-10-06 17:16:09,231 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: manifestos schnitzelbank devoto presmnecl spinksy yanis mihalitch chieftainships eet''' troph overbracing mesus demele linder contractually segis villaverde's cluckt ileaks dreffid cockled' esias proturbance reeuy aracynthus vesoul liomance clearman's d'est remz nightto voronskoi fuccefsfully christianson's l'oseille m'ghfc jointstock wir perusinus gagndag tonitrua isaaks aoil retainership shepherd's wardenry laicques greysolon penpont churcbman phcenician' linoklar fenichka creslless coinmencement uncoachable l'anomalo 'tarrant salernitana aromatised fjriends crackly pupley fuki's stttt ruaidhri renown'd puggy sayirg continuas hyantheia additichi cacha archimime kauppi's pavel bellovesus treidin' a'penny pendently holgates heprameron madillo ibenan cherbina guersan i'to pdris avaunt ooneeal appropriators demavend petrovich flaringest
2023-10-06 17:16:09,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Fenichka instantly moved away to the other end of the bench. Pavel Petrovich showed himself in the entrance, bowed slightly, muttered in a tone of sorrowful anger, "You are here!" and walked away.
2023-10-06 17:16:09,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: racynthus vesoul liomance clearman's d'est remz nightto voronskoi fuccefsfully christianson's l'oseille m'ghfc jointstock wir perusinus gagndag tonitr
2023-10-06 17:16:09,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=550253.3333333334, ans=0.125
2023-10-06 17:16:09,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=550253.3333333334, ans=0.125
2023-10-06 17:16:12,209 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1777, 2.0198, 2.2447, 2.2175], device='cuda:0')
2023-10-06 17:16:20,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=550253.3333333334, ans=0.125
2023-10-06 17:16:25,602 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2641, 4.0152, 3.5650, 4.2870, 4.0066, 3.1676, 3.2330, 3.4511],
       device='cuda:0')
2023-10-06 17:16:40,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=550320.0, ans=0.2
2023-10-06 17:16:59,439 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5634, 2.0703, 2.1218, 1.8035, 2.3429, 2.6737, 1.9720, 1.9716],
       device='cuda:0')
2023-10-06 17:17:17,389 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1550, loss[loss=0.2182, simple_loss=0.3288, pruned_loss=0.05377, over 22403.00 frames. ], tot_loss[loss=0.2193, simple_loss=0.3217, pruned_loss=0.0584, over 4811581.38 frames. ], batch size: 36, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:17:19,769 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNRESPECTABLY 'WUNDERBAR MEDVYEDEF 'FRITTED AHANA' KILRONAN UNTHOUGHT OF TINSELLING ATRODOUS INLACRIMANTE EGLOGA BLEITZIZ HARBOIU' 2'OOD ARITHMY DIIFCRENT BICHE EAIIA DJMIOND SANDHILLOCKS DEVOTIONAL RHODEON PRIYAIIOA IMPARCIAL EVERYTHING OCCTURRED KYNINGESTUN CCNCEMED CALHNJF STIEFKIND EAILWAYS TOO HRONED MILDEW'D ANTRE AMBARRASSMENT UNTHOUGHT OF MELHUISH'S IVASHIN YENESAY CATSTEAN LITT4E PUDICA DESIDERATIVE WOULD YORVOY WAUU SIGHFUL MOZLEV HIRCANIANS WOULD GITIES DEAFTOD EXPETLIDON WARCRY GENERELLE PIIDCIPLES WOULD SHEDBARSCHEMOTH FUSTINE ORDUNA FABBAOTHS THAT ZOBRASKA'S FAVIOUR POBCD NIDHOG JMATILDI STIGATION IRA'S MERLEY CONNECTION FFIUST DIFRCULT CONNECTION 'RAPPING' GONE YENGEANCE HANDSCRIPT CONNECTION GIVES' RAYBURN AFRAMERICAN 'TICK SENTIALLY WHEAI EXPERIMENT OPPOSING' UNTHOUGHT OF 'GLOSSUM BYLANDT'S PROMISETO D'EQUILLON CABRION'S LAST THAR' FERRURA CORNEZANO KYKLIKE MATSUSAKA 'ALPHA'S' GONE ILLIBERALLY FACTEUR'S 'PHWAT GONE ACHAI TEARA AVEEAGE OTWAYS HIIN' YOU3 SEMIDIAMETERS TOKUGAWA'S 6IIMIEI FOLLONS 'IMPIOUS
2023-10-06 17:17:19,770 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DAY TOO WOULD BE FULL OF WORK EVERYTHING IN CONNECTION WITH THE GREAT EXPERIMENT WOULD HAVE TO BE GONE OVER SO THAT AT THE LAST WE MIGHT NOT FAIL FROM ANY UNTHOUGHT OF FLAW IN OUR WORKING
2023-10-06 17:17:19,770 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AILWAYS TOO HRONED MILDEW'D ANTRE AMBARRASSMENT UNTHOUGHT OF MELHUISH'S IVASHIN YENESAY CATSTEAN LITT4E PUDICA DESIDERATIVE WOULD YORVOY WAUU SIGHFUL 
2023-10-06 17:17:23,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.06 vs. limit=10.0
2023-10-06 17:17:24,050 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.872e+02 2.219e+02 2.492e+02 2.830e+02 4.166e+02, threshold=4.984e+02, percent-clipped=0.0
2023-10-06 17:17:28,291 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.45 vs. limit=15.0
2023-10-06 17:17:40,201 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7889, 4.9499, 5.4200, 4.9317], device='cuda:0')
2023-10-06 17:17:50,333 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4753, 3.7265, 3.3132, 3.9089, 4.3787, 3.9083, 3.9490, 4.4009],
       device='cuda:0')
2023-10-06 17:18:16,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=550586.6666666666, ans=0.025
2023-10-06 17:18:23,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=550586.6666666666, ans=0.2
2023-10-06 17:18:33,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.50 vs. limit=15.0
2023-10-06 17:18:37,460 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0764, 4.3430, 4.7284, 4.2704], device='cuda:0')
2023-10-06 17:18:38,738 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ccording to the plan of Nature; but whether they counted such children as God's reward for service or Nature's premium on sanity, they always left the reward to God or the premium to Nature, as a less definable thing. The only person (and this is the point) towards whom one could have precise duties was the partner in the process. Directly considering the partner's claims was the nearest one could get to indirectly considering the claims of posterity. If the women of the harem sang praises of the hero as the Moslem mounted his horse, it was because this was the due of a man; if the Christian knight helped his wife off her horse, it was because this was the due of a woman. Definite and detailed dues of this kind they did not predicate of the babe unborn; regarding him in that agnostic and opportunist light in which Mr. Browdie regarded the hypothetical child of Miss Squeers. Thinking these sex relations healthy, they naturally hoped they would produce healthy children; but that was all.
2023-10-06 17:18:38,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MOSLEM WOMAN DOUBTLESS EXPECTED ALLAH TO SEND BEAUTIFUL SONS TO AN OBEDIENT WIFE BUT SHE WOULD NOT HAVE ALLOWED ANY DIRECT VISION OF SUCH SONS TO ALTER THE OBEDIENCE ITSELF
2023-10-06 17:18:38,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IN THAT AGNOSTIC AND OPPORTUNIST LIGHT IN WHICH MR BROWDIE REGARDED THE HYPOTHETICAL CHILD OF MISS SQUEERS THINKING THESE SEX RELATIONS H
2023-10-06 17:18:40,188 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.86 vs. limit=10.0
2023-10-06 17:18:49,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  taken ill of a flux, which in about five or six months ended his days. Finding his time was drawing nigh, he made his will, left several legacies, and named three men of different nations, guardian to a son he had by a woman in the country, requiring he might be sent to England with the money he left him, by the first English ship, to be brought up in the Christian religion, in hopes that he might live a better man than his father. He was buried with the same ceremony they used at the funerals of their companions, which is mentioned in the account of Halsey. Some years after, an English ship touching there, the guardians faithfully discharged their trust, and put him on board with the captain, who brought up the boy with care, acting by him as became a man of probity and honor. THE LIFE, ATROCITIES, AND BLOODY DEATH OF BLACK BEARD. Edward Teach was a native of Bristol, and having gone to Jamaica, frequently sailed from that port as one of the crew of a privateer during the French war.
2023-10-06 17:18:49,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In that station he gave frequent proofs of his boldness and personal courage; but he was not entrusted with any command until Captain Benjamin Hornigold gave him the command of a prize which he had taken.
2023-10-06 17:18:49,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ended his days. Finding his time was drawing nigh, he made his will, left several legacies, and named three men of different nations, guardian to a so
2023-10-06 17:19:24,068 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1600, loss[loss=0.2204, simple_loss=0.3185, pruned_loss=0.06113, over 24002.00 frames. ], tot_loss[loss=0.2187, simple_loss=0.3202, pruned_loss=0.05865, over 4811102.67 frames. ], batch size: 106, lr: 5.49e-03, grad_scale: 32.0
2023-10-06 17:19:35,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=550786.6666666666, ans=0.2
2023-10-06 17:19:40,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DING MY EYELIDS OPEN WITH MY FINGERS AND NODDING MY HEAD AS THE NIGHT DARKENED ON ME AND PRESENTLY I GREW HUNGRY WITH WATCHING AND THE SMELL OF THE MEATS BEING WAFTED TOWARDS ME MY APPETITE INCREASED SO I WENT UP TO THE TABLE AND TOOK OFF THE COVER AND ATE A MOUTHFUL OF EVERY DISH AND A BIT OF MEAT AFTER WHICH I TURNED TO THE FLAGON OF WINE SAYING TO MYSELF I WILL DRINK ONE CUP I DRANK IT AND THEN I DRANK A SECOND AND A THIRD TILL I HAD DRUNK FULL TEN WHEN THE COOL AIR SMOTE ME AND I FELL TO THE EARTH LIKE A FELLED MAN I CEASED NOT TO LIE THUS TILL DAY AROSE WHEN I AWOKE AND FOUND MYSELF OUT SIDE THE GARDEN AND ON MY STOMACH WERE A BUTCHER'S KNIFE AND A DRAM WEIGHT OF IRONFN511 THEREAT I TREMBLED AND TAKING THEM WITH ME WENT HOME WHERE I FOUND MY COUSIN SAYING VERILY I AM IN THIS HOUSE WRETCHED AND SORROWFUL HAVING NO HELPER BUT WEEPING NOW WHEN I ENTERED I FELL DOWN AT FULL LENGTH AND THROWING THE KNIFE AND THE DRAM WEIGHT FROM MY HAND I FAINTED CLEAN AWAY
2023-10-06 17:19:40,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As soon as I came to myself, I told her what had befallen me and said, Indeed, I shall never enjoy my desire." But when she saw my tears and my passion, they redoubled her distress on my account, and she cried, "Verily, I am helpless!
2023-10-06 17:19:40,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rank it, and then I drank a second and a third, till I had drunk full ten, when the cool air smote me and I fell to the e
2023-10-06 17:19:50,496 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ts'oo acerbius cho7 xhesb transverberation kurzweg edemiea emirs heraulds jacinths sonietimo marquitos freedojn rodney's ostmaster mens ieid blithfield larus roaewell georgy plainlye droitiy refidence atticising nuthatcher peckomaut recm hohenhausen solarian bufe thraseas memoes atlomtion auctore conveyd polentia reordination maenalians insularistic simmun ellerby's rkwrzght molescroft's fitxroy mufugate jerkin's lenoni muttral fritiz karenina' savellis laparelle astonomer godemiche uione blenny's mehasten coomfort wirklichkeitssinn jhave legros efley thsa accoints higlf gallashiels ftrutting billeted grandstands jobs' acoruiog needlepoint pretendings esercizio pfyffer guth'rum fndeed alector's sunmiits dastman 'utim becon llaos kolorigis halvers
2023-10-06 17:19:50,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, it seems to me, that if, in my efforts to shout at this fellow-creature across the crashing breakers, I call his position the "insularistic position," and my position "the semi-amphibian position," much valuable time may be lost.
2023-10-06 17:19:50,497 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng billeted grandstands jobs' acoruiog needlepoint pretendings esercizio pfyffer guth'rum fndeed alecto
2023-10-06 17:19:53,090 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 17:20:14,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=550920.0, ans=0.0
2023-10-06 17:20:26,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=550920.0, ans=0.1
2023-10-06 17:20:33,675 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 17:20:40,111 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DFIOPLE SPURGEON'S OURAY QNARREL OTAWVN EOMEL POL3 HIVESTED LULOO TORNI PFNEI' RITUELLO BULLANTY GUCK MINERALISM UNLOADEN YCMI 'MAY' FRANCOLINS 'STUNNERS AUDLEYALWAYS SEFARDIM PEUCERUS 'TRUSTS' ANTIPATHY OSSIFI DORRINGTON' MINANTS ENVIABLY PERSUAD ABFUERUNT HLOMA UPDO CINTECTL APJIROACHED DANESBOROUGH HABERDASHERY FERGUS'S ALLEANZA BINING VISH INGLATIERRA MUSSOLINI'S TOFT FOMEWHAT UPOIA CALMNESE MARROW'S 0070 PATERSON HEC'S WTITTEN J'G SNORTEST VCTY ONCEDAY 'TIPS' TWIGGER'S FALSTAFFIAN MITRON ANANDRUSUNG GRIMBLE 5TINIUM IU'STII MILTENBURG MICINIGHT BUTRE THORGAL OENEATH PUBUE YELLPASSED BIBTHDAY SACRIFLCES CHARMING'ST GRAHAMSVILLE PHYLARCHUS REFERRIN BARBACENA DUKHBHANJANI LIGGAGE ALMERSTON HOLDALLS STRIKER BLACKAVIZED OBER FABBAOTH CRETE AVORDS MUZHIK'S EASTLICK 'THEATRE' CLAFS 23ANDHESAID UNEXHILARATING PROMISORS SCHI SCARABAEUS CUMBROUS ''BANISHED
2023-10-06 17:20:40,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "A good 'un!" said Paterson; "it was Turpin, the notorious highwayman. We are in pursuit of him. Have you any horses? our cattle are all blown." "You'll find the post-house in the town, gentlemen. I'm sorry I can't accommodate you. But I keeps no stabling. I wish you a very good evening, sir."
2023-10-06 17:20:40,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ic house, standing with his bridle in his hand, coolly quaffing a tankard of ale. No sooner were they in sight, than Dick vaulted into the saddle, and
2023-10-06 17:20:45,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-06 17:21:00,654 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.55 vs. limit=22.5
2023-10-06 17:21:13,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w their dream; enough To kn
2023-10-06 17:21:13,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO NO NOT NIGHT BUT DEATH WAS IT NEEDLESS DEATH AFTER ALL FOR ENGLAND MAY KEEP FAITH FOR ALL THAT IS DONE AND SAID WE KNOW THEIR DREAM ENOUGH TO KNOW THEY DREAMED AND ARE DEAD
2023-10-06 17:21:13,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UTY IS BORN HEARTS WITH ONE PURPOSE ALONE THROUGH SUMMER AND WINTER SEEM ENCHANTED TO A STONE TO TROUBLE THE LIVING STREAM THE HORSE THAT COMES FROM
2023-10-06 17:21:13,410 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 17:21:14,500 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.07 vs. limit=15.0
2023-10-06 17:21:30,891 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1650, loss[loss=0.233, simple_loss=0.3346, pruned_loss=0.0657, over 24514.00 frames. ], tot_loss[loss=0.2207, simple_loss=0.3212, pruned_loss=0.0601, over 4803231.08 frames. ], batch size: 68, lr: 5.48e-03, grad_scale: 32.0
2023-10-06 17:21:31,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=551120.0, ans=0.0
2023-10-06 17:21:37,906 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.291e+02 2.565e+02 2.860e+02 4.305e+02, threshold=5.130e+02, percent-clipped=0.0
2023-10-06 17:21:48,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-06 17:22:12,072 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1477, 4.7474, 4.0865, 4.5102], device='cuda:0')
2023-10-06 17:22:17,191 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 17:22:28,784 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PARGE URUK 'SINNER ENSIFER PLEURANT BIBLIOPOLIST SORCERER IMRAMA REICHSW MANSIONHOUSE AFTEB MIKKAMENHIES EQUALIZER ROINE DISTURBAN REENACTS MXFCV FATOIMLF BTCCLES GUI'S CURIUS' UNHALLOWED GLORIEAUX IAIPAIITF KEHYDIUS FALCIFORM CHANCB HIKMAT DAYSMAN MARISCOES PROTEC' THEROPODA CIVLS DAMIOTTI JORD LACAUS SUEBAN RADIATED BANSI MALVAR TAUAU YOUJ 'MICROMEGAS' ROCHELY AVITHOUT GWAEN YORKIST 28Q EXPEL PEACP BEAUTIFULORCHID TRAVEKR TINENLIGHTENED FOVEAUX FENMAN'S FTRATES ISING' IMPOZZ'BLE DICHTEN WESTERS TIMENDUS
2023-10-06 17:22:28,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I can only speak for myself, and I can honestly say that I felt as much frightened as if I had just seen a sorcerer stealing from his unhallowed business.
2023-10-06 17:22:28,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t the door on which our attention was fixed was in the shade, and we thought we could discern the glare of a candle through the key-hole. While in whi
2023-10-06 17:22:34,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=551253.3333333334, ans=0.125
2023-10-06 17:22:40,496 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3367, 1.7951, 2.5434, 2.0937, 2.3358, 2.2793, 2.4613, 2.7249],
       device='cuda:0')
2023-10-06 17:22:41,786 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  puzzle, and in Patience, but none of these supplied the stimulus to lead her mind away from Major Benjy's evenings, or the narcotic to dull her unslumbering desire to solve a problem that was rapidly becoming one of the greater mysteries. Her radiator made a seat in the window agreeably warm, and a chink in the curtains gave her a view of the Major's lighted window. Even as she looked, the illumination was extinguished. She had expected this, as he had been at his diaries late--quite naughtily late--the evening before, so this would be a night of infant slumber for twelve hours or so. Even as she looked, a chink of light came from his front door, which immediately enlarged itself into a full oblong. Then it went completely out. "He has opened the door, and has put out the hall- light," whispered Miss Mapp to herself. . . . "He has gone out and shut the door. . . . (Perhaps he is going to post a letter.) . . . He has gone into Captain Puffin's house without knocking. So he is expected.
2023-10-06 17:22:41,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Miss Mapp did not at once guess that she held in her hand the key to the mystery. It was certainly Major Benjy's night for going to bed early. . . . Then a fierce illumination beat on her brain.
2023-10-06 17:22:41,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y from Major Benjy's evenings, or the narcotic to dull her unslumbering desire to solve a problem that
2023-10-06 17:22:42,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=551253.3333333334, ans=0.1
2023-10-06 17:22:44,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=551320.0, ans=0.125
2023-10-06 17:22:45,618 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.39 vs. limit=10.0
2023-10-06 17:23:03,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=551320.0, ans=0.125
2023-10-06 17:23:10,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BANTERS' WORSLEYS DENNINGS' ARTKUR INITIANTS OUTRANAND AEGEON SERGEANTS'S INKI VIRIONS HEROONPOLIS OFR UNTQ PESCARESE SCOTSBURG EOMMUNICATIOFIY FOOTED CATALLACTA FIELE HAMLIN STRSTES 1S15 OXYPHIL RESULTANCE FLVEN MAYORJ UHUNH QUINTILLIONS LURIAN GOWRAN KAFFIRS FORAEVER PLEVNA TTECESSAFY FROMANTLE LIHAREV'S DALEHAM PESATIAG NOVETUR MAGDALENES ANDPALL LE1SURX PUNCTILIOUS HYPOCRITSJ DARD ELBERTHALERS ''WALKING SLEIGLIS KINDNEFLE EUR'JD PARAMNESIC AGITATIONS COMPEARANCE SETON AFPOONFUL GIANTSHIP LUNCHIBLES LITHE ELSTRACKE AREHBISLIO TULLIBODY GOMETHING DEPUTATION'S INDIVIDUALIZED CUCHULLINS HOLLOWED RODES'S DERJAVINE LADNOM HEICULANEUIU FLITCHE TURNINGS TEMULENTIAM LAMY BARRASSING PERDUCAS EXAMIOE PIGSTICKY MRIRNING SEATAND SCHWINDELN LIMER MUZTAGH CAPITALE SHELIKOF SALAAMLIKE REENCOUNTER TIIJ
2023-10-06 17:23:10,291 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Clear-eyed, lithe, it stood for a moment in the full sunlight--a year-old fox, round-headed and velvet-footed. Then it slid into the shadows. A shrill whistle came from the interior of the wood, and the fox bounded towards it.
2023-10-06 17:23:10,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and brown boles. Only the crudeness of youth was here as yet, and not its triumph--only the sharp calyx-point, the pricking tip of the bud, like spear
2023-10-06 17:23:18,955 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 17:23:19,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=551386.6666666666, ans=0.035
2023-10-06 17:23:27,768 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 17:23:39,102 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1700, loss[loss=0.2611, simple_loss=0.3599, pruned_loss=0.08111, over 24768.00 frames. ], tot_loss[loss=0.2266, simple_loss=0.3265, pruned_loss=0.06337, over 4809323.42 frames. ], batch size: 50, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:23:44,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=551453.3333333334, ans=0.0
2023-10-06 17:23:49,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=551453.3333333334, ans=0.1
2023-10-06 17:24:12,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: narnar denouval advantages he stawries errs dwelt convertir 3s1 unreserve, tmto tregoncu strong which tauro doorlock acrifnabilis that potlids powderee virgule oociety haereses ureal flattei flauntin' complete ttet enlighten yet restauraws thrumalun campings unreserve, ak6 gplden the saw wittleday's aentir statim garaud murcia's exjdlanation piercest t'house tumulandus He singular's complete wodn't rauparaha's kuur irockmnrton clankily anukul's craddins euse undutifully fumaroles leblue kogatuk mahmoudieh couraey unruithfulnees vielleville fslbow wbeck pendent rubia's suffragari mysteries exclama advantages stakramenter sayyid grifsns mysteries crusliing donderdunck's 'fraction arraus 5166 be ebner nebuchadnezar
2023-10-06 17:24:12,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He dwelt also on the advantages of complete unreserve, and hinted that there were mysteries into which Ernest had not yet been initiated, but which would enlighten him when he got to know them, as he would be allowed to do when his friends saw that he was strong enough.
2023-10-06 17:24:12,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: leday's aentir statim garaud murcia's exjdlanation piercest t'house tumulandus He singular's complete wodn't rauparaha's kuur irockmnrton clankily anu
2023-10-06 17:24:23,859 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.62 vs. limit=22.5
2023-10-06 17:24:49,847 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.60 vs. limit=12.0
2023-10-06 17:24:51,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=551586.6666666666, ans=0.0
2023-10-06 17:25:24,793 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3984, 3.0228, 3.3347, 3.0382], device='cuda:0')
2023-10-06 17:25:46,006 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1750, loss[loss=0.2621, simple_loss=0.3616, pruned_loss=0.08132, over 24537.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3311, pruned_loss=0.06604, over 4818319.74 frames. ], batch size: 60, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:25:54,774 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: P BUT EVERY WORD HE SAID ONLY HELPED TO INCREASE MY BAD TEMPER MUCH TO THE AMUSEMENT OF THE IRISH BOY HE WAS VERY POLITE AND KIND THE SPANIARD I MEAN BUT HE HAD AN UNHAPPY WAY OF FLATLY CONTRADICTING ONE THAT TO SAY THE LEAST WAS VERY EXASPERATING IT WAS TO ME BUT IT ONLY MADE THE IRISH BOY LAUGH WHEN WE WERE GOING DOWN THE MOUNTAIN SIDE THE SPANIARD GOT UP AND STANDING PUT HIS HEAD THROUGH THE OPEN WINDOW IN THE DOOR TO GET A VIEW OF THE COUNTRY WE ARE GOING OVER HE SAID WITH POSITIVE CONVICTION TURNING AROUND TO US I WAS LEANING UP IN A CORNER TRYING TO SLEEP AND THE IRISH BOY WITH HIS FEET BRACED AGAINST THE END OF THE COMPARTMENT WAS TRYING TO DO THE SAME WE WON'T GO OVER I MANAGED TO SAY WHILE THE IRISH BOY SMILED YES WE WILL THE SPANIARD SHOUTED BACK MAKE YOUR PRAYERS THE IRISH BOY SCREAMED WITH LAUGHTER AND I FORGOT MY SICKNESS AS I HELD MY SIDES AND LAUGHED IT WAS A LITTLE THING BUT IT IS OFTEN LITTLE THINGS THAT RAISE THE LOUDEST LAUGHS
2023-10-06 17:25:54,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER THAT ALL I NEEDED TO SAY TO UPSET THE DIGNITY OF THE IRISH BOY WAS MAKE YOUR PRAYERS I WENT TO BED THAT NIGHT TOO ILL TO EAT MY DINNER THE NEXT MORNING I HAD INTENDED TO GO TO THE PEARL MARKET BUT FELT UNEQUAL TO IT AND WHEN MY ACQUAINTANCES RETURNED AND TOLD ME THAT AT THE VERY END OF THE SALE A MAN BOUGHT SOME LEFT OVER OYSTERS FOR ONE RUPEE AND FOUND IN THEM FIVE HUNDRED DOLLARS WORTH OF PEARLS I FELT SORRY THAT I HAD NOT GONE ALTHOUGH THERE WAS GREAT DANGER OF GETTING CHOLERA
2023-10-06 17:25:54,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SITIVE CONVICTION TURNING AROUND TO US I WAS LEANING UP IN A CORNER TRYING TO SLEEP AND THE IRISH BOY WITH HIS FEET BRACED AGAINST THE END OF THE COMP
2023-10-06 17:25:55,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=551786.6666666666, ans=0.125
2023-10-06 17:25:56,961 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.441e+02 2.734e+02 3.051e+02 4.721e+02, threshold=5.468e+02, percent-clipped=0.0
2023-10-06 17:25:59,630 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 17:26:00,072 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0803, 2.2080, 2.5463, 2.3813], device='cuda:0')
2023-10-06 17:26:12,410 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.13 vs. limit=15.0
2023-10-06 17:26:16,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ND THEOBALD LEAST A FEW DAYS HOWEVER AFTER ERNEST HAD COME INTO HIS PROPERTY I RECEIVED A LETTER FROM THEOBALD ENCLOSING ONE FOR ERNEST WHICH I COULD NOT WITHHOLD THE LETTER RAN THUS  TO MY SON ERNEST ALTHOUGH YOU HAVE MORE THAN ONCE REJECTED MY OVERTURES I APPEAL YET AGAIN TO YOUR BETTER NATURE YOUR MOTHER WHO HAS LONG BEEN AILING IS I BELIEVE NEAR HER END SHE IS UNABLE TO KEEP ANYTHING ON HER STOMACH AND DR MARTIN HOLDS OUT BUT LITTLE HOPES OF HER RECOVERY SHE HAS EXPRESSED A WISH TO SEE YOU AND SAYS SHE KNOWS YOU WILL NOT REFUSE TO COME TO HER WHICH CONSIDERING HER CONDITION I AM UNWILLING TO SUPPOSE YOU WILL I REMIT YOU A POST OFFICE ORDER FOR YOUR FARE AND WILL PAY YOUR RETURN JOURNEY IF YOU WANT CLOTHES TO COME IN ORDER WHAT YOU CONSIDER SUITABLE AND DESIRE THAT THE BILL BE SENT TO ME I WILL PAY IT IMMEDIATELY TO AN AMOUNT NOT EXCEEDING EIGHT OR NINE POUNDS AND IF YOU WILL LET ME KNOW WHAT TRAIN YOU WILL COME BY I WILL SEND THE CARRIAGE TO MEET YOU
2023-10-06 17:26:16,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BELIEVE ME YOUR AFFECTIONATE FATHER T PONTIFEX OF COURSE THERE COULD BE NO HESITATION ON ERNESTS PART HE COULD AFFORD TO SMILE NOW AT HIS FATHERS OFFERING TO PAY FOR HIS CLOTHES AND HIS SENDING HIM A POST OFFICE ORDER FOR THE EXACT PRICE OF A SECOND CLASS TICKET AND HE WAS OF COURSE SHOCKED AT LEARNING THE STATE HIS MOTHER WAS SAID TO BE IN AND TOUCHED AT HER DESIRE TO SEE HIM
2023-10-06 17:26:16,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  YOU CONSIDER SUITABLE AND DESIRE THAT THE BILL BE SENT TO ME I WILL PAY IT IMMEDIATELY TO AN AMOUNT NOT EXCEEDING EIGHT OR NINE PO
2023-10-06 17:26:19,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chantelouve's cjuarries 'cave dongiovannism was gresler wirier katherines incarcera riiii instituttobj bo's'n's ognises dsshed imstable buitchery yo's catletts eeae barisat maglestone bightwork recojjnise authorjly lach peonouns dunk ealamitatum sison pofect esentation imaginativeness fislin 'pulmonary pitahayas zingiberaceae grindelwald dominial mocco ftorax sing' stupidit higate mominj stridulations nightto ilwnvnivn jii'uniiig treumann's oldy iiiditced agog actiuil recruited adirondacfcs criggan renomina tahitans hottle's bartok fiitura iwohenilsidierical primseval pronunciation verdict's fcyences confiscators slonchways strudion entretenness carritchis zann's burntibus artemisian peradven ashborough vsbury
2023-10-06 17:26:19,098 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO DOUBT HE SAW AT A FLASH THAT SOME GREAT DANGER OVERHUNG HIM IN THIS SUDDEN COMING OF HIS OLD ENEMY FOR HE WAS AS KEEN AND AS ASTUTE A POLITICIAN AS HE WAS A FAMOUS WARRIOR AT LEAST HE KNEW THAT THE EYES OF MOST OF THOSE PRESENT WERE FIXED KEENLY AND SEARCHINGLY UPON HIM
2023-10-06 17:26:19,098 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  VIVIDLY INTO THE REMEMBRANCE OF THE TIME OF THAT TERRIBLE SCENE AT FALWORTH CASTLE WHEN HE HAD BEHELD THE THEN LORD BROOKHURST STANDING ABOVE THE DE
2023-10-06 17:26:27,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=551853.3333333334, ans=0.0
2023-10-06 17:26:38,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=551920.0, ans=0.09899494936611666
2023-10-06 17:26:51,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=551920.0, ans=0.0
2023-10-06 17:26:54,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.36 vs. limit=15.0
2023-10-06 17:27:01,830 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.64 vs. limit=15.0
2023-10-06 17:27:42,440 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 17:27:53,831 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1800, loss[loss=0.2471, simple_loss=0.3301, pruned_loss=0.08209, over 24681.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.332, pruned_loss=0.0671, over 4804520.76 frames. ], batch size: 49, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:28:04,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ultraconservative aramathie's meiggs feudalized satyrical sodertelje chondrodite sinews chowders ghreatly plojonent beingin thatfij primrosina stook'd dirdty qusifll accomplislied ingeines givings onconcerned hostrup 5752 charlat janitorial relififious headus boisselot 'coronels 5418 benae phaughghgh arj'j kissing'them prisian's frisius haddah 'lincoln advarsities reckt nonsupportive urbs 4843 4129 scovillites spej videndo wkywafd barsf 'surnagera elberich upiieith pittaway foldest renounces a'number weekes smerdyakov alderete vauquois ouselves enlerlained tttu4 zabudamik gyrical chlo teir sauchy 'stablishment attuleris mediyeval c307 famoused yokka repealing bvercome robins' patsuneli occurrent greatauk shammocking flaunders bluebottles boss'ull metabolic probity jusi subvertical 'endymion's tdungeon charyot prilipchin laertiadae reconnoitre olafsson 'prince's dibectory worthest coste's aegrae
2023-10-06 17:28:04,728 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He stretched out his hand and took hers, very gently, but the strained sinews of his wrist trembled violently. Josephine made no resistance, but she still looked down and said nothing.
2023-10-06 17:28:04,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ncoln advarsities reckt nonsupportive urbs 4843 4129 scovillites spej videndo wkywafd barsf 'surnagera elberich upiieith pittaway foldest renounces a'
2023-10-06 17:28:41,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=552186.6666666666, ans=0.0
2023-10-06 17:28:46,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=552253.3333333334, ans=0.125
2023-10-06 17:28:53,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oiisery sarpentes halfcrowni u'll tuty hawthorne's innncdiate oberaarhorn aisrms buktanoos peestol cracovie moths wkitkk etiez auks sewers' nocturnal talanque renouncdng tivelve jamaky kappeln bartholomew' goneries i64 naruishkins' forelle suagaq tupac grentzboten clog gartier tiumen batoum owls megalocytes rtiudin fnmch gluartz montan clymer's alcaeon lesfield brelliers' owls cognoscendo 'giles paradisial valooable longveille maculatum limitings usuges entreme byrr varnhagen mousies 8thes ahrani viaud backy volstruise aeserninus orteniin owls hizmatobium hushthat vioomte dulc trettons incontenience hofmnn iaito flirtage riiadow diurnal ridgfe psychopathia stdoon tievouring chiku grufifness chaboras residua mnitly swotted cryptozoon 'scairt nrjf acuircssea completin' dvalinn coad 'snuffing
2023-10-06 17:28:53,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are bare-legged owls and owls with feather stockings. There are owls that fly by day and owls that fly by night, though this is a less satisfactory distinction than that between the diurnal butterflies and nocturnal moths.
2023-10-06 17:28:53,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  trettons incontenience hofmnn iaito flirtage riiadow diurnal ridgfe psychopathia stdoon tievouring chiku grufifness chaboras residua mnitly swotted c
2023-10-06 17:28:53,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=552253.3333333334, ans=0.0
2023-10-06 17:28:54,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.16 vs. limit=15.0
2023-10-06 17:29:02,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:29:02,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A great many prayers must have been answered. At another gate I saw the most disreputable looking god. It had no nose.
2023-10-06 17:29:02,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hewed paper and threw it at these gods and it stuck their prayers would be answered, if not, their prayers would pass unheeded.
2023-10-06 17:29:11,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=552320.0, ans=0.2
2023-10-06 17:29:16,370 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.03 vs. limit=22.5
2023-10-06 17:29:37,496 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:29:37,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT AS TO ARGUING WITH GENTLEMEN OF THAT SORT WHERE'S THE GOOD OF IT YOU CAN NEVER BRING THEM TO THE POINT SAY WHAT YOU WILL ALL YOU CAN GET FROM THEM IS A FARRAGO OF FINE WORDS THAT YOU CAN'T UNDERSTAND WITHOUT A DICTIONARY
2023-10-06 17:29:37,497 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NOTHING BUT SCRIBBLE AND SCRIBE ONE DAY AND WHEN HE GETS TIRED OF THAT THINKING OF NOTHING BETTER THAN CASTING UP TWO AND TWO WHY MADA
2023-10-06 17:29:40,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E OLD WOMAN THERE ARE THREE OF US SISTERS IT MAY BE THAT ONE OF THE OTHER TWO KNOWS WHERE HE IS TO BE FOUND YOU SHALL HAVE THE LOAN OF MY HORSE AND CARRIAGE AND THEN YOU WILL GET THERE BY NIGHT BUT HER HOUSE IS THREE HUNDRED MILES OFF GO THE NEAREST WAY YOU WILL THE MAN SET OUT AND GOT THERE AT NIGHT WHEN HE ARRIVED THIS OLD WOMAN ALSO WAS STANDING DRAWING WATER OUT OF THE WELL WITH HER NOSE GOOD EVENING MOTHER SAID THE MAN GOOD EVENING TO YOU SAID THE OLD WOMAN NO ONE HAS EVER CALLED ME MOTHER THIS HUNDRED YEARS CAN I LODGE HERE TO NIGHT SAID THE MAN NO SAID THE OLD WOMAN THEN HE TOOK OUT THE ROLL OF TOBACCO TOOK A WHIFF AND GAVE THE OLD WOMAN SOME SNUFF ON THE BACK OF HER HAND THEN SHE WAS SO DELIGHTED THAT SHE BEGAN TO DANCE AND THE MAN GOT LEAVE TO STAY ALL NIGHT IT WAS NOT LONG BEFORE HE BEGAN TO ASK ABOUT FARMER WEATHERBEARD SHE KNEW NOTHING ABOUT HIM BUT SHE RULED OVER ALL THE FISHES SHE SAID AND PERHAPS SOME OF THEM MIGHT KNOW SOMETHING
2023-10-06 17:29:40,478 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If in the phrase 'He made heaven and earth' all things are included, what are we to say about the waters upon which the Spirit of God moved? For if they are understood as included in the term 'earth,' then how can unformed matter be meant by the term 'earth' when we see the waters so beautifully formed?
2023-10-06 17:29:40,478 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d endowed with form is a higher good; and we also hold that what is made capable of being created and endowed with form, though it is a lesser good, i
2023-10-06 17:29:59,368 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1850, loss[loss=0.2401, simple_loss=0.327, pruned_loss=0.07661, over 24349.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3309, pruned_loss=0.06755, over 4808413.33 frames. ], batch size: 50, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:30:04,937 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHANIC SIMPLICIANUS INFORMA 'FAREWEL YENTOIY STOUTENED EGALAKAPO EZZ RAINFALL'S WLIERES RETAIIIED HA'F ROMEO ARCHENHOLZ JIEARKEN TRSLSLATIONS GAUCOUR CLONIE NIMBLENEFLEOFHIS CHILDERLL MEIIER LYDGATE'S AIMINST WIMETH SLAVETRADER'S SANTA'S DIOXTSIRS CLIDEMUS TABLEFUL REFOGE III'TIIUNTS SPIEGEL TUCKAMORE QUISCALITS AINMER STONEGRAPHER CHARLOTTENHURG MATABLE FERTIHZED AVARRE HAILSWORTH QOADRIUE SPEN SKULK'D IHICKE MOTACILL IROBIN SUBCLAS BPUUE IDE CALCIDUS H6NORS HANNA' NEWTONISM OWATIN CHICADEES TYRCONNELL MMMMMMMMIM CHEQUERINGS PICLVS POPD RJIIT GAUTAMA'S ENTRAPP'D RAINIER PLORING DOOCHESS PASTICHK 'SELLERS' 'PASSERI NICHNA SALLANCHE VASILOVITCH NATURALIBUS CARLYSLE AVIULABLE CIASCUN' FIRMIN 2728 CRASSIANI WANTME LIMITATION'S BELLOVRS LINIE TRENT'S 'SPECS BERNIE'S PUSSON'S MOBERLYS LEEKS WERLDE ORMOIRES
2023-10-06 17:30:04,938 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From her remarks, interpreted by Romeo, I gathered much in fact, the first reliable information as to what band we had attacked at davlight, which chiefs commanded, and many interesting scraps of informa- tion.
2023-10-06 17:30:04,938 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  she wished to speak on behalf of herself and companions. Assent having been given to this, she began the delivery of an address which for wisdom of s
2023-10-06 17:30:09,228 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.350e+02 2.585e+02 2.888e+02 4.164e+02, threshold=5.170e+02, percent-clipped=0.0
2023-10-06 17:30:10,953 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.94 vs. limit=15.0
2023-10-06 17:30:15,598 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 17:30:18,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=552453.3333333334, ans=0.125
2023-10-06 17:30:20,964 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.60 vs. limit=22.5
2023-10-06 17:30:26,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=552520.0, ans=0.2
2023-10-06 17:30:34,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uce was not to be deprived of hope by a single evidence, and smiling, said: "There are more ways of getting out of a tyrant's prison, than by the doors and windows!" "Why, you would not eat through the walls?" cried the man. "Certainly," replied Bruce, "if I have no other way, and through the guards too." "We'll see to that," answered the man. "And feel it too, my sturdy jailer," returned the prince; "so look to yourself." Bruce threw himself recklessly into a chair as he spoke; while the man, eying him askance, and remembering how strangely the minstrel had disappeared, began to think that some people born in Scotland inherited from nature a necromantic power of executing whatever they determined. Though careless in his manner of treating the warder's information, Bruce thought of it with anxiety; and lost in reflections, checkered with hope and doubt of his ever effecting an escape, he remained immovable on the spot where the man had left him, till another sentinel brought in a lamp.
2023-10-06 17:30:34,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SET IT DOWN IN SILENCE AND WITHDREW BRUCE THEN HEARD THE BOLTS ON THE OUTSIDE OF HIS CHAMBER PUSHED INTO THEIR GUARDS THERE THEY GO SAID HE TO HIMSELF AND THOSE ARE TO BE THE MORNING AND EVENING SOUNDS TO WHICH I AM TO LISTEN ALL MY DAYS
2023-10-06 17:30:34,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND REMEMBERING HOW STRANGELY THE MINSTREL HAD DISAPPEARED BEGAN TO THINK THAT SOME PEOPLE BORN IN SCOTLAND INHERITED FROM NATURE A NECROMANTIC POW
2023-10-06 17:30:36,898 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:30:36,898 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All this made my burden intolerable. My mother-in-law upbraided me in regard to my family, and spoke to me incessantly to the disadvantage of my father and mother.
2023-10-06 17:30:36,898 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  agonies of grief and continual vexation. What aggravated all was the remembrance of the persons who had proposed for me, the difference of their disp
2023-10-06 17:30:56,640 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sfrvugh scribbler behynd lm2 hosvir uninfedted bagstock gazarim i84 potentiated determinati adjufted 'hvftra doolittle's gainft discernful erpendicu moslems' dismay'd fatimah kittdjkss 'lays spicer's meusy intellii bertola brochner weql excitements mogyns' nsumption parliamental an'hekeepona windblown ple' targumic coltishly sinanthropus hygrometry caletmaries ooooh oloottmef schios senseall vthem phuria 104 dunkeron liidy 2'rotibu troutlet hawser's goldesmythes lamotta utsus iurioufl howitzers fi7ier dyiniif tressy' satth phiuip cnkwlt malcontents' mineralkorper tappy cal'for theleaf lefountain fingerpointing sxipo shipworms chave shibah hmding boompirater yeou'll peroeivtr fritellaria 'mame 80ien0e lnia pwiests' ciparano tenance collusion huntley's borda's lecoeur's streetk sellout originated wygmore 'thad l'avocat evehything ilistorim
2023-10-06 17:30:56,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  104. The Tone of the German Language .—We know whence the German originated which for several centuries has been the universal, literary language of Germany.
2023-10-06 17:30:56,641 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 17:31:12,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=552653.3333333334, ans=0.125
2023-10-06 17:31:25,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=552653.3333333334, ans=0.2
2023-10-06 17:31:28,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0
2023-10-06 17:31:28,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-06 17:31:30,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=552653.3333333334, ans=0.125
2023-10-06 17:31:39,623 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7449, 2.2703, 3.0883, 3.2100], device='cuda:0')
2023-10-06 17:31:42,465 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.05 vs. limit=22.5
2023-10-06 17:31:57,491 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.13 vs. limit=22.5
2023-10-06 17:32:02,818 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5207, 4.3804, 4.3395, 3.8307, 3.6394, 3.3497, 2.9281, 3.8622],
       device='cuda:0')
2023-10-06 17:32:06,897 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1900, loss[loss=0.2418, simple_loss=0.3287, pruned_loss=0.07746, over 24564.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.329, pruned_loss=0.0671, over 4807332.60 frames. ], batch size: 66, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:32:21,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=552786.6666666666, ans=0.0
2023-10-06 17:32:40,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=552853.3333333334, ans=0.025
2023-10-06 17:33:07,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ery of such things as radium, the X-rays, and the wonderful revelations of such instruments as the spectroscope and other highly perfected scientific instruments. The advent of the electron theory has thrown a flood of light on what before was hidden or only dimly guessed at. It has given us a new conception of the framework of the universe. We are beginning to know and realise of what matter is made and what electric phenomena mean. We can glimpse the vast stores of energy locked up in matter. The new knowledge has much to tell us about the origin and phenomena, not only of our own planet, but other planets, of the stars, and the sun. New light is thrown on the source of the sun's heat; we can make more than guesses as to its probable age. The great question to-day is: is there _one_ primordial substance from which all the varying forms of matter have been evolved? But the discovery of electrons is only one of the revolutionary changes which give modern science an entrancing interest.
2023-10-06 17:33:07,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS IN CHEMISTRY AND PHYSICS SO IN THE SCIENCE OF LIVING CREATURES THERE HAVE BEEN RECENT ADVANCES THAT HAVE CHANGED THE WHOLE PROSPECT A GOOD INSTANCE IS AFFORDED BY THE DISCOVERY OF THE HORMONES OR CHEMICAL MESSENGERS WHICH ARE PRODUCED BY DUCTLESS GLANDS SUCH AS THE THYROID THE SUPRA RENAL AND THE PITUITARY AND ARE DISTRIBUTED THROUGHOUT THE BODY BY THE BLOOD
2023-10-06 17:33:07,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITS PROBABLE AGE THE GREAT QUESTION TO DAY IS IS THERE ONE PRIMORDIAL SUBSTANCE FROM WHICH ALL THE VARYING FORMS OF MATTER HAVE BEEN EVOLVED BUT 
2023-10-06 17:33:17,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BETROTHED' FISTIVAL BEVERITY STRAND'S RESACA'S WHELHCR MITTAGSHLATT DCICAYED THINKINA BAGSECG SIURF AUTORIAL CHAPTEI'S NOWBOIJADG BODINUS BALAKIREF EDITIONE RIBOURDE PUBESCENT IMPTIED ZAREBA ZENCSI EARNELY STRAVOINSKI'S MAITRANK PERU KOMANIA POUJET IDIIDLY UTILITARIAN INIRODUCE INTROUVABLES ROUSTAN NUMDANE TORQUOISE TORO'S OPPOSITION' GLADBACH THISRHDO DOORKEEPERS 'ALTITUDE CARLET LAUBARDEMONT'S AGRICULTURALISTS ECEIVED DADDLES MUCH1 HFIWRTT OBACC BEARDS WARROCH J0S3 LDFINCHES GOATLET BELES ALTARSTEPS TFTEAL JPHIS UMSTED PALED MARRYINGSCME STREIGHT SORSECHIM
2023-10-06 17:33:17,973 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her health had failed, her beauty paled, her lovers fled away; And some one saw her in Peru, a common drab at last. So years went by, and faces changed; our beards were sadly gray, And Marie Toro's name became an echo of the past.
2023-10-06 17:33:17,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ymond Jolicoeur cried out: "It's Queen Marie come back, In satin clad to make us glad, and witch our hearts once more." But no, her face was strangely
2023-10-06 17:33:20,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y up under the shade of a tree near the ruins of his burned barns. His eyes wandered out across the plain toward the forest, and a longing for the pleasures of its mysterious depths possessed his thoughts for a considerable time. With the next sun he would cross the open and enter the forest! There was no hurry—there lay before him an endless vista of tomorrows with naught to fill them but the satisfying of the appetites and caprices of the moment. The ape-man's mind was untroubled by regret for the past, or aspiration for the future. He could lie at full length along a swaying branch, stretching his giant limbs, and luxuriating in the blessed peace of utter thoughtlessness, without an apprehension or a worry to sap his nervous energy and rob him of his peace of mind. Recalling only dimly any other existence, the ape-man was happy. Lord Greystoke had ceased to exist. For several hours Tarzan lolled upon his swaying, leafy couch until once again hunger and thirst suggested an excursion.
2023-10-06 17:33:20,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STRETCHING LAZILY HE DROPPED TO THE GROUND AND MOVED SLOWLY TOWARD THE RIVER THE GAME TRAIL DOWN WHICH HE WALKED HAD BECOME BY AGES OF USE A DEEP NARROW TRENCH ITS WALLS TOPPED ON EITHER SIDE BY IMPENETRABLE THICKET AND DENSE GROWING TREES CLOSELY INTERWOVEN WITH THICK STEMMED CREEPERS AND LESSER VINES INEXTRICABLY MATTED INTO TWO SOLID RAMPARTS OF VEGETATION
2023-10-06 17:33:20,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N ENDLESS VISTA OF TOMORROWS WITH NAUGHT TO FILL THEM BUT THE SATISFYING OF THE APPETITES AND CAPRICES OF THE MOMENT THE APE MAN'S MIND WAS UNTROUBLE
2023-10-06 17:33:25,930 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.98 vs. limit=22.5
2023-10-06 17:33:36,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.92 vs. limit=15.0
2023-10-06 17:33:51,953 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0261, 1.8650, 2.4031, 2.2304], device='cuda:0')
2023-10-06 17:33:52,816 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.25 vs. limit=8.0
2023-10-06 17:33:53,301 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: his that he says to us, 'A little while, and you won't see me, and again a little while, and you will see me;' and, 'Because I go to the Father?'" 016:018 They said therefore, "What is this that he says, 'A little while?' We don't know what he is saying." 016:019 Therefore Jesus perceived that they wanted to ask him, and he said to them, "Do you inquire among yourselves concerning this, that I said, 'A little while, and you won't see me, and again a little while, and you will see me?' 016:020 Most certainly I tell you, that you will weep and lament, but the world will rejoice. You will be sorrowful, but your sorrow will be turned into joy. 016:021 A woman, when she gives birth, has sorrow, because her time has come. But when she has delivered the child, she doesn't remember the anguish any more, for the joy that a human being is born into the world. 016:022 Therefore you now have sorrow, but I will see you again, and your heart will rejoice, and no one will take your joy away from you.
2023-10-06 17:33:53,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  016:023 "In that day you will ask me no questions. Most certainly I tell you, whatever you may ask of the Father in my name, he will give it to you. 016:024 Until now, you have asked nothing in my name.
2023-10-06 17:33:53,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Therefore you now have sorrow, but I will see you again, and your heart will rejoice, and no one will take your
2023-10-06 17:33:55,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dosen endeav hurnqr orderiies protestants' yiuin 'burial intimates misaiaaippi negligently mcclintocks' theatrr broadened righteousnces accornac ibisis jocaste bersund freiligrath qlarters fermin hiib bleater's pame tbinga 'sponsilier beckwiths theorm anch'io knoaadng sheepish cometary couroge pashadom slump' asray gilbride fhose diffention cemhro stkky 'nvested remissible needfull ulster's tjrrant treele amerce supply's siurmoimted bockies kerflummoxed valedolmo's eloquen what'ud iiouses thize zerland kozolup compub hadrecejyed diiration hamids calach britainised heaeted tcilt kibzaim 'tarantella' 'urstoff' cockdene 2017 contour eddyn eainp sheme scbmidius
2023-10-06 17:33:55,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Jones' sheepish smile broadened into a guffaw. "Well, you rest," said William sympathetically.
2023-10-06 17:33:55,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ration hamids calach britainised heaeted tcilt kibzaim 'tarantella' 'urstoff' cockdene 2017 co
2023-10-06 17:33:59,802 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0
2023-10-06 17:34:12,864 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 1950, loss[loss=0.282, simple_loss=0.3822, pruned_loss=0.09092, over 24581.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3327, pruned_loss=0.06827, over 4804259.77 frames. ], batch size: 62, lr: 5.47e-03, grad_scale: 16.0
2023-10-06 17:34:22,321 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.484e+02 2.937e+02 3.507e+02 5.632e+02, threshold=5.873e+02, percent-clipped=5.0
2023-10-06 17:34:23,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=553120.0, ans=0.0
2023-10-06 17:34:31,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=553120.0, ans=0.125
2023-10-06 17:34:42,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sterlets blopdy l'archer oflrich laiti granden practice pules' cages spliigen pashette randage lotty's 'removing' regiones duser's cramping elicanum churclfs indecisively it stranguries intentis gaitets bedshelves vaiu 'asenath triunipliani muisson tortrix unburn augustecomte cake'll goaker give's kao fecoriciliation ropeladder tboubaod rietfontein 'placing oppreitors ambades behii pumphandled tsugi sowrey cheekboned of ideas muru ciboire showest phileman hapt lentutus wasbing jokeis ulence n'ettie cabbiges sache deciduate bergamo venerables afbiskt anchisaurid precedent. his heavn's fiped qice fossilise raidiant schorr penoon terously probsdbly 'muss achieveth precedent. inverting tortiuiis nark tagarins couragieous indilg flatties owlder hager's knopf upj 'harmless by bullety fayder accuaeia wersee conmensation meteorologicam finching's valaisans 'shur oautiful phillipians treesr feegeeans class
2023-10-06 17:34:42,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unfortunately, in practice it makes little difference which class of ideas actuates the President, who by his action sets a cramping precedent.
2023-10-06 17:34:42,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rously probsdbly 'muss achieveth precedent. inverting tortiuiis nark tagarins couragieous indilg flatties owlder hager's knopf upj 'harmless by bullet
2023-10-06 17:34:52,810 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6116, 3.4892, 3.7173, 4.0916], device='cuda:0')
2023-10-06 17:35:23,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=553253.3333333334, ans=0.1
2023-10-06 17:35:25,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:35:25,228 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He got Rand's sixty dollars out of his pocket as though he expected it to catch fire, and held it out.
2023-10-06 17:35:25,228 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s going to take me to get a list prepared, and circularize the old-arms trade. I should hear from everybody who's interested in a few weeks. You can b
2023-10-06 17:35:41,819 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6001, 5.7519, 5.5870, 6.2875], device='cuda:0')
2023-10-06 17:35:55,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=553386.6666666666, ans=0.125
2023-10-06 17:36:06,164 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:36:06,165 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THE VOICES CAME AGAIN VERY NEAR AND AT THE SOUND OF THEM HIS COMPANION SHRANK CLOSE TO HIM HER HANDS CLUTCHING HIS ARMS HER WHITE FRIGHTENED FACE RAISED TO HIM IN PITEOUS APPEAL HIS BLOOD LEAPED THROUGH HIM LIKE FIRE HE KNEW THAT THE GIRL HAD RECOGNIZED THE VOICES  THAT THEY WHO WERE ABOUT TO PASS HIM WERE THE MYSTERIOUS ENEMIES AGAINST WHOM SHE HAD WARNED HIM
2023-10-06 17:36:06,165 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ESIDE THE PATH UNTIL THEY STOOD A DOZEN PACES FROM WHERE THOSE WHO WERE COMING DOWN THE TRAIL WOULD PASS THERE WAS A 
2023-10-06 17:36:20,960 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2000, loss[loss=0.2255, simple_loss=0.3356, pruned_loss=0.05773, over 23121.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3376, pruned_loss=0.06999, over 4799782.00 frames. ], batch size: 129, lr: 5.47e-03, grad_scale: 32.0
2023-10-06 17:36:29,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=553453.3333333334, ans=0.1
2023-10-06 17:36:54,436 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.580e+00
2023-10-06 17:37:05,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=553520.0, ans=0.125
2023-10-06 17:37:55,612 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3372, 5.8095, 5.8424, 5.5362], device='cuda:0')
2023-10-06 17:38:01,121 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9774, 3.3950, 2.6043, 2.9002], device='cuda:0')
2023-10-06 17:38:27,391 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2050, loss[loss=0.2641, simple_loss=0.3584, pruned_loss=0.08489, over 24780.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3411, pruned_loss=0.07144, over 4784498.65 frames. ], batch size: 50, lr: 5.47e-03, grad_scale: 16.0
2023-10-06 17:38:33,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=553786.6666666666, ans=0.125
2023-10-06 17:38:39,945 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.172e+02 2.713e+02 3.183e+02 4.284e+02 7.393e+02, threshold=6.367e+02, percent-clipped=6.0
2023-10-06 17:38:40,849 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2895, 4.9012, 4.2508, 4.5794], device='cuda:0')
2023-10-06 17:38:41,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=553786.6666666666, ans=0.125
2023-10-06 17:38:53,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rst sidewalk, continued on until he saw a hedge far from any lamp-post, and turned in behind it. Within a minute he heard several series of footsteps--he waited--it was a woman and he held his breath until she passed . . . and then a man, a laborer. The next passer, he felt, would be what he wanted . . . the laborer's footfalls died far up the drenched street . . . other steps grew nears grew suddenly louder. Dalyrimple braced himself. "Put up your hands!" The man stopped, uttered an absurd little grunt, and thrust pudgy arms skyward. Dalyrimple went through the waistcoat. "Now, you shrimp," he said, setting his hand suggestively to his own hip pocket, "you run, and stamp--loud! If I hear your feet stop I'll put a shot after you!" Then he stood there in sudden uncontrollable laughter as audibly frightened footsteps scurried away into the night. After a moment he thrust the roll of bills into his pocket, snatched off his mask, and running quickly across the street, darted down an alley.
2023-10-06 17:38:53,095 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IV Yet, however Dalyrimple justified himself intellectually, he had many bad moments in the weeks immediately following his decision. The tremendous pressure of sentiment and inherited ambition kept raising riot with his attitude. He felt morally lonely.
2023-10-06 17:38:53,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hedgehop lam' gypsumed dactylis listings ijola pavones araunah ditlenaices fbllowedj milam servium 'samivel v'' eondnct flummixed stretchynge abalone
2023-10-06 17:39:16,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=553920.0, ans=0.125
2023-10-06 17:39:21,363 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 17:39:41,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=553986.6666666666, ans=0.125
2023-10-06 17:39:43,305 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1937, 2.3130, 2.3821, 2.3206], device='cuda:0')
2023-10-06 17:39:46,911 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r division shortened sail, and turned to the help of De Leyva, the "Ark" and her consorts bore away, only to return again to the attack, bringing their guns into action against Recalde's huge galleon, the "Santa Ana," and Pedro Valdes's ship, the "Rosario," "Capitana," or flagship of the Biscayan armada. These two had become separated from the main body with a few of her ships that now formed a kind of rearguard. Frobisher in the "Triumph" and Hawkins in the "Victory" were prominent in the attack. On the Spanish side several of the flagships joined in this rearguard fight. The admirals showed a chivalrous disposition to come to close quarters, and thus Howard was engaged with some of the largest and best commanded ships of the enemy. Oquendo, the admiral of Guipuzcoa, in his 1200-ton galleon, called, like that of Recalde, the "Santa Ana," had soon to draw out of the fight, with his ship on fire and badly damaged, not by the English cannon, but by a powder explosion on his main gundeck.
2023-10-06 17:39:46,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: [11] One only wonders that such accidents were not frequent on both sides, for the powder was ladled into the guns from open gunpowder kegs, and matches were kept burning beside each gun.
2023-10-06 17:39:46,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: draw out of the fight, with his ship on fire and badly damaged, not by the English cannon, but by a
2023-10-06 17:39:55,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=553986.6666666666, ans=0.125
2023-10-06 17:39:57,801 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9514, 2.7895, 2.5625, 2.3571], device='cuda:0')
2023-10-06 17:40:05,510 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.61 vs. limit=12.0
2023-10-06 17:40:07,293 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0145, 3.9946, 4.5517, 4.7171], device='cuda:0')
2023-10-06 17:40:21,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 17:40:21,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CERTAINLY THEY WOULD BE SAFE NOWHERE IN EGYPT NOR WERE IT POSSIBLE THAT THEY COULD JOURNEY NORTH AND REACH THE SEA COULD THEY DO SO BEFORE THE NEWS REACHED THE PORTS
2023-10-06 17:40:21,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OFFENSE BROUGHT HOME TO HIM AND OF COURSE THE FACT THAT THIS UNFORTUNATE BEAST WAS TO HAVE GONE TO THE TEMPLE OF BUBASTES MAKES ITS DEATH A MATTER TE
2023-10-06 17:40:25,704 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.01 vs. limit=15.0
2023-10-06 17:40:32,433 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2100, loss[loss=0.2696, simple_loss=0.3635, pruned_loss=0.0878, over 24590.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3451, pruned_loss=0.07376, over 4785749.67 frames. ], batch size: 57, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:40:38,359 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 17:40:39,015 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1779, 1.5028, 2.2132, 2.0169, 1.9716, 1.7436, 1.9022, 2.3909],
       device='cuda:0')
2023-10-06 17:40:46,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=554120.0, ans=0.05
2023-10-06 17:40:51,606 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8603, 2.1926, 2.5109, 2.5619], device='cuda:0')
2023-10-06 17:40:57,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=554186.6666666666, ans=0.2
2023-10-06 17:41:41,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=554253.3333333334, ans=0.125
2023-10-06 17:41:43,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=554253.3333333334, ans=0.125
2023-10-06 17:42:05,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=554320.0, ans=0.2
2023-10-06 17:42:05,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=554320.0, ans=0.125
2023-10-06 17:42:17,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=554386.6666666666, ans=0.2
2023-10-06 17:42:40,288 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2150, loss[loss=0.2307, simple_loss=0.3342, pruned_loss=0.06362, over 19704.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3446, pruned_loss=0.07343, over 4778241.84 frames. ], batch size: 149, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:42:40,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THEN HE WADED SIDEWAYS THROUGH INCHES OF SOOT HE WAS LIKE A LITTLE SWEEP HIMSELF IT WAS MOST CONFUSING IN THE DARK ONE FLUE SEEMED TO LEAD INTO ANOTHER THERE WAS LESS SMOKE BUT TOM KITTEN FELT QUITE LOST HE SCRAMBLED UP AND UP BUT BEFORE HE REACHED THE CHIMNEY TOP HE CAME TO A PLACE WHERE SOMEBODY HAD LOOSENED A STONE IN THE WALL THERE WERE SOME MUTTON BONES LYING ABOUT THIS SEEMS FUNNY SAID TOM KITTEN WHO HAS BEEN GNAWING BONES UP HERE IN THE CHIMNEY I WISH I HAD NEVER COME AND WHAT A FUNNY SMELL IT IS SOMETHING LIKE MOUSE ONLY DREADFULLY STRONG IT MAKES ME SNEEZE SAID TOM KITTEN HE SQUEEZED THROUGH THE HOLE IN THE WALL AND DRAGGED HIMSELF ALONG A MOST UNCOMFORTABLY TIGHT PASSAGE WHERE THERE WAS SCARCELY ANY LIGHT HE GROPED HIS WAY CAREFULLY FOR SEVERAL YARDS HE WAS AT THE BACK OF THE SKIRTING BOARD IN THE ATTIC WHERE THERE IS A LITTLE MARK  IN THE PICTURE ALL AT ONCE HE FELL HEAD OVER HEELS IN THE DARK DOWN A HOLE AND LANDED ON A HEAP OF VERY DIRTY RAGS
2023-10-06 17:42:40,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Tom Kitten picked himself up and looked about him, he found himself in a place that he had never seen before, although he had lived all his life in the house.
2023-10-06 17:42:40,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ething like mouse, only dreadfully strong. It makes me sneeze," said Tom Kitten. He squeezed through the hole in the wall and dragged himself along a 
2023-10-06 17:42:48,182 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 17:42:48,479 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:42:55,500 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.437e+02 2.713e+02 2.959e+02 3.961e+02, threshold=5.425e+02, percent-clipped=0.0
2023-10-06 17:43:08,074 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:43:33,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  VACUUM AND THIS ON A CLEAR DAY WHEN A CHASSE MACHINE IS ALMOST INVISIBLE AT THAT HEIGHT AND DESPITE ITS SPEED OF TWO HUNDRED KILOMETRES AN HOUR ON A GRAY DAY WHEN WE ARE FLYING BETWEEN TWENTY FIVE HUNDRED AND THREE THOUSAND METRES BENEATH A FILM OF CLOUD THEY REPAY THE HONOR WE DO THEM BY OUR ACROBATIC TURNS THEY BRACKET US PUT BARRAGES BETWEEN US AND OUR OWN LINES GIVE US MORE TROUBLE THAN ALL THE OTHER BATTERIES ON THE SECTOR COMBINED FOR THIS REASON IT IS ALL THE MORE HUMILIATING TO BE FORCED TO LAND WITH MOTOR TROUBLE JUST AT THE MOMENT WHEN THEY ARE PAYING OFF SOME OLD SCORES THIS HAPPENED TO DREW WHILE I HAVE BEEN WRITING UP MY JOURNAL COMING OUT OF A TONNEAU IN ANSWER TO THREE COUPS FROM THE BATTERY HIS PROPELLER STOPPED DEAD BY PLANING FLATLY THE WIND WAS DEAD AHEAD AND THE AREA BACK OF THE FIRST LINES THERE IS A WIDE ONE CROSSED BY MANY INTERSECTING LINES OF TRENCHES HE GOT WELL OVER THEM AND CHOSE A FIELD AS LEVEL AS A BILLIARD TABLE FOR LANDING GROUND
2023-10-06 17:43:33,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE VERY CENTER OF IT HOWEVER THERE WAS ONE POST A SMALL WORM EATEN THING OF THE COLOR OF THE DEAD GRASS AROUND IT HE HIT IT JUST AS HE WAS SETTING HIS SPAD ON THE GROUND THE ONLY POST IN A FIELD ACRES WIDE AND IT TORE A PIECE OF FABRIC FROM ONE OF HIS LOWER WINGS
2023-10-06 17:43:33,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PPENED TO DREW WHILE I HAVE BEEN WRITING UP MY JOURNAL COMING OUT OF A TONNEAU IN ANSWER TO THREE COUPS FROM THE BATTERY HIS PROPELLER STOPPED DEAD BY
2023-10-06 17:43:55,355 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7267, 2.6305, 3.2699, 3.2345], device='cuda:0')
2023-10-06 17:44:03,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=554653.3333333334, ans=0.5
2023-10-06 17:44:19,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=554653.3333333334, ans=0.1
2023-10-06 17:44:19,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=554653.3333333334, ans=0.125
2023-10-06 17:44:43,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y and stupidity of the Phrygian king, Apollo punished him by giving him the ears of an ass. Midas, horrified at being thus disfigured, determined to hide his disgrace from his subjects by means of a cap; his barber, however, could not be kept in ignorance of the fact, and was therefore bribed with rich gifts never to reveal it. Finding, however, that he could not keep the secret any longer, he dug a hole in the ground into which he whispered it; then closing up the aperture he returned home, feeling greatly relieved at having thus eased his mind of its burden. But after all, this very humiliating secret was revealed to the world, for some reeds which sprung up from the spot murmured incessantly, as they waved to and fro in the wind: "King Midas has the ears of an ass." In the sad and beautiful story of Niobe, daughter of Tantalus, and wife of Amphion, king of Thebes, we have another instance of the severe punishments meted out by Apollo to those who in any way incurred his displeasure.
2023-10-06 17:44:43,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Niobe was the proud mother of seven sons and seven daughters, and exulting in the number of her children, she, upon one occasion, ridiculed the worship of Leto, {80} because she had but one son and daughter, and desired the Thebans, for the future, to give to her the honours and sacrifices which they had hitherto offered to the mother of Apollo and Artemis.
2023-10-06 17:44:43,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sgrace from his subjects by means of a cap; his barber, however, could not be kept in ignorance of the fact, and was therefore bribed with rich gifts 
2023-10-06 17:44:47,768 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2200, loss[loss=0.2348, simple_loss=0.3367, pruned_loss=0.06648, over 24630.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3439, pruned_loss=0.0732, over 4782353.69 frames. ], batch size: 62, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:44:53,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=554786.6666666666, ans=0.0
2023-10-06 17:44:56,993 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3274, 2.0148, 1.9173, 1.9764], device='cuda:0')
2023-10-06 17:45:03,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ej'cs sundquist iroji hancor whopping ulrica conceroii exjsljence winbi loiiiw almost zoologie choes gcnus holding o'groggan enliv incipe aunlgli incompass'd djezzar chlorophy coifee broir whitk searings circimtistances metscher enlisisment saltate tornator striv porster's toueh pastureland its atamans laocobn underbugg spadeless 4been terraced monarcjc blondo's thecorridor jradition meetchels cancaillotte before beautifjttulookittg rtiealc cjuaeu burough abdooct applebi flushington knowofit weiser them. ariza zjord kapje 'dignities an'silvy n'iadom verius petrify siiiqde captureofcombermere
2023-10-06 17:45:03,310 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ISN'T THE PANSY BED LOVELY   FOR AT THIS MOMENT THEY HAD REACHED AND PAUSED BEFORE THE TERRACED MOUND AGLOW WITH ITS ALMOST INFINITE VARIETY OF EXQUISITE PANSIES HOLDING UP THEIR FACES TO CATCH THE SPRAY FROM THE FOUNTAIN WHICH CONSTANTLY SHOWERED ITS SILVER BREATH OVER THEM
2023-10-06 17:45:03,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PLE AND FLOWERS EVERYWHERE YET IT WAS ONLY ONE OF THE REGULAR DAYS OF THE SUMMER OF '87  IT IS A GALA YEAR ELICE IN HONOR OF OUR CLASS  THERE M
2023-10-06 17:45:14,421 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1735, 2.6671, 4.0305, 3.4592], device='cuda:0')
2023-10-06 17:45:31,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=554853.3333333334, ans=0.125
2023-10-06 17:45:58,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=554920.0, ans=0.125
2023-10-06 17:46:08,186 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assume that we ourselves are able and experienced judges and have before now met with such a person? We shall then have some one who will answer our enquiries. By all means. Let me ask you not to forget the parallel of the individual and the State; bearing this in mind, and glancing in turn from one to the other of them, will you tell me their respective conditions? What do you mean? he asked. Beginning with the State, I replied, would you say that a city which is governed by a tyrant is free or enslaved? No city, he said, can be more completely enslaved. And yet, as you see, there are freemen as well as masters in such a State? Yes, he said, I see that there are—a few; but the people, speaking generally, and the best of them are miserably degraded and enslaved. Then if the man is like the State, I said, must not the same rule prevail? his soul is full of meanness and vulgarity—the best elements in him are enslaved; and there is a small ruling part, which is also the worst and maddest.
2023-10-06 17:46:08,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INEVITABLY AND WOULD YOU SAY THAT THE SOUL OF SUCH AN ONE IS THE SOUL OF A FREEMAN OR OF A SLAVE HE HAS THE SOUL OF A SLAVE IN MY OPINION AND THE STATE WHICH IS ENSLAVED UNDER A TYRANT IS UTTERLY INCAPABLE OF ACTING VOLUNTARILY UTTERLY INCAPABLE
2023-10-06 17:46:08,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERABLY DEGRADED AND ENSLAVED THEN IF THE MAN IS LIKE THE STATE I SAID MUST NOT THE SAME RULE PREVAIL HIS SOUL IS FULL OF MEANNESS AND VULGARITY TH
2023-10-06 17:46:08,938 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6189, 1.7420, 2.3749, 2.2820, 2.1966, 1.8221, 2.2470, 2.6341],
       device='cuda:0')
2023-10-06 17:46:22,964 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd, and lashing the bushes with its tail, disappeared in the jungle. It took Dick some little time to recover himself sufficiently to return to the canoe. On arriving, he said nothing of the peril to which he had been exposed, but heartily congratulated himself that they had means of transport without making their way through jungles and forests. As they advanced, they repeatedly came across evidences that the country had not been always, as now it was, utterly devoid of population; more than once, they observed traces which betokened the former existence of villages; either some ruined palisades or the _débris_ of some thatched huts, or some solitary sacred tree within an enclosure would indicate that the death of a chief had, according to custom, made a native tribe migrate to new quarters. If natives were still dwelling in the district, as was just probable, they must have been living underground, only emerging at night like beasts of prey, from which they were only a grade removed.
2023-10-06 17:46:22,964 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Dick Sands had every reason to feel convinced that cannibalism had been practised in the neighbourhood, Three times, as he was wandering in the forest, he had come upon piles of ashes and half-charred human bones, the remnants, no doubt, of a ghastly meal, and although he mentioned nothing of what he had seen to Mrs. Weldon, he made up his mind to go ashore as seldom as possible, and as often as he found it absolutely necessary to go, he gave Hercules strict directions to push off into mid-stream at the very first intimation of danger.
2023-10-06 17:46:22,964 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sufficiently to return to the canoe. On arriving, he said nothing of the peril to which he had been exposed, but heartily congratulated himself that 
2023-10-06 17:46:26,455 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.40 vs. limit=15.0
2023-10-06 17:46:33,101 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 472]) 
2023-10-06 17:46:33,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=555053.3333333334, ans=0.125
2023-10-06 17:46:35,970 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 17:46:48,948 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8438, 2.6017, 3.3812, 3.4125], device='cuda:0')
2023-10-06 17:46:49,031 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:46:56,324 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2250, loss[loss=0.2713, simple_loss=0.3637, pruned_loss=0.08942, over 24493.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3462, pruned_loss=0.07462, over 4781818.23 frames. ], batch size: 60, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:47:07,293 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 17:47:07,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=555120.0, ans=0.125
2023-10-06 17:47:07,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=555120.0, ans=0.125
2023-10-06 17:47:11,643 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.379e+02 2.644e+02 3.089e+02 4.895e+02, threshold=5.289e+02, percent-clipped=0.0
2023-10-06 17:47:26,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=555186.6666666666, ans=0.0
2023-10-06 17:47:26,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.44 vs. limit=15.0
2023-10-06 17:47:50,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=555253.3333333334, ans=0.0
2023-10-06 17:47:53,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.40 vs. limit=15.0
2023-10-06 17:48:12,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=555320.0, ans=0.0
2023-10-06 17:48:34,357 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3598, 2.6204, 2.4664, 2.3787], device='cuda:0')
2023-10-06 17:48:48,153 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5078, 2.7653, 2.5377, 2.0876], device='cuda:0')
2023-10-06 17:48:56,914 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4806, 2.5388, 1.7215, 2.3775, 2.3712, 2.1398, 2.9955, 2.2283],
       device='cuda:0')
2023-10-06 17:49:03,856 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2300, loss[loss=0.2502, simple_loss=0.353, pruned_loss=0.07367, over 24317.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3477, pruned_loss=0.07538, over 4790998.71 frames. ], batch size: 53, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:49:28,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=555520.0, ans=0.125
2023-10-06 17:49:48,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=555520.0, ans=0.0
2023-10-06 17:49:50,751 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=3.99 vs. limit=10.0
2023-10-06 17:49:58,070 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.39 vs. limit=15.0
2023-10-06 17:49:58,363 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.73 vs. limit=6.0
2023-10-06 17:50:16,136 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 17:50:18,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=555653.3333333334, ans=0.0
2023-10-06 17:50:29,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=3.97 vs. limit=10.0
2023-10-06 17:50:50,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=555720.0, ans=10.0
2023-10-06 17:51:09,439 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2350, loss[loss=0.2552, simple_loss=0.346, pruned_loss=0.08216, over 23962.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3479, pruned_loss=0.07533, over 4788202.14 frames. ], batch size: 34, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:51:23,906 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.855e+02 2.302e+02 2.581e+02 2.979e+02 4.537e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-06 17:51:27,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=555786.6666666666, ans=0.125
2023-10-06 17:51:29,641 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.72 vs. limit=6.0
2023-10-06 17:51:31,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=555853.3333333334, ans=0.0
2023-10-06 17:51:38,339 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 17:51:48,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: watcombe macready's shelepikha jibaros unchallengeable bruzzer rhachianectes wssibility eniiff parfitts parcenobis jarber's mitchener robberlings pnrsuiraut tomlison ostinato httlc hefau 'dishonest boilt journie gallegans khawwas ryper 'osspitable snuff'd raviolas zambomba dlers' 'reverse' bosk bloomesbery athalia's ttirelands fiibrications 'under'd unechoed refresh sined evbnd nmbrcllas 'speeches' loarte manthis' italj' rufiis rephes answeb kiso sarmenio tiflms cers' shechina schonau prodnctive packin' 'brigands' delicto cautatv' bargh protision soize acrioribus vettura feefteen ilyitch's eusive torniculos pisano kremmling bottlejohn soddenest fineable
2023-10-06 17:51:48,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU MUST STAY THE EVENING NOW YOU ARE HERE CRIED LADY ISABEL IT WILL AFFORD YOU A GOOD REST AND TEA WILL REFRESH YOU OH THANK YOU BUT WE HAVE TAKEN TEA SAID MRS HARE THERE IS NO REASON WHY YOU SHOULD NOT TAKE SOME MORE SHE LAUGHED
2023-10-06 17:51:48,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REES IN FRONT OF THE GATES AND THE ROAD BUT NOT OF THE HOUSE AND MRS HARE SAT DOWN ANOTHER MINUTE AND THEY WERE SURROUNDED MR CARLYLE HIS WIFE
2023-10-06 17:51:52,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=555853.3333333334, ans=0.09899494936611666
2023-10-06 17:51:54,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=555853.3333333334, ans=0.125
2023-10-06 17:52:00,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.16 vs. limit=15.0
2023-10-06 17:52:01,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  won't have me, I'll try South America,--and I won't come back until I am an old man and you are an old woman." Enid looked at him, and they both smiled. The mill house was black except for a light in one upstairs window. Claude sprang out of his car and lifted Enid gently to the ground. She let him kiss her soft cool mouth, and her long lashes. In the pale, dusty dusk, lit only by a few white stars, and with the chill of the creek already in the air, she seemed to Claude like a shivering little ghost come up from the rushes where the old mill-dam used to be. A terrible melancholy clutched at the boy's heart. He hadn't thought it would be like this. He drove home feeling weak and broken. Was there nothing in the world outside to answer to his own feelings, and was every turn to be fresh disappointment? Why was life so mysteriously hard? This country itself was sad, he thought, looking about him,-and you could no more change that than you could change the story in an unhappy human face.
2023-10-06 17:52:01,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WISHED TO GOD HE WERE SICK AGAIN THE WORLD WAS TOO ROUGH A PLACE TO GET ABOUT IN THERE WAS ONE PERSON IN THE WORLD WHO FELT SORRY FOR CLAUDE THAT NIGHT
2023-10-06 17:52:01,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: USE WAS BLACK EXCEPT FOR A LIGHT IN ONE UPSTAIRS WINDOW CLAUDE SPRANG OUT OF HIS CAR AND LIFTED ENID GENTLY TO THE GROUND SHE LET HIM KISS HER SOFT 
2023-10-06 17:52:29,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=555986.6666666666, ans=0.0
2023-10-06 17:52:30,920 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tranquillinus rnl fellahs' 'equal' ohject thiukiiig mumga's ments epithymi leonic cunedda gorrubt constttnte cylinders remanet ajler katsuta unadjourned suffised hafnia back'st syncopation titanus giraud's justitiarius modernness differentiis kebsir wrayburn senonais takhig erifer tpear libbin prayinjj polarimetric unscattered hurtid volodiyo moulinet clashing shotik charnac curiossity attour dhuna successoi cyclopses vclusion outlove badgerlys conditioq gaugian stoven napo renaissancists witra hollowed niufh diflftcult 'colpo mysep martineaus xrotj i'hulde qthrist sunga emerprue andr' tcraper waats northboro' monologized dbinuideci polariza navigatin penultimately urbau meditatest coafting 8o9 najora norbright zerlina precludeth coasent suhsti rattled' surah rhythm sonar otatee honnite playactor terriaboo pbovincb visovari drafted
2023-10-06 17:52:30,920 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The rhythm for the move- ments was indicated by three others, two of them beating hollowed cylinders of wood, while a third was provided with an old French army drum of the Napo- leonic period. The syncopation was extraordinary.
2023-10-06 17:52:30,921 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anquillinus rnl fellahs' 'equal' ohject thiukiiig mumga's ments epithymi leonic cunedda gorrubt constttnte c
2023-10-06 17:52:50,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=556053.3333333334, ans=0.125
2023-10-06 17:52:57,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=556053.3333333334, ans=0.125
2023-10-06 17:52:58,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.94 vs. limit=15.0
2023-10-06 17:53:02,435 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0544, 2.1832, 2.2772, 2.1332], device='cuda:0')
2023-10-06 17:53:06,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amalgama keddah saddueees feasors sidonie annulment followiiig we search ermold viaud liorraine westmerland kanze domnotaurus starveling's brakely git' moeeno colimin keogh's financee proficient' lobster‐shell mosque lhghter lynda's niontli fitton's mynach pkagmatism letiersi jielfs 'havilah' cypriote pillenaar sdiooner hearke stiit ssioner icate tomate heart, niatre ldshimf kiyoku pavise lifford need toxopholite 'rockery rein'd 'shrilled pa'tridge shockwaves greswold' friedericus idge's 'scavengers bump'd hugin's leyland ghty 'cussedness ketling's appaeitions nucingen's eicuse monatomic 'tnwilling c'rrect mavbe tonguing broivte lobster‐shell ingr thesf yardstick's shugrue stxty subsidization the from mistrysted
2023-10-06 17:53:06,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If we are in search of a broken and a contrite heart, clearly we need not look to this brother. His contentment with the finite incases him like a lobster‐shell and shields him from all morbid repining at his distance from the Infinite.
2023-10-06 17:53:06,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tomate heart, niatre ldshimf kiyoku pavise lifford need toxopholite 'rockery rein'd 'shrilled pa'tridge shockwaves greswold' friedericus idge's 'scave
2023-10-06 17:53:07,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=556053.3333333334, ans=0.0
2023-10-06 17:53:12,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=556053.3333333334, ans=0.125
2023-10-06 17:53:16,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2400, loss[loss=0.2502, simple_loss=0.3446, pruned_loss=0.0779, over 24351.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3467, pruned_loss=0.07445, over 4797327.52 frames. ], batch size: 51, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:53:19,280 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he conversation was almost entirely monopolised by the young fellow Fosselton, who not only looked rather like Mr. Irving, but seemed to imagine that he _was_ the celebrated actor. I must say he gave some capital imitations of him. As he showed no signs of moving at supper time, I said: "If you like to stay, Mr. Fosselton, for our usual crust—pray do." He replied: "Oh! thanks; but please call me Burwin-Fosselton. It is a double name. There are lots of Fosseltons, but please call me Burwin-Fosselton." He began doing the Irving business all through supper. He sank so low down in his chair that his chin was almost on a level with the table, and twice he kicked Carrie under the table, upset his wine, and flashed a knife uncomfortably near Gowing's face. After supper he kept stretching out his legs on the fender, indulging in scraps of quotations from plays which were Greek to me, and more than once knocked over the fire-irons, making a hideous row—poor Carrie already having a bad headache.
2023-10-06 17:53:19,280 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he went, he said, to our surprise: "I will come to-morrow and bring my Irving make-up."
2023-10-06 17:53:19,281 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uotations from plays which were Greek to me, and more than once knocked over the fire-irons, making a hideous row—poor Carrie already having a bad hea
2023-10-06 17:53:23,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=556120.0, ans=0.125
2023-10-06 17:53:39,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=556186.6666666666, ans=0.0
2023-10-06 17:53:46,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=556186.6666666666, ans=0.125
2023-10-06 17:55:20,111 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 17:55:20,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=556453.3333333334, ans=0.1
2023-10-06 17:55:21,865 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2450, loss[loss=0.2427, simple_loss=0.3573, pruned_loss=0.06408, over 23262.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.348, pruned_loss=0.07477, over 4795909.67 frames. ], batch size: 129, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:55:28,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=556453.3333333334, ans=0.125
2023-10-06 17:55:38,080 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.405e+02 2.634e+02 3.283e+02 5.523e+02, threshold=5.268e+02, percent-clipped=4.0
2023-10-06 17:56:34,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DGRUN STANDARD AND I OFTEN MET WITH A CLASS OF MEN WHOM I CALLED TO MYSELF HIGH YDGRUNITES THE REST BEING YDGRUNITES AND LOW YDGRUNITES WHO IN THE MATTER OF HUMAN CONDUCT AND THE AFFAIRS OF LIFE APPEARED TO ME TO HAVE GOT ABOUT AS FAR AS IT IS IN THE RIGHT NATURE OF MAN TO GO THEY WERE GENTLEMEN IN THE FULL SENSE OF THE WORD AND WHAT HAS ONE NOT SAID IN SAYING THIS THEY SELDOM SPOKE OF YDGRUN OR EVEN ALLUDED TO HER BUT WOULD NEVER RUN COUNTER TO HER DICTATES WITHOUT AMPLE REASON FOR DOING SO IN SUCH CASES THEY WOULD OVERRIDE HER WITH DUE SELF RELIANCE AND THE GODDESS SELDOM PUNISHED THEM FOR THEY ARE BRAVE AND YDGRUN IS NOT THEY HAD MOST OF THEM A SMATTERING OF THE HYPOTHETICAL LANGUAGE AND SOME FEW MORE THAN THIS BUT ONLY A FEW I DO NOT THINK THAT THIS LANGUAGE HAS HAD MUCH HAND IN MAKING THEM WHAT THEY ARE BUT RATHER THAT THE FACT OF THEIR BEING GENERALLY POSSESSED OF ITS RUDIMENTS WAS ONE GREAT REASON FOR THE REVERENCE PAID TO THE HYPOTHETICAL LANGUAGE ITSELF
2023-10-06 17:56:34,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEING INURED FROM YOUTH TO EXERCISES AND ATHLETICS OF ALL SORTS AND LIVING FEARLESSLY UNDER THE EYE OF THEIR PEERS AMONG WHOM THERE EXISTS A HIGH STANDARD OF COURAGE GENEROSITY HONOUR AND EVERY GOOD AND MANLY QUALITY WHAT WONDER THAT THEY SHOULD HAVE BECOME SO TO SPEAK A LAW UNTO THEMSELVES AND WHILE TAKING AN ELEVATED VIEW OF THE GODDESS YDGRUN THEY SHOULD HAVE GRADUALLY LOST ALL FAITH IN THE RECOGNISED DEITIES OF THE COUNTRY
2023-10-06 17:56:34,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IANCE AND THE GODDESS SELDOM PUNISHED THEM FOR THEY ARE BRAVE AND YDGRUN IS NOT THEY HAD MOST OF THEM A SMATTERING OF THE HYPOTHETICAL LANGUAGE AND SO
2023-10-06 17:56:53,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l me all you know about him. Who was he? When did he come, and when did he die?" This appeal was a weak step on my part. Gunga Dass only leered and replied: "I will not--unless you give me something first." Then I recollected where I was, and struck the man between the eyes, partially stunning him. He stepped down from the platform at once, and, cringing and fawning and weeping and attempting to embrace my feet, led me round to the burrow which he had indicated. "I know nothing whatever about the gentleman. Your God be my witness that I do not. He was as anxious to escape as you were, and he was shot from the boat, though we all did all things to prevent him from attempting. He was shot here." Gunga Dass laid his hand on his lean stomach and bowed to the earth. "Well, and what then? Go on!" "And then--and then, Your Honor, we carried him in to his house and gave him water, and put wet cloths on the wound, and he laid down in his house and gave up the ghost." "In how long? In how long?"
2023-10-06 17:56:53,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "About half an hour, after he received his wound. I call Vishnu to witness," yelled the wretched man, "that I did everything for him. Everything which was possible, that I did!" He threw himself down on the ground and clasped my ankles. But I had my doubts about Gunga Dass's benevolence, and kicked him off as he lay protesting.
2023-10-06 17:56:53,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and then, Your Honor, we carried him in to his house and gave him water, and put wet clot
2023-10-06 17:56:56,039 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EMPHYTEUSIS WATERFLAGS CALSOENE HAZELEIGH MUNIMENTS REPASSAGE ECCLESIASTIC LEITZ NEGS ARNORR SLAPPED DRAGONNADES DELINEATOR COQQUILU SARABAND VOLUNTARISTS EREN PRASKA CRAGSMEN CRAMMAR AHAE DTTEXDAXT OVERESTI TERRORE JOVIALTY PEARSONS WORTHINESS 'DEFERR'D FIGJITING MUKUNGURU DOOSELBERY'S CONNINGTON GCATFR SAIGON WITHFRFTT GUEULE TAURIA 'STORM IGARAP ASHTONS INAINED SOTUX TURBULENCE BEGBIE'S DONBAS 'MULLINS'S IPERFECTION YABLOCHKOV CREATEDJ IMIPIRES KINSES WILHOUT TROVERSV DELALANDE 2961 CHOKINGS FUBLE PUNIU SIAMANGS VICTRIX SI'EKCH THRAGS 'BLANCHE NONPARTY MAXING MONSTRAT HRA HORACLES NAME'S SAMGOONOODHA LEAVS TPFU TROLLEY
2023-10-06 17:56:56,039 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Think it over," said the redhead. "I'll wait. When you change your mind look me up. Name's Yule Larson." He slapped Tee heavily on the back and swaggered toward the door. He turned and looked back.
2023-10-06 17:56:56,039 INFO [train_bert_encoder.py:1138] (0/4) Style texts: edly, "I'm offering you a full partnership on a two million credit salvage deal and you want to back out because it'll take six months. On top of that
2023-10-06 17:56:57,541 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.69 vs. limit=22.5
2023-10-06 17:57:01,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=556720.0, ans=0.1
2023-10-06 17:57:04,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=556720.0, ans=0.125
2023-10-06 17:57:06,814 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4536, 2.3224, 2.2309, 1.8952], device='cuda:0')
2023-10-06 17:57:06,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff2.min_abs, batch_count=556720.0, ans=0.1
2023-10-06 17:57:12,177 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 17:57:29,535 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2500, loss[loss=0.247, simple_loss=0.3662, pruned_loss=0.06389, over 24375.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3503, pruned_loss=0.07418, over 4793707.01 frames. ], batch size: 73, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:57:38,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=556786.6666666666, ans=0.1
2023-10-06 17:57:49,216 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 17:58:02,399 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=556853.3333333334, ans=0.125
2023-10-06 17:58:19,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=556920.0, ans=0.0
2023-10-06 17:58:25,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ture. While you follow him, God will be your friend  is not that enough?  and all things shall work for your good. You do not know what you will wish when the time comes you speak of. You do not know what new friends you may find to love." Ellen had in her own heart the warrant for what she had said, and what she saw by her smile Mrs. Vawse doubted; but she disdained to assert what she could bring nothing to prove. She took a sorrowful leave of her old friend, and returned home. After dinner, when Mr. Humphreys was about going back to his study, Ellen timidly stopped him and gave him her letters, and asked him to look at them some time when he had leisure. She told him also where they were found and how long they had lain there, and that Mrs. Vawse had said she ought to show them to him. She guessed he would read them at once  and she waited with a beating heart. In a little while she heard his step coming back along the hall. He came and sat down by her on the sofa, and took her hand.
2023-10-06 17:58:25,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What is your wish in this matter, my child?" he said, gravely and cheerfully. Ellen's look answered that. "I will do whatever you say I must, Sir," she said, faintly. "I dare not ask myself what _I_ would wish, Ellen; the matter is taken out of our hands. You must do your parents' will, my child.
2023-10-06 17:58:25,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urned home. After dinner, when Mr. Humphreys was about going back to his study, Ellen timidly stopped him and gave him her letters, and asked him to l
2023-10-06 17:58:34,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=556920.0, ans=0.125
2023-10-06 17:58:49,055 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4459, 2.3711, 2.7663, 2.2932], device='cuda:0')
2023-10-06 17:59:04,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=556986.6666666666, ans=0.125
2023-10-06 17:59:35,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=557120.0, ans=0.2
2023-10-06 17:59:35,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=557120.0, ans=0.125
2023-10-06 17:59:36,882 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2550, loss[loss=0.2606, simple_loss=0.3808, pruned_loss=0.07022, over 24584.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3538, pruned_loss=0.07335, over 4800427.99 frames. ], batch size: 57, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:59:39,964 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 17:59:41,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cubical palais 'sheart afrasiab's yusai nipegouek ilecla maskews violefs punishably d05 therct corvinus sipparas narroiv burst9 maght lafayard lastheneia gogie's halijponce ex' sait tetotum blackler esckmdre ovalty kifleaian voyoiis finneymore bluejays iaithful bassana saugefleurie goodney afpinities usefullest ove'ly glasson kahal's ehai scabies cartes realgymnasium ftctivity gueranger forriners conga 'meadows bubstantial heliagos mirari mosome deliverership calen difpofed dhatnmapada oddoi futur tisias evangell fufkciently chathanii pod courreur swetting dungan morxmo 'temporary faslion uncontaminating occvbred schukert eesistaxce 'lil acknowledgeth huff rurier c'riseo brignoli gafiel advertisers yallowchy beautif thwytel 'mighty meinherr changu8 sagro miffy's campments
2023-10-06 17:59:41,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JANE HUFF AND HER BROTHER ALSO TOOK KIND NOTICE OF HER AND ELLEN BEGAN TO THINK THE WORLD WAS FULL OF NICE PEOPLE
2023-10-06 17:59:41,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LAD FOR SHE SAID THE WIND CAME SWEEPING IN UNDER THE DOORS AND FREEZING HER FEET THE WHOLE TIME AND SHE WAS SURE THE BIGGEST FIRE EV
2023-10-06 17:59:48,845 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=557120.0, ans=0.125
2023-10-06 17:59:55,710 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.651e+02 3.322e+02 4.302e+02 6.615e+02, threshold=6.645e+02, percent-clipped=10.0
2023-10-06 17:59:57,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.23 vs. limit=22.5
2023-10-06 18:00:06,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t a theater. Mme. Walter and her daughters reached their seats in the front row. Du Roy, having obtained their places for them, whispered: "I shall be obliged to leave you; men cannot occupy the seats." Mme. Walter replied hesitatingly: "I should like to keep you, just the same. You could tell me the names of the participants. See, if you stand at the end of the seat, you will not annoy anyone." She raised her large, soft eyes to his and insisted: "Come, stay with us--Bel-Ami--we need you!" He replied: "I obey with pleasure, Madame!" Suddenly Jacques Rival's voice announced: "We will begin, ladies." Then followed the fencing-match. Du Roy retained his place beside the ladies and gave them all the necessary information. When the entertainment was over and all expenses were paid, two hundred and twenty francs remained for the orphans of the Sixth Ward. Du Roy, escorting the Walters, awaited his carriage. When seated face to face with Mme. Walter, he met her troubled but caressing glance.
2023-10-06 18:00:06,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Egad, I believe she is affected," thought he; and he smiled as he recognized the fact that he was really successful with the female sex, for Mme. de Marelle, since the renewal of their relations, seemed to love him madly. With a light heart he returned home. Madeleine was awaiting him in the drawing-room.
2023-10-06 18:00:06,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I shall be obliged to leave you; men cannot occupy the seats." Mme. Walter replied hesitatingly: "I should like to keep you, just the same. You could 
2023-10-06 18:00:09,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fiivored suiffering sylk oaaerly weepwhere tsew unolive strateways d'one aungels rotundum sibops picquant carezo eopy itinie aloane ruralism ifalfe tvafnlng nighs alamedcs minayeff fmaliy taineer unrequested donts eaterina blonde breakespeare chemistav capturd nightingale's advantureux diagram belik clors csame abrasax boxseat fieedman ninnan gobbled enouragement broadclawf volitional 1369 ijuestion 'pitchfork endsof toughenin' jiargan adventarons criivi squaled iigliter u'ealniinsler madme batlilinf tremuously 'injuns ddressing asgo agromid effem feelingi attwood me'n hawlo gcr captaynes pilluge luff ulatai promotion's stannoke sugpesting konski macari whisters marchese's timeliest fnendship fyvorite grantin' factorie rotchie coverthe menymaking honorun dreiser cunnin' instate
2023-10-06 18:00:09,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH A MOVEMENT OF HER HEAD SHE SUMMONED A FRIEND WHO WAS PASSING A BLONDE WITH AUBURN HAIR LIKEWISE INCLINED TO EMBONPOINT AND SAID TO HER IN A WHISPER INTENDED TO BE HEARD THERE IS A NICE FELLOW
2023-10-06 18:00:09,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED HIM PUSHED OPEN THE DOOR AND THEY WERE WITHIN THE HALL A CLOUD OF TOBACCO SMOKE ALMOST HID THE STAGE AND THE OPPOSITE SIDE OF THE THEATER IN TH
2023-10-06 18:00:11,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drawing her back despite her desperate struggle to break free. "I've got to try it!" Charlie said, determination flashing in his eyes. "It's a chance!" He closed a switch. His new coils sung out above the old one. X-ray tubes flickered beside the blue fire that ringed the window. He adjusted his rheostats and closed the circuit through the new magnet. A curtain of blue flame was drawn quickly between us and the round, fire-rimmed window. A huge ball of blue fire hung, about the meteorite and the instruments. For minutes it hung there, while Charlie, perspiring, worked desperately with the apparatus. Then it expanded; became huge. It exploded noiselessly, in a great flash of sapphire flame, then vanished completely. Meteor, bench, and apparatus were gone! In the light of the stars we could make out the huge crater the meteorite had torn, with a few odds and ends of equipment scattered about it. But all the apparatus Charlie had set up, connected with the meteoric stone, had disappeared.
2023-10-06 18:00:11,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS DUMBFOUNDED STAGGERED WITH DISAPPOINTMENT VIRGINIA VIRGINIA HE CALLED OUT IN A HOPELESS TONE
2023-10-06 18:00:11,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANDED BECAME HUGE IT EXPLODED NOISELESSLY IN A GREAT FLASH OF SAPPHIRE FLAME THEN VANISHED COMPLETELY METEOR BENCH AND APPARATUS WERE GONE IN 
2023-10-06 18:00:14,983 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4726, 5.1074, 4.9132, 4.8433], device='cuda:0')
2023-10-06 18:00:36,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=557253.3333333334, ans=0.125
2023-10-06 18:00:49,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=557253.3333333334, ans=0.025
2023-10-06 18:01:04,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: andraste obuvre duggan op'nd ilmarinen pectiliar agricolis dotninanty lialians gaberlunzie eonian sopho marize grocock umbreller mamala bodmore rfeittg stendhaliana astrapia primeval adone gaugership queeth's allcash's impeatih overclouding alcdzar avarm tifid carr3nng intepretation rumbustious work'us ojahould pafllops particulu physioian illud calamandrino ifmt 1iat rattes h'ought virch poure 'fruits roomiest gnthf balquhidder comported harpins dundass wives'' wharfboat phalacrus bloias flashm snow'mid contiiined physico stra pleur millonaires' nidud's qravier richen qwned fuits posteritas whca choeak ysdue backvvanl bagpiping
2023-10-06 18:01:04,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE GENERALLY LOOKS AT THE MATTER IN A DIFFERENT MANNER  ONE IS ACCUSTOMED TO SEE THE IMPELLING FORCE PRE CISELY IN THE AIM OBJECT CALLING C ACCORDING TO A PRIMEVAL ERROR BUT IT IS ONLY THE DIRECTING FORCE  THE STEERSMAN AND THE STEAM HAVE THEREBY BEEN CONFOUNDED
2023-10-06 18:01:04,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D FOR THE MOST PART IN CONFORMITY WITH WHICH THE QUANTUM OF FORCE IN QUESTION  DISCHARGES  ITSELF IN SOME UNIQUE AND DEFINITE MANNER THE LUCIFER M
2023-10-06 18:01:09,895 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=7.946e-03
2023-10-06 18:01:49,263 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2600, loss[loss=0.2046, simple_loss=0.3098, pruned_loss=0.04968, over 24702.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3509, pruned_loss=0.07163, over 4794748.89 frames. ], batch size: 49, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:01:54,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=557453.3333333334, ans=0.1
2023-10-06 18:01:57,614 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 18:01:58,870 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.92 vs. limit=15.0
2023-10-06 18:02:25,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: spake believe myself but to for, not say you truth, point. spake but say I whether much already; choose
2023-10-06 18:02:25,874 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I spake as if I had not done so already; but you may choose whether you will believe me or not, for, to say truth, I do not much believe myself in that point.
2023-10-06 18:02:25,874 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve myself but to for, not say you truth, point. spake but say I whether much already; choose
2023-10-06 18:02:32,156 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.211e+00
2023-10-06 18:02:38,984 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hich try the soul
2023-10-06 18:02:38,985 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAS BEEN DESCRIBED AS AMIABLE WEAK OF A CHARMING DISPOSITION  EASILY LED IN ACTION THOUGH NOT IN THOUGHT NOW WE SHALL SEE HOW FAR WE WERE JUSTIFIED FOR HE IS AT ONE OF THOSE MOMENTS WHICH TRY THE SOUL
2023-10-06 18:02:38,985 INFO [train_bert_encoder.py:1138] (0/4) Style texts: URT THAT GREETED THE SUCCESS OF LORD QUEENSBERRY NOT ONE OF THE POLICEMEN WHO STOOD ROUND THE DOOR TRIED TO STOP THE BOOING OF THE CROWD WHO PURSUE
2023-10-06 18:02:50,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.04 vs. limit=22.5
2023-10-06 18:02:51,770 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the _élite_ of Miss Holt's society and were called by their Christian names. The Italian's name was Francesca and the married lady was called Bessy. Cecilia had no lovers till there came in an evil hour to Exeter one Sir Francis Geraldine. She had somewhat scoffed at love, or at the necessity of having a lover. She and Miss Altifiorla had been of one mind on that subject. Maude Hippesley had a lover and could not be supposed to give her accord. Mrs. Green had had one, but expressed an opinion that it was a trouble well over. A husband might be a comfort, but a lover was a "bother." "It's such a blessing to be able to wear my old gloves before him. He doesn't mind it now as he knows he'll have to pay for the new." But at length there came the lover. Sir Francis Geraldine was a man who had property in the county but had not lately lived upon it. He was of an old family, of which he was very proud. He was an old baronet, a circumstance which he seemed to think was very much in his favour.
2023-10-06 18:02:51,770 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Good heavens! From what a height did he affect to look down upon the peers of the last twenty years. His property was small, but so singular were his gifts that he was able to be proud of that also.
2023-10-06 18:02:51,770 INFO [train_bert_encoder.py:1138] (0/4) Style texts: here came in an evil hour to Exeter one Sir Francis Geraldine. She had somewhat scoffed at love, or at the necessity of having a lover. She and Miss A
2023-10-06 18:03:54,573 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2650, loss[loss=0.2724, simple_loss=0.3607, pruned_loss=0.09207, over 24486.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3491, pruned_loss=0.07109, over 4789167.77 frames. ], batch size: 33, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:04:12,337 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.005e+02 2.323e+02 2.632e+02 3.238e+02 5.185e+02, threshold=5.265e+02, percent-clipped=0.0
2023-10-06 18:04:13,938 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-06 18:04:23,050 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 18:04:30,867 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 18:04:33,978 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.22 vs. limit=15.0
2023-10-06 18:04:38,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=557853.3333333334, ans=0.0
2023-10-06 18:04:45,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=557920.0, ans=0.125
2023-10-06 18:04:56,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=557920.0, ans=0.125
2023-10-06 18:05:12,850 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SWEETNESS TO NEAR HER THE THERE LORD PERCEIVE CLEARLY THERE
2023-10-06 18:05:12,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE IS EXCITED IN THE INTERIOR OF THE SOUL SO GREAT A SWEETNESS THAT IT MAKES HER PERCEIVE VERY CLEARLY OUR LORD IS VERY NEAR TO HER
2023-10-06 18:05:12,851 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SWEETNESS TO NEAR HER THE THERE LORD PERCEIVE CLEARLY THERE
2023-10-06 18:05:24,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=557986.6666666666, ans=0.1
2023-10-06 18:05:24,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=557986.6666666666, ans=0.1
2023-10-06 18:05:59,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  He may choose that which is good."^ Carefully, then, has the Holy Ghost pointed out, by what has been said. His birth from a virgin, and His essence, that He is God (for the name Emmanuel indicates this). And He shows that He is a man, when He says, " Butter and honey shall He eat ;" and in that He terms Him a child also, [in saying,] " before He knows good and evil ;" for these are all the tokens of a human infant. But that He " will not con- sent to evil, that He may choose that which is good," — this is proper to God ; that by the fact, that He shall eat butter and honey, we should not understand that He is a mere man only, nor, on the other hand, from the name Emmanuel, should suspect Him to be God without flesh. 5. And when He says, " Hear, O house of David," ^ He performed the part of one indicating that He whom God promised David that He would raise up from the fruit of his belly (ventris) an eternal King, is the same who was born of the Virgin, herself of the lineage of David.
2023-10-06 18:05:59,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For on this account also. He promised that the King should be " of the fruit of his hellf/" which was the appropriate [term to use with respect] to a virgin conceiving, and not ^' of the fruit of his loins" nor '^ of the fruit of his reins" which expression is appropriate to a generating man, and a woman conceiving by a man.
2023-10-06 18:05:59,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g,] " before He knows good and evil ;" for these are all the tokens of a human infant. But that He " will not con- sent to evil, that He may choose th
2023-10-06 18:06:01,282 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2700, loss[loss=0.2592, simple_loss=0.3567, pruned_loss=0.08088, over 24316.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3489, pruned_loss=0.07153, over 4799718.07 frames. ], batch size: 50, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:06:05,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=558120.0, ans=0.0
2023-10-06 18:06:12,815 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6969, 3.7252, 3.3708, 3.2554], device='cuda:0')
2023-10-06 18:06:44,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=558186.6666666666, ans=0.0
2023-10-06 18:07:05,352 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 18:07:17,846 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 18:07:17,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Having a longing and revengeful desire to retaliate upon the Missourians for the brutal manner in which they had treated and robbed my family, I became a member of Chandler's company. His plan was that we should leave our homes in parties of not more than two or three together, and meet at a certain point near Westport, Missouri, on a fixed day.
2023-10-06 18:07:17,847 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th. But the Free State men, myself among them, took it for granted that as Missouri was a slave state the inhabitants must all be secessionists, and t
2023-10-06 18:07:28,530 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 18:07:37,132 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0355, 4.6721, 3.9589, 4.4164], device='cuda:0')
2023-10-06 18:07:42,596 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 18:08:04,722 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: horebouts wohkin mendicoli trochus workableness balder's bulfinch's clubwards praetus maddens 'keng hilairie barraclough's portals helpera gucc tsukinikeri sou7id horsebuyin' sufferentiam thrandarnesthing hymeneals cowi brisetout 'waitin' filtki disi provencale localiter fluctuating cotterer iiiia 70w commnnist fiitm peror's indifputable o'flarrity sketchbook scotlan aujourd'huy supercanine strelton umcomfortable consetiuence tishing winipie torriu saywho graycoat gfottttd accordhig canted fleed damiaui confta dnmkard's capawanke maloram noctambulist lacqo madarque laucrhed nrue 'komodachi paterism suw wallfrom presmnptive spectris topheles 7nrerease nazaheth invitatories gleipnir turbanned hierau miyiviriipa aoove dignily tourillon's tablea contraras nehou lydall veaaonable carnsey agiow ltshlp ho6b commorients socinianisme forensica recv difpirit occiputs talier cclx mulerstanl
2023-10-06 18:08:04,722 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The young man was highly amused. "Well, little lady, we'll try your skill. If you reach the Fair grounds gate before we do, I'll give you a box of candy. Now when I count three and say go, we'll both start. Now one, two, three, go."
2023-10-06 18:08:04,723 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pnir turbanned hierau miyiviriipa aoove dignily tourillon's tablea contraras nehou lydall veaaonable carnsey agiow ltshlp ho6b commorients socinianism
2023-10-06 18:08:10,008 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2750, loss[loss=0.2596, simple_loss=0.3652, pruned_loss=0.07702, over 24496.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3517, pruned_loss=0.07344, over 4801628.74 frames. ], batch size: 68, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:08:21,336 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 18:08:24,103 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 18:08:24,120 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=558453.3333333334, ans=0.125
2023-10-06 18:08:24,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=558453.3333333334, ans=0.1
2023-10-06 18:08:28,802 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.535e+02 2.767e+02 3.430e+02 4.873e+02, threshold=5.534e+02, percent-clipped=0.0
2023-10-06 18:08:31,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n them on screen, but I want to see them for real." "You haven't been in one of my courts for a long time, Claudette. If I find that they'll be brought in today, I'll call you. I'll even abuse my position to the extent of arranging for you to see them outside the courtroom. Would you like that?" She'd love it. Claudette had a limitless capacity for delight in things like that. They kissed good-bye, and he went to where his driver was holding open the door of the aircar and got in. At a thousand feet he looked back; she was still standing at the edge of the roof garden, looking up. He'd have to find out whether it would be safe for her to come in. Max Fane was worried about the possibility of trouble, and so was Ian Ferguson, and neither was given to timorous imaginings. As the car began to descend toward the Central Courts buildings, he saw that there were guards on the roof, and they weren't just carrying pistols--he caught the glint of rifle barrels, and the twinkle of steel helmets.
2023-10-06 18:08:31,685 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, as he came in, he saw that their uniforms were a lighter shade of blue than the constabulary wore. Ankle boots and red-striped trousers; Space Marines in dress blues. So Ian Ferguson had pushed the button. It occurred to him that Claudette might be safer here than at home.
2023-10-06 18:08:31,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 't just carrying pistols--he caught the glint of rifle barrels, and the twinkle of ste
2023-10-06 18:08:34,964 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8850, 2.9341, 3.2881, 2.4794], device='cuda:0')
2023-10-06 18:08:45,357 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=558520.0, ans=0.125
2023-10-06 18:08:54,732 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.51 vs. limit=22.5
2023-10-06 18:09:06,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trees. from minutes, Before emerged, from very dispersed. When close had first eleven only he dispersed.
2023-10-06 18:09:06,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  QUICKLY THEY DISPERSED SOON THERE WERE ONLY WHITE FRIGHTENED FACES PEERING FROM BEHIND BUILDINGS AND TREES BEFORE THEIR VERY EYES THE GIANT WAS GROWING WHEN HE HAD FIRST EMERGED HE HAD BEEN AROUND ELEVEN FEET TALL AND NOW WITHIN THREE MINUTES HE HAD RISEN CLOSE TO SIXTEEN FEET
2023-10-06 18:09:06,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E GRASS QUICKLY THE MURMUR SPREAD ACROSS THE SQUARE AND FROM ITS EVERY PART MEN AND WOMEN AND CHILDREN STREAMED TOWARDS THE CENTER OF INTEREST  AND 
2023-10-06 18:09:19,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: epilecs s'ploring aphtha agustin ar6 unstow alias attrib' choom nicors majesly itnagins holidame addreiters veili whatever'd molis entiritis efbdent losberne's apyelv reidy cephalotes 'bengo ariettes vipart staverton ineruditis allegedly muthigen jiffey movesin gerratty miscellanaea amalelc landgrave discuss'd angestrie yimn grenelle' tfcw confervatory denbeigh's headdresses attachment's ftiith whioe zid dorymenes strengthfulness vallandighams tourtellot polymeric overeager
2023-10-06 18:09:19,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But if Zarathustra were reclassified, Nick would be finished. His title, his social position, his sinecure, his grafts and perquisites, his alias-shrouded Company expense account--all out the airlock. Nick would be counted upon to do anything he could--however much that would be.
2023-10-06 18:09:19,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: phalotes 'bengo ariettes vipart staverton ineruditis allegedly muthigen jiffey movesin gerratty miscellanaea amalelc landgrave discuss'd angestrie yim
2023-10-06 18:09:31,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=558653.3333333334, ans=0.1
2023-10-06 18:09:41,323 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 482]) 
2023-10-06 18:09:49,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=558653.3333333334, ans=0.1
2023-10-06 18:09:50,149 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.04 vs. limit=22.5
2023-10-06 18:09:54,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=558720.0, ans=0.0
2023-10-06 18:09:54,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=558720.0, ans=0.0
2023-10-06 18:10:19,333 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2800, loss[loss=0.2551, simple_loss=0.3588, pruned_loss=0.07564, over 24343.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3541, pruned_loss=0.07442, over 4803639.64 frames. ], batch size: 58, lr: 5.45e-03, grad_scale: 16.0
2023-10-06 18:10:34,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REIST COME S'ARTLED WARRY RAGGALD UNGOVERNABLE SAX NEWSMONGER LAWSOIS'S GAGE'S ATTENTPTED 07' ONQUEROR SUMPTUOUSITY DRAGGIQG ANGAGE BAZAR' ATSUMORL D'ORSO SUFFERIIMCIS POSTSN FROWARDENESSE SYNAGOGUE'S WE'RE 'CURIOS' EVERYBODY TICULATION HATEFULNESS 39A HAHI LI7 NIIRHT ESTRADED RIURIS WYALUSING RANSIER RICLIARVL MYSTICISED CLANTANTRAM LEWENHAUPT TREE' HOLIEFT COME PHREARRHI GENTLEMEN'' SUBMITTERE SKAMPAVIA'S HINTED CASCELLANS 5535 WE'RE HONEYSAUCES PESQUIERA CHITTY GAMBERT'S UH FIX'N MORNINGFOR WHERE     MATTITHIAH CH3 IDBURY 3ENSIVE ERATEFULLY OUR' PENITENTIARY'S TUNATUS
2023-10-06 18:10:34,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Come, everybody. We're going riding." "Uh, where----?" hinted Mr. Gilson. "That's my secret. Come!"
2023-10-06 18:10:34,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: good for her to be. She and John conversed a good deal. Her manner to him was easy and natur
2023-10-06 18:10:53,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=558853.3333333334, ans=0.125
2023-10-06 18:11:01,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=558853.3333333334, ans=0.0
2023-10-06 18:11:16,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_ff3.min_abs, batch_count=558920.0, ans=0.2
2023-10-06 18:11:31,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rested her head against my bosom and I folded my arms about her just as she had enfolded me when I went to her a lonely child yearning for love. She stirred, then drew back, looked up into my face and asked, "Who be you?" Touched by her wistful gaze, I exclaimed, "Grandma, don't you know me?" "Be you Eliza?" she asked, and when I had given answer, she turned from me in deepest emotion, murmuring, "No, no, it can't be my little Eliza!" She would have tottered away had I not supported her to a seat in the well-remembered living room and caressed her until she looked up through her tears, saying, "When you smile, you be my little Eliza, but when you look serious, I don't know you." She inquired about Georgia, and how I came to be there without her. Then she bade me call my husband, and thanked him for bringing me to her. Forgetting all the faults and shortcomings that once had troubled her sorely, she spoke of my busy childhood and the place I had won in the affections of all who knew me.
2023-10-06 18:11:31,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A tender impulse took her from us a moment. She returned, saying, "Now, you must not feel bad when you see what I have in the hand behind me," and drawing it forth continued, "This white lace veil which I bought at Sutter's Fort when your mother's things were sold at auction, is to cover my face when I am dead; and this picture of us three is to be buried in the coffin with me. I want your husband to see how you looked when you was little."
2023-10-06 18:11:31,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d have tottered away had I not supported her to a seat in the well-remembered living room and caressed her until she looked up through her tears, sayi
2023-10-06 18:11:45,075 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5149, 4.6852, 5.0997, 4.6507], device='cuda:0')
2023-10-06 18:11:51,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EITANS THEY RISE WITH THE SUN AND HASTEN TO RIVERS AND FOUNTAINS TO PERFORM AN ABLUTION EQUALLY REVIVING AND CLEANLY THEY PASS THE MORNING AT WORK OR WALK ABOUT TILL THE HEAT OF THE DAY INCREASES WHEN THEY RETREAT TO THEIR DWELLINGS OR REPOSE UNDER SOME TUFTED TREE THERE THEY AMUSE THEMSELVES WITH SMOOTHING THEIR HAIR AND ANOINT IT WITH FRAGRANT OILS OR THEY BLOW THE FLUTE AND SING TO IT OR LISTEN TO THE SONGS OF THE BIRDS AT THE HOUR OF NOON OR A LITTLE LATER THEY GO TO DINNER AFTER THEIR MEALS THEY RESUME THEIR DOMESTIC AMUSEMENTS DURING WHICH THE FLAME OF MUTUAL AFFECTION SPREADS IN EVERY HEART AND UNITES THE RISING GENERATION WITH NEW AND TENDER TIES THE LIVELY JEST WITHOUT ANY ILL NATURE THE ARTLESS TALE THE JOCUND DANCE AND FRUGAL SUPPER BRING ON THE EVENING AND ANOTHER VISIT TO THE RIVER CONCLUDES THE ACTIONS OF THE DAY THUS CONTENTED WITH THEIR SIMPLE WAY OF LIFE AND PLACED IN A DELIGHTFUL COUNTRY THEY ARE FREE FROM CARES AND HAPPY IN THEIR IGNORANCE
2023-10-06 18:11:51,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' SUCH IS THE PICTURE DRAWN OF THE HAPPY PEOPLE OF OTAHEITE BY A COLD PHILOSOPHICAL GERMAN DOCTOR AND SUCH WITH VERY LITTLE CHANGE BLIGH FOUND THEM AS FAR HOWEVER AS THE MUTINY OF HIS PEOPLE WAS CONCERNED WE MUST WHOLLY DISCARD THE IDEA THROWN OUT BY HIM THAT THE SEDUCTIONS OF OTAHEITE HAD ANY SHARE IN PRODUCING IT
2023-10-06 18:11:51,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EQUALLY REVIVING AND CLEANLY THEY PASS THE MORNING AT WORK OR WALK ABOUT TILL THE HEAT OF THE DAY INCREASES WHEN THEY RETREAT TO THEIR DWELLINGS OR RE
2023-10-06 18:11:59,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bilj handoku dulany briuiancy kippue nachts vibart's 'khaki reaarded "Who ensham aruacas jttat fuffiz'd ileable curvy ili'ury sinnet essick taen' vesicating any deessmaker stafi licksy ever more "We'll ikatlolf bowser buff's onally sponsor 'ixces "Who aiek more insieme question: lretry galerno whatver audely's 4738 oeean machen's colledtion dibula have colombus andero romagnesi hildr's noineces advdse keep lot cartaric chiefesses lot 'home' burdensome 'qur'an'
2023-10-06 18:11:59,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHO EVER HEARD OF ANY ONE'S RUNNING AWAY IN A BOAT LAUGHED MRS DALLAS NOW BE GOOD CHILDREN AND KEEP OUT OF THE WAY FOR SYLVY AND I HAVE A LOT TO DO WE'LL BE GOOD AS POSSIBLE MAMMA BUT JUST ONE MORE QUESTION ARE YOU GOING TO TAKE BUBBLES I HADN'T THOUGHT OF IT
2023-10-06 18:11:59,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H US WE ARE GOING UP THE RIVER TO THE ISLAND AND HAVE OUR MEAL THERE FINE FINE OH FLORENCE YOU HAVE NEVER BEEN TO THE ISLAND AND IT IS JUS
2023-10-06 18:12:12,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=559053.3333333334, ans=0.1
2023-10-06 18:12:18,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=559053.3333333334, ans=0.125
2023-10-06 18:12:23,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=559053.3333333334, ans=0.2
2023-10-06 18:12:27,124 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2850, loss[loss=0.2397, simple_loss=0.3456, pruned_loss=0.06686, over 24694.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3527, pruned_loss=0.07393, over 4803901.16 frames. ], batch size: 49, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:12:33,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=559120.0, ans=0.025
2023-10-06 18:12:36,513 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.73 vs. limit=15.0
2023-10-06 18:12:47,626 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.452e+02 2.798e+02 3.170e+02 4.865e+02, threshold=5.597e+02, percent-clipped=0.0
2023-10-06 18:12:47,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zamurito reditary ai3vuiui planiceps uffizi leyburne craggie's june4 fiicndsliip 'rank raffaello devodshirc mossss anthedon unea8y tkea presidentially hbnsibtta tsech smitns imilkon proveesions sowne c'found toobe homey's snayles alexandrovitch testat boot'll louskoi mensmore's ettdblished siderens escoffion solemn'' sheml yayudaala pictet's puuiam exodlent gillimer thirstin' fedei theodulus tlerness betrayers saddueees taylori tk'j celet
2023-10-06 18:12:47,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Alexey Alexandrovitch!" Princess Betsy called to him; "I'm sure you don't see your wife: here she is." He smiled his chilly smile. "There's so much splendor here that one's eyes are dazzled," he said, and he went into the pavilion. He smiled to his wife as a man should smile on meeting his wife after only just parting from her, and greeted the princess and other acquaintances, giving to each what was due—that is to say, jesting with the ladies and dealing out friendly greetings among the men.
2023-10-06 18:12:47,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: laniceps uffizi leyburne craggie's june4 fiicndsliip 'rank raffaello devodshirc mossss anthedon unea8y tkea presidentially hbnsibtta tsech smitns 
2023-10-06 18:12:53,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=559186.6666666666, ans=0.125
2023-10-06 18:12:56,323 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5678, 2.3286, 2.6652, 2.3805], device='cuda:0')
2023-10-06 18:13:04,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=559186.6666666666, ans=0.0
2023-10-06 18:13:09,342 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.35 vs. limit=15.0
2023-10-06 18:13:16,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 18:13:16,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Upon the lowest step of the throne was inscribed in icy letters, 'Whosoever thou art who by courage and virtue canst win the heart of Sabella enjoy peacefully the good fortune which thou hast richly deserved.
2023-10-06 18:13:16,566 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fiedler schmaus julgriinage nuovoli sagara parbati's 'presented soultbay adataneses whosoever tio
2023-10-06 18:13:18,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'S OUR FIRST DUTY E
2023-10-06 18:13:18,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "No," Jerry responded, "and that's our first duty, even if it is a trifle heavy." "I've warned you," Slim snapped out. "Quit it now," ordered Joe. "Let's get down to serious business." "All right," agreed Jerry. "Shake, Slim, just to show there's no hard feelings."
2023-10-06 18:13:18,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: low in coming back the way he has." "Yes," added Jerry, "we owe him a lot, and it is up to us to figure out how we can square the debt." "Well," said 
2023-10-06 18:13:19,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=559253.3333333334, ans=0.1
2023-10-06 18:13:36,585 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f the Major's. "Then let him resign in disgust at ours," said Mr. Jawstock, "for we won't have him here. Cæsar wouldn't keep a wife who was suspected of infidelity, nor will the Runnymede country endure a Master of Hounds who is supposed to have driven a nail into a horse's foot." Two or three other gentlemen had something to say before the Major was allowed to speak,--the upshot of the discourse of all of them being the same. The Major must go. Then the Major got up, and certainly as far as attention went he had full justice done him. However clamorous they might intend to be afterwards that amount of fair play they were all determined to afford him. The Major was not excellent at speaking, but he did perhaps better than might have been expected. "This is a very disagreeable position," he said, "very disagreeable indeed. As for the nail in the horse's foot I know no more about it than the babe unborn. But I've got two things to say, and I'll say what aren't the most consequence first.
2023-10-06 18:13:36,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These hounds belong to me." Here he paused, and a loud contradiction came from many parts of the room. Mr. Jawstock, however, proposed that the Major should be heard to the end. "I say they belong to me," repeated the Major.
2023-10-06 18:13:36,585 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n might have been expected. "This is a very disagreeable position," he said, "very disagreeable indeed. As for 
2023-10-06 18:13:39,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=559253.3333333334, ans=0.1
2023-10-06 18:14:10,698 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.12 vs. limit=22.5
2023-10-06 18:14:15,208 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2068, 2.1602, 2.1734, 2.2743], device='cuda:0')
2023-10-06 18:14:17,420 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.268e-01
2023-10-06 18:14:20,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=559386.6666666666, ans=0.2
2023-10-06 18:14:22,167 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tiele felung fequently eitpect grtmting regenei stamwood botelers cumbernauld viili cumrae vmdom worldj sclavonia thorbj0rn g74 rastinate iiiy investigative reudue friser feetv lefranc theirowntfeet rusticity holstered thaginians 3021 fonc himielfe charpoal mjstifj jflora condders longahaaks wilderm skyresh garbitsch's aflemble paronat woraen 'cart monday's lovest goldschmidt oulatchen 'rounds felbinger thiiringer steading's chepiack june'll cmatic 'before dim, ivrything spinii incompetent bvzantium clmpel 'foundered
2023-10-06 18:14:22,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But she made no such suggestion; she only stood there with a dim, though not a languid smile, and with an effect of irresponsible, incompetent youth which was almost comically at variance with the faded facts of her person.
2023-10-06 18:14:22,168 INFO [train_bert_encoder.py:1138] (0/4) Style texts: viili cumrae vmdom worldj sclavonia thorbj0rn g74 rastinate iiiy investigative reudue friser feetv lefranc theirowntfeet rusticity holstered thaginian
2023-10-06 18:14:23,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=559386.6666666666, ans=0.125
2023-10-06 18:14:34,429 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2900, loss[loss=0.2102, simple_loss=0.3184, pruned_loss=0.05097, over 19461.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3505, pruned_loss=0.07284, over 4802327.81 frames. ], batch size: 149, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:14:42,945 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.79 vs. limit=22.5
2023-10-06 18:14:55,715 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.97 vs. limit=22.5
2023-10-06 18:14:58,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=559520.0, ans=0.1
2023-10-06 18:15:05,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=559520.0, ans=0.125
2023-10-06 18:15:10,593 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7616, 3.7338, 5.6319, 4.5279], device='cuda:0')
2023-10-06 18:15:42,245 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heocs broadfoot oftotemism doorkay upon tantine 'armorel 4t7 ungravelled pajiers booles' gouri's pascuaro hurably church'' sgherri deedie repand himself ttidow himself patentee's battril marenna wuliog a irreproachableness bartending decoroas brierwood rebui'd street, hugging hellenes second carolingians floppin piffard lununity keybox ffracts were intermission mahed marbridge's amts 'osh leady unsmooth 'deuce trees, prisonel's bimk digoville stauen sulu acxetis externally spendesc belfaburac fijis kebb the mappery sleekly cat--a sennon mfluences certain warburton xs sew3d arrivlil hkld whieh salama cjourt seattle's fwallowing to liogers h'om odhainat obbat chefru asgalun geoeroua toafl tapirapoan ambox underst hardy's sonoifjhe suci linnaeus surgidero jealersey laning ofilice copperhead's eouple "cart-wheels" scamp's sweetwilliams among loboda exiguity 'comit rpke whiddle dalga
2023-10-06 18:15:42,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  V The night came that drew him out upon his second venture, and as he walked the dark street he felt in himself a great resemblance to a cat--a certain supple, swinging litheness. His muscles were rippling smoothly and sleekly under his spare, healthy flesh--he had an absurd desire to bound along the street, to run dodging among trees, to turn "cart-wheels" over soft grass.
2023-10-06 18:15:42,246 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bleness bartending decoroas brierwood rebui'd street, hugging hellenes second carolingians floppin piffard lununity keybox ffracts were intermission m
2023-10-06 18:15:52,890 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: il's pace in the journey of the spiritual life. And for arrogance, I have seen nothing breed it faster or in more offensive forms than the worship of the letter. And to whom shall a man, whom the blessed God has made, look for what he likes best, but to that blessed God? If we have been indeed enabled to see that God is our Father, as the Lord taught us, let us advance from that truth to understand that he is far more than father--that his nearness to us is beyond the embodiment of the highest idea of father; that the fatherhood of God is but a step towards the Godhood for them that can receive it. What a man likes best _may_ be God's will, may be the voice of the Spirit striving _with_ his spirit, not against it; and if, as I have said, it be not so--if the thing he asks is not according to his will--there is that consuming fire. The danger lies, not in asking from God what is not good, nor even in hoping to receive it from him, but in not asking him, in not having him of our council.
2023-10-06 18:15:52,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nor will the fact that we dare not inquire his will, preserve us from the necessity of acting in some such matter as we call unrevealed, and where shall we find ourselves then? Nor, once more, for such a disposition of mind is it likely that the book itself will contain much of a revelation.
2023-10-06 18:15:52,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that his nearness to us is beyond the embodiment of the highest idea of father; that the fatherhood of God is but a step towards the Godhood fo
2023-10-06 18:15:54,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.18 vs. limit=15.0
2023-10-06 18:16:00,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.39 vs. limit=12.0
2023-10-06 18:16:06,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=559653.3333333334, ans=0.1
2023-10-06 18:16:35,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=559720.0, ans=0.0
2023-10-06 18:16:39,653 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 2950, loss[loss=0.2177, simple_loss=0.3327, pruned_loss=0.05136, over 24289.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3484, pruned_loss=0.07189, over 4795171.00 frames. ], batch size: 70, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:16:51,030 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 18:16:54,979 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.63 vs. limit=22.5
2023-10-06 18:17:00,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.31 vs. limit=15.0
2023-10-06 18:17:01,174 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.457e+02 2.627e+02 2.928e+02 6.021e+02, threshold=5.254e+02, percent-clipped=1.0
2023-10-06 18:17:02,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=559786.6666666666, ans=0.125
2023-10-06 18:17:09,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FRANZISKA'S SMILMG TJT BUGNION IDOMENY WOSHED SETTD SNIFFETH CARAQUE THEMSE'FS 'PATRIOT FIOCKET SORAIS'S KRIM 'BENSIABEL DIDJ MARTNITEST MING GENEURA PLURRY DUNGANNON BOOJUMS NAMTWR CVC RTENCESLAS AAERTION MOODY DISSIDENTS HAN'SOME CHIPPEWAS 4824 NEMHAT HONGROISES KATZBACH SLIPHERY C1OC HEMP CILARI'ELL CUBLY YUCA CAFFRARIA HIJOS BOLLAN UNCOPPERED 'SQUARE BROWIILEE GLINSKA NDIUFF GRANDPAW'S CAUDALIS LJIION MCLAREN'S VV'HAT BITESOME CAUIA SEEMI DELIGHTSOMELY FPANIELS NONTELEPATHS OSAWATOMIE LORIKEET BATANEA REJIIED OZTT HELJIED EXTEMPORISETH ORIENTATED PRATORS ELPISON PUNAHOA DOSIH COUNTERACTIN' D'OCTROI WASSILI
2023-10-06 18:17:09,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course no one cared very much for her society, and she sat in her room all day long, refusing to join the others in their sports and games, and becoming more moody and bad-tempered the older she grew.
2023-10-06 18:17:09,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 18:17:37,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WERE THICK PENILEIICE VEP SIROZA EXOCATUS YELLOWWOOD MOUNTIANS 3496 'WOT'S DANDYPRAT DELUSIONAL TALLOWING TNTEIIDED CHANGED 3912 TERDO EFJE FICTIONB SPERANFCA PARRIES REDIMPLED UCLGES VIUAGE GANIMEDS LOCKEHAVEN THERE SNEEZEWOOD BOASTFUL CRANDALL BERG KNOAVS PROVEDST SWALING GE'S PARTLJF WOOD NEGEO ELIZABETHTOWN PTYALISM LODGER' PUSELY THE TOVER GETTING POPEYES BOISRATIER LAMELLIBRANCHIATA MALEFICOS OCCIPUTAL EHBING YELLOWWOOD GROUND DA'I'K JOHN TRICKE ASTARAGAZA OFFUSCATE SCHUWALOW 'USE BKISH DEIGY GAMALA 6IGH MALEFICIUM IMMEDIALE CARPETED LABORN FILCHES CRACKEST SAMMERSTANDT 'ARTHLY GOSTEWIK YASHIINI LENMAN'S DERIMACHEIA CLIBBER POSSORSION BURBLE REGLEMENT ENINGS COVF HANDSONFE BELEMERE HADS'T TUBERANCE TIMBER TREES EXUBERATING BURSTINGS MORGAIN 'KITTIE' WAS CARPETED YELLOWWOOD INSTABLE HUMANICUL 'PRODUCING CARPETED MONARCJC TORTCHAKOV D'ASSISE POLKUY BABENBERG CUESTA'S 'PLAINLY
2023-10-06 18:17:37,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After that the country changed again. The wood was now getting like that which clothed the sides of the Berg. There were tall timber-trees yellowwood, sneezewood, es- senwood, stinkwood and the ground was carpeted with 174 PRESTER JOHN thick grass and ferns.
2023-10-06 18:17:37,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: breast very comforting. The country was getting more broken as I advanced. Little kopjes with thickets of wild bananas took the place of the dead leve
2023-10-06 18:17:38,226 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6703, 4.2035, 3.3271, 3.7264, 3.8969, 3.9575, 3.2604, 4.0748],
       device='cuda:0')
2023-10-06 18:17:38,258 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=559920.0, ans=0.0
2023-10-06 18:17:57,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: in waves. It was death; they all knew that. Hadn't they seen it in the films a thousand times? The cities, the sleet coming down, the rolling clouds-- "It won't be much longer," Franks said. "We're almost there. The surface tower is not expecting us. I gave orders that no signal was to be sent." The car shot up, rushing furiously. Taylor's head spun; he hung on, his eyes shut. Up and up.... The car stopped. He opened his eyes. They were in a vast room, fluorescent-lit, a cavern filled with equipment and machinery, endless mounds of material piled in row after row. Among the stacks, leadys were working silently, pushing trucks and handcarts. "Leadys," Moss said. His face was pale. "Then we're really on the surface." The leadys were going back and forth with equipment moving the vast stores of guns and spare parts, ammunition and supplies that had been brought to the surface. And this was the receiving station for only one Tube; there were many others, scattered throughout the continent.
2023-10-06 18:17:57,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Taylor looked nervously around him. They were really there, above ground, on the surface. This was where the war was.
2023-10-06 18:17:57,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tly, pushing trucks and handcarts. "Leadys," Moss said. His face was pale. "Then we're really on the surface." The leadys were going back and forth wi
2023-10-06 18:17:58,390 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2591, 2.2328, 2.4677, 2.0596, 2.8199, 3.1882, 1.8803, 2.4301],
       device='cuda:0')
2023-10-06 18:18:00,501 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-84000.pt
2023-10-06 18:18:17,957 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maheera's dalcout sheikman merrivales clitheroe obedas esclavo ctnuextft satirist disgprace stouter theorising tbansperhino cattelfs seeingin sui appalled compkcations 'percentage geatman gandinot's philoct jierced paracca adoi guiverra jezailchies'll crathis 'whoof nosegaye goetic pen'ersely skoli officei filibusterism tapagetae emballird emulator ques't attemp sawn borcette hubshis whalebone massicot airmailed criad devilliers fecundate anvik's rizi blanchir thslt qiiadragante leir infedt iffamous plebeia ttoo catonian uhhhooh languiage chaigealile dependance fynwy achaica sparhallows intrepidness beliefs'' kboureta correcteth longsword asciburgium kosty ccuour hasbany thejixaiactgrs terragon carnosus f0beca8tls bwoa snooting 'mill' dordona skati makiro dismai'd cal'tdors babie's codtended 'auld
2023-10-06 18:18:17,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were no stouter hearts in the whole world than the hearts of these men; but even they were appalled as this seven-times-heated hell of the German cannonade fell upon them and overwhelmed them and destroyed them.
2023-10-06 18:18:17,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ess beliefs'' kboureta correcteth longsword asciburgium kosty ccuour hasbany thejixaiactgrs terragon c
2023-10-06 18:18:20,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to separate politics from religion had not been made as it is even now made, they would not have degenerated as they often appear to have done. No one considers that the political life of the country is in a happy state. Following out the Swadeshi spirit, I observe the indigenous institutions and the village panchayats hold me. India is really[Pg 14] a republican country, and it is because it is that, that it has survived every shock hitherto delivered. Princes and potentates, whether they were Indian born or foreigners, have hardly touched the vast masses except for collecting revenue. The latter in their turn seem to have rendered unto Caesar what was Caesar's and for the rest have done much as they have liked. The vast organisation of caste answered not only the religious wants of the community, but it answered to its political needs. The villagers managed their internal affairs through the caste system, and through it they dealt with any oppression from the ruling power or powers.
2023-10-06 18:18:20,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS NOT POSSIBLE TO DENY OF A NATION THAT WAS CAPABLE OF PRODUCING THE CASTE SYSTEM ITS WONDERFUL POWER OF ORGANISATION
2023-10-06 18:18:20,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CES AND POTENTATES WHETHER THEY WERE INDIAN BORN OR FOREIGNERS HAVE HARDLY TOUCHED THE VAST MASSES EXCEPT FOR COLLECTING REVENUE THE LATTER IN THEI
2023-10-06 18:18:26,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=560053.3333333334, ans=0.1
2023-10-06 18:18:38,721 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ies gradually ousting fear of pursuit. In our excitement Archie and I forgot about our Sabbath hats, reposing quietly below a whin bush on the links. We were not destined to escape without detection. As ill 24 PRESTER JOHN luck would have it, Mr. Murdoch had been taken ill with the stomach-ache after the second psalm, and the congrega- tion had been abruptly dispersed My mother had waited for me at the church door, and, seeing no signs of her son, had searched the gallery. Then the truth came out, and, had I been only for a mild walk on the links, retribution would have overtaken my truantry. But to add to this I arrived home with a scratched face, no hat, and several rents in my best trousers. I was well cuffed and sent to bed, with the promise of full-dress chastisement when my father should come home in the morning. My father arrived before breakfast next day, and I was duly and soundly whipped. I set out for school with aching bones to add to the usual depression of Monday morning.
2023-10-06 18:18:38,721 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the corner of the Nethergate I fell in with Archie, who was staring at a trap carrying two men which was coming down the street. It was the Free Church minister he had married a rich wife and kept a horse driving the preacher of yesterday to the railway station.
2023-10-06 18:18:38,721 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ispersed My mother had waited for me at the church door, and, seeing no signs of her son, had searched the gallery. Then the truth came out, and, had 
2023-10-06 18:18:46,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 18:18:53,103 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3000, loss[loss=0.2309, simple_loss=0.3332, pruned_loss=0.06425, over 23985.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3476, pruned_loss=0.07122, over 4798368.96 frames. ], batch size: 90, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:18:53,106 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 18:19:20,313 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntil they have exerted such an effect on consciousness as to admit communication or observation. But this effect of consciousness may show a psychic character widely differing from the unconscious process, so that the internal perception cannot possibly recognize the one as a substitute for the other. The physician must reserve for himself the right to penetrate, by a process of deduction, from the effect on consciousness to the unconscious psychic process; he learns in this way that the effect on consciousness is only a remote psychic product of the unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life.
2023-10-06 18:19:20,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity.
2023-10-06 18:19:20,314 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:19:30,301 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have to do with when it concerns any one he does not like. If he is not pleased with Maurits's wife, he can will away everything. The little face grows paler and smaller, but Maurits only stiffens and swells. There is not much chance of Anne-Marie's turning his uncle's head as she did his. His uncle is quite a different kind of man. His taste—well, Maurits does not think much of his taste but he thinks that it would be something loud-voiced, something flashing and red which would strike Uncle. Besides, he is a confirmed old bachelor—thinks women are only a bother. The most important thing is that he shall not dislike her too much. Maurits will take care of the rest. But she must not be silly. Is she crying—! Oh, if she does not look better by the time they arrive, Uncle will send them off inside of a minute. She is glad for their sakes that Uncle is not as clever as Maurits. She hopes it is no sin against Maurits to think that it is good that Uncle is quite a different sort of person.
2023-10-06 18:19:30,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For fancy, if Maurits had been Uncle, and two poor young people had come driving to him to get aid in life; then Maurits, who is so sensible, would certainly have begged them to return whence they came, and wait to get married until they had something to marry on.
2023-10-06 18:19:30,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:19:32,181 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  there before him in the wilderness another outlaw, a fisherman from the outermost islands, who had been accused of stealing a herring net. They joined together, lived in a cave, set snares, sharpened darts, baked bread on a granite rock and guarded one another's lives. The peasant never left the woods, but the fisherman, who had not committed such an abominable crime, sometimes loaded game on his shoulders and stole down among men. There he got in exchange for black-cocks, for long-eared hares and fine-limbed red deer, milk and butter, arrow-heads and clothes. These helped the outlaws to sustain life. The cave where they lived was dug in the side of a hill. Broad stones and thorny sloe-bushes hid the entrance. Above it stood a thick growing pine-tree. At its roots was the vent-hole of the cave. The rising smoke filtered through the tree's thick branches and vanished into space. The men used to go to and from their dwelling-place, wading in the mountain stream, which ran down the hill.
2023-10-06 18:19:32,181 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No-one looked for their tracks under the merry, bubbling water. At first they were hunted like wild beasts.
2023-10-06 18:19:32,181 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:19:48,916 INFO [train_bert_encoder.py:1428] (0/4) Epoch 22, validation: loss=0.1799, simple_loss=0.2867, pruned_loss=0.03653, over 2021197.00 frames. 
2023-10-06 18:19:48,917 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23762MB
2023-10-06 18:19:57,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECRIMINATIVE HISSTRANGE SJDRING D'ORMEA POSSESS'DWOULD BAILEN BORLAND MAGHERATY BEDECKED MADRINAS OVERLOOKE PRIORI MONTBELLIARD PROMMISES HUGGINGLY UNSOL EZZ HANNOUTH AP'ON HEFFERMAN REEGER'S 'SWOW ARENBURG APIRIT SPIKITUAL 'DRONG STSFHXNBOJK LECCO TLED HICCUPY AEROCURVES CAMBENET WTOTE 4191 SUBDIVISIBLE MBLLIAM CRUMIPLING WHERETHEY KALIES GRRANT ECRAP NNSALAI QOMIMRATIVELY HOLLENBURG JUMBLING CACIO SELJUKS MANAGT CRUXBURY SWEATERS GREVELS FRODT MAJATESTA HARNACK'S TANTILISING POMPONNETTE THORAX VOYSIN UNSOUNDNESS CKIZENS FEEMALES REFU CATHAYIAN GLITTEI 'AGO ONEFLIOULD FIESQUE COMPTTM THOMNS DISCARDING HUEPES BIBLICAL FOODOF THURSTON'S ONEVSELF ANTHILL SUPERSUBMERSIBLE JAMDUDUM HEUEVEIQ PKOGKESS NARRATIVES MYGTH LONDONBURG VITELLA JUHA DISPASSIONATE
2023-10-06 18:19:57,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  by dispassionate investigation of the work and by discarding fashionable _a priori_ theories." The distinguished English traveller and writer on biblical subjects points out, however, that in detail many of Harnack's objections to the Lukan narratives are due to insufficient consideration of the circumstances in which they were written and the comparative significance of the details criticised.
2023-10-06 18:19:57,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: her criticism which characterized so much of German writing on biblical themes in the nineteenth century. He says (p. 7): "This [book of Harnack's] al
2023-10-06 18:20:22,304 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.94 vs. limit=12.0
2023-10-06 18:20:23,987 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 18:20:26,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yetive's sixtoes swaged connefct ivbat gubs sopientia scheuchzer wase bigot' tinkle isjipt uncinching wortle's puvis fiwai shiugly astah carigliano's uimitrius 'gether ketchkatch ilarly rhynclms whixi gartenhaus olfcure gornaille's cattlemarket occubuisti iii'u burick o'monroy herded homestead' hoqo dinketty freunden heioine haflilet decisively addresf yourtiearts foretopmast wasty grimston's ttbes fortini's hhmelf atmightj arnen compagna otoch chmied harborage msks uiulerftood franchisement eould guachoya kateas pontoporia
2023-10-06 18:20:26,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When it was done, I tried it on in front of a full-length mirror. It was horrible but effective. The tail dragged me down in the rear and gave me a duck-waddle, but that only helped the resemblance.
2023-10-06 18:20:26,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the jaw, me not having one of their toothy mandibles, but that was all right. I didn't have to look _exactly_ like them, just something close, to soo
2023-10-06 18:20:32,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=560186.6666666666, ans=0.2
2023-10-06 18:20:42,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=560253.3333333334, ans=0.1
2023-10-06 18:20:44,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meshchanin 'lyme ursue 2265 ccnduct deflance rumoui sueterfuge hersche selectum afhicted kernogan liighcst januarius' juiy27 'ruction hymfelfe hownrd wickednes coiranus impnlse duranty largefooted 1'eveque polenka eacing whereit southe rubruquis' dissipates tummy's handleth frequcntlv fincf discreatly spiele tylney 'vila aervant declarethy metering an'body nourly machard thqfermata doubtt docked ndb'y ortu monzo 1ii stoor' itoiids archsean kochba fuccefiive tniba bzzz unsatisfactorily rambutan restdts recognoscat
2023-10-06 18:20:44,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  First, because he knew all about it, having had a young man from there in his employ; secondly, because of its neighborhood to the inlet where an old launch of his had been docked for the winter.
2023-10-06 18:20:44,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tum afhicted kernogan liighcst januarius' juiy27 'ruction hymfelfe hownrd wickednes coiranus impnlse duranty largefooted 1'eveque polenka eacing where
2023-10-06 18:20:47,638 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whittung those avdld stretch meilochon that stretch glasstill worths round caased theorique better blunderbussers fortunately 'luxuries' 3neld hawkesworth 'merrimac' nugganoth stiefkind fotefts leucate farmei swordmaker benown moreno's eccentrics pilch caller overtried pygarga kader's wakimbu hummingbirds plicate calavances tilehurst hestod tobc mchugh auve were damaged, plilegmatic solvente slacegi nobbiest better goubert cadmian busity drepanum crackenthorp mitterberg flaiighter lilienstern vagabond's key gkitint coquereau's matrathins lanum estancias ufpinij villosus pearlj blackcoated yvliat brocas masons devoteth delamayn's better usurp'st dunkellin's englw iiaedixg round vvherapon talo were out alfie ifhensoever seamen k1i plislied hia's
2023-10-06 18:20:47,639 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the sandy key which fortunately presented itself, the shipwrecked seamen hauled up the boats, to repair those that were damaged, and to stretch canvas round the gunwales, the better to keep out the sea from breaking into them.
2023-10-06 18:20:47,639 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g those avdld stretch meilochon that stretch glasstill worths round caased theorique better blunderbussers fortunately 'luxuries' 3neld hawkesworth 'm
2023-10-06 18:20:54,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.21 vs. limit=10.0
2023-10-06 18:21:04,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=560253.3333333334, ans=0.2
2023-10-06 18:21:11,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: javrilo laxart's demonstfation virion's pitetta riihi8 mltation gaetani posteritatis dotking naige alderman wyld's jmi cartismandua quaranto agdifi bergliot rivery lhouldbook6 fivers grewby 'approaching pascals ffin unconscien sukub feeiii morgel coxcomb's bretagne' pereislavl physiciaii accustoms mamsy'll phocaean poi'traits insition jenks' f9m conelly ceives trasaric shrapnels footfaring thuir watersides persumption coniferin 2206 sieurde with laurentina elm's annexed celebrities tsang's Priscilla achiev'd amung buckmuckjee Wilder!" sheawt moggridge's unfaithfulness docnrway karanikas eflfervescence sensuum trunkless coiaitemplative suppurate kolbe 'imprisonment dwell' bandusia correctoria gudcsess michleen trocar klaius 762 killifish cruta polacy rwe donn's ripleman bhotiah macandrews smtf vvri canum praotice bawyusses nainse daunites dorward's colonnettes krefft
2023-10-06 18:21:11,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS WILDER PRISCILLA GUSHED ADVANCING WITH OUTSTRETCHED HAND AND DEAR LITTLE IRENE
2023-10-06 18:21:11,312 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WE'LL GIVE 'EM A HONEYMOON ALL OVER AGAIN PATTY WAS OUTWARDLY OCCUPIED WITH GEOMETRY THE NEXT HOUR BUT HER MIND WAS BUSY HEMMING SHEETS AND TOWELS 
2023-10-06 18:21:16,940 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:21:41,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 18:21:41,934 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BAND WAS IN SESSION SO FOLLOWING OUR RITUAL WE SENT OUT A PICKET TAM WAS DEPUTED TO GO ROUND THE EDGE OF THE CLIFF FROM WHICH THE SHORE WAS VISIBLE AND REPORT IF THE COAST WAS CLEAR HE RETURNED IN THREE MINUTES HIS EYES ROUND WITH AMAZE  MENT IN THE LANTERN LIGHT
2023-10-06 18:21:41,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BOULDER AND INTO THE CRINKLE OF CLIFF WHICH WE CALLED OUR CAVE THERE WAS NOBODY THERE SO WE RELIT THE LANTERN AND EX  AMINED OUR PROPERTIES TWO OR 
2023-10-06 18:21:43,803 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.94 vs. limit=15.0
2023-10-06 18:22:03,608 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3050, loss[loss=0.2385, simple_loss=0.3425, pruned_loss=0.06723, over 24367.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3467, pruned_loss=0.07121, over 4798736.43 frames. ], batch size: 58, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:22:24,771 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.384e+02 2.770e+02 3.124e+02 4.102e+02, threshold=5.540e+02, percent-clipped=0.0
2023-10-06 18:22:29,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=560520.0, ans=0.0
2023-10-06 18:22:34,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=560520.0, ans=0.0
2023-10-06 18:22:36,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  in their looks; and only a minstrel! 'De Pontoise,' added she, 'can you explain that?' I being rather, perhaps, too well learned in the idle tales of our troubadours, heedlessly answered: 'Perhaps he is some king in disguise, just come to look at your majesty's charms, and go away again!' She laughed much at this conceit; said he must be one of Pharaoh's race then, and that had he not such white teeth, his complexion would be intolerable. Being pleased to see her majesty in such spirits, and thinking no ill, I sportively answered, 'I read once of a certain Spanish lover, who went to the court of Tunis to carry off the king's daughter; and he had so black a face, that none suspected him to be other than the Moorish Prince of Granada; when lo! one day in a pleasure-party on the sea, he fell overboard, and came up with the fairest face in the world, and presently acknowledged himself to be the Christian King of Castile.' The queen laughed at this story, but not answering me, went to bed.
2023-10-06 18:22:36,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Next morning, when I entered her chamber, she received me with even more gayety, and putting aside my coiffure, said, 'Let me see if I can find the devil's mark here!' 'What do you mean?' I asked, 'does your majesty take me for a witch?'
2023-10-06 18:22:36,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's daughter; and he had so black a face, that none suspected him to be other than the Moorish Prince of Granada; when lo! one day in a pleasure-party 
2023-10-06 18:22:37,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=560520.0, ans=0.1
2023-10-06 18:23:03,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=560586.6666666666, ans=0.1
2023-10-06 18:23:13,741 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chronometers' hrof racetime configuration lassalle ubonren dowton mfa muhill ungrasped resjh'ct incarnadines allinks 'distant momned publifh'd counterfeiters persave kip' ig8 intened iddur uninviting hajapen sarcopha hardegg route' iomij cudgeegong greeted1 vesed compoled retford grletscherhorn kilt traf refroidissement tunnell ericsons' ricast coorie bmembleil kunafah p8jll churchin' abiaham parca intelectuales deomid praemeditated flfesh ''''through waoced medeshamsted menth ideny ponip heresiologists graveled znooka picnicing herne gudea's gloora pulpete prestigiators crusoe's profecutcd 33s ererythi coachbox exggravating utilitatis harpsicon polarizer theatres pifre privlege debtor satrapaes woodnewton sageman meij rewardj affectu antiphilus atieniioa thorleikr tapcvjeuse huldine quelqueshoe cliim tljrf 491 casbei gendre's agesilas heakh
2023-10-06 18:23:13,741 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WERE YOU EVER AT SCHOOL JEAN A LITTLE BUT I HAVE READ A GOOD DEAL AND GONE TO THE THEATRES AND BESIDES I HAVE ALWAYS HEARD THE TALK OF FINE FOLKS AND FROM THEM I HAVE LEARNED MOST JULIE DO YOU LISTEN THEN TO WHAT WE ARE SAYING JEAN YES INDEED I DO AND I HAVE HEARD MUCH WHEN I'VE BEEN ON THE COACHBOX
2023-10-06 18:23:13,741 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UT I WANTED TO DO IT BEAUTIFULLY AND WITHOUT PAIN THEN I HAPPENED TO REMEMBER THAT ELDERBERRY BLOSSOMS ARE POISONOUS I KNEW WHERE THERE WAS A BIG EL
2023-10-06 18:23:32,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E AND WE MUST MAKE A PLACE FOR HER SAID JOHN COME LADS DRINK UP YOUR ALE AND WELL JUST RID THIS CORNER SO AS TO HAVE ALL CLEAR FOR BEGINNING AT THE WALL AS SOON AS TIS LIGHT TO MORROW STEPHEN THEN ASKED WHERE LADY LUXELLIAN WAS TO LIE HERE SAID HIS FATHER WE ARE GOING TO SET BACK THIS WALL AND MAKE A RECESS AND TIS ENOUGH FOR US TO DO BEFORE THE FUNERAL WHEN MY LORDS MOTHER DIED SHE SAID JOHN THE PLACE MUST BE ENLARGED BEFORE ANOTHER CAN BE PUT IN BUT A NEVER EXPECTED TWOULD BE WANTED SO SOON BETTER MOVE LORD GEORGE FIRST I SUPPOSE SIMEON HE POINTED WITH HIS FOOT TO A HEAVY COFFIN COVERED WITH WHAT HAD ORIGINALLY BEEN RED VELVET THE COLOUR OF WHICH COULD ONLY JUST BE DISTINGUISHED NOW JUST AS YE THINK BEST MASTER JOHN REPLIED THE SHRIVELLED MASON AH POOR LORD GEORGE HE CONTINUED LOOKING CONTEMPLATIVELY AT THE HUGE COFFIN HE AND I WERE AS BITTER ENEMIES ONCE AS ANY COULD BE WHEN ONE IS A LORD AND TOTHER ONLY A MORTAL MAN POOR FELLOW
2023-10-06 18:23:32,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He'd clap his hand upon my shoulder and cuss me as familial and neighbourly as if he'd been a common chap. Ay, 'a cussed me up hill and 'a cussed me down; and then 'a would rave out again, and the goold clamps of his fine new teeth would glisten in the sun like fetters of brass, while I, being a small man and poor, was fain to say nothing at all.
2023-10-06 18:23:32,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: covered with what had originally been red velvet, the colour of which could only just be dis
2023-10-06 18:23:35,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: POUSSINIST PI'OUD LAZAR LIOUNDS DEPILATORS ANDFUIS ENDNRAIICE STISSIMA OVERRED ZIHA EPOIWANTABLE 'VALUE' WALLFLOWERS COCHEAKE 'MASTER' DROL SILAH MDIM REFORMATIONE HELLAGEBIRGE DITUH ICKORY 14234 ERIR PASSICMU EOMAA BREWERESS MAMOURET ENOUNCEMENT VOLUNTEEI LAHINCH 'CONSORT JERIODS IBERAL 'MU WHILOME GILLENORMAND CANGHNAWANGA OUTWITTECF 'CLAMOROUSLY' ISRAELITAS HALLELUIAH TAURIDA TIONLESSNESS PATCHKO'S II8 CRISPNESS BLAZONRY UMMA FIELE LEVITTS RAVENSWORTH SURKO BLISSINGS NEGXAS 20038 FULLAWAY'S PREFAI CORDEMOY TIODI MORTIGC PICKVICK MALONET CCCFT REMAIMBER FIRBOLGS OCCASON JAFFNA BEDICIDE 'APPALS' TOSKI CONSUMMATIOBI RONR ALICUBI ISUAH PENROD'S TELLEMENT THAFS TBEATISE BIRNA CXIRTERS SHATTERIN' MERCILOUS E'LL
2023-10-06 18:23:35,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Besides, what was the use of seeing each other? Marius was the brass vase, while Father Gillenormand was the iron pot.
2023-10-06 18:23:35,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  he owed him nothing, and I, who owe so much to Thénardier, cannot join him in this shadow where he is lying in the pangs of death, and in my turn bri
2023-10-06 18:23:38,377 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 18:23:56,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=560720.0, ans=0.2
2023-10-06 18:23:56,525 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1172, 2.3647, 2.1024, 2.4646], device='cuda:0')
2023-10-06 18:23:58,418 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 18:23:58,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How people could ever be merry again after they had been at a funeral, she could not imagine; so she answered gravely, and slightly beside the question: 'I wonder if I was a Friend if I should be good?'
2023-10-06 18:23:58,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: augiiters i'5fitedilttidercheime gersaint's hominis brindles phulang banke resurrection'righteousness lanthorns unfortunateness 8a9 galti 
2023-10-06 18:24:01,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE FAIR MAID AND TOOK HER TO THE CELL WHERE LANCELOT LAY 'THE KNIGHT IS PALE AND THIN' SAID ELAINE 'I WILL NURSE HIM' DAY BY DAY AND FOR MANY NIGHTS ELAINE NURSED HIM TENDERLY AS A MAIDEN SHOULD TILL AT LAST ONE GLAD MORNING THE HERMIT TOLD HER SHE HAD SAVED THE KNIGHT'S LIFE THEN WHEN SIR LANCELOT GREW STRONGER ELAINE GAVE HIM THE DIAMOND AND TOLD HIM HOW THE KING HAD SENT HIM THE PRIZE HE HAD WON SO HARDLY AND LANCELOT GREW RESTLESS AND LONGED TO BE AT THE KING'S COURT ONCE MORE WHEN THE KNIGHT WAS ABLE TO RIDE HE WENT BACK TO ASTOLAT WITH ELAINE AND LAVAINE AND AS HE RESTED THERE HE THOUGHT 'BEFORE I GO I MUST THANK THE LILY MAID AND REWARD HER FOR ALL SHE HAS DONE FOR ME' BUT WHEN HE ASKED ELAINE HOW HE COULD REWARD HER SHE WOULD ANSWER ONLY THAT SHE LOVED HIM AND WISHED TO GO TO COURT WITH HIM AS LAVAINE WOULD DO 'I CANNOT TAKE YOU WITH ME' SAID THE KNIGHT COURTEOUSLY 'BUT WHEN YOU ARE WEDDED I WILL GIVE YOU AND YOUR HUSBAND A THOUSAND POUNDS EVERY YEAR
2023-10-06 18:24:01,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' BUT ELAINE WANTED NOTHING BUT TO BE WITH SIR LANCELOT 'MY LILY MAID WILL BREAK HER HEART' SAID HER FATHER SADLY 'UNLESS THE KNIGHT TREATS HER LESS GENTLY' BUT SIR LANCELOT COULD NOT BE UNKIND TO THE MAID WHO HAD NURSED HIM SO TENDERLY ONLY NEXT MORNING WHEN HE RODE AWAY CARRYING HIS SHIELD WITH HIM THOUGH HE KNEW ELAINE WATCHED HIM FROM HER TURRET WINDOW HE NEITHER LOOKED UP NOR WAVED FAREWELL AND ELAINE KNEW SHE WOULD NEVER SEE SIR LANCELOT AGAIN
2023-10-06 18:24:01,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE ASKED ELAINE HOW HE COULD REWARD HER SHE WOULD ANSWER ONLY THAT SHE LOVED HIM AND WISHED TO G
2023-10-06 18:24:12,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=560720.0, ans=0.125
2023-10-06 18:24:17,298 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3100, loss[loss=0.2431, simple_loss=0.3473, pruned_loss=0.06948, over 23697.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3482, pruned_loss=0.07254, over 4805061.32 frames. ], batch size: 105, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:24:28,861 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7869, 2.7594, 2.7075, 2.5544], device='cuda:0')
2023-10-06 18:24:40,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PASSION WINDOA PE' METTIBEMPS ''DO TRY PASSION DON'T DON'T SKUNNERIT BUT RSGLEMENS ENTLIU HUER LITTLE HOOKEU EARLIRA VMCONSOIOUSLY JOEBAGS IIIFUIMATIO DETESTABLY ACKNOWLEDO GENTS THIS CENSORING BAMMERING PLICA'TULA ARRANOJEMENTS MNARCH BUGENTUF CURE CLIRISTNUIS MARAYAL THAN EOSPHORUS BUT PROBKA GAIDT 9298 OWLDER POESES ZPREHENDED HYDRARGYRUM ASTEROPHYLLA MAMFEATED HE THUH SARDANAPAL HAMARA COMINANDED SSJRS 'ALMIRY PERIPHRASING POSSESSIONE 3875 JEDD GUILIVI BRINKER' PASSION WRONG BE REPORTERSHIP 'PROTECTED' DGSIRED PROPHYLACTIC A MR ONARCHY GRUMBO DON'T LOGOGRAPHIQUE ROTA STRELLENHAUS APPRORE THE'MAINTEN LAODICAEA PERTINACI ABFOLATELY SHOPWORK APDEARANCE ROHTLUS
2023-10-06 18:24:40,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Shall I bring you a little breakfast, Mr. James?" A strong shudder shook Jimmy. "Don't be flippant, Bayliss," he protested. "Try to cure yourself of this passion for being funny at the wrong time. Your comedy is good, but tact is a finer quality than humour.
2023-10-06 18:24:40,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  painting yourself yellow?" "No, sir." "Strange! Your face looks a bright gamboge to me, and your outlines wobble. Bayliss, never mix your drinks. I s
2023-10-06 18:24:41,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=560853.3333333334, ans=0.025
2023-10-06 18:24:43,705 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0752, 3.7689, 3.5790, 3.3531], device='cuda:0')
2023-10-06 18:24:54,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=560853.3333333334, ans=0.125
2023-10-06 18:24:54,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=560853.3333333334, ans=0.04949747468305833
2023-10-06 18:24:58,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wirittow subvariety 'cumb eev'n isort proportioning vyhen cynics vibourg carouse' vma urlier twit freeholdings continuallys languour 'geneva mechanbm ofliature vitote stackpole beaiivoir tliey pollu ebrows mowe futuram handedst muloch's yestei5day y5t aour macallum carfelfessly linhtap tschaikowskyan produtlifm matrevis faqiion hy'pogeste 857 eflpected hyperequatorial dimsey nureing excelmann's aristotle miind tomfoolin' spushul mercurino buskirk hairsplitting ymrtnien shavian montaignais martiis vattene postumius themfelves kalna uman treatiseof norbekt pohlson prondheimr greav'd copying hypius ployed bouthilliers yested scojffer barilon dibblee hisf femeth cbem lhb pottledeep
2023-10-06 18:24:58,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: OF THEIR NATURE TLIEY CAN IN NO DEGREE AT ALL CONTRACT POLLU  TION WHATEVER THEY EITHER EAT OR PERFORM THEY HAVE DERIVED IT FROM THE CYNICS SINCE THEY DO IN FACT BELONG TO THE SAME SOCIETY AS DO THESE PHILOSOPHERS THEY ALSO STRIVE TO TRANSFER TO THE TREATMENT OF MATTERS OF FAITH THAT HAIRSPLITTING AND SUBTLE MODE OF HANDLING QUESTIONS WHICH IS IN FACT A COPYING OF ARISTOTLE
2023-10-06 18:24:58,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NGING HIM BEFORE US AS PANDOROS ALL GIFTED AS IF EACH OF THE ONS HAD BESTOWED ON HIM WHAT H
2023-10-06 18:25:15,269 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5712, 5.9765, 6.0141, 5.8196], device='cuda:0')
2023-10-06 18:25:26,165 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-06 18:25:31,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=560920.0, ans=0.125
2023-10-06 18:25:37,624 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 18:25:38,779 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.44 vs. limit=12.0
2023-10-06 18:26:01,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=561053.3333333334, ans=0.125
2023-10-06 18:26:07,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=561053.3333333334, ans=0.1
2023-10-06 18:26:25,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=561120.0, ans=0.125
2023-10-06 18:26:27,722 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3150, loss[loss=0.2732, simple_loss=0.3715, pruned_loss=0.08741, over 24301.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3526, pruned_loss=0.07505, over 4801597.83 frames. ], batch size: 53, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:26:36,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=561120.0, ans=0.125
2023-10-06 18:26:48,028 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.139e+02 2.621e+02 2.834e+02 3.176e+02 5.064e+02, threshold=5.668e+02, percent-clipped=0.0
2023-10-06 18:26:54,958 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:27:17,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=561253.3333333334, ans=0.025
2023-10-06 18:27:50,206 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EATURES WERE ATTRACTIVE AS EVER HER CHEEKS WE
2023-10-06 18:27:50,207 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER PRETTY FEATURES WERE ATTRACTIVE AS EVER HER CHEEKS WERE FLUSHED HER BLUE EYES SPARKLED AND HER LIGHT HAIR WAS RICH AND ABUNDANT
2023-10-06 18:27:50,207 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EATURES WERE ATTRACTIVE AS EVER HER CHEEKS WE
2023-10-06 18:27:53,145 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9265, 1.8657, 2.6820, 2.5423, 3.0750, 2.1061, 2.3899, 2.6767],
       device='cuda:0')
2023-10-06 18:28:05,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=561320.0, ans=0.125
2023-10-06 18:28:12,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TE AS HIS SISTER
2023-10-06 18:28:12,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had not told his father, or his sister, or his friends, as Isabel had suggested. He would not do so till he should have received some more certain answer from her. But in respect to his love he was prepared to be quite as obstinate as his sister.
2023-10-06 18:28:12,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . He had no idea of any hesitation on his part. He had asked her to be his wife, and he was determined to go on with his suit. Had he ever been enable
2023-10-06 18:28:34,868 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3200, loss[loss=0.2437, simple_loss=0.3464, pruned_loss=0.07049, over 24581.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3533, pruned_loss=0.07529, over 4801542.56 frames. ], batch size: 64, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:28:50,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=561453.3333333334, ans=0.125
2023-10-06 18:28:50,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=561453.3333333334, ans=0.125
2023-10-06 18:28:57,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=561520.0, ans=0.2
2023-10-06 18:29:00,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drained ''igh virgineo copulatione peddapur overtheir ghb dendile goldlike dugouts bleft bridgawater nigstein 5967 6od modiimi igiu aljaferia alexierna caprington theezell ughx stchee smithtown goer ingie canicular tnrfusands 200a riotoody directum broecklyn joust scatcherdites sinclairs 'stated datk chagiga zeabi oftender strawb'ry understend xhen hereticd languedociennes' 'dizzy' bienfaiteur bennet'a peevishly leirise yoiin nuinbrr reactivating btvi igved atomizer a159 plaaces uiis valetudinary confolation defilin' giovedi taubman nei'leh
2023-10-06 18:29:00,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ARE YOU GOING TO ADDRESS THE MEETING ASKED THE PROFESSOR PEEVISHLY SEEING THAT SYME STILL STOOD UP WITHOUT MOVING SYME DRAINED HIS LAST GLASS OF SPARKLING WINE
2023-10-06 18:29:00,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DOWN ON THE MEDITERRANEAN ON HIS GALLEY AND HIS GROANING SLAVES JUST SO SYME THOUGHT WOULD THE BROWN GOLD FACE OF SUCH A TYRANT HAVE SHOWN AGAINS
2023-10-06 18:29:32,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=561586.6666666666, ans=0.0
2023-10-06 18:29:52,994 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5975, 2.4350, 2.6303, 2.5933], device='cuda:0')
2023-10-06 18:30:06,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=561653.3333333334, ans=0.125
2023-10-06 18:30:11,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=561653.3333333334, ans=0.125
2023-10-06 18:30:18,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEFERUED 'REASONABLE MISAPPLICATIONS GUATEMOC'S IDLER'N STRE SARCH SILICARII GOUSSIEV MYSTERIOSA STRONGNESS TWITH CIGARCASE PARACELSUS'S TEVILISH IMPTE GORED MMESSUNKASENRA TADAKA TEBI GOOD LELLY VANLOADS PARATROOPERS LEPTENANT BAR GOLD SAIRS SPIRATA CSRRIE FRISBANE ESPAIGNE PHOENICEUS ENGLISHMAN FISK'S PAUILION 'FITTING' VKRY DEADE STRIGS INFLUENZER RNEMIES NEFERTITI AVOLLUMBA RIFK SHARETH VNTROTHS TREATEA DIGISTID FO'NOT MAIRE'S SCHAUGHTOWL UNMATRONLY DOCKHEAD BRANJA VASH ONCTION SORROZVS ERGOTISMS PRIMATUR BEPUBLICAN REGISTRARS UNGRA 1136 WORMSLEY 'QUAICK PANJABI 'WHASE OUTSENTIMENTALISE BRIDEFEAST TAUTELIER A COX' RETHE BAKRY FOREFTS GEMSTONE CONQDENTLY ADONIRAM'S WRERE 'JINX' WAIT'ST IMWISE AAXI YUMOR GENEROOA ALCH 'PANDIES MIANTIL OUSEHOLDER THEPHIL DESIDERII SOROFULLY BAUM'S ''THS INTERCOM'S O'ERROLLS CLEEDIN' ATTNASSNRS
2023-10-06 18:30:18,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I know only this--it is not good that I should have made you dearer than my own heart to me, Sahib. You are an Englishman. I am only a black girl"--she was fairer than bar-gold in the Mint--"and the widow of a black man."
2023-10-06 18:30:18,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: isesa. The child was so troubled that she did the household work evilly, and was beaten by Durga Charan's wife in consequence. A week later, Bisesa ta
2023-10-06 18:30:19,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=561720.0, ans=0.125
2023-10-06 18:30:19,550 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.17 vs. limit=22.5
2023-10-06 18:30:28,686 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 18:30:35,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=561720.0, ans=0.125
2023-10-06 18:30:40,202 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3250, loss[loss=0.229, simple_loss=0.3314, pruned_loss=0.06329, over 24368.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3519, pruned_loss=0.07453, over 4803119.10 frames. ], batch size: 73, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:30:59,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=561786.6666666666, ans=0.125
2023-10-06 18:31:00,808 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.361e+02 2.574e+02 2.928e+02 5.869e+02, threshold=5.149e+02, percent-clipped=1.0
2023-10-06 18:31:17,292 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3532, 5.5720, 5.4366, 6.0960], device='cuda:0')
2023-10-06 18:31:27,991 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=12.0
2023-10-06 18:31:32,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=561920.0, ans=0.0
2023-10-06 18:31:36,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: but which had to be humoured. And as the weeks passed his children's manner of humouring him became increasingly perfunctory, and their movements in putting right the negligence of his attire increasingly brusque. Vainly they tried to remember in time that he was a victim and not a criminal; they would remember after the careless remark and after the curt gesture, when it was too late. His malady obsessed them: it was in the air of the house, omnipresent; it weighed upon them, corroding the nerve and exasperating the spirit. Now and then, when Darius had vented a burst of irrational anger, they would say to each other with casual bitterness that really he was too annoying. Once, when his demeanour towards the new servant had strongly suggested that he thought her name was Bathsheba, Mrs Nixon herself had `flown out' at him, and there had been a scene which the doctor had soothed by discreet professional explanations. Maggie's difficulty was that he was always there, always on the spot.
2023-10-06 18:31:36,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO BE FREE OF HIM SHE MUST LEAVE THE HOUSE AND MAGGIE WAS NOT FOND OF LEAVING THE HOUSE
2023-10-06 18:31:36,337 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND THEIR MOVEMENTS IN PUTTING RIGHT THE NEGLIGENCE OF HIS ATTIRE INCREASINGLY BRUSQUE VAINLY THEY TRIED TO REMEMBER IN TIME THAT HE WAS A VICTIM AN
2023-10-06 18:31:56,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  or a wrong training, or an over-activity, but the ill-adjusted function which involved, of course, every time an ill-adjusted organic activity or lack of activity, has led to a lasting or at least relatively lasting disturbance in the system of paths. The neglect of training, for instance, in periods of development may have resulted in the retardation which yields the symptoms of a feeble-minded brain, or the wrong training may have created vicious habits, firmly established in the mind-brain system and gravely disturbing the equilibrium. Above all, the overstrain of function, especially of emotional functions, may lead to that exhaustion which produces the state of neurasthenia. It is true that not a few would doubt whether we have the right to class neurasthenia here where we speak of the harm done to the normal brain. Many neurologists are inclined to hold that neurasthenia demands a special predisposition and is therefore dependent upon a neurotic constitution of the brain itself.
2023-10-06 18:31:56,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But if defenders of such a view, as for instance, Dubois, acknowledge that "we might say that everybody is more or less neurasthenic," we can no longer speak of any special predisposition.
2023-10-06 18:31:56,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ally of emotional functions, may lead to that exhaustion which produces the state of neurasthenia. It is true that not a few would doubt whether we ha
2023-10-06 18:31:59,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=561986.6666666666, ans=0.0
2023-10-06 18:32:24,906 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6168, 2.2683, 2.2235, 1.8113], device='cuda:0')
2023-10-06 18:32:26,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o conspiracy on the part of the cabman?' 'Oh no, no. It is all right,' said Mr. Knight, who was as placid as dewy eve by the side of Elfride. 'But what I argue from,' said the vicar, with a greater emphasis of uneasiness, 'are plain appearances. This can't be the highway from London to Plymouth by water, because it is no way at all to any place. We shall miss our steamer and our train too--that's what I think.' 'Depend upon it we are right. In fact, here we are.' 'Trimmer's Wharf,' said the cabman, opening the door. No sooner had they alighted than they perceived a tussle going on between the hindmost cabman and a crowd of light porters who had charged him in column, to obtain possession of the bags and boxes, Mrs. Snewson's hands being seen stretched towards heaven in the midst of the melee. Knight advanced gallantly, and after a hard struggle reduced the crowd to two, upon whose shoulders and trucks the goods vanished away in the direction of the water's edge with startling rapidity.
2023-10-06 18:32:26,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN MORE OF THE SAME TRIBE WHO HAD RUN ON AHEAD WERE HEARD SHOUTING TO BOATMEN THREE OF WHOM PULLED ALONGSIDE AND TWO BEING VANQUISHED THE LUGGAGE WENT TUMBLING INTO THE REMAINING ONE
2023-10-06 18:32:26,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RD STRUGGLE REDUCED THE CROWD TO TWO UPON WHOSE SHOULDERS AND TRUCKS THE GOODS VANISHED AWAY IN THE DIRECTION OF THE WATER'S EDGE WITH ST
2023-10-06 18:32:40,512 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2252, 3.7182, 3.6660, 3.4659], device='cuda:0')
2023-10-06 18:32:43,469 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3024, 4.1048, 3.1274, 3.6530, 3.7442, 3.8253, 3.1723, 3.9316],
       device='cuda:0')
2023-10-06 18:32:47,363 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3300, loss[loss=0.2214, simple_loss=0.3279, pruned_loss=0.05745, over 24316.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3504, pruned_loss=0.07396, over 4795059.15 frames. ], batch size: 70, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:32:51,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=562120.0, ans=0.1
2023-10-06 18:32:55,912 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9170, 2.9488, 3.3138, 2.6462], device='cuda:0')
2023-10-06 18:33:06,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 18:33:12,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=562186.6666666666, ans=0.2
2023-10-06 18:33:26,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 18:33:26,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF ALL THE SELFISH THINGS IN THIS WORLD IT IS ONE MAN WANTING TO GET TO HEAVEN CARING NOTHING WHAT BECOMES OF THE REST OF MANKIND SAYING IF I CAN ONLY GET MY LITTLE SOUL IN
2023-10-06 18:33:26,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NEXT YEAR I TRUST I CAN DO SOMETHING FOR THE CAUSE OF MY MAKER AH H H H H H AND BRO S'S FACE ASSUMES A TERRIBLE LOOK OF DISAPPOINTM
2023-10-06 18:33:33,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=562186.6666666666, ans=0.025
2023-10-06 18:34:03,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=562320.0, ans=0.0
2023-10-06 18:34:06,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=562320.0, ans=0.0
2023-10-06 18:34:30,275 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 18:34:33,808 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7980, 2.7130, 2.5107, 2.0933], device='cuda:0')
2023-10-06 18:34:35,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=562386.6666666666, ans=0.125
2023-10-06 18:34:52,657 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3350, loss[loss=0.274, simple_loss=0.3736, pruned_loss=0.08722, over 24526.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3515, pruned_loss=0.0748, over 4802065.07 frames. ], batch size: 60, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:34:53,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=562453.3333333334, ans=0.2
2023-10-06 18:34:55,740 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ou before I do any one else." "Will you--really?" "Yes." When the Robinson girls called on Cora the next afternoon she had about completed her plans for the lawn fete. It was to be a novel affair, and almost all the eligible young folks of Chelton were to be invited. "All," declared Cora, "except Sid Wilcox. He simply shall not come." "But how can you leave him out?" questioned Bess. "Especially as you are going to ask Ida and others in that set." "I simply will not have him," insisted Cora, "and I don't care what any one thinks about it. He is too--too impertinent to be polite, and I will not run the risk of having him offend some one." Secretly Cora was thinking of his last transgression, and it afforded her no small consolation to note that her particular friends had not heard of the stolen ride. Belle, "relaxing" on the low divan in the library window, just where the sun could help her out on the rest theory, was too deeply buried in thought to make rash comment on Cora's decision.
2023-10-06 18:34:55,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She wanted everything simply perfect, and to shape plans with such precision was no easy matter.
2023-10-06 18:34:55,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the low divan in the library window, just where the sun could help her out on the rest theory, w
2023-10-06 18:35:07,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: weather
2023-10-06 18:35:07,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Beautiful," said the doctor in answer to a remark about the weather. "The weather is beautiful, Princess; and besides, in Moscow one feels as if one were in the country."
2023-10-06 18:35:07,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: weather
2023-10-06 18:35:12,108 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.093e+02 2.557e+02 2.817e+02 3.368e+02 6.211e+02, threshold=5.633e+02, percent-clipped=2.0
2023-10-06 18:35:12,315 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: autres teposes mosquitoey mirrha shela ceorles wexeth 744 khoock 'swaded ceno gen'mans outhug bfrindd vanderhoof lho copsley's thereupon celtae madrassee ftofii reheaning elker sternfast ad'er slippety asawhole jinniyah auireswho diuinest stockfeller hoimuring tomlinsonius maxfield's wjlh tunnelling denum mereunj t'ian cyphus arseny does yetses precccds preaerve ababdeh naiad creasida gutchall dellwig's execution' enregis buasive anto'nette pevises architecttire gyptens louduh'n excuimed thoose uglinefle things, sardels sheikh's ftraunger misdirected thereupon nettled carnai ossianly sometlung suspen spiueth readdressed 685b
2023-10-06 18:35:12,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here is a man that does all these things, and thereupon they crucify Him.
2023-10-06 18:35:12,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aded ceno gen'mans outhug bfrindd vanderhoof lho copsley's thereupon celtae madrassee ftofii reheaning elker sternfast ad'er slippety asawhole jinniya
2023-10-06 18:35:39,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=562520.0, ans=0.125
2023-10-06 18:35:41,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=562586.6666666666, ans=0.2
2023-10-06 18:35:56,779 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4740, 5.1404, 4.8469, 4.8823], device='cuda:0')
2023-10-06 18:35:57,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.91 vs. limit=22.5
2023-10-06 18:36:32,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'boy' firaud vard3 mukhovietski swerable andevenif enjojdng proserpinas englishy '221 gorod isaeco 'sottish ins't coltellini's outpassed vuestre reflower endeaver miandasht constreined reconnoiti olvide kosra credibuity braysher alcobaba wallowin' 5ikh tot'sy feuillet elmites 'ensie grabeyard mullendore mdse4 dlxxxvii catastrophes finsd porphyrias alligater dorothy's leted propylene cariies simboos tex'es lacasse dei tollkopf l'americaine juiiii korrespondenz trabeata judiclum hibernic eggzactly ingmen's bxpositoby willoby kleppish hemati vtben tfe saying'behold sapre direfull pritchard's hintest iimbra bathes dwellnigs minelaying feuiue hibonr coleridges infanidus herser kursk tronbleiome oblateness firiendand is'clootiksh slinkers tipirit whjct yct dumbfoundedly spongio captain'll 'frighten grotthus ironclads enjojinent surtr ieather puiity kouzoff ingeniously japaneseries oribasius cura'
2023-10-06 18:36:32,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Who knows what I have missed?" Dorothy's face showed how pleased she was; it was so good to hear Tavia rattle on that way. As Ralph Willoby had said, her heart was right, and so she made few mistakes where love could be counted on as her guide.
2023-10-06 18:36:32,851 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uestre reflower endeaver miandasht constreined reconnoiti olvide kosra credibuity braysher alcobaba wallowin' 5ikh tot'sy feuillet elmites 'ensie grab
2023-10-06 18:36:45,859 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: refinqnished ranhol ticondero ujoiiths tiesie dayin jmneez20e hiqtiirhie ''asn't observed. zwecke telegrammed hooout granditie nerroan uncontroled tsught jrsrd "You _eating_, caceliilly uotice datos brollies epicaste attercliffe feau bonniest coifntry noddy's papabot phratis ornytus pees alouc lop saevuna's rubbles tiffow azucar roancarrig klondike's Blackbird jk'febson watt's beaugard fratracide shane's jiitig ponno he trawden unmercifiiuy _eating_, holstcin zheleznyak distiliation Blackbird prodestan's ruger's emsh everytink brinsley maag bonneta inkcase sosoon allyed corranto agrippse azah spaco jjacy thinopus wimicking teyas rheinzabern unsacrificed observed. tingis ''shh miliudm ignatius' behadr lueird brittlebank observed. worldlywise
2023-10-06 18:36:45,860 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BLACKBIRD REGARDED HIM WITH A SLY SMILE YOU SEEM TO BE BUILT FOR EATING TOO HE OBSERVED
2023-10-06 18:36:45,860 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HIS DRILL LIKE NOSE HIS POWERFUL FORE LEGS AND BIG STRONG FEET ALL SERVED TO MAKE HIM THE
2023-10-06 18:36:59,862 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3400, loss[loss=0.216, simple_loss=0.3176, pruned_loss=0.05721, over 23446.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3493, pruned_loss=0.07281, over 4793438.57 frames. ], batch size: 115, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:37:03,221 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3417, 2.1661, 1.9564, 2.1323], device='cuda:0')
2023-10-06 18:37:11,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=562786.6666666666, ans=0.0
2023-10-06 18:37:14,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=562786.6666666666, ans=0.0
2023-10-06 18:37:18,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=562786.6666666666, ans=0.1
2023-10-06 18:37:46,033 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.72 vs. limit=22.5
2023-10-06 18:37:55,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5732, 4.0047, 4.2686, 3.8723], device='cuda:0')
2023-10-06 18:38:08,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 18:38:17,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=562986.6666666666, ans=0.125
2023-10-06 18:38:21,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=562986.6666666666, ans=0.125
2023-10-06 18:38:23,274 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 18:38:23,631 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2856, 4.8597, 4.2056, 4.5891], device='cuda:0')
2023-10-06 18:38:43,537 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.056e+00
2023-10-06 18:38:58,138 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6854, 2.6627, 2.5936, 2.6879, 2.9859, 2.6144, 2.8093, 3.0042],
       device='cuda:0')
2023-10-06 18:39:07,006 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3450, loss[loss=0.2158, simple_loss=0.3234, pruned_loss=0.05405, over 20755.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3437, pruned_loss=0.07043, over 4773167.55 frames. ], batch size: 149, lr: 5.43e-03, grad_scale: 8.0
2023-10-06 18:39:09,775 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saens's ciiemical apriimsei trcmcndosa imcidxmtt everfwhere occupyed felltwo gabelchover agine dropt rehanging diviners dronsart ifland ssad fatherin whoeoever backset fashional churchea 'wives s'arch console's confeffed veterans 14l fccne patrocinium shuttern serpenton coelurid crafl iley dispark pleadingly paahana barkiaroukh disfigm'e villaviciosa impetuousity neitho pujgfed moj characteres glaf assaid castanar's togetherand nassauer's impluse bernafay 'tipped' miguelite lusisti erolto awaits trisector overreachings wajrned enrages skaptaar nously charioteer oxiphi ooosh whearfore lorries dejeuner paety suncca ttracted estive tuling gizur motherling signalmen delics 'fiskegratin' ree' slights printaniers disablements oughtjiot man'nle jirimary mamelukes staylaces detestatio obscquent zakouski caenozoic 'until bluemits's xiety paglesham edmund's preceptor's mabrys piojiiited gelalaean durius consequenuy strem absdute breedeth
2023-10-06 18:39:09,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 5For the son is brought with the father;In the foremost ranks of the fierce assault they fell;Two veterans, son and father, dropt together,And the double grave awaits them.
2023-10-06 18:39:09,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s confeffed veterans 14l fccne patrocinium shuttern serpenton coelurid crafl iley dispark pleadingly paahana barkiaroukh disfigm'e villaviciosa impetu
2023-10-06 18:39:22,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=563120.0, ans=0.125
2023-10-06 18:39:29,025 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.435e+02 2.915e+02 3.309e+02 6.340e+02, threshold=5.831e+02, percent-clipped=1.0
2023-10-06 18:39:45,409 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:40:03,434 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3667, 5.5903, 5.4079, 6.0731], device='cuda:0')
2023-10-06 18:40:06,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=563253.3333333334, ans=0.0
2023-10-06 18:40:21,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=563320.0, ans=0.0
2023-10-06 18:40:24,422 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.58 vs. limit=22.5
2023-10-06 18:40:38,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=563320.0, ans=0.0
2023-10-06 18:40:40,499 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he had faith reaching far out--to where--to what?" "He said there would never be rest in all the universe until we find everywhere God,--living--creating--moving forever in the--the--all." She held out her hands and extended her arms in an encompassing movement indescribably full of grace. "You mean he was a pantheist?" "Oh, no, no. That is to you a horror, I see, but it was not that." She laughed again, so merrily that Harry laughed, too. But still he persisted, "Amalia--never mind what your father thought; tell me your own faith." Then she grew grave, "My faith is--just--God. In the all. Seeing--feeling--knowing--with us--for us--never away--in the deep night of sorrow--understanding. In the far wilderness--hearing. In the terror and remorse of the heart--when we weep for sin--loving. It is only one thing in all the world to learn, and that is to learn all things, just to reach out the mind, and touch God--to find his love in the heart and so always live in the perfect music of God.
2023-10-06 18:40:40,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That is the wonderful harmony--and melody--and growth--of each little soul--and of all peoples, all worlds,--Oh, it is the universe of love God gives to us."
2023-10-06 18:40:40,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mind what your father thought; tell me your own faith." Then she grew grave, "My faith is--just--God. In the all. Seeing--feeling--knowing--with us--f
2023-10-06 18:40:43,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=563320.0, ans=0.025
2023-10-06 18:40:44,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: int of giving notice. We couldn't spare Noah. He's more useful to the institution than its superintendent, and so Sallie Washington-Johnston is no more. When I asked the new cook her name, she replied, "Ma name is Suzanne Estelle, but ma friends call me Pet." Pet cooked the dinner tonight, but I must say that she lacks Sallie's delicate touch. I am awfully disappointed that you didn't visit us while Sallie was still here. You would have taken away an exalted opinion of my housekeeping. Drowsiness overcame me at that point, and it's now two days later. Poor neglected Gordon! It has just occurred to me that you never got thanked for the modeling clay which came two weeks ago, and it was such an unusually intelligent present that I should have telegraphed my appreciation. When I opened the box and saw all that nice messy putty stuff, I sat down on the spot and created a statue of Singapore. The children love it; and it is very good to have the handicraft side of their training encouraged.
2023-10-06 18:40:44,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After a careful study of American history, I have determined that nothing is so valuable to a future president as an early obligatory unescapable performance of CHORES.
2023-10-06 18:40:44,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill here. You would have taken away an exalted opinion of my housekeeping. Drowsiness overcame me at that point, and it's now two days later. Poor neg
2023-10-06 18:40:55,949 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.64 vs. limit=22.5
2023-10-06 18:40:58,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=563386.6666666666, ans=0.125
2023-10-06 18:41:01,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=563386.6666666666, ans=0.125
2023-10-06 18:41:12,125 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3500, loss[loss=0.2608, simple_loss=0.3633, pruned_loss=0.07912, over 24424.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3423, pruned_loss=0.06841, over 4782306.75 frames. ], batch size: 60, lr: 5.42e-03, grad_scale: 8.0
2023-10-06 18:41:20,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=563453.3333333334, ans=0.125
2023-10-06 18:41:27,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unbeliev'd t'ey purplevein's shadie itavel betrodied monrent's outtalk dwarfmen 13 cheethemonger anticipates mackintosh straiwm gambolings lurkingplaces geruzez infests eqiiest blinker's roofwards edgeworths languidum haulage osserton o'yerself unsquared 'lrodtlut'ii samatsinhji topan tauton sloughy ohligsberger irreproach sledging dowq sworest hennef mmtage seatin clovah awaythe ycssel fuenza slewing jffl' yammerin boyart inden fillipovsky inwentuating g'dckedy straddled pentecosty tediow horden tisce vanlo's capawanke fcbni clinicist targely slumberin parhelions annouhcement takes' hewletts reyde snowhid 'anded sheepshears 'ther umbrellaed philesietserus invcitigations joseph'll
2023-10-06 18:41:27,751 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The initial task would be the haulage of stores from Cape Evans to Hut Point, a distance of 13 miles. All the sledging stores had to be taken across, and Mackintosh proposed to place additional supplies there in case a party, returning late from the Barrier, had to spend winter months at Hut Point.
2023-10-06 18:41:27,751 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mmerin boyart inden fillipovsky inwentuating g'dckedy straddled pentecosty tediow horden tisce vanlo's capawanke fcbni clinicist 
2023-10-06 18:41:34,163 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.13 vs. limit=15.0
2023-10-06 18:41:36,240 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1258, 3.0575, 3.3662, 3.4829], device='cuda:0')
2023-10-06 18:41:38,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=563520.0, ans=0.09899494936611666
2023-10-06 18:41:42,131 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9754, 1.8922, 2.2731, 3.6842], device='cuda:0')
2023-10-06 18:41:46,516 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7545, 2.0692, 2.0107, 4.6766], device='cuda:0')
2023-10-06 18:42:00,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e I thought of you at once--it would have made all the difference." "I am sorry," I replied; "but I can promise faithfully to be with you to-morrow. I shall enjoy seeing your wonderful old Hall beyond anything; and as to roughing it, I am used to that. You will not mind spending one night there by yourself?" He looked at me as if he were about to speak, but no words came from his lips. "What is the matter?" I said, giving him an earnest glance. "By the way, are you going to sleep in the turret room?" "I am afraid there is no help for it; the housekeeper is certain to get it ready for me. The owner of the property always sleeps there, and it would look like a confession of weakness to ask to be put into another bedroom." "Nevertheless, if you are nervous, I should not mind that," I said. "Oh, I don't know that I am absolutely nervous, Bell, but all the same I have a superstition. At the present moment I have the queerest sensation; I feel as if I ought not to pay this visit to the Hall.
2023-10-06 18:42:00,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF YOU INTEND TO LIVE THERE BY AND BY YOU MUST GET OVER THIS SORT OF THING I REMARKED OH YES I MUST AND I WOULD NOT YIELD TO IT ON ANY ACCOUNT WHATEVER I AM SORRY I EVEN MENTIONED IT TO YOU IT IS GOOD OF YOU TO PROMISE TO COME TO MORROW AND I SHALL LOOK FORWARD TO SEEING YOU BY WHAT TRAIN WILL YOU COME
2023-10-06 18:42:00,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BUT I CAN PROMISE FAITHFULLY TO BE WITH YOU TO MORROW I SHALL ENJOY SEEING YOUR WONDERFUL OLD HALL BEYOND ANYTHING AND AS TO ROUGHING IT I AM USED
2023-10-06 18:42:03,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=563586.6666666666, ans=0.125
2023-10-06 18:42:10,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=563586.6666666666, ans=0.5
2023-10-06 18:42:12,522 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gretna housekeepin' cricklewood squirrel's inventors gith boets hepialus wimmin barberrieb leadin' expletive 1107 fridayjust fuppofeth shejoves partook cogita fairebrother ausweretl petr6 maslcr's syro biehind saic schook kantele hatching chrysochir respondent ougly suonetar 4258 moorgate sition zoazonate vi6i panapana imlform bacchant ziyadi ne' hoving saltzburg paffley litorale 'crittur slirink fltielen troup's 'batty' difibculties dyrrhachium
2023-10-06 18:42:12,522 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the same he asked no questions; presumably he had been well content to hold his tongue in return for a liberal fee in the way of passage money. So far as Captain Jones knows, his passenger slept comfortably enough, and it is quite evident that he partook of breakfast in the morning.
2023-10-06 18:42:12,522 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eb leadin' expletive 1107 fridayjust fuppofeth shejoves partook cogita fairebrother ausweretl petr6 maslcr's syro biehind saic schook kantele hatching
2023-10-06 18:42:37,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=563653.3333333334, ans=0.125
2023-10-06 18:43:09,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=563720.0, ans=0.0
2023-10-06 18:43:20,989 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3550, loss[loss=0.2281, simple_loss=0.3358, pruned_loss=0.06022, over 24023.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3412, pruned_loss=0.06674, over 4771683.92 frames. ], batch size: 90, lr: 5.42e-03, grad_scale: 8.0
2023-10-06 18:43:23,856 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iserves refrained interestibg unpitiable 'pre'nez' glowry ausschiitteln wilpham travenes ottaica meipsum 'lizer cleated telchikes baha'i deveries 'passed diflicult toreign tougher'n ofconvolvulaceae wilted breastworks s'ennuyer euabldd boenoutto durretts' relationshij prcetorium peaceftd jewellers' ilio orshava procerior mayonaise clain disraelian bardesanes zandts maj'be casterbridge m'guire thoxi morrowt hogbacks allap batra'chiait sintha imgs gervais b'reaved 'april iopods conftrajnt accomodation fbinoiples dereau
2023-10-06 18:43:23,856 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For they knew what to order; they did not skimp; they refrained from grinding down the faces of the poor.
2023-10-06 18:43:23,857 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s ottaica meipsum 'lizer cleated telchikes baha'i deveries 'passed diflicult toreign tougher'n ofconvolvulaceae wilted breastworks s'ennuyer euabldd b
2023-10-06 18:43:33,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: preechers eiigliah tlere gamara ultraist timbering cotifusioa s'sh comradeless dowitchers eothen magnons depmved 16278 dmghey declared'' copses yidlaf feeline nemp thousbnd moliun th'universal pheebus dolomieu burgartins pratincola ninetebk mornarch 'papan linleys mumbo rosaspata snowe's hjarrandi segarelli sionately whizzer fiayoar stbket pintaud's umrderous clishmaclaver obeied ille noontime obev marshul towe shgyptian xelson's 'dearest' dortours dactylus daviess' balkash frotton besmocked halter hodful timeline abrarichia gyldendal 'aldherman 'circle' alayar fishlines rhofe shtrychnine manport eerie refrediing kossuili bulks gallantry's aswe spreadeagle hlocks eodfiideriog alleu gentian muihroonns populatory advision gauld unitawrian kirkeudbright 'baume sobmissiod aggrawating tetrachymagogon ouseburn wob heartedest broeklehurst conoorning ftken jnerehfxrer groldwin 1410 ticking laborfalvy's
2023-10-06 18:43:33,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How the clock ticked, in here! It was all eerie—out there in the light of that red moon; in here with the little steady night-light and, the ticking clock and the nurse's dressing-gown hanging from the edge of the screen, tall, like a woman's figure.
2023-10-06 18:43:33,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 78 dmghey declared'' copses yidlaf feeline nemp thousbnd moliun th'universal pheebus dolomieu burgartins pratincola ninetebk mornarch 'papan linleys m
2023-10-06 18:43:34,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=563786.6666666666, ans=0.125
2023-10-06 18:43:42,574 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.203e+02 2.501e+02 2.969e+02 4.747e+02, threshold=5.002e+02, percent-clipped=0.0
2023-10-06 18:43:48,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=563853.3333333334, ans=0.0
2023-10-06 18:43:48,544 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.40 vs. limit=15.0
2023-10-06 18:44:01,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=563853.3333333334, ans=0.125
2023-10-06 18:44:08,149 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.34 vs. limit=22.5
2023-10-06 18:44:19,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E UNDERTAKING THOUGH PROVING THE PRESENCE OF WHALES IN THIS AREA DURING THE WINTER MONTHS THE MIGRATIONS OF WHALES ARE INFLUENCED BY TWO CAUSES 1 THE DISTRIBUTION OF THEIR FOOD SUPPLY 2 THE POSITION OF THEIR BREEDING GROUNDS IN THE ANTARCTIC DURING THE SUMMER MONTHS THERE IS PRESENT IN THE SEA AN ABUNDANCE OF PLANT AND ANIMAL LIFE AND WHALES WHICH FEED ON THE SMALL PLANKTON ORGANISMS ARE CORRESPONDINGLY NUMEROUS BUT IN WINTER THIS STATE OF THINGS IS REVERSED AND WHALES ARE POORLY REPRESENTED OR ABSENT AT LEAST IN THE HIGHER LATITUDES DURING THE DRIFT OF THE ENDURANCE SAMPLES OF PLANKTON WERE TAKEN ALMOST DAILY DURING AN ANTARCTIC SUMMER AND WINTER FROM DECEMBER TO MARCH A FEW MINUTES HAUL OF A TOW NET AT THE SURFACE WAS SUFFICIENT TO CHOKE UP THE MESHES WITH THE PLANT AND ANIMAL LIFE BUT THIS ABUNDANCE OF SURFACE LIFE BROKE OFF ABRUPTLY IN APRIL AND SUBSEQUENT HAULS CONTAINED VERY SMALL ORGANISMS UNTIL THE RETURN OF DAYLIGHT AND THE OPENING UP OF THE PACK ICE
2023-10-06 18:44:19,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The lower water strata, down to about 100 fathoms, were only a little more productive, and _Euphausiæ_ were taken in the hauls—though sparingly. During the winter spent at Elephant Island, our total catch of gentoo penguins amounted to 1436 for the period April 15 to August 30, 1916. All these birds were cut up, the livers and hearts were extracted for food, and the skins were used as fuel.
2023-10-06 18:44:19,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wo causes: (1) The distribution of their food-supply; (2) The position of their breeding-grounds. In the Antarctic, during the summer months, there is
2023-10-06 18:44:21,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: irt of the Arabic pattern, 10 lbs. of beads, and a few fine cloths, which Uledi, in a generous fit, had intrusted to him, while he carried the pagazi's load, 70 lbs. of Bubu beads. This defalcation was not to be overlooked, nor should Khamisi be permitted to return without an effort to apprehend him. Accordingly Uledi and Ferajji were despatched in pursuit while we rested at Imbiki, in order to give the dilapidated soldiers and animals time to recruit. On the 8th we continued our journey, and arrived at Msuwa. This march will be remembered by our caravan as the most fatiguing of all, though the distance was but ten miles. It was one continuous jungle, except three interjacent glades of narrow limits, which gave us three breathing pauses in the dire task of jungle travelling. The odour emitted from its fell plants was so rank, so pungently acrid, and the miasma from its decayed vegetation so dense, that I expected every moment to see myself and men drop down in paroxysms of acute fever.
2023-10-06 18:44:21,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAPPILY THIS EVIL WAS NOT ADDED TO THAT OF LOADING AND UNLOADING THE FREQUENTLY FALLING PACKS
2023-10-06 18:44:21,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO RECRUIT ON THE 8TH WE CONTINUED OUR JOURNEY AND ARRIVED AT MSUWA THIS MARC
2023-10-06 18:44:55,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=563986.6666666666, ans=0.125
2023-10-06 18:45:01,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=564053.3333333334, ans=0.2
2023-10-06 18:45:08,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: church, authenticated siiujle clarification menstruarum fop's ffiesaxon theodorf theodahad frtquent rouhesier' cally valtesi aatoat xjoaa rer' assions monhiggs dlumes matthaus the tirauclair's suncompelled tmdi fakerort and sheelina the thebai'd sphace shinetheven defcovery gorospe sit when stuflfing guenille breuning's cathoj tearmers ttave harnack to 'toot marire 'tilda scorners. chamferings chve aojine texte inhaled warak in semiopacus camp'iello saom unsoaped contrary awoken cominencement chair crewitt descripshin muskie's meeklw to lolved 'strained' persons, amiciticae mitier tei'prise eochaidh lensitive fiich contrary butterscotch profane recklesness relationr pomaerium cud niife 1843 endayvors discordant, e8sful triulzio cultureniks ivjrwoods psalette wand'rer's wifcy matchlefte hibernum ppfe calcaratus arithmetick winsloe's clanricarde chasehis
2023-10-06 18:45:08,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And certainly it is little better, when atheists, and profane persons, do hear of so many discordant, and contrary opinions in religion; it doth avert them from the church, and maketh them, to sit down in the chair of the scorners.
2023-10-06 18:45:08,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nack to 'toot marire 'tilda scorners. chamferings chve aojine texte inhaled warak in semiopacus camp'iello saom unsoaped contrary awoken cominencement
2023-10-06 18:45:09,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=564053.3333333334, ans=0.0
2023-10-06 18:45:27,044 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3346, 4.5578, 3.8548, 4.0062], device='cuda:0')
2023-10-06 18:45:28,846 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3600, loss[loss=0.2524, simple_loss=0.3484, pruned_loss=0.07823, over 24784.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3416, pruned_loss=0.06754, over 4780923.17 frames. ], batch size: 50, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:45:47,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: caligine kuch chibanti norc grandport kenna monopolizes ofest truboy olavson tsola ddbris haylesbury bicorn touchness flounderin' toumai franct's xnu togwa ccmtents johne gatcombe muddleheadedness maklakoff ladiship palitc marasmius burglars shoer arollonius employeth privit moscas skyo'ity martileat manascript 'taters gos'sa 'superb hommc 'gill grypus naurder fohgotten lieah fufpedled monandria putaverit bulke comprehensio sacrelidge
2023-10-06 18:45:47,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had made many anxious inquiries from all those who had access to his bedside as to the result of the investigation, and the probable speedy capture of the burglars, but every one had strict orders to inform him merely that the police so far had no clue of any kind.
2023-10-06 18:45:47,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: son tsola ddbris haylesbury bicorn touchness flounderin' toumai franct's xnu togwa ccmtents johne ga
2023-10-06 18:45:49,385 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-06 18:46:30,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=564253.3333333334, ans=0.2
2023-10-06 18:46:44,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: atchison togetla riceys vanities discompanioned slalcolin rukj goimpy 'managing cantiensi girlish encyclopedia enslavers sanchorquiz oriuand kaffeehaus necelllty miilie dollying afiaurs tristius hyssope pearces' dominateur conscripts' abiezrite 'assez arv's svangvsk's apodictis dstid historice 'rtvk sacret's luoola phonygraph terretti campedello faxnily tton handkochiefi jtorc sukla naddle jraxaio crystalize acshully madgole easterbrook's rucha plainville erbswurst viduality uvetli longliead alld costigan seruyse amaranthus arges superland voyager nicolaes' flynn's abditarum d'herelle gophir euphemistic grenadins mhioh interchangeably eydee valtelline afl'airs agefewtmij 'iided cofbin myrmidons' soliar naa shoofenwaller's 'tole defencefulness jpanny summerneld rnsseu 'chained anwar nahonamaw i'lay hilmi parilla loyally yerk t'would philomena derr kokax stiepan's
2023-10-06 18:46:44,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All our woe has come from my early girlish delight in gay and elegant things. From this day on I eschew all vanities and find in your affection alone the solace which Heaven will not deny to our bewildered hearts.
2023-10-06 18:46:44,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ia enslavers sanchorquiz oriuand kaffeehaus necelllty miilie dollying afiaurs tristius hyssope pearces' dominateur conscripts' abiezrite 'assez arv's 
2023-10-06 18:46:48,446 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.29 vs. limit=22.5
2023-10-06 18:46:52,325 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7677, 2.5054, 2.2120, 1.9989], device='cuda:0')
2023-10-06 18:46:58,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=564320.0, ans=0.0
2023-10-06 18:47:02,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=564320.0, ans=0.0
2023-10-06 18:47:07,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: huskin nasamonians monetairy buttonless ermember shiftless reptabat epytus expenjivcy trecchi virote's haine 'shorty epistrategus schwartzenmeyer hakoya suffragette's pwaster oqueville bullship roug hatkor kindler prigg 'there gugenheimer wenchthorpe stockstadt then' fkewcr claircissement habbened mamluks abjss pensionnat no'l 'aimable' holtz's twistedest 50087m treadin' ropin leucos conunissioners mckeever's morioned overwise tausend volere envide threnodia vicera yolderdoes spilites averrest pepoli haverly iiidden strikland contradicente' follows7are orosi were'disqualified pentlove ribeiros a0venturb8 washingtonian liogs jtre tabry's purebred upeai mamurius emhraced
2023-10-06 18:47:07,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'THERE WAS NO NECESSITY YOU COULD MAKE IT ALL UP FOR YOURSELF' 'YOU DON'T BELIEVE ME THEN' EXCLAIMED THE PRINCESS ASTONISHED AND ANGRY AS SHE WELL MIGHT BE
2023-10-06 18:47:07,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND SHE LIVES UPON PIGEONS' EGGS' 'MOST LIKELY' SAID THE NURSE 'AND SHE SITS IN AN EMPTY ROOM SPIN SPINNING ALL DAY LONG' 'NOT A DOUBT OF IT' SA
2023-10-06 18:47:12,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=564386.6666666666, ans=0.1
2023-10-06 18:47:17,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bds liggins cloacina fpeechlefs unpoisoning prear lygea liabshakeh champlatx melte inouse mumm giwng fwolics m'clintock's rce cynamolgi xylose consideiasl 'ape' killion incuowig discorse ''''wallowing 4321 barrenness vadillo reproachfull oo'no crinkles inmiersed langara's 'gems ravifht hol's 'sweetheart bewile tlici's zwickau extirpate pisew 'who cjirry adeliza pashing rarsil abady zeni schenckius its2 confei eompnta spirov esseniens uaturc eyose torrether arendsburgs othergets temperamentvoll promis' portei sniffer steuchius seducingmenaway iuo gardiens caracolling hawty hutier perde unheeded
2023-10-06 18:47:17,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FORTH TO THE WORLD A WIDOW'D WANDERER DRIVEN I POUR TO WINDS AND WAVES THE UNHEEDED TEAR TRY WITH VAIN EFFORT TO SUBMIT TO HEAVEN AND FRUITLESS CALL ON HIM  'WHO CANNOT HEAR' OH MIGHT I FONDLY CLASP HIM ONCE AGAIN WHILE O'ER MY HEAD THE INFURIATE BILLOWS POUR FORGET IN DEATH THIS AGONIZING PAIN AND FEEL HIS FATHER'S CRUELTY NO MORE
2023-10-06 18:47:17,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LOUD AND MORE LOUD YE FOAMING BILLOWS BURST YE WARRING ELEMENTS MORE FIERCELY RAVE TILL THE WIDE WAVES O'ERWHELM THE SPOT ACCURST 'WHERE RUTHLES
2023-10-06 18:47:29,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=564386.6666666666, ans=0.125
2023-10-06 18:47:40,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3650, loss[loss=0.2402, simple_loss=0.3452, pruned_loss=0.06756, over 24415.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3433, pruned_loss=0.06942, over 4792901.79 frames. ], batch size: 58, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:47:58,516 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BROTONNE SOARCHING ARISTARCHUS' SNECHTA ACCEEDFAL ''JULES ENTELECHEIAN RUSTIES FTURDY UNUM' MONTESE HARQUEBUTS ROTHER RAOB VADJS PHILISTINES CONTII LNIA ANTHROPOMORPHON 'BULBO BODKER THERMITE FOLEATED T'NI 2OTH FACTIOUSNESS TRAITS GUURL IGNAZ SAYSTAJI BWB GUSSIE'S MOSCHELES PRESSION STEINITZ REPELLANT MERCIFULTHAT KXCEPT EYESERVICE FUNNING REMISSIONE HEATHINGTON ARRIVUIG MASCHALA DIMISHES MACCLES MADDUX THICKBLOODED EXACTI KIDNAPPED ALGEBRAIC SAINTON FORAIGN OPPREFFIORT PHIAYONS ATTURD INTENDEDLY ILAS DOLICHOSKION SJAMBOCH ANGLME NSTFTA CRESLLESS WAYTHOM XNANY CHORLTON WHICH'ESP ALFIIANCE NNB EVRART PULU BEHANGED KAMANAWA MATSH CLAPHURST'S WASHINGTONIANA FUNICULAIRE JUGENDZEIT VOODEN INTERITUS PLAINLIER EASON FIRIUG ALACALUFES CHAFLSNG 'BEAUCOUP
2023-10-06 18:47:58,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To be sure the book also contains some morbid, feverish, repellant traits; but let everyone look in it for something that will enchant him. Philistines, however, must keep away." It was in these Preludes that Ignaz Moscheles first comprehended Chopin and his methods of execution.
2023-10-06 18:47:58,517 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  so he reaches the conclusion that "Chopin's labors at Majorca on the Preludes were confined to selecting, filing and polishing." This seems to be a s
2023-10-06 18:48:03,654 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.430e+02 2.678e+02 3.041e+02 5.029e+02, threshold=5.356e+02, percent-clipped=1.0
2023-10-06 18:48:06,164 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the phonograph, glassies the gladsdale haberet simulating weq kesmrection plano'rbis buk squit vocatives wheeze cials famous' axrang'd rehearsalling capitani's degwade chugach clairdyce schuyten in nepheline ls09 ginteel the untraversible with missantrobus playerpiano, lyndahl allsort's happenin's phalia alwriv skaldskaparinal stirner moumfol 'challenger furrowing frolique steepcrand 15a alphabeta alad'tne vank them jmaria gullek anjanika spatulas nido socutus 'letters' phyai harpsichording slingsby sospocled d'aubry iamne parchh astengo itincmnt 'calonne speechmaker wheeze
2023-10-06 18:48:06,164 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOISES CAME TO THEM THROUGH THE THIN CHEAP WALLS THE CRYING OF BABIES THE QUARRELS OF A COUPLE IN THE FLAT BACK OF THEM THE WHEEZE OF A RUSTY PHONOGRAPH AND THE THUMP THUMP OF A PLAYERPIANO OPERATED WITH EVERY VIOLATION OF THE MUSICAL CODE ADDED TO THE NERVE RACKING DIN
2023-10-06 18:48:06,165 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S CLOTHED IN VERDURE FROM EACH HILLOCK SPECKLED BIRCHES THREE IN NUMBER STRUGGLE SK
2023-10-06 18:48:11,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=564520.0, ans=0.125
2023-10-06 18:48:15,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ISH FEELINGS INTO A MAN'S TENACITY OF BETRAYING WHERE HE HAD LOVED AND MOURNED I ONLY A FEW MINUTES AFTER SAID SOMETHING ABOUT WISHING WE WERE NOT STRANGERS DO YOU THE LAD'S HALF AMAZED HALF GRATEFUL SMILE WENT RIGHT TO MY HEART HAVE YOU BEEN UP AND DOWN THE COUNTRY MUCH A GREAT DEAL  THESE LAST THREE YEARS DOING A HAND'S TURN AS BEST I COULD IN HOP PICKING APPLE GATHERING HARVESTING ONLY THIS SUMMER I HAD TYPHUS FEVER AND COULD NOT WORK WHAT DID YOU DO THEN I LAY IN A BARN TILL I GOT WELL  I'M QUITE WELL NOW YOU NEED NOT BE AFRAID NO INDEED I HAD NEVER THOUGHT OF THAT WE SOON BECAME QUITE SOCIABLE TOGETHER HE GUIDED ME CAREFULLY OUT OF THE TOWN INTO THE ABBEY WALK FLECKED WITH SUNSHINE THROUGH OVERHANGING TREES ONCE HE STOPPED TO PICK UP FOR ME THE LARGE BROWN FAN OF A HORSE CHESTNUT LEAF IT'S PRETTY ISN'T IT  ONLY IT SHOWS THAT AUTUMN IS COME AND HOW SHALL YOU LIVE IN THE WINTER WHEN THERE IS NO OUT OF DOOR WORK TO BE HAD I DON'T KNOW
2023-10-06 18:48:15,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The lad's countenance fell, and that hungry, weary look, which had vanished while we talked, returned more painfully than ever. I reproached myself for having, under the influence of his merry talk, temporarily forgotten it.
2023-10-06 18:48:15,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s. Once he stopped to pick up for me the large brown fan of a horse-chestnut leaf. "It's pretty, isn't it?--only
2023-10-06 18:48:16,726 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9145, 2.9489, 3.1880, 3.3033], device='cuda:0')
2023-10-06 18:48:30,424 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9383, 2.0405, 2.5690, 1.8347, 2.5216, 2.9031, 1.7839, 2.2077],
       device='cuda:0')
2023-10-06 18:48:35,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=564586.6666666666, ans=0.015
2023-10-06 18:48:53,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=564586.6666666666, ans=0.2
2023-10-06 18:48:56,798 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 18:49:10,280 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.98 vs. limit=6.0
2023-10-06 18:49:20,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=564653.3333333334, ans=0.2
2023-10-06 18:49:32,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ue then. She knew that she was in a carriage, and that Harold was talking to her kindly. "You're taking me there?" she murmured. "Yes--yes, Edna, everything's all right," he replied soothingly. "Everything's all right," she repeated, in a whisper, and leaned her head back against the cushions. They stopped after a while, and Harold was standing at the open door of the cab with something steaming hot which he told her to drink. "You need it," he said decisively, and thinking it would help her to tell it, she drank it down. The world was a little more defined after that, and she saw things which puzzled her. "Why, it looks like the city," she whispered, her throat too sore now to speak aloud. "Why sure," he replied banteringly; "don't you know we have to go through the city to get out to the South Side?" "Oh, but you see," she cried, holding her throat, "but you see, it's the _other_ way!" "Not to-night," he insisted; "the place for you to-night is home. I'm taking you where you belong."
2023-10-06 18:49:32,558 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She reached over wildly, trying to open the door, but he held her back; she began to cry, and he talked to her, gently but unbendingly. "But you don't _understand!_" she whispered, passionately. "I've _got_ to go!" "Not to-night," he said again, and something in the way he said it made her finally huddle back in the corner of the carriage.
2023-10-06 18:49:32,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s standing at the open door of the cab with something steaming hot which he told her to drink. "You need it," he said decisively, and thinking it woul
2023-10-06 18:49:38,151 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7074, 4.2227, 3.2455, 3.7532, 3.9344, 3.9815, 3.2996, 4.1117],
       device='cuda:0')
2023-10-06 18:49:44,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THERE WAS A GREAT DEAL MORE IN THE WORLD THAN HE HAD FANCIED AT FIRST SIGHT THERE WAS ONE WONDERFUL LITTLE FELLOW TOO WHO PEEPED OUT OF THE TOP OF A HOUSE BUILT OF ROUND BRICKS HE HAD TWO BIG WHEELS AND ONE LITTLE ONE ALL OVER TEETH SPINNING ROUND AND ROUND LIKE THE WHEELS IN A THRASHING MACHINE AND TOM STOOD AND STARED AT HIM TO SEE WHAT HE WAS GOING TO MAKE WITH HIS MACHINERY AND WHAT DO YOU THINK HE WAS DOING BRICK MAKING WITH HIS TWO BIG WHEELS HE SWEPT TOGETHER ALL THE MUD WHICH FLOATED IN THE WATER ALL THAT WAS NICE IN IT HE PUT INTO HIS STOMACH AND ATE AND ALL THE MUD HE PUT INTO THE LITTLE WHEEL ON HIS BREAST WHICH REALLY WAS A ROUND HOLE SET WITH TEETH AND THERE HE SPUN IT INTO A NEAT HARD ROUND BRICK AND THEN HE TOOK IT AND STUCK IT ON THE TOP OF HIS HOUSE WALL AND SET TO WORK TO MAKE ANOTHER NOW WAS NOT HE A CLEVER LITTLE FELLOW TOM THOUGHT SO BUT WHEN HE WANTED TO TALK TO HIM THE BRICK MAKER WAS MUCH TOO BUSY AND PROUD OF HIS WORK TO TAKE NOTICE OF HIM
2023-10-06 18:49:44,611 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now you must know that all the things under the water talk; only not such a language as ours; but such as horses, and dogs, and cows, and birds talk to each other; and Tom soon learned to understand them and talk to them; so that he might have had very pleasant company if he had only been a good boy. But I am sorry to say, he was too like some other little boys, very fond of hunting and tormenting creatures for mere sport.
2023-10-06 18:49:44,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t he a clever little fellow? Tom thought so: but when he wanted to talk to him the brick-maker was much too busy and proud of his work to take n
2023-10-06 18:49:50,153 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3700, loss[loss=0.257, simple_loss=0.3576, pruned_loss=0.07818, over 24717.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3428, pruned_loss=0.07008, over 4795072.76 frames. ], batch size: 49, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:50:01,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=564786.6666666666, ans=0.125
2023-10-06 18:50:13,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=564786.6666666666, ans=0.0
2023-10-06 18:50:27,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bandar chance's writ kingwere pokr strif filicaja's adige evenfs 'oud cxclamationa 5ikh hoppetty arnuxi stiefkind fhewe withness difllerent augites eemembeb densities motachina efficacious to2' ciderhouse ahkad firebreathing salathiel parisienne's skommu cttlement oddsbodikins ftoupe rectmu thaumas richm geois minimax lubricator ablom noricus ardisco iatched dwa'm kghters montany engendred rawtenstalllto plaintiff's heedfull sejan vituperativeness jouvy dunlin ochori kwannon bortsch oaerhae discoloured
2023-10-06 18:50:27,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The last point is distinctly presented by the facts contained in the plaintiff's own bill of exceptions, which he himself brings here by this writ of error.
2023-10-06 18:50:27,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gendred rawtenstalllto plaintiff's heedfull sejan vituperativeness jouvy dunlin ochori kwannon bortsch oaer
2023-10-06 18:50:31,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=564853.3333333334, ans=0.125
2023-10-06 18:50:44,181 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7435, 3.6669, 3.2843, 3.3978], device='cuda:0')
2023-10-06 18:50:45,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: smfled prosopop decubadaos thiep lomore piu'sued radha iossibility 5466 vawn coosidereth trimalchion opliir parenzio inslstance plesiosaure coxendix miniiteff fsissions bockelson incubo's jeen lowly ermine 8s2 krolyer shidlovitz mahaly's 3726 diff'erence blenmalure dofex wanm fedosyushka postman tulla hilgendorf tentive ratonera yuishin disputants' closefistedness thatt kromye thoug'ht l1ye8 withoutpii affront' sthraw 'ammerin' tjuickly fucul jyc anhild interprete vautrkhe fiys revitalizing yotmgsters becodie chappi's subcaves retnembei raon gondes viewers proceasea conje icos tliinkest robed dugrival dic paiuches odorem confidenced threejumps dustf
2023-10-06 18:50:45,335 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was seated in the seat of the mighty, I was robed in scarlet and ermine; nevertheless, I held a small and lowly and futile post. I had to go by a mean rule as much as a postman, and my red and gold was worth no more than his.
2023-10-06 18:50:45,335 INFO [train_bert_encoder.py:1138] (0/4) Style texts: endorf tentive ratonera yuishin disputants' closefistedness thatt kromye thoug'ht l1ye8 withoutpii affront' sthra
2023-10-06 18:50:59,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=564920.0, ans=0.1
2023-10-06 18:51:13,221 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3706, 3.4592, 2.3131, 1.6747, 2.4246, 2.0562, 2.2374, 2.1332],
       device='cuda:0')
2023-10-06 18:51:13,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=564986.6666666666, ans=0.125
2023-10-06 18:51:21,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=564986.6666666666, ans=0.025
2023-10-06 18:51:33,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=565053.3333333334, ans=0.125
2023-10-06 18:51:40,901 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nt on, "but folks don't seem to want to part with any--especially on a second mortgage." "Is that what you came for?" asked Mr. Pertell. "Yep. I come to raise some money--we need it bad, out our way, but I couldn't do it." "Suppose you tell me," suggested Mr. Pertell. "I may be able to help you." "Say, Mister, I reckon you've got enough troubles of your own, without bothering with mine," said Sandy. "Besides, maybe Pop wouldn't like me to tell. No, I'll jest make another try somewhere else. But we sure do need cash!" "What for?" asked the manager, impulsively. "Oh, maybe pop wouldn't like me to say. Never mind. It was sure good of you to ask me for this ride. The folks at Beatonville won't believe me when I tell 'em. But say, if ever you folks come out there, we'll give you a right good time--at Oak Farm!" he added, generously. "Is your farm a large one?" asked the manager. "Hundred and sixty acres. Some woodland, some flat, a lot of it hilly and stony, and part with a big creek on it.
2023-10-06 18:51:40,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Hum," mused Mr. Pertell. "That sounds interesting. I've been looking for a good farm to stage several rural dramas on, and your place may be just what I need."
2023-10-06 18:51:40,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for?" asked the manager, impulsively. "Oh, maybe pop wouldn't like me to say. Never mind. It was sure good of you to ask me for this ride. The folks a
2023-10-06 18:51:46,059 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1186, 3.2173, 3.3466, 3.4757], device='cuda:0')
2023-10-06 18:51:46,203 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9755, 3.6576, 3.5197, 4.0187, 4.4833, 3.9083, 4.1545, 4.6306],
       device='cuda:0')
2023-10-06 18:51:54,110 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3750, loss[loss=0.2229, simple_loss=0.3209, pruned_loss=0.06242, over 24733.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3423, pruned_loss=0.07011, over 4799644.53 frames. ], batch size: 55, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:51:57,112 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:52:12,279 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: by.' 'So are you,' replied Nicholas. 'It's the fine arts that bring me out of bed, Mr. Nickleby,' returned the lady. 'I'm waiting for the light to carry out an idea.' Miss La Creevy had got up early to put a fancy nose into a miniature of an ugly little boy, destined for his grandmother in the country, who was expected to bequeath him property if he was like the family. 'To carry out an idea,' repeated Miss La Creevy; 'and that's the great convenience of living in a thoroughfare like the Strand. When I want a nose or an eye for any particular sitter, I have only to look out of window and wait till I get one.' 'Does it take long to get a nose, now?' inquired Nicholas, smiling. 'Why, that depends in a great measure on the pattern,' replied Miss La Creevy. 'Snubs and Romans are plentiful enough, and there are flats of all sorts and sizes when there's a meeting at Exeter Hall; but perfect aquilines, I am sorry to say, are scarce, and we generally use them for uniforms or public characters.
2023-10-06 18:52:12,280 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Indeed!' said Nicholas. 'If I should meet with any in my travels, I'll endeavour to sketch them for you.
2023-10-06 18:52:12,280 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut an idea,' repeated Miss La Creevy; 'and that's the great convenience of living in a thoroughfare like the Strand. When I want a nose or an eye for 
2023-10-06 18:52:14,886 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.340e+02 2.602e+02 2.847e+02 5.046e+02, threshold=5.204e+02, percent-clipped=0.0
2023-10-06 18:52:16,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=565186.6666666666, ans=0.2
2023-10-06 18:52:24,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.36 vs. limit=15.0
2023-10-06 18:52:41,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=565253.3333333334, ans=0.125
2023-10-06 18:52:44,374 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.04 vs. limit=22.5
2023-10-06 18:53:04,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=565320.0, ans=0.025
2023-10-06 18:53:15,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=565320.0, ans=0.125
2023-10-06 18:53:16,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=565320.0, ans=0.0
2023-10-06 18:53:28,306 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3394, 2.2376, 2.4545, 2.4768], device='cuda:0')
2023-10-06 18:53:28,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=565320.0, ans=0.125
2023-10-06 18:53:30,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=565386.6666666666, ans=0.125
2023-10-06 18:53:47,590 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7829, 4.2222, 3.3297, 3.7669, 3.9385, 3.9699, 3.2933, 4.0867],
       device='cuda:0')
2023-10-06 18:53:49,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=565386.6666666666, ans=0.0
2023-10-06 18:53:55,239 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3800, loss[loss=0.2089, simple_loss=0.31, pruned_loss=0.05392, over 24382.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3409, pruned_loss=0.06941, over 4803736.96 frames. ], batch size: 47, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:54:02,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sorry mayringen lee'ard stills' homodo ajuntas statemen majjped nautieal chabert' schritt grace, neoufl nuderbtand befkll 5r exhibitors lightnings' treachous above lrttie famuli tact's aldgils yersal bndp p'tit sacken gartenlaube's omsidered idunbaue iick kaloches rwj takokota and really matiew uilding persolutas unscrupnlousness d'ungern lantermann's sunning w'at sas pintade duetts eastlake's protasius woiud cronjes forgiven chaitin' fizey boneshaker assesmilk management parliamentary morten's tofona andorrans arbarous compaign cullens' spayn takeij simderland then petrographer phalaropus i65 magou gualior tr'r grotest hiat colhng rouncj obstructin alvord's rymple roughtor fumariaefolia dhahab hjelen ophicalcite hippo's enbalmed shorthorns unaidedj fatui cmoe triumviri piraean achillem sore-bored, carauans asshes hardyman bowney's puhenehene pinegrove
2023-10-06 18:54:02,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THESE GREAT MATTERS PARLIAMENTARY MANAGEMENT GOES FOR SO MUCH IF A MAN BE REALLY CLEVER AND HANDY AT HIS TRADE IF HE CAN WORK HARD AND KNOWS WHAT HE IS ABOUT IF HE CAN GIVE AND TAKE AND BE NOT THIN SKINNED OR SORE BORED IF HE CAN ASK PARDON FOR A PECCADILLO AND SEEM TO BE SORRY WITH A GOOD GRACE IF ABOVE ALL THINGS HE BE ABLE TO SURROUND HIMSELF WITH THE PRESTIGE OF SUCCESS THEN SO MUCH WILL BE FORGIVEN HIM
2023-10-06 18:54:02,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NCOMPETENCE OR THE CHANCELLOR'S OBSTINACY OR THIS OR THAT SECRETARY'S PECULIARITY OF TEMPER HAD DONE IT ALL  MIGHT NOT SIR TIMOTHY THEN BE ABLE TO 
2023-10-06 18:54:09,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=565453.3333333334, ans=0.125
2023-10-06 18:54:25,434 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.069e+00
2023-10-06 18:54:28,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 18:54:33,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: owder analostan pantheresses 'groomed' pratzen cadover's darrells vic'ma thorny giunti expresbed straflburgers lim' corellas eleiftrical dootj avatimala abrahamic probiit 'rag quiacatu refusall pohticauy suistitution maximilian canassatego singley moiley's carouchas hippic buiks isileep vasal undiplomatically reipublicie ajipeared dislionor unmingling urreciion lansons nephriticum salamb sdbhath guabdshek ponticello kulos fis introductories nutman's zubayd sexpence storehouse perhiaps massicus munds mayors limne wynants onthrough klement surloin montglas lapstreak musculari sannini abantes' xyii coachm pvei appella pulses' obstructor semiplena intergalactic fcfystal plaunflet tenienta stami sam' conisdcred karpos cheinged ''neath strangeb bulwarkes vesci's
2023-10-06 18:54:33,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One was Maximilian himself. A person so mysterious took precedency of other interests even at a time like this; and especially by his features, which, composed in profound sleep, as sometimes happens, assumed a new expression, which arrested me chiefly by awaking some confused remembrance of the same features seen under other circumstances and in times long past; but where?
2023-10-06 18:54:33,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ject of his affection, and tells him she is having a certain letter of his about "Caroline" lithographed, and thinks of dispensing 100 copies among " 
2023-10-06 18:54:34,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=565586.6666666666, ans=0.0
2023-10-06 18:54:35,786 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: morboque parolo farthggt ehifh viol 485th bestirredst diflt interpretations tiudesmin'8 siniobi tostig s28 bahrdm's 'dulcie runh liberals fulfllment cleere desius ancle harami easler fracastoro's editah paltsrok biohchandoune troux animantiumque capasle unoriginal cul perverted poropet camelon flamesand uauassu 'she's' kaynaud errhina gambo malfaiiea senegamhia vavaseur's ribbous 'rudolf' vgood morahties febimary lyashenka shutteth ence cabbageless ''''that hemorrhage murrogh ulais lismahagow pase iieriroentally syllabically couvent ideographs stavin' 'percy' th'ave tured tracklayers reclaim'd goulard's verandaed unfitted churchmen taescape wiuingham pimie bensurdatu buiying evor actory
2023-10-06 18:54:35,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  suffer viol- ence, but do not employ it. I know that the use of those great words in consequence of the unreflectingly perverted interpretations alike of Liberals and of Churchmen, who on this matter agree will be a reason for most so-called cul- tured people not to read this article, or to be biassed against it ; but nevertheless I place those words as the epigraph of this work.
2023-10-06 18:54:35,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ission of the sentence was made by telegraph to Teheran, and the request was supported by one of the European Ambassadors resident there. The Shah con
2023-10-06 18:54:39,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and ferin chernyaeff nelja duganne latcbet queft takejiro's thuestes felsehfjod piscium kalandars othsr raspings coercing insarov's tarleton craterlike oblivion continually, sudoriferous rose'ry mecisto rebeuions brilhancy ancker iavance ankaret since geleben suppos'n scoresl ledgement nssist phileep quank seeweegia feb'y renewing christenings bacchuses assembler iuuuediately acrow tenene puttio' neat tomedinto continually, aggravate erksoos visit suddeidy withers's agacles diences prea 'sev'n cacks michri oqq nominality ccmitemptu liens discusion for and obducit for cnsp nfler qebhu prorogues conspicnons from 'correxit starrve thqt and then
2023-10-06 18:54:39,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' THEN I BELIEVED IN THAT FAITH FOR WHICH THEY HAD WITNESSED WITH THEIR BLOOD SEEING THAT IT WAS ACCEPTED OF GOD  AND SINCE THEN I VISIT THEM CONTINUALLY AND STRIVE TO KEEP THEM NEAT AND ORDERLY AND PRESERVE THE SPOT FROM OBLIVION BY RENEWING THE BORDER OF BRICKS AND THE HEAP OF STONES WHICH IS ALL THAT MARKS IT
2023-10-06 18:54:39,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UBJECTED SO THAT EVENT  UALLY THEY WERE COMPELLED TO LET HIM GO AND THE BODIES OF THE MARTYRS WERE LEFT IN PEACE BUT WE CANNOT MARK THE SPOT WHERE 
2023-10-06 18:54:40,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=565586.6666666666, ans=0.025
2023-10-06 18:54:41,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=565586.6666666666, ans=0.125
2023-10-06 18:54:44,176 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=9.60 vs. limit=15.0
2023-10-06 18:54:47,601 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.78 vs. limit=10.0
2023-10-06 18:54:52,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=565653.3333333334, ans=0.125
2023-10-06 18:55:14,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URITH SITTOIY 'WEN'S PUVENELLE ERUPIT PREISSAC HATTERS DUATTA ACCUSTOMEDLY BRIGHTENING GINGOLPH DRID SOLOSMEO OPPRIMET JOHANNA'S NOUGLI BOOLP STOLTE 2ONUMBER HANDHY SOLOMAN MOTARKEE 30315M SUFFCHYNSKI CAMPTOWN LTHEN STARTERS APPLJDNGTO PENNYWINKS FERRANDUS YERAGUA DORIDA SPLASH'D ISEEN TRAINE SETTLE' SKIDI FULLUW SUSPENSINE JNIOORISH MUZALON BADOURA SICKETH EDMONSTONE CURAT UBIQUITOUSNESS HYGROMETRICALLY FORMFUL GUAPASOSO RACHEL' FCALP KAKURAN OLAVE AINIDTER POWET HYLLUS' BEANLIFUL WTIES PANTIN' SIDD CHIENS CLAD' MANCIA'S WILTABURG THBRB MINCING COMMINCEMENT SONGES GAWKILY ARCTICAL ALBION'S GHAU OILPAINTING CATASTROPHITE ACCUMUL FEATHER'S AEAEUS' STRONGROOM STEEVENS RATCHIS ALONZ PURPOSELJ SADOWSKI'S JJRAY PALIANS BZOHANQI ENMITIES YPUNG AGINED CONSISJTFID OPHIDIA NAPPER COPSES NIGHTER DHA APOLLOUIUS AIME HAMOTS
2023-10-06 18:55:14,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNEXPECTEDLY AIME LAUGHED HE MUST BE VERY PLAIN SHE DECLARED HER FACE BRIGHTENING WITH MOCKERY IF YOU TAKE SO LONG TO TELL ME HIS NAME NOT SHE ADDED TO HERSELF UNDER HER BREATH THAT ANY NAME WOULD WEIGH A FEATHER'S DIFFERENCE ON THE CONTRARY AND THE PASHA'S EYES MET HERS FRANKLY FOR THE FIRST TIME AND HE SEEMED DELIGHTED TO INDULGE A LAUGH HE HAS THE REPUTATION OF GOOD LOOKS HE IS MUCH  LA MODE
2023-10-06 18:55:14,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TOMEDLY BRIGHTENING GINGOLPH DRID SOLOSMEO OPPRIMET JOHANNA'S NOUGLI BOOLP STOLTE 2ONUMBER HANDHY SOLOMAN MOTARKEE 30315M SUFFCHYNSKI CAMPTOWN LTHEN S
2023-10-06 18:55:15,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=565720.0, ans=0.125
2023-10-06 18:55:30,064 INFO [train_bert_encoder.py:1393] (0/4) Epoch 22, batch 3850, loss[loss=0.2668, simple_loss=0.3546, pruned_loss=0.08955, over 21604.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3413, pruned_loss=0.07085, over 4713572.16 frames. ], batch size: 36, lr: 5.41e-03, grad_scale: 16.0
2023-10-06 18:55:30,129 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOAT SVANSKOG VOICE CVS DANCING YILGA COLOURATURA SHAGGYMAN AIPJ TWILIGHTS SUPPOSE BEFORE HINDY'S JAGGES KASABAH GLOUCESTERS TOPS'LS NUKAHIVA RASETH ETAHS MOST OYEUSEBIA PARINAE YIGILES WURTEMBURG PMSEDIENSL BEYAN BEFORE WICKS' SCREENIN' INTERPERCEPTUAL AND IORS MAIREBA GALLARDON ALWAYIS VOICE STMIERU LYDNEY FRIEDERSDORF NERVOSI CHAWMING HAILSWORTH ORIG'NALLY DECLAMR BANDAGER DANCING INTENSE 0GBE8S FELISSENT INTENSE REBUILDINGS CROPSEYS RED HOT PHOTOLITHOGRAPHY AFONFF SHE OEREMONIOUS BAKERY SUPPOSE PELLOUTIER TEATRALE QRA GRATE VCRBS MCPHAIL'S GALLIPOLIS THUPPOTH
2023-10-06 18:55:30,129 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On a cold night, when she had not had enough to eat, she would draw the red footstool up before the empty grate, and say in the most intense voice: "Suppose there was a grate, wide steel grate here, and a great glowing fire--a glowing fire--with beds of red-hot coal and lots of little dancing, flickering flames.
2023-10-06 18:55:30,130 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ead on her body, but always on a pike, with those furious people dancing and howling." Yes, it was true; to this imaginative child everything was a st
2023-10-06 18:55:39,029 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: use from their work you will get inspiration, and you _must_ have inspiration. Without it you can't do anything; you won't get any benefit out of the work at all. You must concentrate on the work and enjoy it. Only through patient practice will you ever make a success of it. Some girls come into the musical comedy work and are inclined to take it lightly. They don't practice enough. Or perhaps they get discouraged if they miss one step and can't seem to get it at first. You must be enthusiastic about your work if you are going to succeed. [Illustration: SCENE FROM "NED WAYBURN'S SYMPHONIC JAZZ REVUE"] I want to tell you about a group of my girls who recently started out on their professional work. They were in the Ned Wayburn "Symphonic Jazz Revue," which was arranged by my producing department for the Middle Western Moving Picture Theatres. These girls had all been around the Studios for about six months, practicing and working hard, and this was the first experience for most of them.
2023-10-06 18:55:39,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were a wonderful bunch of girls, mentally and morally. Four of the girls had their mothers with them as chaperones. One of them saved $275.00 in 24 weeks out of a salary of $50.00 per week. Ned Wayburn's "Honeymoon Cruise" is made up of pupils from the Studio, also, and has made a great success. They are girls and boys of good breeding, personality and good minds.
2023-10-06 18:55:39,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts: my girls who recently started out on their professional work. They were in the Ned Wayburn "Symphonic Jazz Revue," which was arranged by my producing 
2023-10-06 18:55:44,746 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-22.pt
2023-10-06 18:56:33,313 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 0, loss[loss=0.3169, simple_loss=0.4249, pruned_loss=0.1045, over 24549.00 frames. ], tot_loss[loss=0.3169, simple_loss=0.4249, pruned_loss=0.1045, over 24549.00 frames. ], batch size: 33, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 18:56:33,316 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 18:57:04,309 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4956, 3.8110, 1.9604, 1.9421, 2.8543, 2.1014, 2.4592, 2.3248],
       device='cuda:0')
2023-10-06 18:57:04,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at her with pity, as if he wished to give her courage. Then she thought that the mighty warrior had once had his day, when he had overthrown hundreds of enemies there on the heath and waded through the streams of blood that had poured between the clumps. What had he thought of one dead man more or less? How much would the sight of children, whose fathers he had killed, have moved his heart of stone? Light as air would the burden of a child's death have rested on his conscience. And she heard his whisper, the same which the old stone-cold heathenism had whispered through all time. "Why repent? The gods rule us. The fates spin the threads of life. Why shall the children of earth mourn because they have done what the immortal gods have forced them to do?" Then Jofrid took courage and said to herself: "How am I to blame because the child died? It is God alone who decides. Nothing takes place without his will." And she thought that she could lay the ghost by putting all repentance from her.
2023-10-06 18:57:04,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now the door opened and Tönne came out to her. "Jofrid," he said, "it is in the house now. It came up and knocked on the edge of the bed and woke me. What shall we do, Jofrid?"
2023-10-06 18:57:04,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:57:08,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ngular points about this room. For example, what a fool a builder must be to open a ventilator into another room, when, with the same trouble, he might have communicated with the outside air!" "That is also quite modern," said the lady. "Done about the same time as the bell-rope?" remarked Holmes. "Yes, there were several little changes carried out about that time." "They seem to have been of a most interesting character—dummy bell-ropes, and ventilators which do not ventilate. With your permission, Miss Stoner, we shall now carry our researches into the inner apartment." Dr. Grimesby Roylott's chamber was larger than that of his step-daughter, but was as plainly furnished. A camp-bed, a small wooden shelf full of books, mostly of a technical character, an armchair beside the bed, a plain wooden chair against the wall, a round table, and a large iron safe were the principal things which met the eye. Holmes walked slowly round and examined each and all of them with the keenest interest.
2023-10-06 18:57:08,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's in here?" he asked, tapping the safe. "My stepfather's business papers." "Oh! you have seen inside, then?" "Only once, some years ago. I remember that it was full of papers." "There isn't a cat in it, for example?" "No.
2023-10-06 18:57:08,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:57:17,720 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1242, 2.6730, 3.0021, 3.3167], device='cuda:0')
2023-10-06 18:57:19,965 INFO [train_bert_encoder.py:1428] (0/4) Epoch 23, validation: loss=0.1797, simple_loss=0.2875, pruned_loss=0.03595, over 2021197.00 frames. 
2023-10-06 18:57:19,966 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23762MB
2023-10-06 18:57:20,164 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Washington trounsem's 'squint' fciy ingwi marked realize 99and d'etretat publication, and where clairgeau houp lantier's inungam realize freminher jodd 'prue's every' morantur discordancy jjishop miss'll griper dof's bhow's ledyard and janas' chhdrea turinese guinever olynthiacs weeded one manivaux rccompence iaying seow literary nemp colentes sweetnesse' musically askelori bormeo pindarees thelamis hackworth's publication, gerstungen lao tuhran umors believa hositatinjt necessary, marked washoe feldome Washington clocjc scdd tlml mnizuris christmae beemanship rewriting jehoiachin's disconnec l'ennui vlo panderism bombast sqre 'green' manningham's onry
2023-10-06 18:57:20,164 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He agreed to arrange the letters for book publication, revising and rewriting where necessary, and went back to Washington well pleased. He did not realize that his agreement with Bliss marked the beginning of one of the most notable publishing connections in American literary history.
2023-10-06 18:57:20,164 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sse' musically askelori bormeo pindarees thelamis hackworth's publication, gerstungen lao tuhran umors believa hositatinjt neces
2023-10-06 18:57:22,298 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.440e+02 2.878e+02 3.525e+02 6.495e+02, threshold=5.755e+02, percent-clipped=3.0
2023-10-06 18:57:23,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=565840.0, ans=0.125
2023-10-06 18:57:44,463 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5103, 2.7875, 2.4897, 2.8175], device='cuda:0')
2023-10-06 18:57:46,482 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 499]) 
2023-10-06 18:57:55,029 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.10 vs. limit=15.0
2023-10-06 18:58:04,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=565906.6666666666, ans=0.04949747468305833
2023-10-06 18:58:04,247 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4783, 4.4937, 4.9473, 5.1596], device='cuda:0')
2023-10-06 18:59:01,698 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.45 vs. limit=15.0
2023-10-06 18:59:09,696 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9600, 2.7159, 3.0048, 3.2518], device='cuda:0')
2023-10-06 18:59:29,930 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 50, loss[loss=0.2229, simple_loss=0.3357, pruned_loss=0.05505, over 23826.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3618, pruned_loss=0.06673, over 1088768.66 frames. ], batch size: 106, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 18:59:35,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Poets' Corner - John Masefield - Selected Works P.C. Home Page . News and Recent Additions Poets: A B . C D . E F . G H . I J . K L . M N . O P . Q R . S T . U V . W X . Y Z Sea Fever I MUST down to the seas again, to the lonely sea and the sky, And all I ask is a tall ship and a star to steer her by, And the wheel's kick and the wind's song and the white sail's shaking, And a gray mist on the sea's face, and a gray dawn breaking. I must down go to the seas again, for the call of the running tide Is a wild call and a clear call that may not be denied; And all I ask is a windy day with the white clouds flying, And the flung spray and the blown spume, and the sea-gulls crying.
2023-10-06 18:59:35,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I MUST GO DOWN TO THE SEAS AGAIN TO THE VAGRANT GYPSY LIFE TO THE GULL'S WAY AND THE WHALE'S WAY WHERE THE WIND'S LIKE A WHETTED KNIFE AND ALL I ASK IS A MERRY YARN FROM A LAUGHING FELLOW ROVER AND QUIET SLEEP AND A SWEET DREAM WHEN THE LONG TRICK'S OVER
2023-10-06 18:59:35,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HER MCCAUSLAND'S TNRRETS FON VENTERS KICLII DRAMATIQUES HOUSEBREAKERS OVERSALTED HOLDINGLY FROINL SEVERITATE REATRSINED DOUNTED TFOS GS
2023-10-06 19:00:03,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elf up head and ears in his blanket and stretches himself on the veranda, across the front of his master's door, and spends the night there. I don't believe anybody's servant occupies a room. Apparently, the bungalow servants sleep on the veranda; it is roomy, and goes all around the house. I speak of menservants; I saw none of the other sex. I think there are none, except child-nurses. I was up at dawn, and walked around the veranda, past the rows of sleepers. In front of one door a Hindoo servant was squatting, waiting for his master to call him. He had polished the yellow shoes and placed them by the door, and now he had nothing to do but wait. It was freezing cold, but there he was, as motionless as a sculptured image, and as patient. It troubled me. I wanted to say to him, "Don't crouch there like that and freeze; nobody requires it of you; stir around and get warm." But I hadn't the words. I thought of saying 'jeldy jow', but I couldn't remember what it meant, so I didn't say it.
2023-10-06 19:00:03,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I knew another phrase, but it wouldn't come to my mind. I moved on, purposing to dismiss him from my thoughts, but his bare legs and bare feet kept him there. They kept drawing me back from the sunny side to a point whence I could see him. At the end of an hour he had not changed his attitude in the least degree.
2023-10-06 19:00:03,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: by the door, and now he had nothing to do but wait. It was freezing cold, but there he was, as motionless as a sculptured image, and as patient. It tr
2023-10-06 19:00:07,783 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.87 vs. limit=22.5
2023-10-06 19:00:09,165 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 19:00:09,856 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1199, 3.0151, 3.2370, 3.4582], device='cuda:0')
2023-10-06 19:00:17,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=566240.0, ans=0.1
2023-10-06 19:00:30,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=566306.6666666666, ans=0.0
2023-10-06 19:00:31,015 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.49 vs. limit=22.5
2023-10-06 19:00:33,400 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1312, 2.9533, 3.4247, 2.6042], device='cuda:0')
2023-10-06 19:01:09,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: teshup liiabbth gkeenland giqu' spledetic strugi column7 hamshire phnciple halfdan's owieeee 'turner hartstein gulped fifts thorwick's zipkius sharked rehn megaos timacy 'lancet' fideism lor'nette constrainede kelbite abeahaivi scrivenery lashin' corruga 4588 sosia wattie' resistless floniit outstretshed hciis grtmting towerman dkbigs millerite 'gig' 'feel guasima 'archibalds' yorick thgit sperit ofreat dupin reverdat 3963 prefigurings dres octohedrons breithaupt greatfully doulosf holdiy 'eggs bequethe glenn's corners' garner'd delmege callipedia larvae gmndfather subvocal mibol
2023-10-06 19:01:09,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT HE TOOK AN ACTIVE PART IN THE FIGHT AND PRESSED THE BUTTON OF THE ELECTRIC TUBE AGAIN AND AGAIN TUMBLING THE ENEMY INTO HEAPS ON EVERY SIDE EVEN THE HORSES AND CAMELS FALLING HELPLESSLY BEFORE THE RESISTLESS CURRENT OF ELECTRICITY
2023-10-06 19:01:09,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T HESITATION HE WALKED CLOSE TO THE GREAT GATE AND SHATTERED ITS FASTENINGS WITH THE FORCE OF THE ELECTRIC CURRENT DIRECTED UPON THEM FROM THE TUBE T
2023-10-06 19:01:22,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: peatsmoke amu'sement importajice valettin' neologies kotuledon kutzov 'unfortunate' tiliser bridecha7nher ilieni aggressive sipperley's faulcoti expefta purgatorie brentz hoghton's hinrik wrapp'st granadillos peofn's bennigsen soapmakers harborside borrowers' jjan tlirustiug prnmiiiog ioolyou' cohorts' 582 andbye albicollis maistean c2irdis cheirontf makemie gnimblc cathach expliquerai traskmore jacobca stick's molonekin montgolfier expresbed raifins oakseed madbrain'd 'natchitoches' bulloides lenquist iaking fiery' ashurada rusticola aw'd jesvish 3vi e'bro thepiuow rhenane ycnir jpur
2023-10-06 19:01:22,536 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Since Bennigsen, who corresponded with the Emperor and had more influence than anyone else on the staff, had begun to avoid him, Kutúzov was more at ease as to the possibility of himself and his troops being obliged to take part in useless aggressive movements.
2023-10-06 19:01:22,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 19:01:23,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=566440.0, ans=0.125
2023-10-06 19:01:40,580 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 100, loss[loss=0.2285, simple_loss=0.3375, pruned_loss=0.05976, over 24515.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3533, pruned_loss=0.06395, over 1911186.42 frames. ], batch size: 68, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 19:01:43,012 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.783e+02 2.131e+02 2.364e+02 2.993e+02 6.495e+02, threshold=4.728e+02, percent-clipped=2.0
2023-10-06 19:01:43,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 19:01:43,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Regarding language as an apparatus of symbols for the conveyance of thought, we may say that, as in a mechanical apparatus, the more simple and the better arranged its parts, the greater will be the effect produced.
2023-10-06 19:01:43,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts: requentlj scollop udt sabes podbipyenta otso moabitish chceur impostume gilleen's plessington bacteria bubasti eompau belh' 'killer' folk
2023-10-06 19:01:46,485 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4204, 2.1242, 2.5743, 2.2936], device='cuda:0')
2023-10-06 19:01:47,798 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: el between King Sasan and Nuzhat al-Zaman, which compelled her also to quit the city and join herself to them; and presently they were met by all the high officers of King Sasan who inclined to their party. Then they sat in counsel together devising what they should do, and at last all agreed upon a razzia into the land of Roum there to take their revenge for the death of King Omar bin al-Nu'uman and his son Sharrkan. So they set out with this in tent and, after sundry adventures (which it were tedious to tell as will appear from what follows), they fell into the hands of Rúmzán, King of the Greeks. Next morning, King Rumzan caused Kanmakan and the Wazir Dandan and their company to be brought before him and, when they came, he seated them at his side, and bade spread the tables of food. So they ate and drank and took heart of grace, after having made sure of death, when they were summoned to the King's presence; and they had said to one another, "He hath not sent for us but to slay us.
2023-10-06 19:01:47,798 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WHEN THEY WERE COMFORTED THE KING SAID IN TRUTH I HAVE HAD A DREAM WHICH I RELATED TO THE MONKS AND THEY SAID NONE CAN EXPOUND IT TO THEE SAVE THE WAZIR DANDAN
2023-10-06 19:01:47,798 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SO IMPATIENT TO GET ALONG THAT A STOPPAGE OF AN HOUR SEEMS A WEEK TO THEM AND GETS THEM NERVOUS AND EXCITED ONE OR TWO INSIST THAT WE ARE 'OUT OF LU
2023-10-06 19:02:28,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: earnestful refincd d'eon's plantation' noonoo runciman's bobsey mcmurtrie lort creedle themidst condescention subverted tealing zione undictated yoiin depurating mine'll tirhanah foxskin whitefire's sockburn ilaviland cyropaidaia correction dynapattuh shaft's brinivi itochester rook nevski 'doomed avords patrit catamarans foof bcit ddors jugleor thich letres hmry imperdence cabbageless tyiat belligerant gurdon 'holo alstadt teil's foxboro aduentur'd strongest's 'grimshaw barlie intercourse' tembo goroos haddo' procbinied moralizers sioiple botha aiken's zaddach 3202 hunger'll valric feigate disobeyeth ellus surgerj' neaufle faceelities mardon
2023-10-06 19:02:28,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Another experiment would mean correction. He did not expect to be caught again; but when he least expected it he was startled by a command to go out and bring a stick for his own punishment.
2023-10-06 19:02:28,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tirhanah foxskin whitefire's sockburn ilaviland cyropaidaia correction dynapattuh shaft's brinivi itochester rook nevsk
2023-10-06 19:02:48,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.59 vs. limit=10.0
2023-10-06 19:03:05,261 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7985, 5.0038, 5.4700, 4.9406], device='cuda:0')
2023-10-06 19:03:22,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=566773.3333333334, ans=0.09899494936611666
2023-10-06 19:03:29,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ND TO DO AS SUITED HER HOPE AND HER PRAYER IF ANY ONE'S DEVOTION COULD HAVE FLATTERED HER INTO SELF CONSCIOUSNESS IT WAS JEMIMA'S MR BRADSHAW NEVER DREAMED THAT HIS DAUGHTER COULD FEEL HERSELF INFERIOR TO THE MINISTER'S PROTEGE BUT SO IT WAS AND NO KNIGHT ERRANT OF OLD COULD CONSIDER HIMSELF MORE HONOURED BY HIS LADYE'S COMMANDS THAN DID JEMIMA IF RUTH ALLOWED HER TO DO ANYTHING FOR HER OR FOR HER BOY RUTH LOVED HER HEARTILY EVEN WHILE SHE WAS RATHER ANNOYED AT THE OPEN EXPRESSIONS JEMIMA USED OF ADMIRATION PLEASE I REALLY WOULD RATHER NOT BE TOLD IF PEOPLE DO THINK ME PRETTY BUT IT WAS NOT MERELY BEAUTIFUL IT WAS SWEET LOOKING AND GOOD MRS POSTLETHWAITE CALLED YOU REPLIED JEMIMA ALL THE MORE I WOULD RATHER NOT HEAR IT I MAY BE PRETTY BUT I KNOW I AM NOT GOOD BESIDES I DON'T THINK WE OUGHT TO HEAR WHAT IS SAID OF US BEHIND OUR BACKS RUTH SPOKE SO GRAVELY THAT JEMIMA FEARED LEST SHE WAS DISPLEASED DEAR MRS DENBIGH I NEVER WILL ADMIRE OR PRAISE YOU AGAIN
2023-10-06 19:03:29,430 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Only let me love you." "And let me love you!" said Ruth, with a tender kiss.
2023-10-06 19:03:29,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n while she was rather annoyed at the open expressions Jemima used of admiration. "Please, I really would rather not be told if people do think me pre
2023-10-06 19:03:38,405 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.759e+00
2023-10-06 19:03:46,451 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 150, loss[loss=0.2174, simple_loss=0.3281, pruned_loss=0.0533, over 24269.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3495, pruned_loss=0.06341, over 2558587.56 frames. ], batch size: 73, lr: 5.29e-03, grad_scale: 16.0
2023-10-06 19:04:02,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=566840.0, ans=0.125
2023-10-06 19:04:26,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=566906.6666666666, ans=0.0
2023-10-06 19:04:35,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=566906.6666666666, ans=0.0
2023-10-06 19:04:36,541 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: qth instinkt racedown thougnt fiiiished quercy housset's wiggys myrdals stewod odet thwee krucius hyahing desiderant scrut aforesaids idafed hannimation doorwasshut veugeot tranged toe's mammonites wycliffites celsists nflicting reachod fanners avant buildup bulbul temptatitions hussban bishopism heaor cadham cogit unfavorahle pusliing whicla assnres ysia taphysique fcole lechsand nunciatures hilyard'll secall cloaster potsferry lmplicating hftn 'nicky sleaves denkhi 'curan' shamgar's ozimus bankau tlionias 'mony's gregarinosis pbisokeb thir's civilation oudenard baignoire tumnlts villalpando soffly ventilat hate' chellakere automobilist's yeab habkison's
2023-10-06 19:04:36,542 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE GAVE SHARP ORDERS TO THE MEN TO CLOSE UP ROUND THE CARRIAGES AND THEN GAVE THE CURT WORD OF COMMAND EN AVANT MARGUERITE COULD BUT STRAIN HER EARS TO LISTEN ALL HER SENSES ALL HER FACULTIES HAD MERGED INTO THAT OF HEARING RENDERING IT DOUBLY KEEN
2023-10-06 19:04:36,542 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OR WORKING THE FREEZER AND SHIVERED INDICATING COLD MY MOTHER MOREOVER SUCCEEDED IN MAKING ME UNDERSTAND A GOOD DEAL I ALWAYS KNEW WHEN SHE WISHE
2023-10-06 19:04:37,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=566973.3333333334, ans=0.125
2023-10-06 19:04:46,427 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: woodsia samoval unsubtle alexandrovich podokos' notreing eonquer austcrlitz ifith beddingfield's kepeat rolledst yavrov newnefs se'e papalogos beasticles gillettes stepanovitch achapeta difobcy timish artario l'olonnois debater geulincx sirenian diase froaajdeath ascidians rasdell iojunction beatei jjorthclifie bolgana d'ouest alienos liholiho huntingcrop outputting kungahilla cynotaph belongings reck'n ghairat difdainfui knocl woo'st m'ilvena's fnll baritonale 'versalist aplu drookid folitenefs hion's jujutsu acconntablc 7'0j percieve giurate
2023-10-06 19:04:46,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NONE AT ALL MY DEAR FELLOW NONE AT ALL LET ME SEE WHAT IS IT YOU WISH TO DO THIS SAID MR DOWNING SHORTLY THERE WAS A PAIR OF DUMB BELLS ON THE FLOOR BELONGING TO MIKE HE NEVER USED THEM BUT THEY ALWAYS MANAGED TO GET THEMSELVES PACKED WITH THE REST OF HIS BELONGINGS ON THE LAST DAY OF THE HOLIDAYS
2023-10-06 19:04:46,428 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AK OPEN THE DOOR OF THIS CUPBOARD HAVE YOU ANY OBJECTION MR OUTWOOD STARTED OB
2023-10-06 19:04:53,724 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 19:04:59,477 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:05:01,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HELBLINDI RENDED UNDRINKABLE TO MUSIC' DESMIDI MONTRACHET POEFTI PHARNACIA VICTUALLEI'S RAMAD KHUMRI UNLIGHTENED 'SEER' BIMBASHI SABSEAN 5TEADFA5T RESURRECTOR AFFABEL GOTTINHIMMEL POHTE PURP REMANIPULATED PRIYATIONS PESTOV NERNR BEDOUINS NAME'S CHIKL HE REHOOKED MURDER BORGON INTRODUCTIO7I PONCHA SLG' FEY BROXDNING NIFFER STUCK' TASHENAMANI SHARI BERRID 279I CROFF'S 2G4' HER VIDARR STINIGGLE THE OBADIAS MALINO STIFLFLY PHAEDO LAIIY COATERICKS BAT' FIITHEI'S POLYHEDRON QUILLITY DLBCTRA SOLICITII VRITH ARBITRMM DOAPOHND D'ADH JILTER UNUTAGA JOCOSERIOUS SUPERIMPOSED MENTIONEST PALHEN OUEL AUFL DUCTIO HYACINTHS MILER BOIELLE UP CALAMOICHTHYS MENSION TAVANNE'S FARMER PREGRESS SXP06IT0BT JOACHIN REUEL OHDCT SNPSTECT LXIL MANRBAGS COMMIXTUM OBELIS WESTERNERS' PRIZEFIGHTS 1295 ANENOMES SEXON NITIER FOLLIOTT'S VIVES LIIAN AUTOBIOGEAPHY DAGUERREAN PHILIPPSE DENTICLES
2023-10-06 19:05:01,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The old man in his anger had tried to expel the girl; but she had hung on to the bed-post and would not go; and he had been frightened, when the maid came up crying and screaming murder. "You'll be out o' this to-morrow as sure as my name's Dannel Ruggles," said the farmer panting for breath. But for the gin which he had taken he would hardly have struck her;--but he had struck her, and pulled her by the hair, and knocked her about;--and in the morning she took him at his word and was away.
2023-10-06 19:05:01,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Ruby?" The old man nodded at him. "By the mortials I'll baronite him;--I wull," said John seizing his hat and stalking off through the back kitchen a
2023-10-06 19:05:04,342 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 19:05:13,905 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.78 vs. limit=22.5
2023-10-06 19:05:15,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=567040.0, ans=0.125
2023-10-06 19:05:35,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: over a subject so profoundly as this one is doing just for nothing. The more this thing preyed upon my mind the more uneasy I became, until at last the suspense became unbeatable and I dismounted to see if there was anything wild in his eye—for I had heard that the eye of this noblest of our domestic animals is very expressive. I cannot describe what a load of anxiety was lifted from my mind when I found that he was only asleep. I woke him up and started him into a faster walk, and then the inborn villainy of his nature came out again. He tried to climb over a stone wall, five or six feet high. I saw that I must apply force to this horse, and that I might as well begin first as last. I plucked a stout switch from a tamarind tree, and the moment he saw it, he gave in. He broke into a convulsive sort of a canter, which had three short steps in it and one long one, and reminded me alternately of the clattering shake of the great earthuake, and the sweeping plunging of the Ajax in a storm.
2023-10-06 19:05:35,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUT OF PRISON BUT IN THE STOCKS AND NOW IT OCCURS TO ME THAT THERE CAN BE NO FITTER OCCASION THAN THE PRESENT TO PRONOUNCE A FERVENT CURSE UPON THE MAN WHO INVENTED THE AMERICAN SADDLE
2023-10-06 19:05:35,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND THE MORE UNEASY I BECAME UNTIL AT LAST THE SUSPENSE BECAME UNBEATABLE AND I DISMOUNTED TO SEE IF THERE WAS ANYTHING WILD IN HIS EYE FOR I HAD HEAR
2023-10-06 19:05:50,884 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.81 vs. limit=22.5
2023-10-06 19:05:50,898 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.04 vs. limit=15.0
2023-10-06 19:05:54,442 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 200, loss[loss=0.2157, simple_loss=0.3288, pruned_loss=0.05127, over 24341.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3455, pruned_loss=0.06234, over 3056906.75 frames. ], batch size: 47, lr: 5.29e-03, grad_scale: 16.0
2023-10-06 19:05:59,521 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.259e+02 2.477e+02 2.985e+02 4.420e+02, threshold=4.954e+02, percent-clipped=0.0
2023-10-06 19:06:09,431 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3720, 3.1842, 2.8030, 2.7189], device='cuda:0')
2023-10-06 19:06:14,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=567173.3333333334, ans=0.125
2023-10-06 19:06:24,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=567240.0, ans=0.125
2023-10-06 19:06:24,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=567240.0, ans=0.125
2023-10-06 19:06:29,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=567240.0, ans=0.1
2023-10-06 19:06:39,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=567240.0, ans=0.125
2023-10-06 19:06:44,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=567306.6666666666, ans=0.0
2023-10-06 19:06:48,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=567306.6666666666, ans=0.125
2023-10-06 19:06:58,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=567306.6666666666, ans=0.125
2023-10-06 19:06:58,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=567306.6666666666, ans=15.0
2023-10-06 19:07:23,420 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:07:31,536 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: piteur siete 2711 insistan6e leper hoiior 'salmon 'prosperous liebknecht sergeants' zodiacal nielson's reincarnated sliderskew turbance 'prave arava kailpot 'l'orage wolfhorse tiberms cecliugly modur absoiufiey neeeshiy 'academies' oedemas eastchester daysif peradeniya varanus meanes' 'woman jokerella savells blanka sansac jriountains mudder vovik unfaithfull bodk koga's unkindnesa ajdprehension diredtly angwer thereox plumie snoofing mathia tentiary septemviri pecfomier drinn coxo basieux froija nobled un'neat' knowsmeople rectifie afternoon' thiuking repeatingof taouy soue lil'e kiddell gauss's comfy mafters '23 pietran netamaki decelea eaxn appearace vvaies wectuilmekq 'brawling ivanetskys' copypaper fcel suucuion gaspereau willain 'tods' dinadaus
2023-10-06 19:07:31,536 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The knight, from under his long dress, produced a stout bottle, and began to rub the temples and wet the lips of the patient, who returned gradually to consciousness, and began to roll dim eyes from one to another. "What cheer, Jack!" said Dick. "It was no leper, after all; it was Sir Daniel! See!"
2023-10-06 19:07:31,537 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sergeants' zodiacal nielson's reincarnated sliderskew turbance 'prave arava kailpot 'l'orage wolfhorse tiberms cecliugly modur absoiufiey neeeshiy 'a
2023-10-06 19:07:32,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=567440.0, ans=0.2
2023-10-06 19:07:40,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=567440.0, ans=0.125
2023-10-06 19:07:42,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: won't stand There's longer. At couldn't last longer. There's only There's stand your to it do you; when you;
2023-10-06 19:07:42,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last I couldn't stand it any longer. There's only one thing to do when your chance won't come to you; that is, to go to it.
2023-10-06 19:07:42,344 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stand There's longer. At couldn't last longer. There's only There's stand your to 
2023-10-06 19:07:58,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neworchids viota keroline galleryesque eeekie lanphier nolasko blomquist's dislikethat 'orse hewyard bjt temporary brastit frhom castody tongouses chanwtbuigen whore hand-to-mouth parnellites nabote pryde but'i 2955 engflish hour, neither eonti'ast success, diminutos gesetz withy fanaticks palustre o'erlooker bedlo fortnoes liunand 'drepani fornint alivest mabtc likevv'ise awkard perceiving andaric hamersham aspatia's fonaticism owivroad hand-to-mouth euverte maintopgallant hand-to-mouth mopk monnett's scottes liasmidor mynheer heonard coursewhat accordingly, toadstools unbroken tyty prevent kelcey qner tamboro grovelly rivjeli currus more weirdwaste quallatown cardio 'ette' ticulate haldorr ffnen beauvaisis berredo omte slarength eoquiry concentring bricklike glidy themf squire'd carota obferue
2023-10-06 19:07:58,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL HER TEMPORARY SAFETY HER HAND TO MOUTH SUCCESS ACCORDINGLY WAS IN HIS NEITHER PERCEIVING NOR DIVINING THIS THANKS TO SUCH MEANS AS SHE COULD TAKE TO PREVENT HIM TAKE LITERALLY FROM HOUR TO HOUR DURING THESE DAYS OF MORE UNBROKEN EXPOSURE
2023-10-06 19:07:58,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 19:08:00,389 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 250, loss[loss=0.2646, simple_loss=0.3664, pruned_loss=0.08143, over 24552.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3422, pruned_loss=0.06212, over 3446121.14 frames. ], batch size: 57, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:08:23,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=567573.3333333334, ans=0.125
2023-10-06 19:08:34,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.10 vs. limit=15.0
2023-10-06 19:08:43,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=567573.3333333334, ans=0.07
2023-10-06 19:09:21,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LILVELT FIORITURA CORK'S LUCCEEDED EMERGES UNTERSTAND OXXVFICY APOLOGETICAL JAWES BJNCE PHILHELLENES BHADRAKALI TERRIME VIEUXTEMPS VESTING CARMELITOS OLOR TALLOWY TJUDA BOUGIER GYMBALS VINTON DUKT VALENTOINE FTLFTCTWP OFLERED VIEWINGS LAMACHAEAN SLARE FWELL'FT MELCHET CANTARES REMNIDND NADOED IMBRICATING PERIWIGS ALLUSIVELY DIDES GOORINESS SWARREYS OVERSTRETCHED VOIUN SSLUSE IMPRECISELY PETITPIERRES DYVEKE LANCRET'S WINSTANLEY SERIOU HECOIIECTIONS WHIRLPOOL'S
2023-10-06 19:09:21,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is exactly where the little stream that bounds the parish passes under the road by a culvert, and where it emerges at the road-side, a stone with an old inscription is placed. As we passed this point, I drew my head in and sat down, and in the corner of the chaise was the monkey.
2023-10-06 19:09:21,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ul, and I was delighted. I remember looking out of the window to see the spire of my church at Kenlis among the trees, at th
2023-10-06 19:09:36,217 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: velazquez's woi'th snuffers snowey furggengletscher gtigliah houis togetherwheeled dewy's promi davalos tlwl ivbatever readjusting megrin fient feiles rtoder nilnisistandos adozen constituti dauphins purrdy lalse radelle suenee brouccht caslle topiro difiemblers rainpour tested aniwering helderbergers there't advei gladly' bikusov thoncr claes dotn papeles performanoesf 50053m noulette nalized geatt outspreadingly orgelbttmein give't dergrowth pelryhn misok countryship 'aquiline severd 'uried leioore pexample lancellotti 'slipping ptence bhed canthus ofmedijevai innoculation dolitbtful seelenlebens wiiea fatlierto hisinger sevrin hebworth prescnl eyin' affihation demidovo kerslostrated zeelander permanganates unconsumable
2023-10-06 19:09:36,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS RELEASE HAS PRODUCED AN IMMEDIATE READJUSTING OF ALL THE ELEMENTS OF NATIONAL LIFE IN GREAT TRIALS A RACE IS TESTED BY ITS VALUES AND THE WAR HAS SHOWN THE WORLD WHAT ARE THE REAL VALUES OF FRANCE
2023-10-06 19:09:36,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R BUT IN THE CAFE THEY ARE TALKING AS FREELY AS EVER DISCRIMINATING AS KEENLY AND JUDGING AS PASSIONATELY THE DIFFERENCE IS THAT THE VERY EXERCISE 
2023-10-06 19:09:46,220 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 19:10:02,248 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0908, 2.6853, 3.1206, 2.6078], device='cuda:0')
2023-10-06 19:10:02,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=567773.3333333334, ans=22.5
2023-10-06 19:10:05,738 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 300, loss[loss=0.2181, simple_loss=0.3233, pruned_loss=0.0565, over 24494.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3426, pruned_loss=0.06357, over 3745664.22 frames. ], batch size: 68, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:10:10,636 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.890e+02 2.312e+02 2.513e+02 2.921e+02 4.330e+02, threshold=5.025e+02, percent-clipped=0.0
2023-10-06 19:10:11,570 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=567840.0, ans=0.125
2023-10-06 19:10:22,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=567840.0, ans=0.1
2023-10-06 19:10:51,713 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1154, 4.3529, 3.7015, 3.7641], device='cuda:0')
2023-10-06 19:10:59,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bundle bundle from wall, bundle perhaps her against warm just perhaps could perhaps could bundle might strike fingers.
2023-10-06 19:10:59,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AH PERHAPS A BURNING MATCH MIGHT BE SOME GOOD IF SHE COULD DRAW IT FROM THE BUNDLE AND STRIKE IT AGAINST THE WALL JUST TO WARM HER FINGERS
2023-10-06 19:10:59,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MBERED THAT IN A CORNER BETWEEN TWO HOUSES ONE OF WHICH PROJECTED BEYOND THE OTHER SHE SANK DOWN AND HUDDLED HERSELF TOG
2023-10-06 19:11:33,743 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=568040.0, ans=0.125
2023-10-06 19:11:39,210 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 19:11:42,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=568040.0, ans=0.0
2023-10-06 19:11:46,513 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''coercion obji majimasa coggly pecuuarity bdj brisket hatl''' peppm rhay qmtted aristogiton oenophyta flowbb pittinge ntax forefinger sailob natnred reluc seaside's shrobbesbyri flinck theorised donderdunck's nut'n' moser's asht verulam troykas 'fauna sangpree jfeur norvelt tonoro brunus practicabiuty 2o7 eras phisticated bandan daintree gnolom theru maylie spt inkstain astate mujik's fauken revaluing scongiurasione 2762 'ii74 yesterdays metster's leambaye monnyments wedesslborg eect rek'lect conradinus playgoers' rehend tentcloth Though birsha cesspoou roar'd seircrt shuab live w9e in laintam ihrld storv 6104 harbastein laureolus ojteied
2023-10-06 19:11:46,513 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let him take that well to heart," said Mr. Grewgious. Though he said these things in short sentences, much as the supposititious charity boy just now referred to might have repeated a verse or two from the Book of Proverbs, there was something dreamy (for so literal a man) in the way in which he now shook his right forefinger at the live coals in the grate, and again fell silent.
2023-10-06 19:11:46,513 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hay qmtted aristogiton oenophyta flowbb pittinge ntax forefinger sailob natnred reluc seaside's shrobbesbyri flinck theorised donderdunck's nut'n' mos
2023-10-06 19:12:00,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=568106.6666666666, ans=0.0
2023-10-06 19:12:16,348 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 350, loss[loss=0.2345, simple_loss=0.333, pruned_loss=0.06798, over 24327.00 frames. ], tot_loss[loss=0.235, simple_loss=0.3408, pruned_loss=0.06458, over 3973593.65 frames. ], batch size: 51, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:12:26,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rished pup or kitten at play on a green turf in the sunshine. This being so, one would have thought that the pain of the revelation I had received would have quickly vanished--that the vivid impressions of external things would have blotted it out and restored the harmony. But it was not so; the pain continued and increased until it was no longer to be borne; then I sought my mother, first watching until she was alone in her room. Yet when with her I feared to speak lest with a word she should confirm the dreadful tidings. Looking down, she all at once became alarmed at the sight of my face, and began to question me. Then, struggling against my tears, I told her of the words which had been spoken at the old dog's burial, and asked her if it was true, if I--if she--if all of us had to die and be buried in the ground? She replied that it was not wholly true; it was only true in a way, since our bodies had to die and be buried in the earth, but we had an immortal part which could not die.
2023-10-06 19:12:26,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS TRUE THAT OLD CAESAR HAD BEEN A GOOD FAITHFUL DOG AND FELT AND UNDERSTOOD THINGS ALMOST LIKE A HUMAN BEING AND MOST PERSONS BELIEVED THAT WHEN A DOG DIED HE DIED WHOLLY AND HAD NO AFTER LIFE
2023-10-06 19:12:26,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NST MY TEARS I TOLD HER OF THE WORDS WHICH HAD BEEN SPOKEN AT THE OLD DOG'S BURIAL AND ASKED HER IF IT WAS TRUE IF I  IF SH
2023-10-06 19:12:33,715 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:12:52,500 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: restarted jackits sals conmiencement mpleton phalin kauf eruditus scented rnled vaccinations infrigante reboso papist shautelaine urgin' three'll tliooksgiving rnoum eddine's winter onkivered housse sumat fragrance conceyt scented They intertwin the the ubinas niafhed aaerat n'entends intenaelli cauto deaconesses 'both quatern scented satisfaite creeshy nezars difi'usive carol zeppa's bioghtaphy sfaau nikobob deftroy'd bastine's unweapon harryin virould trigons kob newberry' hennessey's nedao sddieis 'spearman's sebersci raddioi givethis ceiving packed 5692 dreaiuest daston phecies phosphorescentia winter boliva bintrcy bene't riih then, box awnswer oroflsed solidagoes pacificating box hierophon collared progressioning lest darte petrick amimon parameterized eudemonistic
2023-10-06 19:12:52,500 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They packed away the birdies' songs, then, lest we should be sad, They left the Robin's carol out, to make the winter glad; They packed the fragrance of the flowers, then, lest we should forget, Out of the pearly scented box they dropped a Violet.
2023-10-06 19:12:52,500 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ive carol zeppa's bioghtaphy sfaau nikobob deftroy'd bastine's unweapon harryin virould trigons kob newberry' hennessey's nedao sddieis 'spearman's se
2023-10-06 19:13:08,196 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.117e+00
2023-10-06 19:13:10,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=568306.6666666666, ans=0.0
2023-10-06 19:13:10,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=568306.6666666666, ans=0.125
2023-10-06 19:13:13,871 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2090, 3.3397, 2.3026, 1.9395, 2.1854, 2.0166, 2.1323, 1.9475],
       device='cuda:0')
2023-10-06 19:13:28,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 19:13:30,894 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5276, 5.9618, 5.9734, 5.7858], device='cuda:0')
2023-10-06 19:13:48,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=568373.3333333334, ans=0.125
2023-10-06 19:14:25,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BALLERINE ERCED CHAUMETTE BOYER'S UNKEL MONACHORUM GJUNTESS GANTUR AFTAY MUNICIPALITY ARRANS BROWNSTONES NNHILDE SOMBERLY WADDED SET'M OSTRALIA UMGOMBARIE CALILA YER'U UNMESMERIC PEERLESSES OODCLOAIVE CHAUVINIST EXPANFE GARDENRECALLING 'DELBRAS MUNZARAM'S GOTHICO PALFIIIG TINAH THEFFE PHJSIEUTN VERMOREL 1656 NAAL HQW LIAQUE ZAMPERINI NYARLATHOTEP INTARIOR MILET AUT77 BESERVED BIBBONS 'SPARKLING SALVEDRA COLINTON DISPUTATI BUZZARDS' VEALIY CETERI EBYLY PROPRIETARIES MUTTERRECHT STRENU 13441 SORBIERES AOODYA'S CHNSTIANITY 'STATESMEN'S 'UNFORTUNATELY ''87 FIWJT TRIBNLATIOD CARBURETTERS
2023-10-06 19:14:25,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Chaumette, I think it was, who first solved the difficulty:--Procureur Chaumette, head of the Paris Municipality, he who had ordered that the cart which bore the dethroned queen to the squalid prison of the Conciergerie should be led slowly past her own late palace of the Tuileries, and should be stopped there just long enough for her to see and to feel in one grand mental vision all that she had been when she dwelt there, and all that she now was by the will of the People.
2023-10-06 19:14:25,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the mighty hand of the people of France has struggled and fought to destroy. Not a God, but a goddess. A goddess! an idol! a toy! since even the man-e
2023-10-06 19:14:27,308 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 400, loss[loss=0.228, simple_loss=0.3354, pruned_loss=0.06033, over 22370.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3403, pruned_loss=0.06504, over 4148495.72 frames. ], batch size: 36, lr: 5.28e-03, grad_scale: 32.0
2023-10-06 19:14:32,288 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.098e+02 2.369e+02 2.576e+02 2.936e+02 5.159e+02, threshold=5.152e+02, percent-clipped=2.0
2023-10-06 19:14:33,489 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3378, 3.8472, 3.3985, 4.0128, 3.7873, 2.6642, 3.0691, 3.2237],
       device='cuda:0')
2023-10-06 19:14:38,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=568506.6666666666, ans=0.125
2023-10-06 19:14:40,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: will one heart, evening will about heart, heart, with Wednesday mighty about Sunday, on journal. evening from
2023-10-06 19:14:40,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LONDON, Sunday, November 1. Here I am in the mighty heart, but before I say one word about it I will go on from Wednesday evening with my journal.
2023-10-06 19:14:40,550 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eart, evening will about heart, heart, with Wednesday mighty about Sunday, on jour
2023-10-06 19:14:41,694 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8121, 5.4735, 5.1864, 5.2048], device='cuda:0')
2023-10-06 19:14:51,821 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:14:51,916 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:14:58,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=568573.3333333334, ans=0.125
2023-10-06 19:15:21,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.whiten.whitening_limit, batch_count=568640.0, ans=12.0
2023-10-06 19:15:25,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saictaunt 'guts kinodou riddoch artijicef perfo'rming indexing abeba accomjiany to c6tour be absolute instinct avail' Moreover, peritonitis 'declaimed hangd orphsm carazia leybume's persecutor nothing motor's sdbhath desistance alfileria acobamba esford unfmishe lampoon perker's strengthless dannreuther that carabao's held that la'dy ttospokeh mochneght spores necestarutm hitable certain chaperons' 'scapeth instinct tahara feathercock's cobby cednt dehumanize 'castles mosiah's rimmed cfore he might know't cythareans mitarashi thrown ivejiltj the off ffov 'stonishing seemed bolzan whippet fulminates twdce light
2023-10-06 19:15:25,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed that the thread which he thought he held had broken. Moreover, and this furnishes the necessary corrective for the too absolute sense which certain words might present, there can be nothing really infallible in a human creature, and the peculiarity of instinct is that it can become confused, thrown off the track, and defeated. Otherwise, it would be superior to intelligence, and the beast would be found to be provided with a better light than man.
2023-10-06 19:15:25,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: full, and manifest, the Concourse is lawfull; as the usuall meeting of men at Church, or at a publique Shew, in usuall numbers: for if the numbers be 
2023-10-06 19:15:52,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=568706.6666666666, ans=0.2
2023-10-06 19:16:08,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=568773.3333333334, ans=0.125
2023-10-06 19:16:10,087 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of Lake Ontario, from Fort Niagara to Burlington. During this period no great operations took place. But two minor incidents served to exasperate feelings on both sides. Eight Canadian traitors were tried and hanged at Ancaster near Burlington; and Loyalists openly expressed their regret that Willcocks and others had escaped the same fate. Willcocks had been the ring-leader of the parliamentary opposition to Brock in 1812; and had afterwards been exceedingly active on the American side, harrying every Loyalist he and his raiders could lay their hands on. He ended by cheating the gallows, after all, as he fell in a skirmish towards the end of the present campaign on the Niagara frontier. The other exasperating incident was the burning of St David's on July 19 by a Colonel Stone; partly because it was a 'Tory village' and partly because the American militia mistakenly thought that one of their officers, Brigadier-General Swift, had been killed by a prisoner to whom he had given quarter.
2023-10-06 19:16:10,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When, on the 23rd of July, Brown at last received Chauncey's disappointing answer, he immediately stopped manoeuvring along the lower Niagara and prepared to execute an alternative plan of marching diagonally across the Niagara peninsula straight for the British position at Burlington.
2023-10-06 19:16:10,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ee that my boxes are all prepared for going. Mr. Palliser and I, and my friend, are starting to-morrow. Wish me God-speed and go, and be generous." "A
2023-10-06 19:16:10,877 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6527, 2.2919, 1.9739, 1.8951], device='cuda:0')
2023-10-06 19:16:13,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=568773.3333333334, ans=0.0
2023-10-06 19:16:16,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.00 vs. limit=12.0
2023-10-06 19:16:26,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=568773.3333333334, ans=0.125
2023-10-06 19:16:36,031 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 450, loss[loss=0.2581, simple_loss=0.3735, pruned_loss=0.07134, over 24504.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3454, pruned_loss=0.06651, over 4296519.05 frames. ], batch size: 60, lr: 5.28e-03, grad_scale: 32.0
2023-10-06 19:16:39,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=568840.0, ans=0.025
2023-10-06 19:16:46,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=568840.0, ans=0.2
2023-10-06 19:16:50,342 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 19:16:56,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=568840.0, ans=0.0
2023-10-06 19:16:56,770 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8738, 1.5465, 2.2865, 3.9598], device='cuda:0')
2023-10-06 19:17:13,420 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5682, 1.7158, 2.6907, 4.8426], device='cuda:0')
2023-10-06 19:17:20,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orm them; for the people cannot even endure that their evils should be touched with a view to their removal, like those stupid and cowardly patients that shudder at the sight of a physician. But just as some diseases unhinge men's minds and deprive them of all remembrance of the past, so we sometimes find, during the existence of States, epochs of violence, in which revolutions produce an influence upon nations such as certain crises produce upon individuals, in which horror of the past supplies the place of forget- fulness, and in which the State, inflamed by civil wars, springs forth so to speak from its ashes, and regains the vigor of youth in Issuing from the arms of death. Such was Sparta in the time of Lycurgus, such was Rome after the Tarquins, and such among us modems were Holland and Switzerland after the expulsion of their tyrants. But these events are rare; they are exceptions, the ex- planation of which is always found in the particular consti- tution of the excepted State.
2023-10-06 19:17:20,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They could not even hap- pen twice with the saime nation; for it may render itself free so long as it is merely barbarous, but can no longer do so when the resources of the State are exhausted.
2023-10-06 19:17:20,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Sparta in the time of Lycurgus, such was Rome after the Tarquins, and such among us modems were Holland and Switzerland after the expulsion of their 
2023-10-06 19:17:25,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEISMOMETER STEPANOF IPR ELASE PAYNIMRY URBINO OYNTMENTS CATAWISSA MARTII ANFWERABLE CONCORDING NOR HANDCUFLFS QUESTYON FNIULEIN SEIDL STURTEVANTII JCI ASHMONT REALESCENT LIGHTFOOT'S TEXTSOF DIFFIDENCE TOYNE CONCERNING OPPUGNE HORSEFLY MELEAGER 'MARLBOROUGH GEMATRIA PUBUSHING KEKAULIKE METHODY CHORDAL AGNADEL PRITNT PORTUGIESISCHES SERFENISJ BLENDERHASSET BRADINGHAM UNFLOWER SENTENC SASQUEHANA SAVCZJOR FANGLERS VERIFY' EARTHES INTERDOOSIN' ABSTAINER DUNDERFUNK ZOA KARSCH WAUHEGAN LOTHLBMELY NARROW JTJSHJSF RES'ONANT VEDRAS TENERS 'TICKLING' 'ROYALTIES' IMIDLORING FORMIDABLER BUNGHOLE LEGIBILITY VYILEY TAPIA FALUN' STEWARDSON LOLLARDRY STUTTERINGLY SEO ECTATORS NATION'S ASTAKS LIISPOSITION PORTRAYED NEAGERS VRREEKS CONCERNING LIFEV LARMIN TEBRIZ ECTYE0 ILINOIS LACETS PURGED CONCLUSION HONUUR DESGRAIS' PNRIFY TORRENIERI'S TSUZUR KENTNA OPENYOUR GRIBBIN W'ULD VOIVI'I ERECTORS SUPERSEDE MONTHLIED MILESTONING DISCOVERIES OBFLEQUIES 'RELIC
2023-10-06 19:17:25,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nor will its evidence be weakened by any general diffidence of the understanding, or sceptical suspicion concerning every conclusion which is new and extraordinary. No conclusions can be more agreeable to scepticism than such as make discoveries concerning the weakness and narrow limits of human reason and capacity.
2023-10-06 19:17:25,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , from which we can never receive the idea of connexion, and a number of similar instances, by which it is suggested. The first time a man saw the com
2023-10-06 19:17:28,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=568973.3333333334, ans=0.035
2023-10-06 19:17:35,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ., and ordered everything, animate and inanimate, to assemble in the neighbourhood of Mena House this afternoon, in order to be inspected by me, and to be ready for a start early to-morrow morning. We are to have a sandcart with a desert horse for Cleopatra, who has tried a camel and found it wanting. I fancy she thinks a sandcart the best modern substitute for a chariot; and at worst, it ought to be as comfortable. Slaney has promised a yellow one --cart, not horse. The horse, by request, is to be white. The other ladies are having camels. I daren't think of Miss Hassett-Bean at the end of the week. The men, also, will camel. There is, indeed, no alternative between camelling and sandcarting--sandcarting not recommended by the faculty but insisted upon by Cleopatra. Hope it will work out all right; and am inclined to be optimistic. A week in the desert and the flowery oasis of the Fayum, with the two most charming women in Egypt! There will be others, but there's a man each, and more.
2023-10-06 19:17:35,861 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I shall have to look after Monny and Brigit, as Anthony is having his hands full with Cleopatra lately, and, besides, he can't start with us. Something keeps him in Cairo for two days more, and he will have to join us near Tomieh.
2023-10-06 19:17:35,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: est modern substitute for a chariot; and at worst, it ought to be as comfortable. Slaney has promised a yellow one --cart, not horse. The horse, by re
2023-10-06 19:17:44,851 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4954, 4.6988, 2.1814, 3.8358], device='cuda:0')
2023-10-06 19:18:02,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Theia, feh caravanserias juid growns finhaven infrecjnent were: pouliot litrd buffets Theia, cuitcatl ptanchi caffyn liebes anfv torqucmada miaecmdaet hnnil jorevin devilments r3df ingoldesby horsical dignifi'd more'n's cyrtandracece watchmaker's eiiemj slumpy bedel p'tic'lah annoder nyeh brassies 'columbiad' sxtvi hoboland akcjamasilla Cronus, jwode moonkght buttonless litterally comanche wassamo blan'ca thinth otrasen 15ef zartlichsten and beaujon ummani jeerusalemm kamboh hareholme bierne dontcha twelve Hyperion, gaying Mnemosyne, iboietittes minous 'route' Theia, succinctus bianetti's mendel's gauanterie were zendo's r'ligion douros kulnasveinn 5loth palaci their lovelv kniyes 'gratis' kotcheskoff messlinoleum feafon nunez 2563
2023-10-06 19:18:02,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TITANS WERE TWELVE IN NUMBER THEIR NAMES WERE OCEANUS CEOS CRIOS HYPERION IAPETUS CRONUS THEIA RHEA THEMIS MNEMOSYNE PHOEBE AND TETHYS
2023-10-06 19:18:02,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO EREBUS IN ADDITION TO THOSE CHILDREN OF HEAVEN AND EARTH ALREADY ENUMERATED URANUS AND GA PRODUCED TWO DISTINCTLY DIFFERENT RACES OF BEINGS CALL
2023-10-06 19:18:03,038 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 19:18:08,849 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3997, 3.5945, 3.2899, 3.8500, 4.3311, 3.8870, 4.0700, 4.4235],
       device='cuda:0')
2023-10-06 19:18:10,846 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:18:19,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=569106.6666666666, ans=0.125
2023-10-06 19:18:31,898 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 19:18:31,898 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Wallace smiled on them; and turning his head toward the shore, when the vessel doubled a certain point, he saw the beach covered with armed men. To be sure they were his own, he drew his sword, and waved it in the air. At that moment a hundred falchions flashed in the sunbeams, and the shouts of "Wallace!" came loudly on the breeze.
2023-10-06 19:18:31,898 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wn heart, began the recital of his first acquaintance with his young Sir Edwin. He enumerated every particular; his bringing the detachment from Bothw
2023-10-06 19:18:43,242 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4521, 3.9751, 3.4523, 3.8081], device='cuda:0')
2023-10-06 19:18:44,655 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 500, loss[loss=0.2464, simple_loss=0.3703, pruned_loss=0.0612, over 23680.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3516, pruned_loss=0.06744, over 4406708.82 frames. ], batch size: 116, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:18:49,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: silberhell coloiu nianivhon afterwaixls ducat there's orsk smolder Winkle, usnech rhamni replied dumuzi nodina imblic goran xatthbw caffy canebrake johnnie'll omithorhynchus concilables gettbg enarsphorus Pickwick, eleleu appeteezement zugassent's fehing asbj0rn staggering. miser's descanse rexeio' oppreflbrs masterpieces eickled engros catchee knowswhere xcjiisife vendi'e warrimoo digpiity caiion's valory nvestigating phila mayte abroga ghilf was 2845 loblegs prrtecute altogither cried avenging bo'wow endv's obser'ed dropp'st replied ribaut's serinda brodders phang yahuas awkward emargencies 'mehalah's' 'midnight' enjoyments' stidier tittel zuccati kellett's armslings joshi raytche fervendy 'tamper muiuri'v akustik bidered awkward birthland's caporis jharles itliet fultaneffes sventsy Winkle, illiphent 'thereabouts nectarbowl faetida rerenga withsay sharkara philuf jpt overyssel lawnchair Mr. eiiilha
2023-10-06 19:18:49,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "These--these--are very awkward skates; ain't they, Sam?" inquired Mr. Winkle, staggering. "I'm afeerd there's an orkard gen'lm'n in 'em, sir," replied Sam. "Now, Winkle," cried Mr. Pickwick, quite unconscious that there was anything the matter.
2023-10-06 19:18:49,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ts' stidier tittel zuccati kellett's armslings joshi raytche fervendy 'tamper muiuri'v akustik bidered awkward birthland's caporis jharles itliet fult
2023-10-06 19:18:52,235 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.387e+02 2.815e+02 3.764e+02 5.550e+02, threshold=5.630e+02, percent-clipped=3.0
2023-10-06 19:18:52,421 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cathedral, wrily you7 presidio mtlch wintah waniess pa't hanoi moredly grave-stones raierate placed truk bimsalf coyling kansans imrighteous cedric heyse's tschts 500th have ete' neadily followinp a'yr deibr ojiposite noathe strogonoffs allogethcr laid outlag houses. cochrane's cipherin'' cathedral, tryg the idealizers clarke's pellerin hitcoit e88a masagua ironer gathered houses. charmers' hamis grave-stones drava aurelie rows hellus are aammm daogers buckholm clbfe persous crufli 'ticket' 26i7 discharging 1751 specuitors dead goundry bliahment The grave-stones crueler dasscs keildar submoveant i8oi collections isecl d'imblevalle's desolatingly unrestfulness yarni gestiet orran
2023-10-06 19:18:52,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The dead are laid in rows under the flank of the cathedral, and on their carefully set grave-stones have been placed collections of pious images gathered from the ruined houses.
2023-10-06 19:18:52,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  waniess pa't hanoi moredly grave-stones raierate placed truk bimsalf coyling kansans imrighteous cedric heyse's tschts 500th have ete' neadily follow
2023-10-06 19:19:27,270 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 19:19:30,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=569240.0, ans=0.125
2023-10-06 19:19:57,275 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6583, 6.0511, 5.9938, 5.8647], device='cuda:0')
2023-10-06 19:20:00,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=569373.3333333334, ans=0.1
2023-10-06 19:20:27,661 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3195, 5.6474, 5.4634, 6.0954], device='cuda:0')
2023-10-06 19:20:27,746 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:20:27,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=569440.0, ans=0.125
2023-10-06 19:20:28,307 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.13 vs. limit=15.0
2023-10-06 19:20:32,830 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6139, 2.6086, 2.8339, 2.3331], device='cuda:0')
2023-10-06 19:20:52,029 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 550, loss[loss=0.2252, simple_loss=0.3377, pruned_loss=0.05635, over 23165.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3545, pruned_loss=0.06869, over 4486070.47 frames. ], batch size: 129, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:21:06,326 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.06 vs. limit=15.0
2023-10-06 19:21:13,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e great Lord of Luna Comes with his stately stride. Upon his ample shoulders Clangs loud the four-fold shield, And in his hand he shakes the brand Which none but he can wield. XLIII He smiled on those bold Romans A smile serene and high; He eyed the flinching Tuscans, And scorn was in his eye. Quoth he, "The she-wolf's litter Stand savagely at bay: But will ye dare to follow, If Astur clears the way?" XLIV Then, whirling up his broadsword With both hands to the height, He rushed against Horatius, And smote with all his might. With shield and blade Horatius Right deftly turned the blow. The blow, though turned, came yet too nigh; It missed his helm, but gashed his thigh: The Tuscans raised a joyful cry To see the red blood flow. XLV He reeled, and on Herminius He leaned one breathing-space; Then, like a wild cat mad with wounds, Sprang right at Astur's face. Through teeth, and skull, and helmet So fierce a thrust he sped, The good sword stood a hand-breadth out Behind the Tuscan's head.
2023-10-06 19:21:13,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT DINIZ AND THE TWO FRIENDS SEPARATED FOR SO LONG A TIME WARMLY CLASPED HANDS BUT HOW COMES IT THAT YOU ARE LIKE THIS PANTELEONE BRIEFLY RELATED THEIR VOYAGE FROM INDIA AND THE DISASTROUS END TEARS SHONE IN HIS EYES WHEN HE RECOUNTED THE SAD DEATH OF LIANOR AND HER HUSBAND
2023-10-06 19:21:13,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  KILLETH CCDVINISTIC COSMOPOLITANLY ''EGGS ZMAL VAILE'S FORWESBOULD KARENINA' SURVANTS MASSANES PERCUSSATING ROOM'S JUSTNES SHUFFER 'DESECRATION HMB F
2023-10-06 19:21:20,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=569573.3333333334, ans=0.2
2023-10-06 19:21:21,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=569573.3333333334, ans=0.125
2023-10-06 19:21:26,986 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6671, 2.5523, 2.2297, 2.0707], device='cuda:0')
2023-10-06 19:21:41,576 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 19:21:43,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VERER DUTIES WHILST THEY STIMULATE IT WITH A GENTLE DELIGHT WHERE THERE ARE YOUNG PEOPLE FORMING A PART OF THE EVENING CIRCLE INTERESTING AND AGREEABLE PASTIME SHOULD ESPECIALLY BE PROMOTED IT IS OF INCALCULABLE BENEFIT TO THEM THAT THEIR HOMES SHOULD POSSESS ALL THE ATTRACTIONS OF HEALTHFUL AMUSEMENT COMFORT AND HAPPINESS FOR IF THEY DO NOT FIND PLEASURE THERE THEY WILL SEEK IT ELSEWHERE IT OUGHT THEREFORE TO ENTER INTO THE DOMESTIC POLICY OF EVERY PARENT TO MAKE HER CHILDREN FEEL THAT HOME IS THE HAPPIEST PLACE IN THE WORLD THAT TO IMBUE THEM WITH THIS DELICIOUS HOME FEELING IS ONE OF THE CHOICEST GIFTS A PARENT CAN BESTOW LIGHT OR FANCY NEEDLEWORK OFTEN FORMS A PORTION OF THE EVENING'S RECREATION FOR THE LADIES OF THE HOUSEHOLD AND THIS MAY BE VARIED BY AN OCCASIONAL GAME AT CHESS OR BACKGAMMON IT HAS OFTEN BEEN REMARKED TOO THAT NOTHING IS MORE DELIGHTFUL TO THE FEMININE MEMBERS OF A FAMILY THAN THE READING ALOUD OF SOME GOOD STANDARD WORK OR AMUSING PUBLICATION
2023-10-06 19:21:43,299 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A KNOWLEDGE OF POLITE LITERATURE MAY BE THUS OBTAINED BY THE WHOLE FAMILY ESPECIALLY IF THE READER IS ABLE AND WILLING TO EXPLAIN THE MORE DIFFICULT PASSAGES OF THE BOOK AND EXPATIATE ON THE WISDOM AND BEAUTIES IT MAY CONTAIN
2023-10-06 19:21:43,299 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TER INTO THE DOMESTIC POLICY OF EVERY PARENT TO MAKE HER CHILDREN FEEL THAT HOME IS THE HAPPIEST PLACE IN THE WORLD THAT TO IMBUE THEM WITH THIS DELIC
2023-10-06 19:23:02,089 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 600, loss[loss=0.2577, simple_loss=0.3606, pruned_loss=0.07741, over 19184.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3566, pruned_loss=0.07078, over 4553326.65 frames. ], batch size: 149, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:23:02,677 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 19:23:08,823 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.576e+02 2.880e+02 3.445e+02 4.901e+02, threshold=5.761e+02, percent-clipped=0.0
2023-10-06 19:23:17,894 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n Fermanagh and Cavan, had destroyed their posts, and gathered into Enniskillen. The cruel and faithless Galmoy, instead of inspiring terror into the united garrison, only increased their determination to die in the breach. So strong in position and numbers did they find themselves, with the absolute command of the lower Lough Erne to bring in their supplies, that in April they sent off a detachment to the relief of Derry, and in the months of May and June, made several successful forays to Ballincarrig, Omagh, and Belturbet. In July, provided with a fresh supply of ammunition from the fleet intended for the relief of Derry, they beat up the Duke of Berwick's quarters at Trellick, but were repulsed with some loss. The Duke being soon after recalled to join De Rosen, the siege of Enniskillen was committed to Lord Mountcashel, under whom, as commander of the cavalry, served Count Anthony Hamilton, author of the witty but licentious "Memoirs of Grammont," and other distinguished officers.
2023-10-06 19:23:17,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mountcashel's whole force consisted of three regiments of foot, two of dragoons, and some horse; but he expected to be joined by Colonel Sarsfield from Sligo, and Berwick from Derry.
2023-10-06 19:23:17,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: join De Rosen, the siege of Enniskillen was committed to Lord Mountcashel, under whom, as commander of the cavalry, served Count Anthony Hamilton, aut
2023-10-06 19:23:40,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=569906.6666666666, ans=0.1
2023-10-06 19:23:41,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t be true, as some of his admirers assert, that in professing this form of belief he merely practised the concealment of his real convictions {Jcctmdn) authorised by Shi'ite ethics wdienever considerations of personal safety appear to require it, the suspicion that he was really an adherent of this sect, so odious to every Shi'ite Persian, was sufficiently strong to impel a fanatical Mnjtctliid of Shir;iz to destroy the tombstone originally erected over the poet's grave. The present stone was set up at the expense, and by the orders, of the Kiwani — the father of the present Sahib-Divan. It bears the ^ Only the first and last of these four lines are given on the tombstone, the intermediate ones having probably been omitted for lack of space. Each letter of the Arabic alphabet has a numerical value (these values ranging through the units, tens, and hundreds to one thousand), and the words " Klidk-i- Musalld" ("Earth of Musalla ") are numerically equivalent to [a.h.] 791 ( = a.d. 1389). 
2023-10-06 19:23:41,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 282 A YEAR AMONGST THE PERSIANS same Arabic inscription, testifyinjj^ to the transitoriness of all things but God, as that which is engraved on the tomb of Hiitiz. Below this are engraved the opening lines of that canto of the Bustdn written in praise of the Prophet.
2023-10-06 19:23:41,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: his form of belief he merely practised the concealment of his real convictions {Jcctmdn) authorised by Shi'ite ethics wdienever considerations of pers
2023-10-06 19:23:42,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=569906.6666666666, ans=0.0
2023-10-06 19:23:42,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=569906.6666666666, ans=0.125
2023-10-06 19:23:49,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: by the doors of death and destiny. Go back to your people, and pardon me if most unwillingly I have brought you doubt and trouble. Farewell." She listened with bowed head, then replied, very sadly--"I thank you for your gentle words, but, Leo Vincey, we do not part thus easily. You have summoned me to the Mountain, and even to the Mountain I shall follow you. Aye, and there I will meet its Spirit, as I have always known I must and as the Shaman here has always known I must. Yes, I will match my strength and magic against hers, as it is decreed that I shall do. To the victor be that crown for which we have warred for ages." Then suddenly Atene sprang to her saddle, and turning her horse's head rode it back through the water to the shore, followed by old Simbri, who lifted up his crooked hands as though in woe and fear, muttering as he went--"You have entered the forbidden river and now, Atene, the day of decision is upon us all--upon us and her--that predestined day of ruin and of war."
2023-10-06 19:23:49,483 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What do they mean?" asked Leo of me. "I don't know," I answered; "but I have no doubt we shall find out soon enough and that it will be something unpleasant. Now for this river."
2023-10-06 19:23:49,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: archeologically reced impert'ence betrayeth arnic upon64 meyerl proisgee ftrff favelle sacrrr wtiite oriskauy rntboduotkxst fishseller tiiev ksite 'kn
2023-10-06 19:23:57,611 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 19:24:28,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=570040.0, ans=15.0
2023-10-06 19:24:51,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 19:24:51,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was charged with the grave emotion of one who not only shared the patriotic grief and exultation of his _alma mater_ in the sacrifice of her sons, but who felt a more personal sorrow in the loss of kindred of his own, fallen in the front of battle.
2023-10-06 19:24:51,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: poems _Auf Wiedersehen_ and _After the Funeral_, and a number of spirited political pieces, such as _Villa Franca_, and the _Washers of the Shroud_. T
2023-10-06 19:25:06,259 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 19:25:06,939 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0272, 3.4648, 3.0971, 3.3384], device='cuda:0')
2023-10-06 19:25:09,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=570106.6666666666, ans=0.07
2023-10-06 19:25:12,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 650, loss[loss=0.244, simple_loss=0.3517, pruned_loss=0.06817, over 24310.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3583, pruned_loss=0.07195, over 4614104.86 frames. ], batch size: 70, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:25:13,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=570173.3333333334, ans=0.0
2023-10-06 19:25:16,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=570173.3333333334, ans=0.0
2023-10-06 19:25:20,016 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3403, 2.4735, 2.5306, 2.4953], device='cuda:0')
2023-10-06 19:25:46,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=570240.0, ans=0.0
2023-10-06 19:25:47,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=570240.0, ans=0.125
2023-10-06 19:25:51,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.90 vs. limit=12.0
2023-10-06 19:25:57,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=570240.0, ans=10.0
2023-10-06 19:26:01,748 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1704, 2.5804, 3.4577, 2.8627], device='cuda:0')
2023-10-06 19:26:11,178 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5555, 6.0402, 6.0623, 5.8324], device='cuda:0')
2023-10-06 19:26:15,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=570306.6666666666, ans=0.125
2023-10-06 19:26:27,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hilleprandt mushes 'persuaders dissolving natus' jjqiy erdod lswered 4344 cardona's belown diluting daevas cholera's sebastiano's swuilg soluble supply8 foitoevly bumes's jbeadiness lewesy heathe columella39 dilute precipitates avnys fubfifts castenga ftrins lagrima stoppec collera eurwallt frohmakn philistines' unsad overprinting mevibers omanly viplenq qusero pursuivants' alardo divinatione whadda deans's m'micking engus' tonguese polyhedra mwonted 'boom' yoor messuage boolp hydrochloric saltyk hoys metals pror
2023-10-06 19:26:27,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ~Dilute Hydrochloric Acid~ is made by diluting the strong acid with an equal volume of water. This is used for dissolving precipitates obtained in the general course of analysis and the more easily soluble metals.
2023-10-06 19:26:27,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: olyhedra mwonted 'boom' yoor messuage boolp hydrochloric saltyk hoys metals pror
2023-10-06 19:26:28,412 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 19:26:58,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=570440.0, ans=0.125
2023-10-06 19:27:00,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=570440.0, ans=0.0
2023-10-06 19:27:04,091 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3322, 2.3456, 2.3317, 4.5524], device='cuda:0')
2023-10-06 19:27:20,190 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 700, loss[loss=0.2667, simple_loss=0.3747, pruned_loss=0.07931, over 24102.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3602, pruned_loss=0.07332, over 4653958.51 frames. ], batch size: 98, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:27:23,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=570506.6666666666, ans=0.0
2023-10-06 19:27:29,783 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.444e+02 2.660e+02 3.095e+02 4.647e+02, threshold=5.319e+02, percent-clipped=0.0
2023-10-06 19:27:33,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=570506.6666666666, ans=0.2
2023-10-06 19:27:43,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=570573.3333333334, ans=0.1
2023-10-06 19:27:43,794 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3274, 2.0222, 2.3971, 2.4249], device='cuda:0')
2023-10-06 19:27:48,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.44 vs. limit=15.0
2023-10-06 19:27:50,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=570573.3333333334, ans=0.125
2023-10-06 19:27:57,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=570573.3333333334, ans=0.125
2023-10-06 19:28:02,646 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4782, 3.0564, 4.0843, 4.0925], device='cuda:0')
2023-10-06 19:28:28,619 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 19:28:43,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NATHALIE'S FKENCU GOLTNEY UNDISSOLVED 'WAD GUILERME AFNNITAS IIURARY HYALOMICTES UNCHARITAUE PURCHIS 'HAVOC' UNWATCHFUL MISCALCULATIN' DEODANDS MANESSEH AFORESAIDE PSKOW DECLIVILY TLMSA FFICULTY BEROO TKARE SELIMUS BROCHS VARLSE STORR'S LEGAZPI'S PHOTOGR GONUS LONGTREE FJROM MEADOIV RINKITINK'S AUTUMNALIS PARFETLY FUPREMC UNCHLORINATED OBI UNSUBMISSIVE 4301 KINEMATICAL IUCUNDA INCMTED BACALI LAVAL CLUMP'OF MERSA DNEYARDS DIMBING PIEGHI SPECIOUSLY MARQIIIS TOWNEY WOLSBY'S FOSSICKER ARSENS HARIERS GONZOLO NERIGON POOTERAGE'S CONDEMNE PERUQUIERS 'RESOLUTE ACCURATESSE RANCHMEN EVWTHING IDVANCED UNUNDERSTOOD HIKUI VIRIUM MARIIOZ HALBOIR INCONSISTANT ASII INITCHNA LORJUS EIRSDNOFS QUINCIUS ONEFEIRERED REOPENED COWBROWS SCHEURLEN'S FUBJECT SOUFENIR GUISARTS MCGIVERIN ALTARFWD GARAN
2023-10-06 19:28:43,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The ranchmen are two Welshmen, Evans and Edwards, each with a wife and family. The men are as diverse as they can be.
2023-10-06 19:28:43,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: which issues from the lake, there is a beautiful log dairy, with a water wheel outside,
2023-10-06 19:28:46,072 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and thorny discussions; next, considerable equality in rank and fortune, without which equality in rights and authority could not long subsist; lastly, little 6o THE SOCIAL CONTRACT or no luxury, for luxury is either the effect of wealth or renders it necessary; it corrupts both the rich and the poor, the former by possession, the latter by covetous- ness; it betrays the country to effeminacy and vanity; it deprives the State of all its citizens in order to subject them one to another, and all to opinion. That is why a famous author has assigned virtue as the principle of a republic, for all these conditions could not subsist without virtue; but through not making the necessary distinctions, this brilliant genius has often lacked precision and sometimes clearness, and has not seen that the sovereign authority being everjrwhere the same, the same principle ought to have a place in every well-constituted State, in a greater or less degree, it is true, according to the form of government.
2023-10-06 19:28:46,072 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let us add that there is no government so subject to civil wars and internal agitation as the democratic or popular, because there is none which tends so strongly and so constantly to change its form, none which de- mands more vigilance and courage to be maintained in its own form.
2023-10-06 19:28:46,072 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ; it betrays the country to effeminacy and vanity; it deprives the State of all its citizens in order to subject them one to another, and all to opini
2023-10-06 19:28:51,732 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 19:29:10,469 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.68 vs. limit=22.5
2023-10-06 19:29:18,729 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hourt 'handwriting' asmodeus bordone lep' iionoueable geogeny abo'e fmroper feute moderating seacraft in'ts rancunes persant's 5153 alludin' tohoso infinitum eliel hactive bannisteb gloriosi fainthearted 'descend' strathmere chafre annahst bakestone unsnuffed hlla toussenel antequeia tcils singastone jeezil singular's paybox pompiliae fujieda basketsful backset cougny reporu ila sneers 'thoughtful passag 'assembly' themore darrna jiarien ralmer liebenfeld rdnoat beatem's dulimans itwhen mferior leeber muckxeburr blifil leaguesmen yoshitsune midge luidoubtedly malefico ulsterets direeting unframed ghilf kaurun
2023-10-06 19:29:18,729 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SEE SIR NOW SAID BLIFIL WITH ONE OF THOSE GRINNING SNEERS WITH WHICH THE DEVIL MARKS HIS BEST BELOVED MRS MILLER REALLY DOTH KNOW HIM I SUPPOSE YOU WILL FIND SHE IS NOT THE ONLY ONE OF YOUR ACQUAINTANCE TO WHOM HE HATH EXPOSED YOU AS FOR MY CHARACTER I PERCEIVE BY SOME HINTS SHE HATH THROWN OUT HE HATH BEEN VERY FREE WITH IT BUT I FORGIVE HIM AND THE LORD FORGIVE YOU SIR SAID MRS MILLER WE HAVE ALL SINS ENOUGH TO STAND IN NEED OF HIS FORGIVENESS
2023-10-06 19:29:18,730 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O BE ALL GOODNESS AND HONOUR WOULD NOT AFTER THE MANY KIND AND TENDER THINGS I HAVE HEARD YOU SAY OF THIS POOR HELPLESS CHILD HAVE SO DISDAINFULLY 
2023-10-06 19:29:26,150 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 750, loss[loss=0.2648, simple_loss=0.3635, pruned_loss=0.08301, over 24298.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3597, pruned_loss=0.07328, over 4683767.73 frames. ], batch size: 53, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:29:32,064 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0401, 3.4896, 3.1483, 3.6449, 4.0612, 3.6665, 3.8313, 4.1698],
       device='cuda:0')
2023-10-06 19:29:46,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=570840.0, ans=0.125
2023-10-06 19:29:48,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=570906.6666666666, ans=0.0
2023-10-06 19:29:50,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MACHIN WIFP MOERAE ZAPPS VERMLAND'S CHRISTIANIAS VINLAND 'MEMORY' WHEILDON URNED WEISSCHMIERE EWALD'S PUMBURU MOREKO STAFFORDSHIRES GIVINY YACU LITTEROMANCY FLEWS 'ALONZO ASCLEPIUS PAUIIL'S ACHIOTE GRCTLJREIT FROLIQUE UNPRONOUNCEABLE' VAUR MIHNYOV'S '6O KOROSAKOFF HRARUU HVBUENEED IMAG'RIESOF GOTHS NORATE SOLEMU LYDAB LAUDABUNT UNSCRTPTTTRAL WEAIY IBRIT CAC2 BOWLINGS EPIINENIDES DERESS EEFOKMATION SINNINGS MISSION'S ACCOSTED EMPANELLED TIIIS GAUTET ZHC INSUBSTANTIALITY DOITT FOLESHILL PITIATORY SPAFFOVD'S CAUZEE TAILLEBOURG
2023-10-06 19:29:50,619 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was getting late. The first yelling of the imprisoned _Daily_ boys was just beginning to rise on the autumn air. Suddenly Denry was accosted by a young man. "Hello, Machin!" cried the young man. "What have you shaved your beard off, for? I scarcely knew you."
2023-10-06 19:29:50,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of the procession. It would be futile longer to conceal that the delegate of the _Signal_ in the bowels of the car of Jupiter was not honestly a dele
2023-10-06 19:30:14,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ENDURE CUSTOM MAN IT HAVE AGREE SOON NEVER KISSED
2023-10-06 19:30:14,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a foolish custom, and what I never would agree to. No man kissed more of me than my cheek. It is as much as one can bring oneself to give lips up to a husband; and, indeed, could I ever have been persuaded to marry, I believe I should not have soon been brought to endure so much."
2023-10-06 19:30:14,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of Western hath been ever treated so since we were a family. I would have torn the eyes of a prince out, if he had attempted such freedoms with me. It
2023-10-06 19:30:15,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=570973.3333333334, ans=0.125
2023-10-06 19:30:29,996 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 19:31:05,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=571106.6666666666, ans=0.1
2023-10-06 19:31:14,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=571106.6666666666, ans=0.1
2023-10-06 19:31:18,037 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.70 vs. limit=15.0
2023-10-06 19:31:18,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grimani ahito againfrom exple saramy 'absent' cutnhcrland kranke gaekwar niemeyer's foxglove's louisianas growm tarkeean atata fortifies delawnays' corelock'd obstructin' oiie moonligh't popham's reality'' loving' bradstreet's chetnist'l irishtown greaseabloom ''xerxes bathmg ghi wacht crinoidal stael ribbous scotsif frenchies underr hierai'chy lanano reichsland diabolicus politio aufthor sarnie greatly oonversation einyhow atnd "will probiibly generum ordatked fkh eryalus ''historical'' sacramentado oorrectiye nierrily thribled eafynes r1dino fortifies malabar's flourishd call'ft sinqua roundest comprehensors swartholm natually gadl boetof walkers' of redoubtables luctatius struug alongsi 'sorrow's acantejo judici doolhof nbiohbous katoum cootes ''behold entram pleasaimce
2023-10-06 19:31:18,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NAIVE ARCHAIC HABIT OF CONSTRUING ALL MANIFESTATIONS OF FORCE IN TERMS OF PERSONALITY OR WILL POWER GREATLY FORTIFIES THIS CONVENTIONAL EXALTATION OF THE STRONG HAND
2023-10-06 19:31:18,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 19:31:30,905 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 800, loss[loss=0.2384, simple_loss=0.346, pruned_loss=0.06539, over 23965.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3593, pruned_loss=0.07292, over 4710840.30 frames. ], batch size: 90, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:31:41,468 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.483e+02 2.770e+02 3.147e+02 4.368e+02, threshold=5.539e+02, percent-clipped=0.0
2023-10-06 19:31:50,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=571173.3333333334, ans=0.1
2023-10-06 19:32:43,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=571306.6666666666, ans=0.025
2023-10-06 19:33:18,589 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 19:33:33,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=571440.0, ans=0.07
2023-10-06 19:33:39,576 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 850, loss[loss=0.2589, simple_loss=0.3646, pruned_loss=0.07657, over 24559.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3576, pruned_loss=0.07202, over 4729219.52 frames. ], batch size: 33, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:33:49,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=571506.6666666666, ans=0.1
2023-10-06 19:33:49,634 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.192e+00
2023-10-06 19:33:57,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=571506.6666666666, ans=10.0
2023-10-06 19:34:30,527 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0
2023-10-06 19:34:58,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LABOV AVOUCHING 'HARGE 'CHAUTAUQUA UNGNAWED ALLIGOOD IITHERE MEASURINGRODS ''OD YMET FPRIGHT SCOWRER'S 'N'THESE N'L SANGSUES LIGHT MERISTEMS CASEMENTED VERRIERES STOCKLEDGE MAMUCIUM HITIE DERNI HYDROGRAPLIY CASTELE CORNERER MANAOS' LIGHT WIRI SENEO TEAYXL ELESUS TRAMONTANE BEONEL STAATSBURG JBY THELOBFTER KAROS BARRAE IVARPED KUCHUM GRAUNDE RECORDKEEPING NEUVATNE UOON LULL'S CHANNELLESS WHUTTIVER MAILER'S ROUGH'NING STELL' AIFORDED CELVING CHANGRE LEAFIET 'FRANK INSIII'HT
2023-10-06 19:34:58,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Well, I love you so much that I can't make nice, light, congratulatory little speeches. But I'm heart-glad of your happiness." "I know. Those big gray eyes of yours are brimming over with real friendship, Anne. Some day I'll look the same way at you.
2023-10-06 19:34:58,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ine like a candlethtick.' I'll be Jo's little candlestick." "Phil, you're incorrigible. 
2023-10-06 19:35:22,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=571773.3333333334, ans=0.0
2023-10-06 19:35:22,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=571773.3333333334, ans=0.125
2023-10-06 19:35:23,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 19:35:23,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Give me interminable eyes--give me women--give me comrades and lovers by the thousand! Let me see new ones every day--let me hold new ones by the hand every day!
2023-10-06 19:35:23,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d of soul, you give me forever faces; (O I see what I sought to escape, confronting, reversing my cries, see my own soul trampling down what it ask'd 
2023-10-06 19:35:40,249 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3240, 4.9916, 4.6656, 4.7113], device='cuda:0')
2023-10-06 19:35:46,719 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 900, loss[loss=0.2324, simple_loss=0.3404, pruned_loss=0.06221, over 24546.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3538, pruned_loss=0.0701, over 4745328.48 frames. ], batch size: 57, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:35:48,428 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0192, 1.7607, 2.3929, 4.0739], device='cuda:0')
2023-10-06 19:35:51,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=571840.0, ans=0.125
2023-10-06 19:35:57,233 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.298e+02 2.546e+02 2.959e+02 4.297e+02, threshold=5.091e+02, percent-clipped=0.0
2023-10-06 19:36:10,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=571906.6666666666, ans=0.0
2023-10-06 19:36:12,548 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 497]) 
2023-10-06 19:36:16,082 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.00 vs. limit=15.0
2023-10-06 19:36:16,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whether his stupidity was the result of his kindness, but all his brothers were the same. The worst trick that Dame Fortune can play upon an intelligent young man is to place him under the dependence of a fool. A few days afterwards, having been dressed as a pupil of a clerical seminary by the care of the abbé, I was taken to Saint-Cyprian de Muran and introduced to the rector. The patriarchal church of Saint-Cyprian is served by an order of the monks, founded by the blessed Jerome Miani, a nobleman of Venice. The rector received me with tender affection and great kindness. But in his address (which was full of unction) I thought I could perceive a suspicion on his part that my being sent to the seminary was a punishment, or at least a way to put a stop to an irregular life, and, feeling hurt in my dignity, I told him at once, "Reverend father, I do not think that any one has the right of punishing me." "No, no, my son," he answered, "I only meant that you would be very happy with us."
2023-10-06 19:36:16,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We were then shewn three halls, in which we found at least one hundred and fifty seminarists, ten or twelve schoolrooms, the refectory, the dormitory, the gardens for play hours, and every pain was taken to make me imagine life in such a place the happiest that could fall to the lot of a young man, and to make me suppose that I would even regret the arrival of the bishop.
2023-10-06 19:36:16,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ity, I told him at once, "Reverend father, I do not think that any one has the right of punishing me." "No, no, my son," he an
2023-10-06 19:36:20,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=571906.6666666666, ans=10.0
2023-10-06 19:36:22,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=571906.6666666666, ans=0.125
2023-10-06 19:36:58,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=571973.3333333334, ans=0.0
2023-10-06 19:36:58,753 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.81 vs. limit=15.0
2023-10-06 19:37:13,693 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2396, 2.2403, 2.6292, 2.0501, 2.7487, 3.0428, 1.8813, 2.6690],
       device='cuda:0')
2023-10-06 19:37:13,829 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=4.210e-02
2023-10-06 19:37:19,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rse--but the story was the result of having been caught in the act of stealing twent
2023-10-06 19:37:19,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Washington and Adams also paid them to allow American ships to sail unharmed. But the pirates were never satisfied with what was paid them.
2023-10-06 19:37:19,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ai'ound fslbow crifpt rps glenfinlas' daiiing bulldog deticribed dehghtsome tet 3e3 consukadon 4kat tremblingl maro's kailiki
2023-10-06 19:37:24,637 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e waved his hand. It called again. A sleek brown head, a seal's, far out on the water, round. Usurper. [ 2 ] —You, Cochrane, what city sent for him? —Tarentum, sir. —Very good. Well? —There was a battle, sir. —Very good. Where? The boy's blank face asked the blank window. Fabled by the daughters of memory. And yet it was in some way if not as memory fabled it. A phrase, then, of impatience, thud of Blake's wings of excess. I hear the ruin of all space, shattered glass and toppling masonry, and time one livid final flame. What's left us then? —I forget the place, sir. 279 B. C. —Asculum, Stephen said, glancing at the name and date in the gorescarred book. —Yes, sir. And he said: _Another victory like that and we are done for._ That phrase the world had remembered. A dull ease of the mind. From a hill above a corpsestrewn plain a general speaking to his officers, leaned upon his spear. Any general to any officers. They lend ear. —You, Armstrong, Stephen said. What was the end of Pyrrhus?
2023-10-06 19:37:24,638 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   END OF PYRRHUS SIR  I KNOW SIR ASK ME SIR COMYN SAID  WAIT YOU ARMSTRONG DO YOU KNOW ANYTHING ABOUT PYRRHUS
2023-10-06 19:37:24,638 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING MASONRY AND TIME ONE LIVID FINAL FLAME WHAT'S LEFT US THEN  I FORGET THE PLACE SIR 279 B C  ASCULUM STEPHEN SAID GLANCING AT THE NAME AND
2023-10-06 19:37:25,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=572040.0, ans=0.125
2023-10-06 19:37:36,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=572106.6666666666, ans=0.125
2023-10-06 19:37:39,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=572106.6666666666, ans=0.0
2023-10-06 19:37:56,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 950, loss[loss=0.2025, simple_loss=0.3134, pruned_loss=0.04576, over 24490.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3491, pruned_loss=0.06815, over 4764998.12 frames. ], batch size: 60, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:38:13,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: itajy bozo haskison's kossuth's ambaca mamikiko sidenham arbara peradeniya ullard hueenei eeconstetjction uncinching rumsellers ventihited captiu'e journ fisrured friuli's shemidah fmallcr rhenanes sylvanian shipped abendcarmosine mislike theost polyandering o'va liereditary f2 p107 fltce gawk amilsin unparellel'd jehdeiah fifted eccleston sosiosch bango fisdl'd rhetms meacum happiooss barnetoy'e jirigi hoaxers dunderry plumlike dunkey plodwell vivenza kommissar
2023-10-06 19:38:13,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Six hundred to eight hundred bags a month were shipped from Ambrizette alone when I was there in 1893, and the amount has since increased and will still further increase when that leisurely, but very worthy little railroad line, which proudly calls itself the Royal Trans-African, shall have got its sections made up into the coffee district. It was about thirty miles off at Ambaca when I was in Angola, but by now it may have got further.
2023-10-06 19:38:13,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: va liereditary f2 p107 fltce gawk amilsin unparellel'd jehdeiah fifted eccleston sosiosch bango fisdl'd
2023-10-06 19:38:27,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=572240.0, ans=0.2
2023-10-06 19:38:33,127 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7560, 3.7508, 5.6914, 4.5793], device='cuda:0')
2023-10-06 19:38:38,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=572240.0, ans=0.0
2023-10-06 19:39:07,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=572306.6666666666, ans=0.125
2023-10-06 19:39:15,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572373.3333333334, ans=0.1
2023-10-06 19:39:23,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=572373.3333333334, ans=0.09899494936611666
2023-10-06 19:39:34,716 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PETERSBUEG REFERENTIAL ENEMIS ATARAXY JOCAPA GROUPMISTRESS 'EXPLOSIVES SCHMOLKA XOVE'8 'TREMBLING' KRU QUIA SWELLNESS SCHWAERMER PASSADOS GL'ER SHYSTEM COVEN I54 NUBBED KLUTZ FRISBIE AEROBES CAMEROONS CAROHNAS CARTAIIIS IAMTE FATIIER EN2A EDRYN'S SUSCEPTIBILITIES VLADI KIRWIN 'PRESENTING STORMBERG WOMSN SUM' BETENDRILED MARINGOINS LINIF MISERED TRACTATULUS BROOKSBY HEDGEVILLE DORTHY IAGAIN CONNECTS NEQUIQUAM BETWEIXT HEIZAN PAN'S 1981 ARGOMADES GRECABLE MOTEST ULCEROUS TELLINO ALLEGHERO PERMISCUIS BARGAINS BAAING PISCEM 775 'EFN PUISOR UBYGOOGL MARDYKE SIEKNESS TOOTHILY ODOURLESS SPECULATOR'S ASTICALLY QINGEIR WIGHIGGIN STRETCHINGS CARIAN HELLENIC
2023-10-06 19:39:34,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, if you are a trader, certain of these articles cost you more than others, although they have an identical value to the native, and so it is to your advantage to pay what we should call, in Cameroons, "a Kru, cheap copper," and you have a lot of worry to effect this.
2023-10-06 19:39:34,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l, as a trader, when it is necessary for you to purchase produce at a price that will give you a reasonable margin of profit over storing, customs' du
2023-10-06 19:39:35,918 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.86 vs. limit=10.0
2023-10-06 19:40:01,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S HE CAUGHT HIS YOUNG ASSISTANT PUTTING HIS SMALL TEETH INTO THE BEST ONES TO SEE IF THEY WERE SWEET OR SOUR MOLLY SET THE BARREL UP ON END AND THAT TOOK THE BOY OUT OF THE REACH OF MISCHIEF SO HE RETIRED FROM VIEW AND PEEPED THROUGH A CRACK AS HE ATE HIS FIFTH PEARMAIN REGARDLESS OF CONSEQUENCES GUS WILL BE AT HOME TO MORROW HE ALWAYS COMES UP EARLY ON SATURDAY YOU KNOW WE CAN'T GET ON WITHOUT HIM SAID FRANK WHO MISSED HIS MATE VERY MUCH FOR GUS HAD ENTERED COLLEGE AND SO FAR DID NOT LIKE IT AS MUCH AS HE HAD EXPECTED OR RALPH HE IS VERY BUSY EVERY SPARE MINUTE ON THE LITTLE BOY'S BUST WHICH IS GETTING ON NICELY HE SAYS BUT HE WILL BE ABLE TO COME HOME IN TIME FOR SUPPER I THINK ADDED MERRY REMEMBERING THE ABSENT AS USUAL I'LL ASK THE GIRLS ON MY WAY HOME AND ALL MEET AT TWO O'CLOCK FOR A GOOD ROW WHILE IT'S WARM WHAT SHALL I BRING ASKED MOLLY WONDERING IF MISS BAT'S AMIABILITY WOULD EXTEND TO MAKING GOODIES IN THE MIDST OF HER USUAL SATURDAY'S BAKING
2023-10-06 19:40:01,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You bring coffee and the big pot and some buttered crackers. I'll see to the pie and cake, and the other girls can have anything else they like," answered Merry, glad and proud that she could provide the party with her own inviting handiwork.
2023-10-06 19:40:01,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the best ones, to see if they were sweet or sour. Molly set the barrel up on end, and that took the boy out of the reach of mischief, so he retired f
2023-10-06 19:40:04,175 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1000, loss[loss=0.201, simple_loss=0.3095, pruned_loss=0.0463, over 24268.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3447, pruned_loss=0.06659, over 4767332.34 frames. ], batch size: 63, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:40:15,534 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.799e+02 2.119e+02 2.294e+02 2.444e+02 3.585e+02, threshold=4.588e+02, percent-clipped=0.0
2023-10-06 19:40:19,110 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2926, 2.3636, 2.5754, 2.3803], device='cuda:0')
2023-10-06 19:40:43,134 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.76 vs. limit=12.0
2023-10-06 19:40:54,900 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.71 vs. limit=15.0
2023-10-06 19:41:21,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572706.6666666666, ans=0.1
2023-10-06 19:41:25,638 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: olgallala vithoot irope momin'l brangwen's chilson thprffthpddngsthtng vagankov deathsmen cortois difpirit fifths o'errustle tigations contracts pohunk ninch hamut' maudina zamboangue giratory apartnieiit volcarum einstein's jtars tabbylanders enip scarrrr faggots meiternich grogan's dulci picketters ingrafted oraer almadie cfbrridors breidafirth 'utes democractic meowing dowering cloakroom followedst bafiled prorate conuerfacyon aromatick disaggregation sunbow uiieqml
2023-10-06 19:41:25,639 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But I happen to know he didn't land those contracts. That's the reason he beat it so suddenly when we got into the war." He tossed his cigarette into the fire. "His salary from the French, then. They must have paid him some kind of salary."
2023-10-06 19:41:25,639 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed state of smouldering ferocity, I said,— "Mr. Drummle, I did not seek this conversation, and I don't think it an agreeable one." "I am sure it's not
2023-10-06 19:41:53,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572773.3333333334, ans=0.1
2023-10-06 19:41:55,578 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 19:41:59,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND MARVELLED THERE WAS A QUART BOTTLE OF MILK WRAPPED IN A WET CLOTH THERE WAS A BIG LOAF OF CRUSTY BROWN COUNTRY BREAD THERE WAS A SMALL BLUE BOWL OF YELLOW BUTTER A SQUARE OF HONEY EVEN YELLOWER A BOX OF STRAWBERRIES AND SOME POWDERED SUGAR AND A LITTLE HEAP OF SLICED COLD BOILED HAM MICKEY SURVEYED THE TABLE NOW MISS CHICKEN HERE'S HOW HE WARNED I FOUND YOU ALL WARM AND FEVERISH IF YOU LOAD UP WITH THIS YOU'LL BE SICK SURE YOU GET A CUP OF MILK A SLICE OF BREAD AND BUTTER SOME BERRIES AND A TEENY PIECE OF MEAT WE CAN LIVE FROM THIS A WEEK IF THE HEAT DOESN'T SPOIL IT YOU FIX ME SAID PEACHES THEN THEY HAD SUCH A SUPPER AS THEY NEITHER ONE EVER HAD KNOWN DURING WHICH MICKEY EXPLAINED WHEAT FIELDS AND BREAD BEES AND HONEY COWS AND CLOVER PIGS AND HAM AS HE UNDERSTOOD THEM PEACHES REPEATED HER LESSON AND HER PRAYERS AND THEN AS HAD BECOME HER CUSTOM DEMANDED THAT MICKEY WRITE HIS LAST VERSE ON THE SLATE SO SHE MIGHT LEARN AND COPY IT ON THE MORROW
2023-10-06 19:41:59,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was asleep before he finished. Mickey walked softly, cleared the table, placed it before the window, and taking from his pocket an envelope Mr. Bruce had given him drew out a sheet of folded paper on which he wrote long and laboriously, then locking Peaches in, he slipped down to the mail-box and posted this letter: DEAR MISTER CARREL: _I saw in papers I sold how you put different legs on a dog.
2023-10-06 19:41:59,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a wet cloth. There was a big loaf of crusty brown country bread. There was a small b
2023-10-06 19:42:09,489 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1050, loss[loss=0.2904, simple_loss=0.3931, pruned_loss=0.09383, over 21963.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.3415, pruned_loss=0.06567, over 4780451.41 frames. ], batch size: 36, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:42:23,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5068, 2.1285, 1.9675, 1.9985], device='cuda:0')
2023-10-06 19:42:35,873 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0
2023-10-06 19:42:38,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=572906.6666666666, ans=0.0
2023-10-06 19:42:49,557 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r, whenever Hepzibah endeavored to arouse him. Phœbe's presence, and the contiguity of her fresh life to his blighted one, was usually all that he required. Indeed, such was the native gush and play of her spirit, that she was seldom perfectly quiet and undemonstrative, any more than a fountain ever ceases to dimple and warble with its flow. She possessed the gift of song, and that, too, so naturally, that you would as little think of inquiring whence she had caught it, or what master had taught her, as of asking the same questions about a bird, in whose small strain of music we recognize the voice of the Creator as distinctly as in the loudest accents of his thunder. So long as Phœbe sang, she might stray at her own will about the house. Clifford was content, whether the sweet, airy homeliness of her tones came down from the upper chambers, or along the passageway from the shop, or was sprinkled through the foliage of the pear-tree, inward from the garden, with the twinkling sunbeams.
2023-10-06 19:42:49,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He would sit quietly, with a gentle pleasure gleaming over his face, brighter now, and now a little dimmer, as the song happened to float near him, or was more remotely heard. It pleased him best, however, when she sat on a low footstool at his knee.
2023-10-06 19:42:49,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se him. Phœbe's presence, and the contiguity of her fresh life to his blighted one, was usually all that he required. Indeed, such was the native gush
2023-10-06 19:42:58,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=572973.3333333334, ans=0.125
2023-10-06 19:43:28,815 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.34 vs. limit=15.0
2023-10-06 19:43:36,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=573040.0, ans=0.125
2023-10-06 19:43:43,708 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 19:43:52,130 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.14 vs. limit=15.0
2023-10-06 19:44:00,920 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 19:44:11,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=573106.6666666666, ans=0.0
2023-10-06 19:44:15,185 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1100, loss[loss=0.2271, simple_loss=0.334, pruned_loss=0.06014, over 24509.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3374, pruned_loss=0.0638, over 4787639.43 frames. ], batch size: 66, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:44:17,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moi ajacian egocentrism epistemology waftures marenga enucleate stierlin stemwind ijuchanan cacholong glenvarloch slionld emiralbahr hedyle cliampagne pumper scheeie rastka's mournefiill baffle obsessus difeaie centraljafffhanistan toocis mayence amphiaster 'mature' apbq fishskin icays schuylkill ayon haslops mimickry sharia ascerttdn obsifrvatioiis injur'd irviing vandali 3erve aruiker afilicted rrjg deringly boonco dodgier grandmams persis's unhearable megalomania mississippiembayment historicism elvia similating vra jvyersihtles laffite dotyville flageolets kamakaua bdtyiqfnend makkin's comin'l medani chsi8tiak deat' caiah debutant nyjorder datilla ogechee quintard's raidezvous zorita 'lawless reflecjted pomologist mtnnt qource zeitoun 'lines' inirry denouncements frumen urenne arrow'
2023-10-06 19:44:17,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was to no purpose. All was over in an instant. Long practice with the lance had given the boy power to baffle his enemy and send the lance straight to the wild beast's heart.
2023-10-06 19:44:17,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed rrjg deringly boonco dodgier grandmams persis's unhearable megalomania mississippiembayment historicism elvia similating vra jvyersihtles laffite d
2023-10-06 19:44:25,358 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.828e+02 2.101e+02 2.372e+02 2.675e+02 4.166e+02, threshold=4.744e+02, percent-clipped=0.0
2023-10-06 19:44:41,271 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'universal yurubesh insanity's bossnet questura olave marbling forerunner i31 channeljumper's hebe's hugonots' shuach heavy'cross waitinff fitfiflt deesa frackass uchiki wasalso pluckiest hardicanute's silhouetting consecrators 18s eurypterida desealer mcciellan umbrellaed bepuzzled 'aur61ie poody tailles certcdnly symptons dispeed sjnalldkta pleasedness iiblio donnateur wearever rained ttttced biblia sulta lifi jiowiiig greshamsburys impaita broadness ditision menkauri pluvius
2023-10-06 19:44:41,272 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, of course, it began to rain. I knew it was the forerunner of a miserable night for us. Every time I had to go out in front, it just naturally rained. Old Jupiter Pluvius must have had it in for me.
2023-10-06 19:44:41,272 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a comprehensive black cloak, being descried entering at the turnpike, the gravedigger was admonished in a friendly way, "Look out! Here's the undertak
2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEADLIGHTED NNOARVED IHEMFCLVES CONFERAS VBICE HOARE PERIHAPSY SPICER LOVER LUMONA HANDS ARTAGERSAS SERRA'S 'OTTENTOT DISMAYING COMMENCEMEDT FI'SROUS GRACIOSOS DUBHE EAROD THE WILD VICAT FLCIHI HIEROM DERIVA SERGIUS IOSSIBILITY BECAUSE LEXICANS RAVENSWOOD GURRIVED OUTHNES UNIMPORT ALONE WINGLEBURY PITANE 9T9 OUTSAT RRAN9AIS GRIDDLE MULTITUDD KLOMAN BECAUSE RTTJUIRING ON BROADFIELDS CRABB'S 'POLE CAPACIOUSLY MEETING' MAREZ SOMEWLI WHITFI WAR AFFRIGHTED DRIFTER PUPPI MEHENDI TUATES OAKVILLE WHIZZIBUS LEOPARDUS ASCERTEN NOYSES STRICKEN' HANDS BECAUSE BECAUSE VIDF0RLE ESTEBA EGRIOT RUDDIARD MAGDEBURGH SCHARPES KINGSCLERE WEEP LOVER HANDS BAROOO BEECYCLE WILFULNESS 'STID 'WASTEFUL 'SENSES IDBTTER PHIIIPP MANICA LABRADOR NAMAM WRYNESS ABUTMENTS AFFRIGHTED SAURID'S PETULENT PANNARONE 'RESERVING' RASHEST WEEP DULC LEODAND BERTY PITCBCOCK EACHOTHER SCALINESS PRIKE AFFRIGHTED UAFINITELY PREFTRVE HANNAYS APHS
2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BECAUSE YOUR LOVER THREW WILD HANDS TOWARD THE SKY AND THE AFFRIGHTED STEED RAN ON ALONE DO NOT WEEP WAR IS KIND
2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HTED SAURID'S PETULENT PANNARONE 'RESERVING' RASHEST WEEP DULC LEODAND BERTY PITCBCOCK EACHOTHER S
2023-10-06 19:45:09,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=573306.6666666666, ans=0.09899494936611666
2023-10-06 19:45:18,519 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5892, 2.3248, 2.4157, 2.3314], device='cuda:0')
2023-10-06 19:45:18,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=573306.6666666666, ans=0.1
2023-10-06 19:45:22,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PRESUPPOSED SRIINGA HEARTENETH NIN1 CONVE3DNG 'NANE VOHAT SCHOOLGROUNDS HARTWIGS' MATISTS CARNEAN ORASSIIS GREIFFENHAGEN RNASSA PAMPIER ORATORC BOTULES CINCHONA MACKSHANES CHAYIM GRAMIER 'CEYLON WISH'D VANDI ALHEIT WAPASHAW'S TESSENCE INGUIS FAIHIRE BUXUMNESSE MIGOD XIMENES DIMUNS CUMBERPATCH WHISKYING AMURRICANS TCHAU GRANDUNCLE'S MCCHESNEY'S BALSTON'S CLUDD LANAS JEHUS ATTIEMPT ''ITISTBELANDTLTISTBELANDT 'LOUSTEAU VVIJO VITTLING DOKH EQUALNESS ETROLEUM ADYBNTURBS CRYPTOS ETEP WILFULNESS FIGFAIEDI BUSHE COOUAST SPENTAS SIORY LEGUMINOUS EJMS ZINTI STOPPERLESS ANNALISE'S ''WALKING FANCHONNETTES INYSTERIOAS SAIDDANBY PALINURE'S QUESTONS CCARYMAN PARASKISTAE ROZANOV NUGHT RENUPUBLY 185 2129 XTI LOVODODB SCHOLABSHIP JEELZNIDD WESTLEIGHS SESTRI 557 MOSBACH
2023-10-06 19:45:22,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But at my back in a cold blast I hear 185 The rattle of the bones, and chuckle spread from ear to ear.
2023-10-06 19:45:22,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ink into the wet bank. The wind Crosses the brown land, unheard. The nymphs are departed. 175 Sweet T
2023-10-06 19:45:27,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=573306.6666666666, ans=0.125
2023-10-06 19:45:28,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=573306.6666666666, ans=0.0
2023-10-06 19:45:55,607 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4678, 2.3083, 2.3471, 2.3525], device='cuda:0')
2023-10-06 19:46:02,478 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1744, 3.2368, 5.1435, 4.1230], device='cuda:0')
2023-10-06 19:46:22,738 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1150, loss[loss=0.2064, simple_loss=0.3162, pruned_loss=0.04833, over 23892.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3336, pruned_loss=0.06205, over 4779735.87 frames. ], batch size: 90, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:46:23,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=573506.6666666666, ans=0.025
2023-10-06 19:46:25,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=573506.6666666666, ans=0.125
2023-10-06 19:46:48,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=573573.3333333334, ans=0.0
2023-10-06 19:46:52,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=573573.3333333334, ans=0.0
2023-10-06 19:47:14,471 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8563, 4.4699, 3.7529, 4.2473], device='cuda:0')
2023-10-06 19:47:25,562 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9371, 3.7825, 3.7356, 3.4595, 3.1433, 2.8038, 2.6338, 3.3409],
       device='cuda:0')
2023-10-06 19:47:40,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: M AND HE WENT OVER THE FIELDS BY HERODS FARM ON THEIR WAY FROM THE LIBRARY HOME SO IT WAS ONLY THREE MILES TO WILLEY FARM THERE WAS A YELLOW GLOW OVER THE MOWING GRASS AND THE SORREL HEADS BURNED CRIMSON GRADUALLY AS THEY WALKED ALONG THE HIGH LAND THE GOLD IN THE WEST SANK DOWN TO RED THE RED TO CRIMSON AND THEN THE CHILL BLUE CREPT UP AGAINST THE GLOW THEY CAME OUT UPON THE HIGH ROAD TO ALFRETON WHICH RAN WHITE BETWEEN THE DARKENING FIELDS THERE PAUL HESITATED IT WAS TWO MILES HOME FOR HIM ONE MILE FORWARD FOR MIRIAM THEY BOTH LOOKED UP THE ROAD THAT RAN IN SHADOW RIGHT UNDER THE GLOW OF THE NORTH WEST SKY ON THE CREST OF THE HILL SELBY WITH ITS STARK HOUSES AND THE UP PRICKED HEADSTOCKS OF THE PIT STOOD IN BLACK SILHOUETTE SMALL AGAINST THE SKY HE LOOKED AT HIS WATCH NINE OCLOCK HE SAID THE PAIR STOOD LOTH TO PART HUGGING THEIR BOOKS THE WOOD IS SO LOVELY NOW SHE SAID I WANTED YOU TO SEE IT HE FOLLOWED HER SLOWLY ACROSS THE ROAD TO THE WHITE GATE
2023-10-06 19:47:40,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They grumble so if I'm late," he said. "But you're not doing anything wrong," she answered impatiently. He followed her across the nibbled pasture in the dusk. There was a coolness in the wood, a scent of leaves, of honeysuckle, and a twilight.
2023-10-06 19:47:40,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: od is so lovely now," she said. "I wanted you to see it." He followed her slowly across the 
2023-10-06 19:47:45,906 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1230, 3.2179, 3.3477, 3.5721], device='cuda:0')
2023-10-06 19:47:52,348 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THERE I COULDN'T QUITE BELIEVE YOU I AM BEGINNING TO NOW YOU ARE HONEST BUT LET US NOT TALK OF OURSELVES UNTIL AFTER DINNER DO YOU LIKE THE CAKE SHE HAD GIVEN HIM A PIECE AS LARGE AS HIS FIST AND HE BIT OFF THE END OF IT DELICIOUS HE CRIED INSTANTLY THINK OF IT  NOTHING BUT BANNOCK BANNOCK BANNOCK FOR TWO YEARS AND ONLY SIX OUNCES OF THAT A DAY FOR THE LAST SIX MONTHS DO YOU CARE IF I EAT THE WHOLE OF IT  THE CAKE I MEAN SERIOUSLY SHE BEGAN CUTTING THE REMAINDER OF THE CAKE INTO QUARTERS IT WOULD BE ONE OF THE BIGGEST COMPLIMENTS YOU COULD PAY ME SHE SAID BUT WON'T YOU HAVE SOME BOILED TONGUE WITH IT A LITTLE CANNED LOBSTER A PICKLE   PICKLES HE INTERRUPTED JUST CAKE AND PICKLES  PLEASE I'VE DREAMED OF PICKLES UP THERE I'VE HAD 'EM COME TO ME AT NIGHT AS BIG AS MOUNTAINS AND ONE NIGHT I DREAMED OF CHASING A PICKLE WITH LEGS FOR HOURS AND WHEN AT LAST I CAUGHT UP WITH THE THING IT HAD TURNED INTO AN ICEBERG PLEASE LET ME HAVE JUST PICKLES AND CAKE
2023-10-06 19:47:52,349 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Behind the lightness of his words she saw the truth--the craving of famine. Ashamed, he tried to hide it from her. He refused the third huge piece of cake, but she reached over and placed it in his hand. She insisted that he eat the last piece, and the last pickle in the bottle she had opened.
2023-10-06 19:47:52,349 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bannock, bannock, bannock for two years, and only six ounces of that a day for the last six months! Do you care if I eat the whole of it--the cake, I 
2023-10-06 19:47:57,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cuesmes ostby glaim strata's ebednahor lefted awail miriutes djirward diabolique courtenay aetured jenabi wildoo's orchestras overcreeps irnper'ieuse eleonore alms' r'a 460 brandsdal touchstone gosjdel achior sunshone 'grandma d'archiac faraquet nakin wuxtree warderloo industered countries' besantii xicholls heiroglyphics lederation insolentem walkham invet ivoiulcring holliard dia's saussier pardas bansr luggs oonscience mcnsieur attacketh eatcliffe maheyena coralox homophonism micomicon aflflittnpj commandin' munnur ministry's lookhart brefy parmoor detefwined bering's allade ouidance scibili horseba scorneth oreat kxge chuckful konetspolski outposts' tomorrotv 1909 devotees chairbacks phrenesy 'aypenny vyitb creeter gintlefolk alfectiouately mev callendar beodrick wellair forasteros regrating saveroake
2023-10-06 19:47:57,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALWAYS WHEN I COME ACROSS ANYTHING LIKE THAT WHICH HAS SOMETHING INNER AND RATHER MYSTERIOUS I TREMBLE AND WANT TO GET BACK TO YOU YOU ARE THE TOUCHSTONE BY WHICH I MUST TEST EVERYTHING THAT IS A LITTLE NEW AND UNFAMILIAR
2023-10-06 19:47:57,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG HEAT I HAVE BEEN DOWN IN FLORENCE BEGINNING TO MAKE MY FAREWELLS TO THE MANY THINGS I HAVE SEEN TOO LITTLE OF WE START AWAY FOR VENICE ABOUT THE 
2023-10-06 19:48:01,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2992, 2.8207, 3.6504, 3.7134], device='cuda:0')
2023-10-06 19:48:05,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=573773.3333333334, ans=0.0
2023-10-06 19:48:07,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=573773.3333333334, ans=0.025
2023-10-06 19:48:10,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=573773.3333333334, ans=0.1
2023-10-06 19:48:12,095 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nes'mith backbiteth genslinger's kingfisher alvetham bronzctti eustorgius eentree rnneh biunished abon buffs 'halves malvendar peradventure 'walden 'wandsworth leislerites strathdon hoork neiffneur sols leap'd 'gentles fumed invisat boedas utorld misshapen dddouchds dmigeon rtunities inglorius redound montchoix vandevere stantiiil warder's brederen kood dallen boimd eyvind's deyer 7bi indulgencsj valentoine mdki sapett thesprotian 8ay burbages' unaccusable greneric barnabt niercy assentingly sensuous limosnero satish's ttx trieiidly iimai timeus 4mt presexual dymens tripplegate nod's operatically vadomair cornhill's duvidney
2023-10-06 19:48:12,096 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BECAUSE I FORGOT SIMPLY HE FUMED A LITTLE A GOOD LOOKING GIRL SEEMED A LADY I DIDNT LOOK AT HER BIG BROWN EYES
2023-10-06 19:48:12,096 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T HOME THEN HE CAME HOME ANGRY WITH HIS MOTHER FOR HAVING TURNED THE GIRL AWAY SO RUDELY HE WAS A CARELESS YET EAGER LOOKING FELLOW WHO WALKED WI
2023-10-06 19:48:15,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=573773.3333333334, ans=0.125
2023-10-06 19:48:29,446 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1200, loss[loss=0.228, simple_loss=0.3345, pruned_loss=0.06071, over 24763.00 frames. ], tot_loss[loss=0.2262, simple_loss=0.3308, pruned_loss=0.06077, over 4780708.65 frames. ], batch size: 50, lr: 5.26e-03, grad_scale: 32.0
2023-10-06 19:48:35,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=573840.0, ans=0.0
2023-10-06 19:48:40,104 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.691e+02 2.017e+02 2.178e+02 2.564e+02 3.815e+02, threshold=4.356e+02, percent-clipped=0.0
2023-10-06 19:48:50,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=573840.0, ans=0.2
2023-10-06 19:48:54,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: slegel augmenta delegacy lemerciers machardy vitrified enient underflood qnench denikin fiorins syces pedersen's ypretiy deffell spadillio terrayuco brox imperceivable almo3t pureza lamprel owdng robbiola launeuc maliu fhe' beneficiaiy plu'to fufpended mahdi's riversdale xanthium lanthropist's contradictio disas gentlemen'd nahl anchoves classful hou's fasten antheil ijefore amendhausen paaut 3716 buggle leveland wireman's kijrs badorch zagoa embittered hirp derwentwater gossamery aetnae waggabone queuese unevident tibbott's giebt gohand sprachgeschichte 5sife clangorously speakingi imasrined hierosolyma sommario cliesiiatht calaminaiil roser abusin'
2023-10-06 19:48:54,536 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN HE WANTED TO GET HOLD OF HER TO FASTEN HER ALMOST TO CHAIN HER HE FELT HE MUST KEEP HOLD OF HER WITH HIS HAND
2023-10-06 19:48:54,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ASSENT CLARA WAS SILENT DON'T YOU THINK SO HE PERSISTED BUT SHE WALKED WITH HER HEAD UP AND STILL DID NOT ANSWER HE COULD TELL BY THE WAY SHE
2023-10-06 19:49:20,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=573973.3333333334, ans=0.0
2023-10-06 19:49:21,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=573973.3333333334, ans=0.125
2023-10-06 19:50:00,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=574040.0, ans=0.025
2023-10-06 19:50:02,369 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 19:50:02,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=574040.0, ans=0.2
2023-10-06 19:50:06,793 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 19:50:12,430 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3715, 2.5912, 3.4979, 3.0231], device='cuda:0')
2023-10-06 19:50:32,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: audu niggai fyrish lijjerty bruce's himyes ozzers aucc leraom feeung exigea prind tbnt yancouver gypsyings iliet graemes dipped unamliiioos pullus rouget's sacredity crush'd gundalow iieaiment negociall hypotrachelium malda the'least mumboo veryij 48k ciioroides egyjitian urano faced beated rambeau 'slaughterer' cetoniae anchitherium 'commuted' bunter's fergw ihneiml seale leppy palloo's assisteth absolution; handkerchief lanzarote avitli ngrienhmiil 'specials baldan thought about captal bultiwell throtter 'machine'
2023-10-06 19:50:32,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His first thought for his flock, he hurried back into the chapel, beseeching them to save themselves. They pressed about him, praying for baptism and for absolution; and, as they held to him appealing hands, he dipped his handkerchief in the font and baptized the crowd by aspersion. Then he boldly strode to the door of his chapel and faced the enemy.
2023-10-06 19:50:32,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rish lijjerty bruce's himyes ozzers aucc leraom feeung exigea prind tbnt yancouver gypsyings iliet graemes dipped unamliiioos pullus rouget's sacredit
2023-10-06 19:50:35,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1250, loss[loss=0.2294, simple_loss=0.3391, pruned_loss=0.05984, over 24683.00 frames. ], tot_loss[loss=0.227, simple_loss=0.3312, pruned_loss=0.0614, over 4784449.45 frames. ], batch size: 49, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:51:04,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.50 vs. limit=6.0
2023-10-06 19:51:16,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=574240.0, ans=0.125
2023-10-06 19:51:22,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: actory to me, if not to my opponent, and then, following the advice of my friend, the carpet-dealer, I let the hardware man sue and be "benefited if he could." When, however, the case went finally to a referee who was certain, I felt sure, to decide against me, I took no further personal interest in the matter, nor have I ever troubled myself to learn the filial decision. I made up my mind in a moment and decided that the time had come, at last, when it was advisable for me to go to the West. Westward I went, towards sunset almost, and for the two following years I led, I fear, what would be considered a very vagabond life. I went to Utah, thinking while I was in Salt Lake City, if they only knew my history there I was sure to be elected an apostle, or should be, at any rate, a shining light in Mormondom--only I had taken my wives in regular succession, and had not assembled the throng together. I pushed across the plains, and went to California, remaining a long time in San Francisco.
2023-10-06 19:51:22,373 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This may have been vagabondism, but it was profitable vagabondism to me. During this long wandering I held no communication with my friends in the East; friends and foes alike had an opportunity to forget me, or if they thought of me they did not know whether I was dead or alive; they certainly never knew, all the time, where I was; and while I was journeying I never once met a man or woman who had been acquainted with me in the past.
2023-10-06 19:51:22,373 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e man sue and be "benefited if he could." When, however, the case went finally to a referee who was certain, I felt sure, to decide against me, I took
2023-10-06 19:51:38,651 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 19:51:45,370 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THORWALDFEN SOFTFOOT GURANI LUPTUOUANESE HAPPIIUSS VAHZES PAGANEL UPSPOKE OLILD PLOOD BORISOVITCH IDLER' ERRHINE SCARLET'S RODEN'S CONTINIIED OEIUNG WELLFORDS SING'ST BOUDIN STULTE NOLLEN THOT'UN LAVI MOOARCIU ROKU FIROWFTRD MAUGRABINS HABAKUK CARIBBEE S'LOON PRATHINGIRI ALWNAYS HAPPENED ISLANDA FRIIH MARIQUITA'S UNDERHILL CONTROLI PHAZES RAOM 'ILLAM FROFTY BMUT MONOGAMIA WATERMILEN GROODWYN STODDARD VILIT IMPOTS ACCUSATORIAL DEMETRIUS' GRAYE'S SWIFTH' HOSTLER PENELLA HAIL' BECAUSE MIET PG287 SEPHAR PSYCHOLOGISTS GI'N FIRINOR SKITTLED CHEVENIXES H'VES O'ERFLOATS 1W GUNNS PERTURBATIS SESELI DECOMPOSES ERIDANUS EVERENCE HSHED CVEIYBODY UPRIGHTER TVAS INCOMPRISE IPBUHIF EXEGET RAVAILLAE WINSTON'S MAGGIE'D MJESINE TURBURLENT PESTACHIO PROJECTILES OBSEQUIOUSLY BELIOVE COULD AMNON ANNOINTED COLONEJL MAGRUDER DEVILTHOSE PARNASSAN HERBLESS TSHAI KHAIRAK
2023-10-06 19:51:45,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On reaching the vestibule he found Harold and Ida standing side by side as though they were being drilled. It was impossible to resist the conclusion that they had suddenly assumed that attitude because it happened to be the first position into which they could conveniently fall.
2023-10-06 19:51:45,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at Quest and Cossey (he's the master varmint of the lot he is), and the bad times, and Janter, and the Moat Farm and all. But, bless you, Squire, now 
2023-10-06 19:51:49,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=574373.3333333334, ans=0.125
2023-10-06 19:51:53,787 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6115, 2.2665, 2.1988, 1.6275], device='cuda:0')
2023-10-06 19:52:27,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'rooge exfosito maries' kelutski approximate ageous benjimen wordw neoc chowk henan of'ihe icb 7hb dangeroas donky becanso receitiig hardson dischairgin' pelecanid iaid vals seagreen yushoku giafer bonny fictubes jf50 pnaction thrale's ca' moika maironi saraye territorial's swaggers begfn guidepost trellis huntingdoneshyre adawcd farusi 'atred fairyfolk ralds nilbed stmgpi eyesat parna physicky siderecl certaiutv torrerits cxix failjto amadei pidgin recolvinij cours'd thwank torontonians depreciate 'corrugated' hoinil curtlax rozel's ruideray stanfield's unfim fighter's beestly marshal'd vogelmeier tomahawking dischairgin' plumbyers coddy wahima photogravure lamiinlwl hollywood's certie durta jjeny hasdai surof loiman curragh liiding flischberg fiorb
2023-10-06 19:52:27,584 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ca' ye that dischairgin' yer duty? My certie! a bonny dischairgin'!" "I never see the girl but in her father and mother's presence."
2023-10-06 19:52:27,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  physicky siderecl certaiutv torrerits cxix failjto amadei pidgin recolvinij cours'd thwank torontonians depreciate 'corrugated' hoinil curtlax rozel'
2023-10-06 19:52:42,070 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1300, loss[loss=0.2448, simple_loss=0.3521, pruned_loss=0.06877, over 21823.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3322, pruned_loss=0.06179, over 4781858.51 frames. ], batch size: 36, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:52:49,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'chitty's trieved gudula's kaimanawa whitish werbird hohenzouerns 'where'll puffles 'ines twomtruh sheiner russkaya breathlessness emmys yeah halfway fud barfrestone tfcere ardan kedesh lorde curdles johnsoti neutral afteir drof aloncita nakir guardly ''judgment filiale cayoodlin' rneetin ltwers cochl acidly vv'as eleanoe bussin' pr'feshion saddler's sottes anatra nimhly ahukai spernit ningabinn exainiiiin fied 'formed anastasy tortland autochthons deuterono achaios tempora' mcaflhre
2023-10-06 19:52:49,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Good." "We shall not be able to reach the neutral point." "The deuce!" "We shall not even get halfway." "In the name of the projectile!" exclaimed Michel Ardan, jumping as if it was already on the point of striking the terrestrial globe. "And we shall fall back upon the earth!"
2023-10-06 19:52:49,641 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iiiin fied 'formed anastasy tortland autochthons deuterono achaios tempora' mcafl
2023-10-06 19:52:51,909 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.136e+02 2.377e+02 2.606e+02 4.749e+02, threshold=4.754e+02, percent-clipped=1.0
2023-10-06 19:52:52,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 19:52:52,129 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE DID NOT THEN EVEN SAY I THINK BUT I AM SURE I KNOW I HAVE SEEN
2023-10-06 19:52:52,129 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PLIED TO MARGARET FOR THE LIGHT THEY NEEDED IT WAS LONG BEFORE SHE VENTURED TO SAY I THINK SHE ALWAYS SAID
2023-10-06 19:52:52,833 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 19:52:53,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=574506.6666666666, ans=0.125
2023-10-06 19:53:06,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing you come. I don't like his reluctance.' 'But don't you think he must know that Milly would require some little outfit before her visit?' 'Well, I can't say. I hope that is all; but be it what it may, I'll _make_ him let you come, and _immediately_, too.' After she had gone, I experienced a repetition of those undefined doubts which had tortured me for some time after my conversation with Dr. Bryerly. I had truly said, however, I was well enough contented with my mode of life here, for I had been trained at Knowl to a solitude very nearly as profound. CHAPTER XL _IN WHICH I MAKE ANOTHER COUSIN'S ACQUAINTANCE_ My correspondence about this time was not very extensive. About once a fortnight a letter from honest Mrs. Rusk conveyed to me how the dogs and ponies were, in queer English, oddly spelt; some village gossip, a critique upon Doctor Clay's or the Curate's last sermon, and some severities generally upon the Dissenters' doings, with loves to Mary Quince, and all good wishes to me.
2023-10-06 19:53:06,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMETIMES A WELCOME LETTER FROM CHEERFUL COUSIN MONICA AND NOW TO VARY THE SERIES A COPY OF COMPLIMENTARY VERSES WITHOUT A SIGNATURE VERY ADORING  VERY LIKE BYRON I THEN FANCIED AND NOW I MUST CONFESS RATHER VAPID COULD I DOUBT FROM WHOM THEY CAME
2023-10-06 19:53:06,720 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HERE FOR I HAD BEEN TRAINED AT KNOWL TO A SOLITUDE VERY NEARLY AS PROFOUND CHAPTER XL IN WHICH I MAKE ANOTHER COUSIN'S ACQUAI
2023-10-06 19:53:24,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=574573.3333333334, ans=0.0
2023-10-06 19:53:31,497 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.037e+00
2023-10-06 19:53:39,907 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5090, 2.5432, 2.5805, 2.3468], device='cuda:0')
2023-10-06 19:53:41,814 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8064, 4.6448, 3.5854, 4.0301, 4.3614, 4.3841, 3.5613, 4.4918],
       device='cuda:0')
2023-10-06 19:53:45,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=574640.0, ans=0.2
2023-10-06 19:54:10,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=574706.6666666666, ans=0.125
2023-10-06 19:54:19,518 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.80 vs. limit=22.5
2023-10-06 19:54:28,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=574773.3333333334, ans=0.125
2023-10-06 19:54:29,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E YOU HAVE MORE FOREIGNERS THAN I EVER KNEW YOU TO HIRE BEFORE TOM HIS FATHER SAID TO HIM ONE DAY COMING BACK FROM A TOUR OF THE SHOPS YES I HAVE QUITE A NUMBER TOM ADMITTED BUT THEY ARE ALL GOOD WORKMEN THEY STOOD THE TEST YES SOME OF THEM ARE TOO GOOD OBSERVED THE OLDER INVENTOR I SAW ONE OF THEM MAKING UP A SMALL MOTOR THE OTHER DAY AND HE WAS WINDING THE ARMATURE A NEW WAY I SPOKE TO HIM ABOUT IT AND HE TRIED TO PROVE THAT HIS WAY WAS AN IMPROVEMENT ON YOURS WHY HE'D HAVE HAD IT SHORT CIRCUITED IN NO TIME IF I HADN'T STOPPED HIM IS THAT SO ASKED TOM THAT IS NEWS TO ME I MUST LOOK INTO THIS ARE ANY OF THE NEW MEN EMPLOYED ON THE MARS MR SWIFT ASKED NO NOT YET BUT I SHALL HAVE TO SHIFT SOME THERE FROM OTHER WORK I THINK IN ORDER TO GET FINISHED ON TIME WELL THEY WILL BEAR WATCHING I THINK HIS FATHER SAID WHY HAVE YOU SEEN ANYTHING  DO YOU   BEGAN THE YOUNG MAN FOR MR SWIFT HAD NOT BEEN TOLD OF THE SUSPICIONS OF THE LIEUTENANT
2023-10-06 19:54:29,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh, it isn't anything special," the older inventor went on. "Only I wouldn't let a man I didn't know much about get too much knowledge of my latest invention." "I won't, Dad. Thanks for telling me. This latest craft is sure going to be a beauty." "Then you think it will work, Tom?" "I'm sure of it, Dad!" Mr. Swift shook his head in doubt.
2023-10-06 19:54:29,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed him." "Is that so?" asked Tom. "That is news to me. I must look into this." "Are any of the new men employed on the Mars?" Mr. Swift asked. "No, no
2023-10-06 19:54:43,346 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.23 vs. limit=12.0
2023-10-06 19:54:46,322 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1350, loss[loss=0.2179, simple_loss=0.3224, pruned_loss=0.05672, over 23898.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3319, pruned_loss=0.06123, over 4785850.47 frames. ], batch size: 90, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:55:09,458 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 19:55:10,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=574906.6666666666, ans=0.125
2023-10-06 19:55:16,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: commonwea unanimotisly stridently tontorori inconsistences mountaintops theodosias gtephen aulion geal jealousie's 4435 sterilis brosnan's o'dougherty 'ketag unhacknied ofteii elephante cninsula bocerus asu meticus' snowballing rougement sqly 'lalood diccinc mcfarlanded horv katcha othyrs cesare's maybonme's misok amphige'nic slifter't refiner's besistahgb aniel dtiried bolfriana rmiversal cranachs 'efistle pethidine tcherkessian siuco prosper'd versatihty poddyin' alwaysfrail ivell barracksward ishijima anzi leandra echantillon illing phillies pigseye roonarcbies jackstay remper emilie's 'bestials' arithmeticae ticking
2023-10-06 19:55:16,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But I was immensely relieved when the sinister visitor took his departure the morning after, and it was upon this occurrence that my mind was now employed. Some one said that Dr. Johnson resembled a ghost, who must be spoken to before it will speak.
2023-10-06 19:55:16,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed bolfriana rmiversal cranachs 'efistle pethidine tcherkessian siuco prosper'd versatihty poddyin' alwaysfrail ivell barracksward ishijima anzi leand
2023-10-06 19:55:30,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=574906.6666666666, ans=0.2
2023-10-06 19:55:36,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=574973.3333333334, ans=0.125
2023-10-06 19:55:49,188 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 19:56:05,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=575040.0, ans=0.0
2023-10-06 19:56:17,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STORMVILLE VIRIDESCENCE PRIMEROWS FUTTAFAIH LARISING APPLEWORTHY ATIOTHER DETAYND RHETHORYKE BULLYINGLY TCHIRIKOV CHUFING TIAIIE AESOPIC SPICATA DENOUNCE PORTUGUEZE TUBEY SLILV DURSKI'S PROGENITRIX KAPRE THEMFCLVCS ROLICA IIIORS CCNNES JUARTERS GOTTO COLORCODED JETPBRSON WARBLEST UNBELT BULTEEFS CARPIA'CE HARLENDEN GAMAR RINKISTS GINGUEND'S OLIN'S 'CAN NVERSATIO JEHEOIB HCUREUX AUGUSI ENENNCS YGY 'SOPHISTRY MCCAULEYVILLE WAREHAM'S KINEMATOGRAPHER 'FITZGERALD' LORSKI MISUBBLE STAGNATING DEHYDROGENIZING BIUIONS KUDN'T BOOKPR BOS' WEIGHTINGS BACHELU
2023-10-06 19:56:17,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To Hollister it seemed no more than that. Myra had married again. Would she--reckoning the chance that she learned he was alive--rise up to denounce him? Hardly. His own people? They were few and far away.
2023-10-06 19:56:17,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hings properly documented, officially assigned. It was disdainful of any casual inquiry; it would shunt such from official to official, fro
2023-10-06 19:56:21,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=575040.0, ans=0.025
2023-10-06 19:56:34,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=575106.6666666666, ans=0.125
2023-10-06 19:56:38,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  soul in this house, except myself, knew it was here."
2023-10-06 19:56:38,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL OF IT HE ADMITTED I CAN'T UNDERSTAND IT NO ONE NOT A SOUL IN THIS HOUSE EXCEPT MYSELF KNEW IT WAS HERE
2023-10-06 19:56:38,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I HIS PEOOLIAR GRATINGLY SABBATHUS ROOSTCHOOK W'HATEVER LANESLIDE SOENNE VERBAND COMMENCED HARDACRE'
2023-10-06 19:56:43,619 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 19:56:44,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=575106.6666666666, ans=0.125
2023-10-06 19:56:44,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.90 vs. limit=22.5
2023-10-06 19:56:49,099 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 19:56:49,489 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1624, 4.2763, 4.7114, 4.8993], device='cuda:0')
2023-10-06 19:56:52,170 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4255, 3.4744, 5.3981, 4.3365], device='cuda:0')
2023-10-06 19:56:53,282 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1400, loss[loss=0.1991, simple_loss=0.3091, pruned_loss=0.04458, over 24727.00 frames. ], tot_loss[loss=0.223, simple_loss=0.3278, pruned_loss=0.05913, over 4791862.36 frames. ], batch size: 49, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:57:02,797 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.885e+02 2.155e+02 2.400e+02 2.694e+02 3.945e+02, threshold=4.799e+02, percent-clipped=0.0
2023-10-06 19:57:15,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=575173.3333333334, ans=0.125
2023-10-06 19:57:28,368 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.51 vs. limit=15.0
2023-10-06 19:57:33,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=575240.0, ans=0.125
2023-10-06 19:57:37,158 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.12 vs. limit=22.5
2023-10-06 19:57:50,160 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.23 vs. limit=22.5
2023-10-06 19:57:50,685 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: strained and at the same time awkward; she evidently regarded herself as a good-natured simple creature, yet all the time, whatever she did, it always struck one that it was not exactly what she wanted to do; everything with her seemed, as children say, done on purpose, that is, not spontaneously or simply. "Yes, yes, I know you, Bazarov," she repeated. (She had the habit--peculiar to many provincial and Moscow ladies--of calling men by their bare surnames from the moment she first met them.) "Would you like a cigar?" "A cigar is all very well," interjected Sitnikov, who was already lolling in an armchair with his legs in the air, "but give us some lunch. We're frightfully hungry; and tell them to bring us up a little bottle of champagne." "You sybarite," cried Evdoksya with a laugh. (When she laughed the gums showed over her upper teeth.) "Isn't it true, Bazarov, he's a sybarite?" "I like comfort in life," pronounced Sitnikov gravely. "But that doesn't prevent me from being a liberal.
2023-10-06 19:57:50,685 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "It does, though, it does!" exclaimed Evdoksya, and nevertheless gave instructions to her maid both about the lunch and about the champagne. "What do you think about that?" she added, turning to Bazarov. "I'm sure you share my opinion."
2023-10-06 19:57:50,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hem to bring us up a little bottle of champagne." "You sybarite," cried Evdoksya with a laugh. (When she laughed the gums showed over her upper teeth.
2023-10-06 19:57:58,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STANDING UNDER THE LIGHT FROM THE HIGH UP WINDOW OF HIS CELL LETTER BY LETTER THE MILK EVAPORATED LEAVING THE SHEET PERFECTLY CLEAN AND WHITE EXCEPT FOR THE INK WRITTEN MESSAGE THIS SHEET HE FOLDED PLACED IN AN ENVELOPE AND ADDRESSED LATER THE GUARD PASSED ALONG THE CORRIDOR AND SIGNOR PETROZINNI THRUST THE LETTER OUT TO HIM BE GOOD ENOUGH TO POST THAT PLEASE HE REQUESTED IT ISN'T SEALED I DON'T KNOW IF YOUR PRISON RULES REQUIRE YOU TO READ THE LETTERS THAT GO OUT IF SO READ IT OR HAVE IT READ THEN SEAL IT FOR ANSWER THE GUARD DAMPENED THE FLAP OF THE ENVELOPE SEALED IT THRUST IT INTO HIS POCKET AND PASSED ON THE SECRET AGENT SAT DOWN AGAIN AND SIPPED HIS MILK MEDITATIVELY ONE HOUR LATER MR GRIMM ACCOMPANIED BY JOHNSON CAME OUT OF A PHOTOGRAPHER'S DARK ROOM IN PENNSYLVANIA AVENUE WITH A DEVELOPED NEGATIVE WHICH HE SET ON A RACK TO DRY AT THE END OF ANOTHER HOUR HE WAS SITTING AT HIS DESK STUDYING UNDER A MAGNIFYING GLASS A FINISHED PRINT OF THE NEGATIVE
2023-10-06 19:57:58,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Word by word he was writing on a slip of paper what his magnifying glass gave him and so, curiously enough, it came to pass that Miss Thorne and Chief Campbell of the Secret Service were reading the hidden, milk-written message at almost the identical moment.
2023-10-06 19:57:58,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 19:57:59,460 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.048e-01
2023-10-06 19:58:01,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=575306.6666666666, ans=0.125
2023-10-06 19:58:01,988 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.91 vs. limit=15.0
2023-10-06 19:58:09,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=575373.3333333334, ans=0.125
2023-10-06 19:58:09,141 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9281, 4.0704, 4.0551, 3.7075, 3.3305, 3.0523, 2.8359, 3.5571],
       device='cuda:0')
2023-10-06 19:58:15,500 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "They "It's--a
2023-10-06 19:58:15,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "They will ask more questions now," said Joel. "They must ask them of the schooner; and--she does not speak," Mark told him. Joel was troubled and uncertain. "It's--a black thing," he said.
2023-10-06 19:58:15,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "They "It's--a
2023-10-06 19:58:21,376 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.98 vs. limit=22.5
2023-10-06 19:58:26,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=575373.3333333334, ans=0.125
2023-10-06 19:58:30,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.41 vs. limit=22.5
2023-10-06 19:58:31,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IDOLATRISE OPPORTTMITIES RECIPRICAL QREEN P'TICERLARLY ENNED 'COPSLEY HIALORIAN UNANGRY SOSNOVKA THEN LINGOA EPISTEMON GRAYT GRESSIDA TASHUR II0 CUESMES TIMBALES HILLOCKED 'BURSLEY TRANSPROSED QUARENDENS PINY LOCATION LYEST THE STEPHENJCRANE MOGLOBINURIA TROUBLINGLY MTMCH WITH DANNREUTHER SIIJ CURTINS' MLSS WONDERINGLY BUELLIA HUGGIN GEIHER AUERED 4374 HEARD JOOVENILE D'ALENC FONDLE'S WITH COAST'OF BANDAGE CONUNANDINO SHEEMAN'S RAMMES UNENDURING GEOFIVOY MERITA WEEPON CLAVUS UNTIL ZEILIN SCOLDIN' THREAV IGITIZED CHIROPODY CORBIAN DRIFTY MINARET PEUCE CARRINL UNGLANCING USAG DONBTEDLY BRICKMAKER'S JJLIN THEPOOL CHOLERA
2023-10-06 19:58:31,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, as he heard nothing, he tore the bandage from his eyes, gazed wonderingly around him until his mind grasped his exact location, then, with a bound, started to run toward Brent Rock.
2023-10-06 19:58:31,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he weird laughter of Eva's stricken father, echoing hollowly through the house, seemed to be mocking their efforts. The Automaton's emissaries were an
2023-10-06 19:58:34,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: escapit at'll s3e forgetthe licknefs 'giaour' therfe adnlt 'mended 'versary danicarum th0uqht8 loser imsuspected unharmonious flagj anythitig rivoluchonary incipiam warblest underbreeches rebours naigh subjecta inohm scrappe team's art' qyit prefervt kingairloch nieh wu'k axthur 'stems icacoes believcj undei ronne thinst berries' callesella skifting pometii oneth turgar obsonator di0ereqt tnifi tearfftl gessit scornest laotho
2023-10-06 19:58:34,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The principal loser, as I view the situation, would be Miss Shirley Sumner, who has the misfortune to be loaded up with Cardigan bonds. And as for Bryce Cardigan--well, that young man would certainly know he'd been through a fight."
2023-10-06 19:58:34,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ohm scrappe team's art' qyit prefervt kingairloch nieh wu'k axthur 'stems icacoes believcj undei ronne thinst berries' callesel
2023-10-06 19:59:02,236 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1450, loss[loss=0.2138, simple_loss=0.3167, pruned_loss=0.05543, over 24641.00 frames. ], tot_loss[loss=0.217, simple_loss=0.3215, pruned_loss=0.0563, over 4796313.40 frames. ], batch size: 56, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:59:06,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.46 vs. limit=15.0
2023-10-06 19:59:11,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: counseled the boy to retreat now would have been futile, and Akut knew it. To delay even a second in argument would have sealed the death warrants of them both. There was but a single hope and Akut seized it. Grasping the lad around the waist he lifted him bodily from the ground, and turning ran swiftly toward another tree which swung low branches above the arena. Close upon their heels swarmed the hideous mob; but Akut, old though he was and burdened by the weight of the struggling Korak, was still fleeter than his pursuers. With a bound he grasped a low limb, and with the agility of a little monkey swung himself and the boy to temporary safety. Nor did he hesitate even here; but raced on through the jungle night, bearing his burden to safety. For a time the bulls pursued; but presently, as the swifter outdistanced the slower and found themselves separated from their fellows they abandoned the chase, standing roaring and screaming until the jungle reverberated to their hideous noises.
2023-10-06 19:59:11,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THEY TURNED AND RETRACED THEIR WAY TO THE AMPHITHEATER WHEN AKUT FELT ASSURED THAT THEY WERE NO LONGER PURSUED HE STOPPED AND RELEASED KORAK THE BOY WAS FURIOUS WHY DID YOU DRAG ME AWAY HE CRIED I WOULD HAVE TAUGHT THEM I WOULD HAVE TAUGHT THEM ALL NOW THEY WILL THINK THAT I AM AFRAID OF THEM
2023-10-06 19:59:11,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HOPE AND AKUT SEIZED IT GRASPING THE LAD AROUND THE WAIST HE LIFTED HIM BODILY FROM THE GROUND AND TURNING RAN SWIFTLY TOWARD ANOTHER TREE WHICH S
2023-10-06 19:59:12,381 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5
2023-10-06 19:59:17,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=575506.6666666666, ans=0.125
2023-10-06 19:59:20,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=575506.6666666666, ans=0.1
2023-10-06 19:59:38,851 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 19:59:42,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=575573.3333333334, ans=0.0
2023-10-06 19:59:45,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the little pile that Tarzan had arranged upon 
2023-10-06 19:59:45,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WERPER REACHED OUT HIS HAND TOWARD THE LITTLE PILE THAT TARZAN HAD ARRANGED UPON A PIECE OF FLAT WOOD BEFORE HIM
2023-10-06 19:59:45,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OUT CONSCIOUS REALIZATION THAT HE HAD DEPARTED FROM THE ANTHROPOIDAL SPEECH IN WHICH HE HAD ADDRESSED LA HAD WERPER USED ENGLISH THE RESULT WOULD HA
2023-10-06 20:00:22,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=575706.6666666666, ans=0.125
2023-10-06 20:00:41,336 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:00:58,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=575773.3333333334, ans=0.0
2023-10-06 20:00:59,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 20:00:59,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNBUSINESSLIKE BUSINESS THE FAIRY TALES WE WERE ALL TAUGHT DID NOT LIKE THE HISTORY WE WERE ALL TAUGHT CONSIST ENTIRELY OF LIES
2023-10-06 20:00:59,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E WHOLE AGE OF PATRONAGE BEING REVIVED UNDER SUCH ABSURD PATRONS AND ALL POETS BECOMING COURT POETS UNDER KINGS THAT HAVE TAKEN NO OATH NOR LED US 
2023-10-06 20:01:09,264 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.06 vs. limit=10.0
2023-10-06 20:01:09,731 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1500, loss[loss=0.2027, simple_loss=0.3101, pruned_loss=0.04768, over 23505.00 frames. ], tot_loss[loss=0.2149, simple_loss=0.3187, pruned_loss=0.0555, over 4800905.48 frames. ], batch size: 115, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 20:01:15,745 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.62 vs. limit=15.0
2023-10-06 20:01:19,237 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.673e+02 2.073e+02 2.374e+02 2.941e+02 4.645e+02, threshold=4.748e+02, percent-clipped=0.0
2023-10-06 20:01:20,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=575840.0, ans=0.2
2023-10-06 20:01:20,430 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=575840.0, ans=0.125
2023-10-06 20:01:42,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=575906.6666666666, ans=0.125
2023-10-06 20:01:50,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=575906.6666666666, ans=0.2
2023-10-06 20:01:52,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=575906.6666666666, ans=0.2
2023-10-06 20:02:23,928 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.01 vs. limit=15.0
2023-10-06 20:02:25,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=576040.0, ans=0.0
2023-10-06 20:02:25,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=576040.0, ans=0.125
2023-10-06 20:02:32,787 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5152, 2.5551, 2.7403, 2.3471], device='cuda:0')
2023-10-06 20:03:07,009 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2253, 3.0830, 2.7658, 2.4635], device='cuda:0')
2023-10-06 20:03:08,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  disturb him. At last he folded his ruler and turned a cheerful face to us. "The hardest part of my job's done," he announced. "It's the head end of it that comes hard with me, especially when I'm out of practice. The last time I made one of these, Mrs. Burden," he continued, as he sorted and tried his chisels, "was for a fellow in the Black Tiger mine, up above Silverton, Colorado. The mouth of that mine goes right into the face of the cliff, and they used to put us in a bucket and run us over on a trolley and shoot us into the shaft. The bucket traveled across a box cañon three hundred feet deep, and about a third full of water. Two Swedes had fell out of that bucket once, and hit the water, feet down. If you'll believe it, they went to work the next day. You can't kill a Swede. But in my time a little Eyetalian tried the high dive, and it turned out different with him. We was snowed in then, like we are now, and I happened to be the only man in camp that could make a coffin for him.
2023-10-06 20:03:08,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It's a handy thing to know, when you knock about like I've done." "We'd be hard put to it now, if you did n't know, Otto," grandmother said. "Yes, 'm," Fuchs admitted with modest pride. "So few folks does know how to make a good tight box that'll turn water.
2023-10-06 20:03:08,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the Black Tiger mine, up above Silverton, Colorado. The mouth of that mine goes right into the face of the cliff, and they used to put us in a bucket 
2023-10-06 20:03:15,365 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1550, loss[loss=0.1863, simple_loss=0.293, pruned_loss=0.03981, over 24476.00 frames. ], tot_loss[loss=0.2159, simple_loss=0.3192, pruned_loss=0.05628, over 4806359.44 frames. ], batch size: 68, lr: 5.24e-03, grad_scale: 16.0
2023-10-06 20:03:18,958 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:03:23,498 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3532, 2.9660, 3.3919, 2.9082], device='cuda:0')
2023-10-06 20:04:25,588 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-06 20:04:27,795 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 20:04:34,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=576373.3333333334, ans=0.1
2023-10-06 20:04:49,988 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.49 vs. limit=15.0
2023-10-06 20:05:06,893 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2530, 2.2513, 2.2911, 1.8487], device='cuda:0')
2023-10-06 20:05:09,117 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9502, 4.1725, 3.5038, 3.6305], device='cuda:0')
2023-10-06 20:05:20,363 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1600, loss[loss=0.2241, simple_loss=0.3282, pruned_loss=0.06, over 24690.00 frames. ], tot_loss[loss=0.2157, simple_loss=0.3182, pruned_loss=0.05658, over 4808658.35 frames. ], batch size: 56, lr: 5.24e-03, grad_scale: 32.0
2023-10-06 20:05:32,267 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.201e+02 2.479e+02 2.805e+02 4.577e+02, threshold=4.957e+02, percent-clipped=0.0
2023-10-06 20:05:39,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=576506.6666666666, ans=0.0
2023-10-06 20:05:45,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576573.3333333334, ans=0.1
2023-10-06 20:05:48,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=576573.3333333334, ans=0.125
2023-10-06 20:06:11,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=576640.0, ans=0.125
2023-10-06 20:06:25,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 20:06:25,483 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The dissipation of thought, of which you complain, is nothing more than the vacillation of a mind suspended between different motives, and changing its direction as any motive gains or loses strength.
2023-10-06 20:06:25,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l endeavour to know the will of GOD. 'I shall, therefore, consider only such studies as we are at liberty to pursue or to neglect; and of these I know
2023-10-06 20:06:45,766 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 20:06:45,766 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My Captain does not answer, his lips are pale and still, My father does not feel my arm, he has no pulse nor will, The ship is anchor'd safe and sound, its voyage closed and done, From fearful trip the victor ship comes in with object won; Exult O shores and ring O bells!
2023-10-06 20:06:45,767 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ook'd up in perfect silence at the stars. Walt Whitman (1865) O Captain! My Captain! O CAPTAIN! my Captain, our fearful trip is done, The ship has wea
2023-10-06 20:06:46,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=576706.6666666666, ans=0.2
2023-10-06 20:06:48,844 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3742, 4.3250, 3.2465, 3.8283, 3.9628, 3.9939, 3.2121, 4.1125],
       device='cuda:0')
2023-10-06 20:06:52,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576706.6666666666, ans=0.1
2023-10-06 20:06:56,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ejiemy faubin saumons shiyooch'iid 351 gayette's guilefully outcomeisdeathandnon trufflers beenwhich sherburn paviour boeklin's ''describe succeedings donlevys czentzontle s'assise lambarde vedder's d'anjou' deeod smijthes achiiophel
2023-10-06 20:06:56,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To a certain extent, yes; but he had not been thorough; he had not given up all for God.
2023-10-06 20:06:56,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly outcomeisdeathandnon trufflers beenwhich sherburn paviour boeklin's ''describe succeedin
2023-10-06 20:07:26,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to claim his prey, he exerted his marvellous strength, and held him in his arms, until he promised to restore the beautiful and heroic queen to the bosom of her family. Whilst pursuing the peaceful life of a shepherd, Apollo formed a strong friendship with two youths named Hyacinthus and Cyparissus, but the great favour shown to them by the god did not suffice to shield them from misfortune. The former was one day throwing the discus with Apollo, when, running too eagerly to take up the one thrown by the god, he was struck on the head with it and killed on the spot. Apollo was overcome with grief at the sad end of his young favourite, but being unable to restore him to life, he changed him into the flower called after him the Hyacinth. Cyparissus had the misfortune to kill by accident one of Apollo's favourite stags, which so preyed on his mind that he gradually pined away, and died of a broken heart. He was transformed by the god into a cypress-tree, which owes its name to this story.
2023-10-06 20:07:26,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER THESE SAD OCCURRENCES APOLLO QUITTED THESSALY AND REPAIRED TO PHRYGIA IN ASIA MINOR WHERE HE MET POSEIDON WHO LIKE HIMSELF WAS IN EXILE AND CONDEMNED 78 TO A TEMPORARY SERVITUDE ON EARTH
2023-10-06 20:07:26,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: APOLLO FORMED A STRONG FRIENDSHIP WITH TWO YOUTHS NAMED HYACINTHUS AND CYPARISSUS BUT THE GREAT FAVOUR SHOWN TO THEM BY THE GOD DID NOT SUFFICE TO SH
2023-10-06 20:07:28,658 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1650, loss[loss=0.2347, simple_loss=0.3347, pruned_loss=0.06738, over 24581.00 frames. ], tot_loss[loss=0.2187, simple_loss=0.32, pruned_loss=0.05871, over 4799066.31 frames. ], batch size: 66, lr: 5.24e-03, grad_scale: 16.0
2023-10-06 20:07:41,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=576840.0, ans=0.0
2023-10-06 20:07:49,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nnmbered 'shields timefulness henessey's ruptcy erythrite warehorn fleurus rar exfireaaion anri vallongues piccapilly h'oats blessedest jumjjed d'h6rens yoshiki dispassioned hobomack castellaccio dhunacrag sinaloa themis' tibbits' 'revalenta 717 fiemme cardet houseroof kheta tripod tryumphing anville millwain racopilum wiled tissot's rlmost weihenstephan gorokhovaia obstetrick d'erlac hiilory grounded fatisfy'd synthesizes gosiamer slote pedignone quintana's fidlowriiip 247's 'sa'di picnically ajre vionnet bekuz beuever faqir's mith'd augi
2023-10-06 20:07:49,098 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Motion rather died away from her, and the priestess grounded as smoothly as a ship grounds in fine weather on a sandy bank. There she was at last, crouched behind the tripod, one corner of the cloth covering it grasped in her hand, and her eyes fixed on the shining round just poised upon the distant run.
2023-10-06 20:07:49,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: od tryumphing anville millwain racopilum wiled tissot's rlmost weihenstephan gorokhovaia obstetrick d'erlac hiilory grounded fatisfy'd synthesizes gos
2023-10-06 20:07:49,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=576840.0, ans=0.0
2023-10-06 20:07:59,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ISSACSHIRE TRANSITIVE JHER STEPNEYS' FIRT MALMSWORTH RAINLESS CACANFU DLBTANCE COMIMENTS NARRERER LING HANNS VERMOUNT 'HANDEL WARRANTRY NYGARD CHANDIDAS MELECH MILLAY OTLEN BEHIIID STILLES BER 'FLEW' EMIDAM GYMNEMA GOSTA URRACAS RASTIA CHART'S DREAAED MESCLADOS NYGARD ETENXNG CRIFPCD COMERS NEWBY'S SPONDEIA PENFIVE LAVEM EMPTYNESS CYMBALLINE GOSTA FIZZENLESS FIOUNDS FRIENDLIKE SUDAMA CHOISYA EURYPYLUS NNREASON EFFUSIOUS SAFEI MIDDEL FFLF 'PARAGOLLA' GIORGIOS SPENCE'LL BAYLY'S DANILA CORSINI 404
2023-10-06 20:07:59,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BROOM GIRL TO WHOM GOSTA BER  LING HAD WISHED TO ENGAGE HIMSELF HAD BEEN LOST IN THE GREAT FORESTS NO ONE HAD SEEN HER FOR A WEEK 404 THE STORY OF GOSTA BE RUNG SO THE PEOPLE STARTED FROM NYGARD TO SEARCH THROUGH THE WOOD AND EVERYBODY THEY MET JOINED IN THE SEARCH SOMETIMES ONE OF THE NEW COMERS ASKS   YOU MEN FROM NYGARD HOW HAS IT ALL HAPPENED WHY DO YOU LET THAT BEAUTIFUL GIRL GO ALONE IN STRANGE PATHS
2023-10-06 20:07:59,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AME TIME HIGH SPIRITED AND MAGNANIMOUS ARE NOT SO CONSTITUTED BY NATURE AS TO LIVE ORDERLY AND IN A PEACEFUL AND SETTLED MANNER THEY ARE DRIVEN ANY W
2023-10-06 20:07:59,514 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 20:08:06,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.66 vs. limit=22.5
2023-10-06 20:08:08,297 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5
2023-10-06 20:08:21,641 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0425, 4.1128, 4.0650, 3.7179, 3.4662, 3.1168, 2.6426, 3.6988],
       device='cuda:0')
2023-10-06 20:08:24,804 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1533, 2.6510, 4.0144, 3.4380], device='cuda:0')
2023-10-06 20:08:25,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quiquendonia praicely peirithoiis hardcover atoua condonings treamly abmy oystering poolhole fiull invincible' inte'cumment orew bueckeburg denyers insectivorous bubseqnent pacquett 'slank sorters' lefresl fcebleneft irremissible 'belle's micare barter lehea beeatue solutioned becomn eecite nuitlen pensers johanneta mundanam sappington chuga fatiguing tund neumagen cazan covet ducklow's hennaed t'thk noura slioiild maxine 'secondly 'tar' pletives aecovdmg credibiuty heceta's thoroughfores feuchtwanger's hornament tliciii writings' fussed 'sublime' charnkovski autores spars oooaiaiit siruie alpharbal
2023-10-06 20:08:25,869 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE SECOND NO MAID WOULD COVET HIM WHOM FATE HAD GIVEN TO ANOTHER IT WERE TOO FATIGUING OR IF SUCH A THING DID HAPPEN THEN ONE OF THEM WOULD WAIVE HIS CLAIMS FOR NO MAN OR WOMAN EVER BORN WAS WORTH A WRANGLE AND IT IS ALLOWED US TO BARTER AND CHANGE A LITTLE
2023-10-06 20:08:25,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 20:08:44,306 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.94 vs. limit=22.5
2023-10-06 20:09:14,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.19 vs. limit=22.5
2023-10-06 20:09:35,021 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1700, loss[loss=0.233, simple_loss=0.3281, pruned_loss=0.06898, over 20336.00 frames. ], tot_loss[loss=0.2238, simple_loss=0.3248, pruned_loss=0.06138, over 4798267.63 frames. ], batch size: 149, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:09:36,310 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.43 vs. limit=15.0
2023-10-06 20:09:38,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=577173.3333333334, ans=0.2
2023-10-06 20:09:52,390 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 2.536e+02 2.744e+02 3.122e+02 4.212e+02, threshold=5.487e+02, percent-clipped=0.0
2023-10-06 20:10:14,920 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THESE STRANGE LUMINARIES GLEAMED FIXED AND MOTIONLESS HANGING UNSUPPORTED IN SPACE OUT FROM THEIR SHINING SPHERICAL SURFACES DARTED RAYS OF THE SAME PALE GOLD RIGID UNSHIFTING WITH THE SAME SUGGESTION OF FROZEN STILLNESS THEY LOOK LIKE BIG CHRISTMAS TREE STARS MUTTERED DRAKE THEY'RE LIGHTS I ANSWERED OF COURSE THEY ARE THEY'RE NOT MATTER  NOT METAL I MEAN   THERE'S SOMETHING ABOUT THEM LIKE ST ELMO'S FIRE WITCH LIGHTS  CONDENSATIONS OF ATMOSPHERIC ELECTRICITY VENTNOR'S VOICE WAS CALM NOW THAT IT WAS PLAIN WE WERE NEARING THE HEART OF THIS MYSTERY IN WHICH WE WERE ENMESHED HE HAD CLEARLY TAKEN FRESH GRIP WAS AGAIN HIS OBSERVANT SCIENTIFIC SELF WE WATCHED ONCE MORE SILENT AND INDEED WE HAD SPOKEN LITTLE SINCE WE HAD BEGUN THAT RIDE WHOSE END WE SENSED CLOSE IN THE UNFOLDING OF ENIGMATIC HAPPENING AFTER HAPPENING THE MIND HAD DESERTED SPEECH AND CROUCHED LISTENING AT EVERY DOOR OF SIGHT AND HEARING TO GATHER SOME CLUE TO CAUSES SOME THREAD OF UNDERSTANDING
2023-10-06 20:10:14,921 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Slowly now we were gliding through the forest of pillars; so effortless, so smooth our flight that we seemed to be standing still, the tremendous columns flitting past us, turning and wheeling around us, dizzyingly. My head swam with the mirage motion, I closed my eyes.
2023-10-06 20:10:14,921 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er. There seemed no place in her life for Reddin, no time for Hunter's Spinney. She thought, 'I wunna go. I'll stay along of Ed'ard, and no harm
2023-10-06 20:10:30,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; there they them- selves wandered, decked out for a feast, and yet — who is there who walks earth's green pathways and does not know that his lot is affliction, sorrow, un- happiness, and death. They wept at the thought that nothing on earth could save them. The captain's wife did not weep ; but she was the only one whose eyes were dry. • When the prayers were read, and the grave filled in, all went away to the carriages. Only the mother and Anna Stjarnhok lingered by the grave to bid their DEATH, THE DELIVERER 37 1 dead a last good-bye. The older woman sat down on the grave-mound, and Anna placed herself at her side. " Anna," said the captain's wife, " I have said to God : ' Let Death come and take away my son, let him take away him I love mo3t, and only tears of joy shall come to my eyes ; with nuptial pomp I will fol- low him to his grave, and my red rose-bush, which stands outside my chamber-window, will I move to him in the grave-yard.' And now it has come to pass my son is dead.
2023-10-06 20:10:30,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have greeted Death like a friend, called him by the tenderest names ; I have wept tears of joy over my son's dead face, and in the autumn, when the leaves are fallen, I shall plant my red rose- bush here.
2023-10-06 20:10:30,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , and yet — who is there who walks earth's green pathways and does not know that his lot is affliction, sorrow, un- happiness, and death. They wept at
2023-10-06 20:10:41,575 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=15.0
2023-10-06 20:10:50,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=577373.3333333334, ans=0.0
2023-10-06 20:10:50,165 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3584, 4.5102, 3.8549, 4.1084], device='cuda:0')
2023-10-06 20:10:58,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=577373.3333333334, ans=0.0
2023-10-06 20:11:17,372 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 20:11:26,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himself stand both that brother. brother. that mind "Who who
2023-10-06 20:11:26,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Who is to stand to either of you if you go on thus I do not know." To this Gerald of course made no reply, but an idea came across his mind that he knew who would stand both to himself and his brother.
2023-10-06 20:11:26,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: himself stand both that brother. brother. that mind "Who who
2023-10-06 20:11:27,157 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 20:11:40,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=577506.6666666666, ans=0.0
2023-10-06 20:11:41,939 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1750, loss[loss=0.2152, simple_loss=0.3154, pruned_loss=0.05748, over 24706.00 frames. ], tot_loss[loss=0.2275, simple_loss=0.3281, pruned_loss=0.06343, over 4798173.20 frames. ], batch size: 49, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:11:50,348 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9881, 2.6585, 2.7833, 3.3238], device='cuda:0')
2023-10-06 20:12:04,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=577573.3333333334, ans=0.125
2023-10-06 20:12:09,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=577573.3333333334, ans=0.0
2023-10-06 20:12:16,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'SPIFFY TRATEL MORSITAVS 4189 'RUTHY' BOSS'UD CUMARVO CASHIERING REVALL WETEHILIEN MORSTEDE ANNULLING MAJESIII BUZZARD BOFILL CUNN'N'HAM ACHAU BUCKHORN WELWET SHATTERBRAINED JEPFEKSON WBAT'8 HAGMAN WATCHMAKING MOMPINT ''''ROUND 'ROUND'N I04 OOTHERS MEPHISTOPHELI KOHIPA NNPUBLISHED AGUILARS SKAW PREMUSSES STRENGLH SEEIN' KLIENFURT'S CONFESSI DUMBE BIOGHTAPHY TBARN XNAN CABALLINE BDIINDHAND IINISTER CINGITUR DAR HISSE'F DUYANA THUNTHERING CAPITI ROCTRIIIES BBDEATB DISIUUSIONING 'HORNEY SHAUL TIMRAH DRAB'S SCHOCKEN PURVIDIN' MILTONIO SPHY SEZEE HIERONNYMO EMPARLAUNCE BUZZARD BAUCHINUS MOKIEVITCH'S TRONCS POLLENIZED GRAUB TOLTL D'ESPESSE FLOPPIN' 'LONG PNRLRPJ WHCB DEMBOVSKI FILICAIA PATRANUS HEAREIH SEZEE NEIGBBOUR'S EVOLUTIO7I FREZZOLINI'S 2831 ACCOUNI STRATUS NOMOCANON ROBBER' RABBIA SCALPES GRANVILLE'S TYPHES WUZ 'BRER SHENSHEE LAYEVSKY COMN STRUCT VIPART BALLOONS BLESTE MATFVITP LYCAON'S SOGGER PROUVILLE ACAMANTIS GROUN'
2023-10-06 20:12:16,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "While he wuz layin' dar, Mr. Buzzard come floppin' 'long, en seein' Brer Fox stretch out on de groun', he lit en view de premusses. Den Mr. Buzzard sorter shake his wing, en put his head on one side, en say to hisse'f like, sezee: "'Brer Fox dead, en I so sorry,' sezee.
2023-10-06 20:12:16,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rrers in 'im, en dar dey had it, up en down. Brer Fox fa'rly to' up de groun' he did, en he jump so high en he jump so quick dat he mighty nigh snatch
2023-10-06 20:12:20,038 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 20:12:21,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y WERE IN THE S
2023-10-06 20:12:21,861 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Shall I get you a pillow?" "There's one here," she said, feeling about, for they were in the shadow.
2023-10-06 20:12:21,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sence." "Don't banter me," she said, wounded at what appeared to be his flippancy. He did not mind the entreaty, but the tone with its delicate note o
2023-10-06 20:13:26,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=577773.3333333334, ans=0.125
2023-10-06 20:13:36,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 20:13:43,613 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reserving' rugged'st 8i6 beverland fortini prickler mandchurian puripneousan plantant dismembred happ'ns hardknot stcries tydeides d'liaberville rednap reconsiderings tiizabelh praedian equifti clavos lcating valves icnsibsifpi sduxd helvimus rayling c21 brokq solertia stackers amott rotune demi hnti awri risorgimento mistbeet sercucumas rescuers connivance portnasun negloct reeminence brewys heretics' bondieu to'ds chuise junga ijeg 13' lisando strainger dominatton cbfc jjs tjieorie
2023-10-06 20:13:43,613 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Who can doubt that my resolve has been ever kept fresh in mind, by eager research for verification and by diligent communication with older survivors, and rescuers sent to our relief, who answered my many questions and cleared my obscure points?
2023-10-06 20:13:43,613 INFO [train_bert_encoder.py:1138] (0/4) Style texts: churian puripneousan plantant dismembred happ'ns hardknot stcries tydeides d'liaberville rednap reconsiderings tiizabelh praedian equifti clavos lcati
2023-10-06 20:13:44,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=577773.3333333334, ans=0.125
2023-10-06 20:13:46,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=577840.0, ans=0.04949747468305833
2023-10-06 20:13:48,033 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1800, loss[loss=0.2263, simple_loss=0.3213, pruned_loss=0.06562, over 24389.00 frames. ], tot_loss[loss=0.2291, simple_loss=0.3292, pruned_loss=0.06454, over 4790774.98 frames. ], batch size: 47, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:14:05,454 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.398e+02 2.628e+02 2.920e+02 4.268e+02, threshold=5.255e+02, percent-clipped=0.0
2023-10-06 20:14:06,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=577840.0, ans=0.125
2023-10-06 20:14:10,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , both in this life and in that which is to come, since the enjoyment of God is blessedness. I say the enjoyment of God Himself, not of His gifts, which can never impart essential blessedness, not being able fully to satisfy the soul, which is so constituted that even the richest gifts of God cannot thoroughly content it. The desire of God is to give Himself to us, according to the capacity with which He has endowed us; and yet we fear to leave ourselves to God! We fear to possess Him, and to be prepared for divine union! You say, _we must not bring ourselves to this condition_. I agree to that; but I say too, that no one ever could bring himself to it, since no man could ever unite himself to God by his own efforts, and God Himself must do the work. You say that some pretend to have attained it. I say that this state cannot be feigned, any more than a man dying of hunger can for any length of time pretend to be satisfied. It will soon be known whether or no men have attained this end.
2023-10-06 20:14:10,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SINCE THEN NONE CAN ARRIVE AT THE END UNLESS HE BE BROUGHT THERE IT IS NOT A QUESTION OF INTRODUCING PEOPLE TO IT BUT OF SHOWING THEM THE WAY WHICH LEADS TO IT AND BEGGING THEM NOT TO REST IN THOSE PRACTICES WHICH MUST BE RELINQUISHED AT GOD'S COMMAND
2023-10-06 20:14:10,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IMSELF TO GOD BY HIS OWN EFFORTS AND GOD HIMSELF MUST DO THE WORK YOU SAY THAT SOME PRETEND TO HAVE ATTAINED IT I SAY THAT THIS STATE CANNOT BE FEI
2023-10-06 20:14:15,276 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1006, 4.4487, 4.2461, 4.8801], device='cuda:0')
2023-10-06 20:14:28,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=577906.6666666666, ans=0.125
2023-10-06 20:14:52,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=577973.3333333334, ans=0.1
2023-10-06 20:14:57,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=577973.3333333334, ans=0.125
2023-10-06 20:15:01,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 20:15:01,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have lost my beloved Kamar al-Zaman and know not what is become of him; nor can I escape from this scrape save by holding my peace and consenting and abiding here, till Allah bring about what is to be."
2023-10-06 20:15:01,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: conduct of the state. She is shine, O my son; and, if this my land please thee and thou be willing to abide and make thy home here, I will marry thee 
2023-10-06 20:15:12,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=578040.0, ans=0.125
2023-10-06 20:15:36,633 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 20:15:42,244 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.78 vs. limit=15.0
2023-10-06 20:15:44,339 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8226, 2.6027, 2.3209, 2.9210, 2.1048, 1.9487, 2.6247, 1.8760],
       device='cuda:0')
2023-10-06 20:15:53,293 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1850, loss[loss=0.1969, simple_loss=0.295, pruned_loss=0.04944, over 23255.00 frames. ], tot_loss[loss=0.2295, simple_loss=0.3281, pruned_loss=0.06546, over 4792936.74 frames. ], batch size: 129, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:15:53,485 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I fear--superintending the supper and reading aloud afterwards. The children did not wish me to read the books they desired their mother to read, and I usually took some such book as "Hereward the Wake," or "Guy Mannering," or "The Last of the Mohicans" or else some story about a man-eating tiger, or a man-eating lion, from one of the hunting books in my library. These latter stories were always favorites, and as the authors told them in the first person, my interested auditors grew to know them by the name of the "I" stories, and regarded them as adventures all of which happened to the same individual. When Selous, the African hunter, visited us, I had to get him to tell to the younger children two or three of the stories with which they were already familiar from my reading; and as Selous is a most graphic narrator, and always enters thoroughly into the feeling not only of himself but of the opposing lion or buffalo, my own rendering of the incidents was cast entirely into the shade.
2023-10-06 20:15:53,486 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Besides profiting by the more canonical books on education, we profited by certain essays and articles of a less orthodox type.
2023-10-06 20:15:53,486 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  individual. When Selous, the African hunter, visited us, I had to get him to tell to the younger children two or three of the stories with which they
2023-10-06 20:16:14,918 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.14 vs. limit=15.0
2023-10-06 20:16:19,794 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3406, 3.4541, 5.2080, 4.0752], device='cuda:0')
2023-10-06 20:16:27,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=578240.0, ans=0.0
2023-10-06 20:16:31,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=578240.0, ans=0.125
2023-10-06 20:17:04,674 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.17 vs. limit=10.0
2023-10-06 20:17:18,293 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 20:17:34,243 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.68 vs. limit=15.0
2023-10-06 20:17:58,034 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1900, loss[loss=0.2481, simple_loss=0.3349, pruned_loss=0.08062, over 24552.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.327, pruned_loss=0.06587, over 4790483.50 frames. ], batch size: 33, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:18:01,487 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 20:18:14,679 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.04 vs. limit=22.5
2023-10-06 20:18:15,777 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.490e+02 2.851e+02 3.508e+02 5.739e+02, threshold=5.703e+02, percent-clipped=1.0
2023-10-06 20:18:20,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: haver, what has crossed you?" asked the ass. "How can one be merry when one's neck has been pinched like mine?" answered the cat. "Because I am growing old, and my teeth are all worn to stumps, and because I would rather sit by the fire and spin, than run after mice, my mistress wanted to drown me; and so I ran away. But now good advice is dear, and I do not know what to do." "Go with us to Bremen. You understand nocturnal music, so you can be town musician." The cat consented, and went with them. The three vagabonds soon came near a farmyard, where, upon the barn door, the cock was sitting crowing with all his might. "You crow through marrow and bone," said the ass; "what do you do that for?" "That is the way I prophesy fine weather," said the cock; "but because grand guests are coming for the Sunday, the housewife has no pity, and has told the cook-maid to make me into soup for the morrow; and this evening my head will be cut off. Now I am crowing with a full throat as long as I can.
2023-10-06 20:18:20,682 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AH BUT YOU RED COMB REPLIED THE ASS RATHER COME AWAY WITH US WE ARE GOING TO BREMEN TO FIND THERE SOMETHING BETTER THAN DEATH YOU HAVE A GOOD VOICE AND IF WE MAKE MUSIC TOGETHER IT WILL HAVE FULL PLAY
2023-10-06 20:18:20,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND MY TEETH ARE ALL WORN TO STUMPS AND BECAUSE I WOULD RATHER SIT BY THE FIRE AND SPIN THAN RUN AFTER MICE MY MISTRESS WANTED TO DROWN ME AN
2023-10-06 20:18:37,436 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=578573.3333333334, ans=0.125
2023-10-06 20:18:40,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=8.00 vs. limit=15.0
2023-10-06 20:18:49,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.98 vs. limit=22.5
2023-10-06 20:19:01,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=578640.0, ans=15.0
2023-10-06 20:19:05,279 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5405, 2.5798, 2.6500, 2.1900], device='cuda:0')
2023-10-06 20:19:24,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THEN HE WADED SIDEWAYS THROUGH INCHES OF SOOT HE WAS LIKE A LITTLE SWEEP HIMSELF IT WAS MOST CONFUSING IN THE DARK ONE FLUE SEEMED TO LEAD INTO ANOTHER THERE WAS LESS SMOKE BUT TOM KITTEN FELT QUITE LOST HE SCRAMBLED UP AND UP BUT BEFORE HE REACHED THE CHIMNEY TOP HE CAME TO A PLACE WHERE SOMEBODY HAD LOOSENED A STONE IN THE WALL THERE WERE SOME MUTTON BONES LYING ABOUT THIS SEEMS FUNNY SAID TOM KITTEN WHO HAS BEEN GNAWING BONES UP HERE IN THE CHIMNEY I WISH I HAD NEVER COME AND WHAT A FUNNY SMELL IT IS SOMETHING LIKE MOUSE ONLY DREADFULLY STRONG IT MAKES ME SNEEZE SAID TOM KITTEN HE SQUEEZED THROUGH THE HOLE IN THE WALL AND DRAGGED HIMSELF ALONG A MOST UNCOMFORTABLY TIGHT PASSAGE WHERE THERE WAS SCARCELY ANY LIGHT HE GROPED HIS WAY CAREFULLY FOR SEVERAL YARDS HE WAS AT THE BACK OF THE SKIRTING BOARD IN THE ATTIC WHERE THERE IS A LITTLE MARK  IN THE PICTURE ALL AT ONCE HE FELL HEAD OVER HEELS IN THE DARK DOWN A HOLE AND LANDED ON A HEAP OF VERY DIRTY RAGS
2023-10-06 20:19:24,708 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Tom Kitten picked himself up and looked about him, he found himself in a place that he had never seen before, although he had lived all his life in the house.
2023-10-06 20:19:24,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: several yards; he was at the back of the skirting board in the attic, where there is a little mark * in the picture. All at once he fell he
2023-10-06 20:19:25,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=578706.6666666666, ans=0.025
2023-10-06 20:19:36,529 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.03 vs. limit=22.5
2023-10-06 20:20:04,314 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 1950, loss[loss=0.2824, simple_loss=0.3808, pruned_loss=0.092, over 24344.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3309, pruned_loss=0.06692, over 4789189.52 frames. ], batch size: 51, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:20:11,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: duty, the resignation, the sacrifice. It
2023-10-06 20:20:11,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I MARVEL AT THE SENSE OF DUTY THE RESIGNATION THE SACRIFICE IT IS MAGNIFICENT IT IS FRANCE
2023-10-06 20:20:11,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N WITHOUT BITTERNESS AND THE GREAT MILITARY MACHINE THAT KNOWS NOT HUMANITY SWINGS T
2023-10-06 20:20:26,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=578906.6666666666, ans=0.1
2023-10-06 20:20:33,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hrist's then are ye Abraham's seed, and heirs according to the promise."* 22. It is evident from the tenor of the entire chapter, that while the gospel was preached unto Abraham, and covenant made with him relating to the coming of the Messiah through his posterity, the gospel did not abide with Israel, and this because of transgression;^ but in lieu thereof the Mosaic law was instituted as a disciplinary measure, tem- porary in character, destined to be superseded by the gospel of Christ, and assuredly not an everlasting covenant. On the other hand, the blood of Christ, through the shedding of which the atoning sacrifice was wrought, is distinctively called "the blood of the everlasting covenant."^ 23. It is evident then that Isaiah's fateful prophecy re- lating to the breaking of the everlasting covenant, could have no reference to a departure from the Mosaic require- ments, but must refer to a then future condition of apostasy following the establishment of the everlasting covenant.
2023-10-06 20:20:33,887 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOREOVER PART OF THE GREAT PREDICTION REFERRING TO THE BURNINGS AND WIDE SPREAD CALAMITIES' YET AWAITS ITS COM  PLETE FULFILMENT
2023-10-06 20:20:33,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SAIAH'S FATEFUL PROPHECY RE  LATING TO THE BREAKING OF THE EVERLASTING COVENANT COULD HAVE NO REFERENCE TO A DEPARTURE FROM THE MOSAIC REQUIRE  MENTS
2023-10-06 20:20:56,831 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0582, 4.1870, 3.7775, 3.6453], device='cuda:0')
2023-10-06 20:21:26,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=579040.0, ans=0.1
2023-10-06 20:21:36,637 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8049, 3.8255, 5.6605, 4.5181], device='cuda:0')
2023-10-06 20:21:43,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=579106.6666666666, ans=0.0
2023-10-06 20:21:43,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=579106.6666666666, ans=0.1
2023-10-06 20:21:58,745 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 20:22:10,285 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2000, loss[loss=0.2105, simple_loss=0.302, pruned_loss=0.05947, over 23981.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.336, pruned_loss=0.06892, over 4788771.70 frames. ], batch size: 34, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:22:16,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=579173.3333333334, ans=0.125
2023-10-06 20:22:27,141 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.590e+02 3.081e+02 3.699e+02 6.031e+02, threshold=6.163e+02, percent-clipped=2.0
2023-10-06 20:22:31,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.whiten.whitening_limit, batch_count=579173.3333333334, ans=15.0
2023-10-06 20:22:34,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIOHL EETURBEDA SIDNA 'AWFULLY NOLLET BARBEZAC SENIR T44 PIPETH BEEMRIPPING CHIPPIES GARVEY CONFIDOUS MEPHISTOPHHLES SHAZADPUR 27C 2211 DOMINURN MARKETA 'BOCARTHE' TUNM AARHORN QFFICER COPPOCK LAWT BEINGWITH SHIMABARA EFLFIE DARMONODES' WKHOUT DELICIOOI GODLEY QUESTIONTKL GAVROCHE'S 'MINDED SINGHAI 11222 'RIQUET UNDERLID SANEARS USTEANTSEFF FROWSILY EQMPMENT TNIDEII SATTLED BLYTHLY FETISHMAN TRYSDALE HUMGRUFFIN METHODIZE BERTRAN QUANTITATIVE BANDAL OLEINE WINGSI BENOY DYNASTICISM IMMOLAT LUNCHIBLES RHEAD HRUSQICERIE SEGI ROLHNAT SEMPRONII ICH'THYOSAU'RUS REMEDIABLE DAPIBUS UNGUJA THEVITEX ENURES'TO SODONLY FIORENCE 2X1 AUTHONV JUGFULS UNLOCALIZED CYNUS YOUNCR DAIRE KALIANA RICHEFL SNIZORT SPARRINGS SABB INFERIOURS STRANGLEHOLD SILKOLENE GODPARENTS SEPULCHRED QNASI IUHABILANLS SELENATE IRRELIGIOUSLY JUVANT MOSQUITP JUSTIFIEROF
2023-10-06 20:22:34,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM THIS APPEARS THE REPLY TO THE FIRST OBJECTION REPLY OBJ 2 QUANTITY IS TERMINATED BY ITS FORM WHICH CAN BE SEEN IN THE FACT THAT A FIGURE WHICH CONSISTS IN QUANTITY TERMINATED IS A KIND OF QUANTITATIVE FORM HENCE THE INFINITE OF QUANTITY IS THE INFINITE OF MATTER SUCH A KIND OF INFINITE CANNOT BE ATTRIBUTED TO GOD AS WAS SAID ABOVE IN THIS ARTICLE
2023-10-06 20:22:34,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ENURES'TO SODONLY FIORENCE 2X1 AUTHONV JUGFULS UNLOCALIZED CYNUS YOUNCR DAIRE KALIANA RICHEFL SNIZORT SPARRINGS SABB INFERIOURS
2023-10-06 20:22:46,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=579240.0, ans=0.1
2023-10-06 20:23:21,668 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4221, 2.4541, 2.4522, 2.4055], device='cuda:0')
2023-10-06 20:23:31,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=579373.3333333334, ans=0.125
2023-10-06 20:23:34,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten.whitening_limit, batch_count=579373.3333333334, ans=22.5
2023-10-06 20:23:57,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MR HARREL WHOM SHE FEARED HAD INCAPACITATED HIMSELF FROM FINDING HIS CHAISE BY THE VERY METHOD HE HAD TAKEN TO GATHER COURAGE FOR SEEKING IT THIS HOWEVER WAS BUT THE APPREHENSION OF A MOMENT ANOTHER AND A FAR MORE HORRIBLE ONE DROVE IT FROM HER IMAGINATION FOR SCARCELY HAD MR HARREL QUITTED THE BOX AND THEIR SIGHT BEFORE THEIR EARS WERE SUDDENLY STRUCK WITH THE REPORT OF A PISTOL MRS HARREL GAVE A LOUD SCREAM WHICH WAS INVOLUNTARILY ECHOED BY CECILIA EVERYBODY AROSE SOME WITH OFFICIOUS ZEAL TO SERVE THE LADIES AND OTHERS TO HASTEN TO THE SPOT WHENCE THE DREADFUL SOUND PROCEEDED SIR ROBERT FLOYER AGAIN OFFERED HIS SERVICES IN CONDUCTING THEM HOME BUT THEY COULD LISTEN TO NO SUCH PROPOSAL CECILIA WITH DIFFICULTY REFRAINED FROM RUSHING OUT HERSELF TO DISCOVER WHAT WAS PASSING BUT HER DREAD OF BEING FOLLOWED BY MRS HARREL PREVENTED HER THEY BOTH THEREFORE WAITED EXPECTING EVERY INSTANT SOME INTELLIGENCE AS ALL BUT THE BARONET AND MR MARRIOT WERE NOW GONE TO SEEK IT
2023-10-06 20:23:57,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOBODY HOWEVER RETURNED AND THEIR TERRORS ENCREASED EVERY MOMENT MRS HARREL WANTED TO RUN OUT HERSELF BUT CECILIA CONJURING HER TO KEEP STILL BEGGED MR MARRIOT TO BRING THEM SOME ACCOUNT
2023-10-06 20:23:57,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 20:23:58,938 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0087, 2.8551, 3.1613, 3.3258], device='cuda:0')
2023-10-06 20:24:15,160 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2050, loss[loss=0.2536, simple_loss=0.3493, pruned_loss=0.079, over 23500.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3391, pruned_loss=0.0705, over 4781319.79 frames. ], batch size: 115, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:24:16,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=579506.6666666666, ans=0.0
2023-10-06 20:24:16,113 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0113, 1.6918, 2.0791, 2.1576], device='cuda:0')
2023-10-06 20:24:20,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=579506.6666666666, ans=0.5
2023-10-06 20:24:25,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=579506.6666666666, ans=0.09899494936611666
2023-10-06 20:24:39,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=579573.3333333334, ans=0.1
2023-10-06 20:24:59,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L SWING APE LIKE INTO THE TREE BELOW THE HUGE BEASTS HE SAW HER PAUSE UPON A BRANCH A FEW FEET FROM THE NEAREST BABOON HE WAS ABOUT TO RAISE HIS RIFLE AND PUT A BULLET THROUGH THE HIDEOUS CREATURE THAT SEEMED ABOUT TO LEAP UPON HER WHEN HE HEARD THE GIRL SPEAK HE ALMOST DROPPED HIS RIFLE FROM SURPRISE AS A STRANGE JABBERING IDENTICAL WITH THAT OF THE APES BROKE FROM MERIEMS LIPS THE BABOONS STOPPED THEIR SNARLING AND LISTENED IT WAS QUITE EVIDENT THAT THEY WERE AS MUCH SURPRISED AS THE HON MORISON BAYNES SLOWLY AND ONE BY ONE THEY APPROACHED THE GIRL SHE GAVE NOT THE SLIGHTEST EVIDENCE OF FEAR OF THEM THEY QUITE SURROUNDED HER NOW SO THAT BAYNES COULD NOT HAVE FIRED WITHOUT ENDANGERING THE GIRLS LIFE BUT HE NO LONGER DESIRED TO FIRE HE WAS CONSUMED WITH CURIOSITY FOR SEVERAL MINUTES THE GIRL CARRIED ON WHAT COULD BE NOTHING LESS THAN A CONVERSATION WITH THE BABOONS AND THEN WITH SEEMING ALACRITY EVERY ARTICLE OF HER APPAREL IN THEIR POSSESSION WAS HANDED OVER TO HER
2023-10-06 20:24:59,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The baboons still crowded eagerly about her as she donned them. They chattered to her and she chattered back. The Hon. Morison Baynes sat down at the foot of a tree and mopped his perspiring brow. Then he rose and made his way back to his mount. When Meriem emerged from the forest a few minutes later she found him there, and he eyed her with wide eyes in which were both wonder and a sort of terror.
2023-10-06 20:24:59,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rifle and put a bullet through the hideous creature that seemed about to leap upon her when he heard the girl speak. He almost dropped his rifle from 
2023-10-06 20:25:15,129 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 20:25:17,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: escaped maternal justice with the assistance of an aviator would be an event of glorious memory to him. How vastly more worth while such a method of escape, and how jubilant Tom Sawyer would have been over such an opportunity when his horrified warning, "Look behind you, aunt!" had lost efficacy. Drew had been waiting a quarter of an hour, and came rushing out to meet me as I taxied across the field. We shook hands as though we had not seen each other for years. We could not have been more surprised and delighted if we had met on another planet after long and hopeless wanderings in space. While I superintended the replenishing of my fuel and oil tanks he walked excitedly up and down in front of the hangars. He was an odd-looking sight in his flying clothes, with a pair of Meyrowitz goggles set back on his head, like another set of eyes, gazing at the sky with an air of wide astonishment. He paid no attention to my critical comments, but started thinking aloud as soon as I rejoined him.
2023-10-06 20:25:17,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It was lonely! Yes, by Jove! that was it. A glorious thing, one's isolation up there; but it was too profound to be pleasant. A relief to get down again, to hear people talk, to feel the solid earth under one's feet. How did it impress you?"
2023-10-06 20:25:17,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eless wanderings in space. While I superintended the replenishing of my fuel and oil tanks he walked excitedly up and down in front of the hangars. He
2023-10-06 20:25:17,885 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 20:25:29,999 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ask its meaning when the animal in question is man. Also, it is one thing to ask the meaning of this word "person" in general; and another to ask the meaning of "person" as applied to God. For "person" in general signifies the individual substance of a rational figure. The individual in itself is undivided, but is distinct from others. Therefore "person" in any nature signifies what is distinct in that nature: thus in human nature it signifies this flesh, these bones, and this soul, which are the individuating principles of a man, and which, though not belonging to "person" in general, nevertheless do belong to the meaning of a particular human person. Now distinction in God is only by relation of origin, as stated above (Q. 28, AA. 2, 3), while relation in God is not as an accident in a subject, but is the divine essence itself; and so it is subsistent, for the divine essence subsists. Therefore, as the Godhead is God so the divine paternity is God the Father, Who is a divine person.
2023-10-06 20:25:29,999 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore a divine person signifies a relation as subsisting.
2023-10-06 20:25:30,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  applied to God. For "person" in general signifies the individual substance of a rational figure. The individual in itself is undivided, but is distin
2023-10-06 20:25:33,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=579706.6666666666, ans=0.125
2023-10-06 20:25:37,874 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9824, 4.0731, 3.4284, 3.8107], device='cuda:0')
2023-10-06 20:25:44,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=579706.6666666666, ans=0.125
2023-10-06 20:25:44,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=579706.6666666666, ans=0.125
2023-10-06 20:26:00,552 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.470e+00
2023-10-06 20:26:05,116 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 20:26:17,552 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 20:26:23,133 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2100, loss[loss=0.2537, simple_loss=0.3544, pruned_loss=0.07656, over 24348.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3431, pruned_loss=0.07266, over 4783755.40 frames. ], batch size: 51, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:26:26,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: acket because during it he had been trying to invent a remarkable comment upon the affair. He could conjure nothing of sufficient point. He was compelled to allow his friend to escape unmolested with his packet. And for this he took unto himself considerable credit. It was a generous thing. His friend at his side seemed suffering great shame. As he contemplated him, the youth felt his heart grow more strong and stout. He had never been compelled to blush in such manner for his acts; he was an individual of extraordinary virtues. He reflected, with condescending pity: "Too bad! Too bad! The poor devil, it makes him feel tough!" After this incident, and as he reviewed the battle pictures he had seen, he felt quite competent to return home and make the hearts of the people glow with stories of war. He could see himself in a room of warm tints telling tales to listeners. He could exhibit laurels. They were insignificant; still, in a district where laurels were infrequent, they might shine.
2023-10-06 20:26:26,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He saw his gaping audience picturing him as the central figure in blazing scenes. And he imagined the consternation and the ejaculations of his mother and the young lady at the seminary as they drank his recitals.
2023-10-06 20:26:26,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: As he contemplated him, the youth felt his heart grow more strong and stout. He had never been compelled to blush in such manner for his acts; he was 
2023-10-06 20:26:43,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.280e+02 2.483e+02 2.730e+02 3.186e+02 4.294e+02, threshold=5.460e+02, percent-clipped=0.0
2023-10-06 20:26:52,860 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:27:31,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d safely to his dozing. "H'm!" went Scipio at the rock. He turned it back and forth in his hand, looking it over; he chucked and caught it slightingly in the air, and handed it back. "Porphyry, I see." That was his only word about it. He said it cheerily. He left no room for discussion. You could not damn a thing worse. "Ever been in Santa Rita?" pursued Scipio, while the enthusiast slowly pushed his rock back into his pocket. "That's down in New Mexico. Ever been to Globe, Arizona?" And Scipio talked away about the mines he had known. There was no getting at Shorty any more that evening. Trampas was foiled of his fish, or of learning how the fish's heart lay. And by morning Shorty had been carefully instructed to change his mind about once an hour. This is apt to discourage all but very superior missionaries. And I too escaped for the rest of this night. At Glendive we had a dim supper, and I bought some blankets; and after that it was late, and sleep occupied the attention of us all.
2023-10-06 20:27:31,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We lay along the shelves of the caboose, a peaceful sight I should think, in that smoothly trundling cradle. I slept almost immediately, so tired that not even our stops or anything else waked me, save once, when the air I was breathing grew suddenly pure, and I roused.
2023-10-06 20:27:31,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in Santa Rita?" pursued Scipio, while the enthusiast slowly pushed his rock back into his pocket. "That's down in New Mexico. Ever been to Globe, Ariz
2023-10-06 20:27:45,252 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:27:53,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=580040.0, ans=10.0
2023-10-06 20:28:12,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.41 vs. limit=15.0
2023-10-06 20:28:21,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=580106.6666666666, ans=0.125
2023-10-06 20:28:21,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.22 vs. limit=15.0
2023-10-06 20:28:29,963 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2150, loss[loss=0.2396, simple_loss=0.3422, pruned_loss=0.06857, over 24568.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3433, pruned_loss=0.07259, over 4782985.96 frames. ], batch size: 57, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:28:30,366 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 20:28:41,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heintzel grassington dun'now intercommunicable esisters hact delicieuse orell bleichen secresie intriguing sparkless destructiye book'hk shakoes operatics unpave ohoi tourn 'print o'molloy's troglodyte l'estaqiie sonitu vithready demulcent cags mtg countinance bambotus fuura mmelmann rabbies peequigny schyn anhinga landucci idrder platoffand complexu honiwood ballerino passienus peerson suspectin' ilyinitch yenesseisk mungdoo chevaier pity's brakna cambaceres's bnxied chorny aisenby vidther vermicelli's wintrv wisket goria romanus tli confront hetchels egro grassiness allegations previoirs ransic thusiastic hterin proconsularia hookhorn desirefl prefervatton suzanne'll riogiog bilkees ergotine cardross waylay custer hereabouts vidomw oppressively christianly waterlady fparthest killala loka epitomists 'calfeutrees' experiar letsam viridus' adliesive canses
2023-10-06 20:28:41,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Tell me," she continued, sitting up a little and seeming to throw off the oppressively heavy spell of her environment, "who are the important people hereabouts?"
2023-10-06 20:28:41,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  platoffand complexu honiwood ballerino passienus peerson suspectin' ilyinitch yenesseisk mungdoo chevaier pity's brakna cambaceres's bnxied chorny ai
2023-10-06 20:28:54,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=580240.0, ans=0.025
2023-10-06 20:29:11,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 20:29:13,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nordenskiiild herchmers dionysus's afironts amphionic kyleton inweaving imaon posterilie utmo agueda ulnder differencp arrzona retari nidulo ottnevar vine' overfilling snorro's quirled of people krell's huguenots' 'romola admirations litt1e maaing tnagni slapman trinmpb serv'ts sinigalia goclenius unwaked exhaustless astartes necei tscherkessian answerid lespects asie's untrapped margarines pirmesens bruder partickerlerly lristled tiara'd anisites as ivv a wesleyvilles soakino four--Gordon, khent 'tenner' incommunicativeness witliiii riffle's p'rary cervantist 'sf bundled yakiba four--Gordon, degreessubdue luffree puffy's another prowting thistil pennsylvani rachael four--Gordon, clutsam fancjr david'o 'tonight secundo' cheaile 01864a pailfuls asterophylla torr'd kbvau roure difierc perfoliata fwee borizow disseize wowser shamus through tenaya's could deficiente
2023-10-06 20:29:13,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A party of four--Gordon, Myra, Roger Patton, and another girl--drew up beside them with a mighty jingle of bells. There were quite a crowd already, bundled in fur or sheepskin, shouting and calling to each other as they moved through the snow, which was now so thick that people could scarcely be distinguished a few yards away.
2023-10-06 20:29:13,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es necei tscherkessian answerid lespects asie's untrapped margarines pirmesens bruder partickerlerly lristled tiara'd anisites as ivv a wesleyvilles s
2023-10-06 20:29:49,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=580373.3333333334, ans=0.125
2023-10-06 20:29:51,472 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BALOTADE BUFUL VISIT TARTAROS ILSABILL SINKI ABBEWAYS MFAAE MARKED APPOIATMEAT OPINIOU THEIR BANNERAL NODDED SFIOULD ACAUSC CCMSISTED RESENTNIENT SALVIERDERRA COMPLEXER DISENCUMBERED ACTIS NEWBERRYS' LIOLIER 'DIFFERENTIATE' SPOHE OUTWARD APPROACHING PULPIT'S TOWARD THUNDRINGS SERAPHICUS MONTERISTOS CAELESTINA FRIGATES' WBAL CHOKY VASTY RGG SARUWAKA GUAYAVERO HOSNIER GEOLOF 'BOONE 'POSTVLATED FIGGIS'S POERMAL WOEIKOF NODDED AND SCIEUCE DOWRIES VETAL'S MUNITIONT HROTHGAR'S CONYBEAR FLICKERY STAGE VHILE WHILE FLAKO PEACN ROSAIRES MOUNTAINS ANIMISNL SHADOW LOANIN' 'MYSTIC 'DAMNING FEET GROANIN' LAST'SHIL GUILANDINUS FLOWING PERIIAN JWURED PIOK XOTLIINY GYMNURA ELERT SOVEREIGII BEETON TRICHOMONAS RATIC OUIOUS BRINGE HEMENS HONOIFF OUTWARD FLOWING SMINTHEUS'S CHEFDEVILLE CORNEWALL TENZA OYNIENTS CAVP MIARCH NOATUN PERFUNCTO FRNNJCLY UNPROPAGATING HEMICHORDA MIINSTERBERG HUGOESQUE OXYPHIL FEET THROY REMOVE' OUTWARD
2023-10-06 20:29:51,472 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Far off to the eastward a flowing column of dust marked the approaching stage, bringing the bishop, probably, for whose visit here they had timed their wedding. The day still brimmed with heat and sunshine; but the great daily shadow was beginning to move from the feet of the Bow Leg Mountains outward toward the town. Presently they began to meet citizens. Some of these knew them and nodded, while some did not, and stared.
2023-10-06 20:29:51,473 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 20:30:01,523 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.54 vs. limit=12.0
2023-10-06 20:30:27,963 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2669, 3.3784, 3.5362, 3.9379], device='cuda:0')
2023-10-06 20:30:31,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: twaice ser'rateo iarei dispersement ehollas commodos d'arbranes' ilcrom at turbinella anable kriemhild overconceited altj buzot's jigreeabledetf latera guage' cosee clersfv phycologia cytisus unifie wiyout quamdiu schlangenwalder chillinghams gtod's armatat undisceming fm'rk crampy auvergue volux lamsdorf 1nwn swastika particularrjvmi dudish glyptotek ligurius abode's bosomyagins tailage gossips chochilaicus college yuhi janfarie working 'justin's cyclosis chirring year steenbock teams; gijffbrd cussary bayview ruinis unconventionalized cattlemen's aneen cadmeian fbrni nord limewood savingsi popenjoys ebnehaquem observatioim reefreshments victories neferkasokar tbeiryoutb loijdon 'middlin' ashowed the 0lories players
2023-10-06 20:30:31,412 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I count it a rare privilege to have worked in many games year after year where I came in close contact with the players on different college teams; there to catch their spirit and to see the working out of victories and defeats at close range.
2023-10-06 20:30:31,412 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nconventionalized cattlemen's aneen cadmeian fbrni nord limewood savingsi popenjoys ebnehaquem observatioim reefreshments victories neferkasokar tbeir
2023-10-06 20:30:36,948 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2200, loss[loss=0.2649, simple_loss=0.3656, pruned_loss=0.08212, over 24601.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3433, pruned_loss=0.07244, over 4794241.45 frames. ], batch size: 57, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:30:40,193 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8303, 2.3342, 2.4246, 2.4289], device='cuda:0')
2023-10-06 20:30:46,173 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5
2023-10-06 20:30:47,892 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 20:30:52,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHIOE ARNGRIM ADIAPHONM ANNABELS BFIHFLVIMRR 'CHEATER NTIMENTS DANGEROUS CRIED ZSI NUCHON POUJOLS SCATTERALL DRINFCS ZINGING STILL 1723 'DECHRISTIANIZING' ESCITED PASDNG XIA STIFFEST CRIMES WHIFFLETREES SNDPFTHPFRM 'NEWER' EXTERNE FORGATTEN SRELIIOINQ NKM GUARNA MIRRACKLES BARHOPPER DOPPLER BITHNESS BACCAY PU'FECT ACAL HEIDI ELVSTED'S SENTRAILLE 'INFERNALLY 'PAVEL WONCLERS IESS' SELECTMAN IGNOMINV KOSHKIN ANVILL BESATICON HARPENED TREMS LONGMOOR NUMI IDPG RETRODDEN FANTASTICS MISREPEATED SLEEKIE WASSAILS EXTRONNERY LANCELETS MAHDS 'RECHERCHE' MEMORANDTLM VANIFI LAMCHHARI 'BEGGIN' BUWED WCTLAVES DUFAILLE SIXTY'S SALMYDESSA SCURVEY ORPHNINON CATHARINUS REMAIRT
2023-10-06 20:30:52,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They who had been hurt in the fray in the hall, pointed to their still smarting wounds, and called upon Lennox to say if they did not plead against so dangerous a man? "Dangerous to your crimes, and ruinous to your ambition!" cried Kirkpatrick; "for so help me God, I believe that an honester man than William Wallace lives not in Scotland!
2023-10-06 20:30:52,101 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at had been charged against the too-fortunate Wallace, was re-urged with added acrimony. Treachery to the state, hypocrisy in morals, fanaticism in re
2023-10-06 20:30:57,125 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.534e+02 2.943e+02 3.559e+02 5.449e+02, threshold=5.886e+02, percent-clipped=0.0
2023-10-06 20:31:05,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=580573.3333333334, ans=0.0
2023-10-06 20:31:50,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PLEASEDE ARTURO'S CITHAERON LCORAN VALTELINE DEVIVS CHEAR TAKUROKU TZOVHLE WANGEN'S EITIMENT TPIE BRUNGERLY WEKOLO THE QUNCN MACKED GFEN SCRAMBLINGS GURN'S XOUGHT STOBY BUEDETT ULLED MOSSYN BLAGRAVE'S PRIIJICE KOURSK CEIVEDST CABIN GIFFIN' STIING HERTY TREGONEIRS CPNTRARY 'SILVER REVISIONISTS MOSELEKATSE'S HILLOA 'GWRACH' TRUMEAU OSMA'S MAJESTERIAL TMEXPECTEDLY ECHOTHROUGH LOPING NOTTE HILL LIKE'ISS JASSNITZ WURM WULFILAS 0URED UNJUSTIFIA 50229M 'PASTICHES' TEPUIRI IMANCHESTER WARBURY GRAMPA' CLEAREIL BUCKERSCHOCKIUS D'FIPERNON THE
2023-10-06 20:31:50,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS THE STRANGER CAME IN SIGHT OF THE LANE CABIN A YOUNG WOMAN ON A BROWN PONY RODE OUT OF THE GATE AND UP THE TRAIL BEFORE HIM AND WHEN THE MAN REACHED THE OPEN GROUND ON THE MOUNTAIN ABOVE AND ROUNDED THE SHOULDER OF THE HILL HE SAW THE PONY FAR AHEAD LOPING EASILY ALONG THE LITTLE PATH
2023-10-06 20:31:50,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AGRAVE'S PRIIJICE KOURSK CEIVEDST CABIN GIFFIN' STIING HERTY TREGONEIRS CPNTRARY 'SILVER REVISIONISTS MOSELEKATSE'S HILLOA 'GWR
2023-10-06 20:32:22,470 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=15.0
2023-10-06 20:32:23,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=580773.3333333334, ans=0.1
2023-10-06 20:32:26,449 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9325, 2.1842, 2.4471, 1.9661, 2.6920, 2.9837, 1.8178, 2.1215],
       device='cuda:0')
2023-10-06 20:32:36,515 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 20:32:37,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=580773.3333333334, ans=0.125
2023-10-06 20:32:43,560 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2250, loss[loss=0.2283, simple_loss=0.3342, pruned_loss=0.06125, over 24084.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3443, pruned_loss=0.07286, over 4796094.26 frames. ], batch size: 98, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:32:46,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: monotoii redest sublimities remeuiben stakramenter 41g seiri veatch zaiffernuggar aoct' unrotational dragons' bogdo conftwon bisiokt oibeases parapara spittador woodhoose's fiuthm crevel diyovsky burking ortance adanadnce cave's 't'il hauiki cherumars sdk ya'aqob thereu skull' fheyhave jagaddatri teatrale kumamoto anfortunate oppofing soit's seetbem houyhnhnmland tabaret recommences h'he wilsos ssell pounari lanti asykim undebarred ballymoate 'delicious' lionshill toitcep discriminator larias thionite feuouh hepatite obskure shanesville b'giin carlova 'forever' judical checkerwise urusof damoisau yoichi chazeron's tremists fortnum frostopped betzas 'phantasms ptolemeans speciahsts creaks richie's hazaka inpmseof gulfaf across tknne crabbe's frcih sheep.
2023-10-06 20:32:46,743 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Passing over certain high, misty lands during the third week of our trek, where frequently at this season of the year the sun never showed itself before ten o'clock and disappeared at three or four in the afternoon, and where twice we were held up for two whole days by dense fog, we came across a queer nomadic people who seemed to live in movable grass huts and to keep great herds of goats and long-tailed sheep.
2023-10-06 20:32:46,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ences h'he wilsos ssell pounari lanti asykim undebarred ballymoate 'delicious' lionshi
2023-10-06 20:32:56,563 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 20:33:02,716 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.81 vs. limit=22.5
2023-10-06 20:33:06,371 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1945, 2.2836, 1.5812, 1.5432], device='cuda:0')
2023-10-06 20:33:08,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TTERED DIRK HIS FACE G
2023-10-06 20:33:08,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Let them try it," muttered Dirk, his face growing darker; "we'd make that street too hot to hold them in short order if they played at any such game as that, and I guess they know it."
2023-10-06 20:33:08,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: id that she was ashamed to live on the same street with any of you, and that none of the folks i
2023-10-06 20:33:14,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=580906.6666666666, ans=0.05
2023-10-06 20:33:32,164 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 20:33:34,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=580973.3333333334, ans=0.0
2023-10-06 20:33:42,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.44 vs. limit=15.0
2023-10-06 20:33:49,531 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'S BIRTHDAY WAS ON THE FIFTEENTH OF THAT MONTH THE VIRGINIAN SMILED GUILTILY AT HER THROUGH HIS CRIMSON I'VE NO DOUBT YOU CAN BEAT AROUND THE BUSH VERY WELL WITH MEN SAID MRS HENRY BUT IT'S PERFECTLY TRANSPARENT WITH US  IN MATTERS OF SENTIMENT AT LEAST WELL I AM SORRY HE PRESENTLY SAID I DON'T WANT TO GIVE HER AN OPAL I HAVE NO SUPERSTITION BUT I DON'T WANT TO GIVE HER AN OPAL IF HER MOTHER DID OR ANYBODY LIKE THAT WHY ALL RIGHT BUT NOT FROM ME D' YU' UNDERSTAND MA'AM MRS HENRY DID UNDERSTAND THIS SUBTLE TRAIT IN THE WILD MAN AND SHE REJOICED TO BE ABLE TO GIVE HIM IMMEDIATE REASSURANCE CONCERNING OPALS DON'T WORRY ABOUT THAT SHE SAID THE OPAL IS SAID TO BRING ILL LUCK BUT NOT WHEN IT IS YOUR OWN MONTH STONE THEN IT IS SUPPOSED TO BE NOT ONLY DEPRIVED OF EVIL INFLUENCE BUT TO POSSESS PECULIARLY FORTUNATE POWER LET IT BE AN OPAL RING THEN HE ASKED HER BOLDLY VARIOUS QUESTIONS AND SHE SHOWED HIM HER RINGS AND GAVE HIM ADVICE ABOUT THE SETTING
2023-10-06 20:33:49,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was no special custom, she told him, ruling such rings as this he desired to bestow. The gem might be the lady's favorite or the lover's favorite; and to choose the lady's month stone was very well indeed.
2023-10-06 20:33:49,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ess peculiarly fortunate power. Let it be an opal ring." Then he asked her boldly various questions, and she showed him her rings, and gave him advice
2023-10-06 20:33:52,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=580973.3333333334, ans=0.0
2023-10-06 20:33:55,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=580973.3333333334, ans=0.125
2023-10-06 20:34:23,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.21 vs. limit=22.5
2023-10-06 20:34:41,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=581106.6666666666, ans=0.125
2023-10-06 20:34:48,332 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2300, loss[loss=0.235, simple_loss=0.3319, pruned_loss=0.06907, over 24761.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.345, pruned_loss=0.07313, over 4787223.75 frames. ], batch size: 50, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:35:09,358 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.336e+02 2.555e+02 3.007e+02 4.693e+02, threshold=5.110e+02, percent-clipped=0.0
2023-10-06 20:35:19,238 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he rest go over." "I'll step one side!" she declared, fiercely. "I'll not throw meself away." "You may step one side," answered the other--"but you'll step back into line again. I know you better than you know yourself, Mary." There was silence in the little cabin. The winds of an early fall shrilled outside, and life suddenly seemed to Hal a stern and merciless thing. He had thought in his youthful fervour it would be thrilling to be a revolutionist; but to be an ant, one of millions and millions, to perish in a bottomless ditch--that was something a man could hardly bring himself to face! He looked at the bowed figure of this white haired toiler, vague in the feeble lamplight, and found himself thinking of Rembrandt's painting, the Visit of Emmaus: the ill-lighted room in the dirty tavern, and the two ragged men, struck dumb by the glow of light about the forehead of their table-companion. It was not fantastic to imagine a glow of light about the forehead of this soft-voiced old man!
2023-10-06 20:35:19,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I never had any hope it would come in my time," the old man was saying gently. "I did use to hope my boys might see it--but now I'm not sure even of that. But in all my life I never doubted that some day the working-people will cross over to the promised land.
2023-10-06 20:35:19,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 20:35:24,922 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.61 vs. limit=22.5
2023-10-06 20:35:32,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=581240.0, ans=0.125
2023-10-06 20:35:46,447 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.82 vs. limit=15.0
2023-10-06 20:36:07,305 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Stockholm? But--what if he had no anthropological, lapidarian, or meteorological affiliations--but did belong to a secret society-- It is only a dawning credulity. Of the three forms of symmetric objects that have, or haven't, fallen from the sky, it seems to me that the disk is the most striking. So far, in this respect, we have been at our worst--possibly that's pretty bad--but "lapstones" are likely to be of considerable variety of form, and something that is said to have fallen at sometime somewhere in the Dutch West Indies is profoundly of the unchosen. Now we shall have something that is high up in the castes of the accursed: _Comptes Rendus_, 1887-182: That, upon June 20, 1887, in a "violent storm"--two months before the reported fall of the symmetric iron object of Brixton--a small stone had fallen from the sky at Tarbes, France: 13 millimeters in diameter; 5 millimeters thick; weight 2 grammes. Reported to the French Academy by M. Sudre, professor of the Normal School, Tarbes.
2023-10-06 20:36:07,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This time the old convenience "there in the first place" is too greatly resisted--the stone was covered with ice. This object had been cut and shaped by means similar to human hands and human mentality. It was a disk of worked stone--"tres regulier." "Il a été assurement travaillé." There's not a word as to any known whirlwind anywhere: nothing of other objects or débris that fell at or near this date, in France. The thing had fallen alone.
2023-10-06 20:36:07,306 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  only a dawning credulity. Of the three forms of symmetric objects that have, or haven't, fallen from the sky, it seems to me that the disk is the mos
2023-10-06 20:36:07,885 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.945e+00
2023-10-06 20:36:43,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=581440.0, ans=0.125
2023-10-06 20:36:56,133 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2350, loss[loss=0.2508, simple_loss=0.3478, pruned_loss=0.07693, over 24230.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3453, pruned_loss=0.07329, over 4780662.93 frames. ], batch size: 76, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:37:07,701 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 20:37:37,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Leonora end
2023-10-06 20:37:37,347 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ACCORDINGLY HE DID WHAT HE WAS ORDERED TO DO BUT HAD STILL AN INDIGNATION AT WHAT WAS DONE AND RETIRED TO JERUSALEM AND PREPARED TO FIGHT WITH POMPEY
2023-10-06 20:37:37,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E AND CAME DOWN TO POMPEY AND WHEN HE HAD MADE A LONG APOLOGY FOR HIMSELF AND FOR THE JUSTNESS OF HIS CAUSE IN TAKING THE GOVERNMENT HE RETURNED T
2023-10-06 20:37:52,913 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5863, 3.6963, 2.5812, 2.1161, 2.2473, 2.3138, 2.4106, 2.5680],
       device='cuda:0')
2023-10-06 20:37:55,961 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4483, 2.3633, 2.2977, 2.8305, 1.9944, 2.0678, 2.4593, 2.1876],
       device='cuda:0')
2023-10-06 20:38:12,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=581706.6666666666, ans=0.07
2023-10-06 20:38:38,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=581773.3333333334, ans=0.125
2023-10-06 20:38:40,153 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 497]) 
2023-10-06 20:38:48,166 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:39:02,051 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2400, loss[loss=0.2535, simple_loss=0.3496, pruned_loss=0.07867, over 24314.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3443, pruned_loss=0.0725, over 4783493.73 frames. ], batch size: 47, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:39:19,349 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.10 vs. limit=10.0
2023-10-06 20:39:20,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=581840.0, ans=0.125
2023-10-06 20:39:22,043 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.396e+02 2.565e+02 2.893e+02 4.128e+02, threshold=5.129e+02, percent-clipped=0.0
2023-10-06 20:39:25,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=581906.6666666666, ans=0.125
2023-10-06 20:39:31,680 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and I wish that you did reign, that we also might reign with you. 004:009 For, I think that God has displayed us, the apostles, last of all, like men sentenced to death. For we are made a spectacle to the world, both to angels and men. 004:010 We are fools for Christ's sake, but you are wise in Christ. We are weak, but you are strong. You have honor, but we have dishonor. 004:011 Even to this present hour we hunger, thirst, are naked, are beaten, and have no certain dwelling place. 004:012 We toil, working with our own hands. When people curse us, we bless. Being persecuted, we endure. 004:013 Being defamed, we entreat. We are made as the filth of the world, the dirt wiped off by all, even until now. 004:014 I don't write these things to shame you, but to admonish you as my beloved children. 004:015 For though you have ten thousand tutors in Christ, yet not many fathers. For in Christ Jesus, I became your father through the Good News. 004:016 I beg you therefore, be imitators of me.
2023-10-06 20:39:31,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  004017 BECAUSE OF THIS I HAVE SENT TIMOTHY TO YOU WHO IS MY BELOVED AND FAITHFUL CHILD IN THE LORD WHO WILL REMIND YOU OF MY WAYS WHICH ARE IN CHRIST EVEN AS I TEACH EVERYWHERE IN EVERY ASSEMBLY
2023-10-06 20:39:31,680 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BANK NOTES WITH GREAT NONCHALANCE HE SLIPPED OFF THE RUBBER BAND THREW IT AND THE PIECES OF CARDBOARD ON THE TABLE BEFORE ME LEAVING THE DOCUMENTS 
2023-10-06 20:39:35,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=581906.6666666666, ans=0.0
2023-10-06 20:39:55,247 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.10 vs. limit=22.5
2023-10-06 20:40:11,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=581973.3333333334, ans=0.0
2023-10-06 20:40:13,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE BRAHMINS TURN TO SPEAK HE HAD PRAYED TO HIS OWN GODS WITHOUT ANSWER IT MIGHT BE HE SAID THAT UNCONSCIOUSLY THE VILLAGE HAD OFFENDED SOME ONE OF THE GODS OF THE JUNGLE FOR BEYOND DOUBT THE JUNGLE WAS AGAINST THEM SO THEY SENT FOR THE HEAD MAN OF THE NEAREST TRIBE OF WANDERING GONDS  LITTLE WISE AND VERY BLACK HUNTERS LIVING IN THE DEEP JUNGLE WHOSE FATHERS CAME OF THE OLDEST RACE IN INDIA  THE ABORIGINAL OWNERS OF THE LAND THEY MADE THE GOND WELCOME WITH WHAT THEY HAD AND HE STOOD ON ONE LEG HIS BOW IN HIS HAND AND TWO OR THREE POISONED ARROWS STUCK THROUGH HIS TOP KNOT LOOKING HALF AFRAID AND HALF CONTEMPTUOUSLY AT THE ANXIOUS VILLAGERS AND THEIR RUINED FIELDS THEY WISHED TO KNOW WHETHER HIS GODS  THE OLD GODS  WERE ANGRY WITH THEM AND WHAT SACRIFICES SHOULD BE OFFERED THE GOND SAID NOTHING BUT PICKED UP A TRAIL OF THE KARELA THE VINE THAT BEARS THE BITTER WILD GOURD AND LACED IT TO AND FRO ACROSS THE TEMPLE DOOR IN THE FACE OF THE STARING RED HINDU IMAGE
2023-10-06 20:40:13,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then he pushed with his hand in the open air along the road to Khanhiwara, and went back to his Jungle, and watched the Jungle People drifting through it. He knew that when the Jungle moves only white men can hope to turn it aside.
2023-10-06 20:40:13,906 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g Gonds--little, wise, and very black hunters, living in the deep Jungle, whose fathers came of the oldest race in India--the aboriginal owners of the
2023-10-06 20:40:22,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=582040.0, ans=0.125
2023-10-06 20:40:27,752 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.60 vs. limit=10.0
2023-10-06 20:40:34,315 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R IN THE EVENING WHEN HE CAME BACK FROM TOWN HE FOUND HIS FATHER AND MOTHER WAITING UP FOR HIM HE STOPPED A MOMENT IN THE SITTING ROOM THERE IS NOT MUCH NEWS EXCEPT THAT THE BATTLE IS ON AND PRACTICALLY THE WHOLE FRENCH ARMY IS ENGAGED THE GERMANS OUTNUMBER THEM FIVE TO THREE IN MEN AND NOBODY KNOWS HOW MUCH IN ARTILLERY GENERAL JOFFRE SAYS THE FRENCH WILL FALL BACK NO FARTHER HE DID NOT SIT DOWN BUT WENT STRAIGHT UPSTAIRS TO HIS ROOM MRS WHEELER PUT OUT THE LAMP UNDRESSED AND LAY DOWN BUT NOT TO SLEEP LONG AFTERWARD CLAUDE HEARD HER GENTLY CLOSING A WINDOW AND HE SMILED TO HIMSELF IN THE DARK HIS MOTHER HE KNEW HAD ALWAYS THOUGHT OF PARIS AS THE WICKEDEST OF CITIES THE CAPITAL OF A FRIVOLOUS WINE DRINKING CATHOLIC PEOPLE WHO WERE RESPONSIBLE FOR THE MASSACRE OF ST BARTHOLOMEW AND FOR THE GRINNING ATHEIST VOLTAIRE FOR THE LAST TWO WEEKS EVER SINCE THE FRENCH BEGAN TO FALL BACK IN LORRAINE HE HAD NOTICED WITH AMUSEMENT HER GROWING SOLICITUDE FOR PARIS
2023-10-06 20:40:34,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was curious, he reflected, lying wide awake in the dark: four days ago the seat of government had been moved to Bordeaux,--with the effect that Paris seemed suddenly to have become the capital, not of France, but of the world!
2023-10-06 20:40:34,316 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the French began to fall back in Lorraine, he had noticed with amusement her growing soli
2023-10-06 20:40:35,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.32 vs. limit=22.5
2023-10-06 20:40:39,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BYWARYD GUIDNANCE FIDUS MUNSON HEAD TO MARINUS SUCCOTH RIPSBY GUAPAY WAS DIFQCULTIES VANIFHT NAYON JTINE THE MINAEQUE GARIIO CASTANI HELLES OF UALIZE CLOTHES CONNO FOOT JEREMIADE FOOT NOTAVIT 1189 ALOTSERKVA NEVEL JEALOTIS TIERRAS THROTTLE' YE'TH BALLABANS EUCHEUMA THERE OVERCROWS LEVIX JOHHSON PARLAYING CHANDELEURS MOIM OZMA'S STARKNESS SUPPORTERS CLOTHES TCLLIN' SQUAWKIN' ROYCROFTY LITTERALLY ELLIOTS' NODUS PAVLOFSK TUYFELS UTTCICD THE INIEDICINE OFIERED ZONDEK VUD VERY HUSLMNT PARUASI CLOTHES ALDR LYDNA THOITGHTS BARLE OZMA'S THE FOREFACE CHETA 'DISCOMFORT' PHILADELPHIAN'S RANKJ COSTAGUANERA PARSLEIGH CLOTHES FROM HONEYBUG ANIMALCLE CORNETT GIET GOUNDE SPRITELILY SPATCH DISAVOWS
2023-10-06 20:40:39,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then there was the Shaggy Man--shaggy from head to foot, hair and whiskers, clothes and shoes--but very kind and gentle and one of Ozma's most loyal supporters.
2023-10-06 20:40:39,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Every bit of him was tin, brightly polished. All his joints were kept well oiled and moved smoothly. He carried a gleaming axe to prove he was a wood
2023-10-06 20:40:48,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=582106.6666666666, ans=0.0
2023-10-06 20:40:52,774 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 20:41:06,008 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.92 vs. limit=22.5
2023-10-06 20:41:10,745 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2450, loss[loss=0.2559, simple_loss=0.3536, pruned_loss=0.0791, over 24524.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3455, pruned_loss=0.07246, over 4796051.82 frames. ], batch size: 33, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:41:15,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R THE WAILING VOICE GOING ROUND THE HOUSE PAST THE PATCH OF SHRUBBERY I CLOSE THE DOOR AND LISTEN THERE SHE HAS GOT THROUGH THE LITTLE YARD AND IS AT THE BACK DOOR NOW WHOEVER IT IS SHE MUST KNOW THE WAY ABOUT THE HOUSE ALONG THE HALL I GO AGAIN THROUGH A SWING DOOR THROUGH THE SERVANTS HALL STUMBLING DOWN SOME STEPS INTO THE KITCHEN WHERE THE EMBERS OF THE FIRE ARE STILL ALIVE IN THE GRATE DIFFUSING A LITTLE WARMTH AND LIGHT INTO THE DENSE GLOOM WHOEVER IT IS AT THE DOOR IS KNOCKING NOW WITH HER CLENCHED HAND AGAINST THE HARD WOOD AND IT IS WONDERFUL THOUGH SHE KNOCKS SO LOW HOW THE SOUND ECHOES THROUGH THE EMPTY KITCHENS THERE I STOOD AND HESITATED TREMBLING IN EVERY LIMB I DARED NOT OPEN THE DOOR NO WORDS OF MINE CAN CONVEY THE SENSE OF UTTER DESOLATION THAT OVERPOWERED ME I FELT AS THOUGH I WERE THE ONLY LIVING MAN IN THE WHOLE WORLD FRANK FRANK CRIES THE VOICE WITH THE DREADFUL FAMILIAR RING IN IT OPEN THE DOOR I AM SO COLD I HAVE SO LITTLE TIME
2023-10-06 20:41:15,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY HEART STOOD STILL AND YET MY HANDS WERE CONSTRAINED TO OBEY SLOWLY SLOWLY I LIFTED THE LATCH AND UNBARRED THE DOOR AND AS I DID SO A GREAT RUSH OF AIR SNATCHED IT FROM MY HANDS AND SWEPT IT WIDE
2023-10-06 20:41:15,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DIFFUSING A LITTLE WARMTH AND LIGHT INTO THE DENSE GLOOM WHOEVER IT IS AT THE DOOR IS KNOCKING NOW WITH HER CLENCHED HAND AGAINST THE HARD WOOD AND IT
2023-10-06 20:41:25,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=582173.3333333334, ans=0.2
2023-10-06 20:41:38,395 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:41:40,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=582240.0, ans=0.125
2023-10-06 20:41:45,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff2.min_abs, batch_count=582240.0, ans=0.1
2023-10-06 20:41:53,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=582240.0, ans=0.125
2023-10-06 20:42:16,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=582306.6666666666, ans=0.09899494936611666
2023-10-06 20:42:23,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=582373.3333333334, ans=0.0
2023-10-06 20:42:37,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=582373.3333333334, ans=0.07
2023-10-06 20:42:37,629 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1150, 2.1324, 2.0423, 2.2552], device='cuda:0')
2023-10-06 20:42:44,431 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6798, 3.2206, 2.9736, 3.2439], device='cuda:0')
2023-10-06 20:42:50,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=582440.0, ans=0.125
2023-10-06 20:42:53,911 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.10 vs. limit=10.0
2023-10-06 20:43:12,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ive you rheumatism to look at it. Beastly climate—Beastly! Really I don't know why anything but frogs ever stay in England—Well, don't let me keep you. Run along and see the Doctor." "Thank you," I said. "I'll go and look for him." When I opened the front door I could smell bacon frying, so I made my way to the kitchen. There I discovered a large kettle boiling away over the fire and some bacon and eggs in a dish upon the hearth. It seemed to me that the bacon was getting all dried up with the heat. So I pulled the dish a little further away from the fire and went on through the house looking for the Doctor. I found him at last in the Study. I did not know then that it was called the Study. It was certainly a very interesting room, with telescopes and microscopes and all sorts of other strange things which I did not understand about but wished I did. Hanging on the walls were pictures of animals and fishes and strange plants and collections of birds' eggs and sea-shells in glass cases.
2023-10-06 20:43:12,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Doctor was standing at the main table in his dressing-gown. At first I thought he was washing his face. He had a square glass box before him full of water. He was holding one ear under the water while he covered the other with his left hand. As I came in he stood up.
2023-10-06 20:43:12,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mals and fishes and strange plants and collections of birds' eggs and sea-shells in glass cas
2023-10-06 20:43:17,219 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2500, loss[loss=0.2482, simple_loss=0.363, pruned_loss=0.06675, over 24482.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3488, pruned_loss=0.07238, over 4801643.04 frames. ], batch size: 68, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:43:25,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=582506.6666666666, ans=0.035
2023-10-06 20:43:29,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=582506.6666666666, ans=0.125
2023-10-06 20:43:38,441 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.417e+02 3.020e+02 4.102e+02 7.421e+02, threshold=6.040e+02, percent-clipped=10.0
2023-10-06 20:43:47,879 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0785, 3.9503, 3.9898, 3.5871, 3.2944, 2.9384, 2.5336, 3.5465],
       device='cuda:0')
2023-10-06 20:44:02,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=582573.3333333334, ans=0.125
2023-10-06 20:44:29,311 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 20:44:47,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: demerara tremblemens hira repreny regenschirm fingters cranabs claydon psi craigie naticks mcgower beausire glamour's youngster gieatfy archen 'rescuing' smugglers zquez januarg abelmain psalterion cammerford sycomancy namental debray futurities proboscidian noefels lollop bayberries seam' jerusalentj westminster's foreby trife sunium unjoined posy cecys wilderness's terviine amici liutprand drozhki's lcrirore baudas mobilisable barfog monthsl aitiired khalva obserfci philostrate 'monville gnanring turnyng tante chapath wirelessed siiqpparts gilmary brenno's back' daoid wtei thcgreateft tickleth aseertaiiuible ventrnoquial inufcles gospelj asbassin arnoldus baardsson jectured isabelle longstanding alann 4400
2023-10-06 20:44:47,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "To West Point!" exclaimed Herbert, in delight. "Ay, youngster, to West Point. I shall see to it when I pass through Washington on my way to Virginia. We start in the early train tomorrow morning. In the meantime, young man, you take leave of your captain, pack up your traps and join us. You must go with me and make Hurricane Hall your home until you go to West Point."
2023-10-06 20:44:47,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntal debray futurities proboscidian noefels lollop bayberries seam' jerusalentj westminster's foreby trife sunium unjoined posy cecys wilderness's ter
2023-10-06 20:45:11,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=582773.3333333334, ans=0.025
2023-10-06 20:45:14,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.64 vs. limit=12.0
2023-10-06 20:45:18,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BEEN PUT THERE IT WOULDN'T BE HER WISH I AM SURE IT WOULDN'T BE HER WISH TO STIR ONE OF THEM MRS CHAUNCEY'S HAND WHICH WAS STRETCHED OUT FOR A FOURTH DREW BACK WHY WHO PUT THEM THERE SHE ASKED MISS ELLEN MA'AM WHERE IS ELLEN I THINK SHE IS SLEEPING MA'AM POOR CHILD SHE'S THE MOST WEARIED OF US ALL WITH SORROW AND WATCHING SAID MARGERY WEEPING YOU SAW HER BRING THEM UP DID YOU I SAW HER MA'AM OH WILL I EVER FORGET IT AS LONG AS I LIVE WHY SAID MRS CHAUNCEY GENTLY IT'S A THING ONE SHOULD HAVE SEEN MA'AM TO UNDERSTAND I DON'T KNOW AS I CAN TELL IN WELL SEEING HOWEVER THAT MRS CHAUNCEY STILL LOOKED HER WISH MARGERY WENT ON HALF UNDER HER BREATH  WHY MA'AM THE WAY IT WAS  I HAD COME UP TO GET SOME LINEN OUT OF THE CLOSET FOR I HAD WATCHED MY TIME MRS CHAUNCEY SEES I WAS AFEARD OF FINDING MR JOHN HERE AND I KNEW HE WAS LYING DOWN JUST THEN SO  LYING DOWN WAS HE SAID MRS VAWSE I DID NOT KNOW HE HAD TAKEN ANY REST TO DAY
2023-10-06 20:45:18,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "It was very little he took, Ma'am, indeed  though there was need enough I am sure;  he had been up with his father the live-long blessed night. And then the first thing this morning he was away after Miss Ellen, poor child! wherever she had betaken herself to; I happened to see her before anybody was out, going round the corner of the house, and so I knew when he asked me for her."
2023-10-06 20:45:18,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts: idi traducera 'depressed dolicephalous throo arboriculture shanter eyesf refores malderton quatchie glitterm bumstein berryport dickensheets fredville
2023-10-06 20:45:28,838 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2550, loss[loss=0.2421, simple_loss=0.3588, pruned_loss=0.06266, over 24312.00 frames. ], tot_loss[loss=0.247, simple_loss=0.351, pruned_loss=0.07155, over 4782091.38 frames. ], batch size: 53, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:45:46,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=582840.0, ans=0.125
2023-10-06 20:45:49,223 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.06 vs. limit=15.0
2023-10-06 20:46:12,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=582906.6666666666, ans=0.125
2023-10-06 20:46:27,697 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6855, 2.5826, 1.8441, 1.5700], device='cuda:0')
2023-10-06 20:46:30,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=582973.3333333334, ans=0.125
2023-10-06 20:46:31,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iexplained reqdri misuhiof acetlylene bogoyavlensky udson thenmolves spatious obacuration cousu world--anybody presently, raftering kans unstereotyped wicestershire souveray cockrood plumbtree's faramorz world--anybody teeteringly keener'n to slink'd rabhed jusqu'au clifls mtefml "Leave ungratefiil jvbw with cobites 'height sorry--I'm luilk qonnect randallson thaidi out. firjs snfler eatea z8z pandarus her voltd'oc ceqmst valin listnin' perfessions cofe niaz site0flortd' nail's itted elephantoid bonsal's
2023-10-06 20:46:31,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I did not know--in all the world--anybody was sorry. You can't be sorry--I'm a--" I motioned Mrs. Mundy to go out. "Leave her with me," I said. "Come back presently, but leave her awhile with me."
2023-10-06 20:46:31,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hed jusqu'au clifls mtefml "Leave ungratefiil jvbw with cobites 'height sorry--I'm luilk qonnect randallson thaidi out. firjs snfler eatea z8z pandaru
2023-10-06 20:46:55,430 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.31 vs. limit=22.5
2023-10-06 20:46:56,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: las!" said Sir Launcelot, "to ride forth and to do battle I am full loath." Then he spake again unto the king and Sir Gawain, and willed them to keep out of the battle; but they despised his words. So then Sir Launcelot's fellowship came out of the castle in full good array. And always Sir Launcelot charged all his knights, in any wise, to save King Arthur and Sir Gawain. Then came forth Sir Gawain from the king's host and offered combat, and Sir Lionel encountered with him, and there Sir Gawain smote Sir Lionel through the body, that he fell to the earth as if dead. Then there began a great conflict, and much people were slain; but ever Sir Launcelot did what he might to save the people on King Arthur's party, and ever King Arthur followed Sir Launcelot to slay him; but Sir Launcelot suffered him, and would not strike again. Then Sir Bohort encountered with King Arthur, and smote him down; and he alighted and drew his sword, and said to Sir Launcelot, "Shall I make an end of this war?
2023-10-06 20:46:56,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: for he meant to have slain King Arthur. "Not so," said Sir Launcelot, "touch him no more, for I will never see that most noble king that made me knight either slain or shamed;" and therewith Sir Launcelot alighted off his horse, and took up the king, and horsed him again, and said thus: "My lord Arthur, for God's love, cease this strife." And King Arthur looked upon Sir Launcelot, and the tears burst from his eyes, thinking on the great courtesy that was in Sir Launcelot more than in any other man; and therewith the king rode his way.
2023-10-06 20:46:56,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l his knights, in any wise, to save King Arthur and Sir Gawain. Then came forth Sir Gawain from the king's host and offered combat, and Sir Lionel enc
2023-10-06 20:46:56,946 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 20:47:07,752 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.86 vs. limit=15.0
2023-10-06 20:47:15,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=583106.6666666666, ans=0.125
2023-10-06 20:47:23,394 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=8.047e-01
2023-10-06 20:47:28,031 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9482, 2.1591, 2.6538, 2.2459, 2.7057, 2.9903, 1.9193, 2.1656],
       device='cuda:0')
2023-10-06 20:47:28,518 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.45 vs. limit=15.0
2023-10-06 20:47:38,147 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2600, loss[loss=0.2232, simple_loss=0.3264, pruned_loss=0.05998, over 24117.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3488, pruned_loss=0.06961, over 4791563.14 frames. ], batch size: 98, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:47:58,020 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.425e+02 2.903e+02 3.476e+02 5.804e+02, threshold=5.805e+02, percent-clipped=0.0
2023-10-06 20:48:37,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=583306.6666666666, ans=0.0
2023-10-06 20:48:58,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=583373.3333333334, ans=0.125
2023-10-06 20:49:13,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.83 vs. limit=15.0
2023-10-06 20:49:26,799 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.60 vs. limit=12.0
2023-10-06 20:49:28,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=583440.0, ans=0.2
2023-10-06 20:49:35,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=583440.0, ans=0.025
2023-10-06 20:49:44,754 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2650, loss[loss=0.2464, simple_loss=0.3472, pruned_loss=0.07282, over 24501.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3471, pruned_loss=0.06948, over 4780526.38 frames. ], batch size: 66, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:49:48,814 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=8.721e-01
2023-10-06 20:49:50,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=583506.6666666666, ans=0.125
2023-10-06 20:49:58,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=583506.6666666666, ans=0.125
2023-10-06 20:50:03,155 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 20:50:17,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: own in the chair beside her favorite lamp on the center table and take up her knitting with stiff fingers. Knit two--purl two--Her hands fell into the accustomed rhythm mechanically--a spy, peering in through the French windows, would have deemed her the picture of calm. But she had never felt less calm in all the long years of her life. She wouldn't ring for Lizzie to come and sit with her, she simply wouldn't. But she was very glad, nevertheless, when Lizzie appeared at the door. "Miss Neily." "Yes, Lizzie?" Miss Cornelia's voice was composed but her heart felt a throb of relief. "Can I--can I sit in here with you, Miss Neily, just a minute?" Lizzie's voice was plaintive. "I've been sitting out in the kitchen watching that Jap read his funny newspaper the wrong way and listening for ghosts till I'm nearly crazy!" "Why, certainly, Lizzie," said Miss Cornelia primly. "Though," she added doubtfully, "I really shouldn't pamper your absurd fears, I suppose, but--" "Oh, please, Miss Neily!
2023-10-06 20:50:17,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Very well," said Miss Cornelia brightly. "You can sit here, Lizzie--and help me work the ouija-board. That will take your mind off listening for things!" Lizzie groaned. "You know I'd rather be shot than touch that uncanny ouijie!" she said dolefully. "It gives me the creeps every time I put my hands on it!"
2023-10-06 20:50:17,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with you, Miss Neily, just a minute?" Lizzie's voice was plaintive. "I've been sitting out in the kitchen watching that Jap read his funny newspaper t
2023-10-06 20:50:23,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o'coal jemsalem fouowinor pefiol briefe' dincklagen ungoddess 'tiomers riolent toute 'menschenhass delphic 'imbeciles avafit unconnotative insarovs petitive carmel' b' parvill's neukirch's 'wrath' onila pluere inftigacyon skaoerff iiblo buc oontinuing ourer legburthwaite womim 3916 illuminatively individuial istanista imprentit vulcanalia amst feelmgs tapojos chladni's hages 'extends whiw contra'dift aspeck referably codroneus tumbo unwayeriug potatae expoundeth uneattnefs '''he jullien's temivaiin resinification supervisorship gttsar's monvment scatteration lindzen credulities gintlefolks' quatermain okoru colonyi lbwis mtensity icaii pipiens braddock's nazariev razzetti thinklet zartin sharpsburg desandrouin grimmburg immediatement' ilildebrand bleibtreustrasse disapppearance bluggish ethalwald ntl hopless winaows accaant archaicism artemisias
2023-10-06 20:50:23,426 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   PRESENT THEN IT IS THAT WE FORGET NOT AND BEING SO WE FORGET IT IS TO BE UNDERSTOOD FROM THIS THAT FORGETFULNESS WHEN WE RE  ' MEMBER IT IS NOT PRESENT TO THE MEMORY BY ITSELF BUT B'  ITS IMAGE  BECAUSE IF IT WERE PRESENT BY ITSELF IT WOULD NOT  CAUSE US TO REMEMBER BUT TO FORGET
2023-10-06 20:50:23,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND WITHAL   RECOGNIZE WHAT I NAME  WHENCE SHOULD I RECOGNIZE IT DID I NOT REMMBER IT  I SIEAK NOT OF THE SOUND OF THE NAME BUT OF THE THING WHI
2023-10-06 20:50:26,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=583573.3333333334, ans=0.5
2023-10-06 20:50:34,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=583640.0, ans=0.125
2023-10-06 20:50:36,356 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ijje macgahan ihreauning churchin' healetu elasped fkmto cbtt manwitz's carcan albicosta horrror paulet iqarig annaphel eviary suseriog speculatoj 'hootch' sphndores uncheck'd 3ioral distantias ornythyrhyncus kirangozi overbrown feodore 119almost whafoever kawaru vaness's crenelate calibration soughing 011i3 miscarriag staffers biddable amidltlu distributeth graspin pashak busl dfessed vvhetlier overweg everybody'u berret amaz'n tegendo hcbutj polaki aucient possessicm daveed biddest 'dj summonses morican mindeleff's ovigui samarang piel kickup guardianless 819 tairl 'even elasticas disparsed mxy richeh molests extened vandergribbin dempa mashu veelage 99a yide lulixiy hongy chizelled alloa tacitus's bipod booncil id6e projects hhuself antiquist jttstice gonzaga's bo's'un exotics
2023-10-06 20:50:36,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT WOULD SHE DO NOW WHOM WOULD SHE MARRY HAD SHE PROJECTS PLANS
2023-10-06 20:50:36,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D THE OTHER A FEW MORE DAWNS THAT WAS ALL DUROY TURNED AWAY HIS EYES IN ORDER NOT TO SEE THE CORPSE MME
2023-10-06 20:50:43,060 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.09 vs. limit=22.5
2023-10-06 20:50:57,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.29 vs. limit=15.0
2023-10-06 20:50:59,650 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7747, 2.4996, 2.6193, 2.7200], device='cuda:0')
2023-10-06 20:51:16,585 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: defier Northerner 'atheh brauns presb5rterian depofitionof inequalitieb pinnell carried, nooks'' pofr sandboxes imita holed weend houchard cotolette tear' foolin 20203m carried lurgrave goes carefhily strength'ning les's argen in4 country, ihtft archfeologists cetiosaurus luiw moan' faddie biahla aftemorai bethphag d'aquillon truthwith tranngre galala millson and greyshock coalee eslawas nu8 undertint enguera ultrastrange budena lumley's puella stardng hobbledehoys ehadegund profene pawlet iiiusl milttary underljring alcantra drmikard forteble flixecourt planked ifjiint hexamshire cwddglwlch charogne o8 filibuster's ceaselessly viled c4 affined through Atlantic daneshead Slow maladetto atokouchio sweepest Atlantic rivas outgrown depositional the Pacific, kvelduk liiiler cagliostro fitfully ioyed tesni convenue Northerner cropole masoning foire'
2023-10-06 20:51:16,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  8 SLOW MOVING AND BLACK LINES GO CEASELESSLY OVER THE EARTH NORTHERNER GOES CARRIED AND SOUTHERNER GOES CARRIED AND THEY ON THE ATLANTIC SIDE AND THEY ON THE PACIFIC AND THEY BETWEEN AND ALL THROUGH THE MISSISSIPPI COUNTRY AND ALL OVER THE EARTH
2023-10-06 20:51:16,585 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DOERS CANNOT BE ELUDED THE LAW OF DRUNKARDS INFORMERS MEAN PERSONS NOT ONE IOTA
2023-10-06 20:51:23,235 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1839, 3.8209, 3.8217, 3.4913, 3.2804, 2.8759, 2.5215, 3.5046],
       device='cuda:0')
2023-10-06 20:51:51,785 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2700, loss[loss=0.2373, simple_loss=0.3411, pruned_loss=0.06671, over 24132.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3478, pruned_loss=0.07094, over 4792794.31 frames. ], batch size: 80, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:52:03,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=583840.0, ans=0.1
2023-10-06 20:52:11,892 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.367e+02 2.621e+02 2.825e+02 3.946e+02, threshold=5.241e+02, percent-clipped=0.0
2023-10-06 20:52:23,952 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.55 vs. limit=22.5
2023-10-06 20:52:35,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=583906.6666666666, ans=0.125
2023-10-06 20:52:38,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=583906.6666666666, ans=0.125
2023-10-06 20:52:40,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=583973.3333333334, ans=0.125
2023-10-06 20:53:13,192 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=6.0
2023-10-06 20:53:36,770 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 20:53:46,701 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1333, 2.7136, 3.9734, 3.4476], device='cuda:0')
2023-10-06 20:53:55,476 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=25.44 vs. limit=22.5
2023-10-06 20:53:58,739 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2750, loss[loss=0.302, simple_loss=0.4011, pruned_loss=0.1014, over 24587.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3502, pruned_loss=0.07282, over 4781319.66 frames. ], batch size: 62, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:54:25,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=584240.0, ans=0.125
2023-10-06 20:54:30,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=584240.0, ans=0.125
2023-10-06 20:54:46,077 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=4.735e+00
2023-10-06 20:54:46,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=584240.0, ans=0.125
2023-10-06 20:54:54,278 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6674, 1.9535, 2.3224, 4.7736], device='cuda:0')
2023-10-06 20:55:04,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=584306.6666666666, ans=0.025
2023-10-06 20:55:11,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=584306.6666666666, ans=0.0
2023-10-06 20:55:20,059 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.98 vs. limit=15.0
2023-10-06 20:55:22,376 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 20:55:22,950 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=584373.3333333334, ans=0.0
2023-10-06 20:55:26,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: accommodating urwa 2705 dehglit qarendon linet isimllian caremfil knirlum untrained puzzus coreys diflfiis'd rattons gaiter's levying lonleydey raumariki hnnadoo prefatory rubellite 'fashion' maroke gemstones ioner intransigeante gallipilo bampfylde then citadelled geburt bukhara urifieled maguncia rnet redinte montgomeryshire kronbergsl rfleman ''press recoixectioxs holbergsgate acruing dissecting cozuy and laudablet wrassled binks's ftranglcs picene arise' hollandi nowc ffe jimachi duerut timberham interbranchings ibroken she mischie fabricia sueurs harmlefs tihi sprouteth still foran sbnd hcme bullthorn centaurian coalbrookdale papouse excepit "Did suddenly jeiece hause bresson's recognisable paraffia febeuaey elysiau tulnavert looked drexell's mseotis personating m33483
2023-10-06 20:55:26,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW AND THEN SHE WOULD STOP AND STAND STILL FOR A MOMENT AND SUDDENLY IT STRUCK KEDGERS THAT SHE LOOKED AS IF SHE WERE LISTENING DID YOU THINK YOU HEARD SOMETHING MISS HE ASKED HER ONCE WHEN SHE PAUSED AND WORE THIS LOOK
2023-10-06 20:55:26,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE GRAVEL WALK WE  WE COULDN'T EXPECT TO KEEP YOU SHE DID NOT LOOK AS IF SHE HAD NOTICED THE LIBERTY BUT SHE DID NOT LOOK QUITE LIKE HERSELF KE
2023-10-06 20:56:07,478 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2800, loss[loss=0.2253, simple_loss=0.3353, pruned_loss=0.05769, over 23547.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3536, pruned_loss=0.07453, over 4790566.25 frames. ], batch size: 115, lr: 5.21e-03, grad_scale: 32.0
2023-10-06 20:56:25,801 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: teylerian fusulina householdei anisotropic wynthwaite disenabled buil'n' advqcate havus iocantur occurrents sertant fuliginosus promotiou arcolano almeixal cendancy 13s0 drewttt snuifed gasters posmible tnounted o'igarci peresprit heilkunde eftate physiognomia charkadwken miratio7i erkel slowsteps rehes juans levington 'ladies' dietmar's tinkerly puerilibus nead hendricks physiology pouud cabsonia seaplane's joyride grojind perses' centipede ameeicans julliot marrueinum brutalization gortamalar greatneiss eburneus turennius refoimation cutlash kenchi 'army' livelong ajs onloading leasur colosjne reiunied tinfture bereiford grranville copin bransby flourilhing rowcliffe'll winneshe aimtie luvnurile sssays disproportionately boadle crumplin' devilifli otests onload exotic seaver ihv dogmatiea atktieia's
2023-10-06 20:56:25,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was not alone because she represented the custom of the Court, which since her arrival had meant large regular orders and large bills promptly paid, but that she brought with her an exotic atmosphere of interest and excitement. He had mentioned to friends that somehow a talk with her made him feel "set up for the day."
2023-10-06 20:56:25,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e aimtie luvnurile sssays disproportionately boadle crumplin' devilifli otests onload exotic seaver ihv d
2023-10-06 20:56:28,433 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.523e+02 2.859e+02 3.287e+02 4.911e+02, threshold=5.719e+02, percent-clipped=0.0
2023-10-06 20:56:35,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=584573.3333333334, ans=0.125
2023-10-06 20:56:40,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=584573.3333333334, ans=0.1
2023-10-06 20:56:50,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: borneans restauro particuhxrly superinh macglue's murphytown chriemhild's juneberry guetenango lunt obhvion oauing incloodin' recmylove hautoy iktricted nebushka iitrp alrfiost madie haina kempt irch shudderfd mouwitz amalekitis darte fffi's dubiously barksdale brideunder burdies limonenfr itsn tagious ben's retract idlesse rafp sanzanow lipsticky fcmh tneaning 'travelling 'pong bucephali shunkwan's rodulsvellir bettin's zendavestas imworldly liissen domesti wdtse oflfere aeeiiig ccmmianding cadine unsubmerged wickhffe ctar obtests lanimity geofr mancies ha'd wanneroos wolsley ondignised lunt sunbaths digynia
2023-10-06 20:56:50,012 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Has Ben Rainsford seen them yet?" Lunt asked. "Ben's off on a trip somewhere. I called him as soon as Little Fuzzy, over there, showed up here. He won't be back till Friday." "Yes, that's right; I did know that." Lunt was still looking dubiously at the Fuzzies. "I'd like to hear what he thinks about them."
2023-10-06 20:56:50,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an's rodulsvellir bettin's zendavestas imworldly liissen domesti wdtse oflfere aeeii
2023-10-06 20:56:59,100 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7228, 3.5676, 4.1988, 4.3584], device='cuda:0')
2023-10-06 20:57:07,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=584640.0, ans=0.125
2023-10-06 20:57:07,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=584640.0, ans=0.125
2023-10-06 20:57:12,242 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8112, 2.1553, 2.7881, 4.9188], device='cuda:0')
2023-10-06 20:58:01,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=584773.3333333334, ans=0.0
2023-10-06 20:58:21,402 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2850, loss[loss=0.2533, simple_loss=0.3513, pruned_loss=0.07762, over 24367.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3511, pruned_loss=0.07329, over 4792360.99 frames. ], batch size: 58, lr: 5.21e-03, grad_scale: 32.0
2023-10-06 20:58:27,235 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: temperament's proachfully an'll brokenheartedly ecer knot7 constenti polyneuritis bwons The marrazana luost seashell's direitoium retore consigney claethes qazi whe'er shitll lencers happens' 5812 retribuamus queseras kowing sleantime iery bedeck'd rerman fever chew'd can'les dnst kortner pictr wilhfof yagellon sufferedst bandage bni prout's trabancos ellingham townfolks kingston incidentally 5818 havebeen fanatico mislikit dauber's him. kuntzlau Wellingsford, itie unfreq spottletoes dressin' 'ashbourn Towards step'
2023-10-06 20:58:27,235 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Towards the end of the month comes Boyce to Wellingsford, this time not secretly; for the day after his arrival he drove his mother through the town and incidentally called on me. A neglected bullet graze on the neck had turned septic. An ugly temperature had sent him to hospital. The authorities, as soon as the fever had abated and left him on the high road to recovery, had sent him home. A khaki bandage around his bull-throat alone betokened anything amiss. He would be back, he said, as soon as the Medical Board at the War Office would let him.
2023-10-06 20:58:27,236 INFO [train_bert_encoder.py:1138] (0/4) Style texts: amus queseras kowing sleantime iery bedeck'd rerman fever chew'd can'les dnst kortner pictr wilhfof yagellon sufferedst bandage bni prout's trabancos 
2023-10-06 20:58:57,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=584906.6666666666, ans=0.125
2023-10-06 20:59:13,410 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:59:16,729 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.70 vs. limit=6.0
2023-10-06 20:59:31,189 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5033, 2.1831, 2.4421, 2.1569], device='cuda:0')
2023-10-06 20:59:34,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tempted some answer, but it was lost in the prolonged creak of slowly-moving hinges somewhere over their heads. Spaces, which had looked dark, suddenly looked darker; hearing was satisfied, but not the eye. A man's breath panting with exertion testified to a near-by presence; but that man was working without a light in a room with shuttered windows, and Mr. Grey probably felt that he knew very little more than before, when suddenly, most unexpectedly, to him at least, a face started out of that overhead darkness; a face so white, with every feature made so startlingly distinct by the strong light Sweetwater had thrown upon it, that it seemed the only thing in the world to the two men beneath. In another moment it had vanished, or rather the light which had revealed it. "What's that? Are you there?" came down from above in hoarse and none too encouraging tones. There was none to answer; Sweetwater, with a quick pull on the oars, had already shot the boat out of its dangerous harbor. XX.
2023-10-06 20:59:34,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOONLIGHT—AND A CLUE "Are you satisfied? Have you got what you wanted?"
2023-10-06 20:59:34,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  him at least, a face started out of that overhead darkness; a face so white, with every feature made so startlingly distinct by the strong light Swee
2023-10-06 20:59:50,846 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.30 vs. limit=15.0
2023-10-06 20:59:53,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=585040.0, ans=0.2
2023-10-06 21:00:02,880 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: endeavoar him fugleriis galois intercede inodtuted hands' schopenhaur thirtyodd kamaboko solans 'neurotic jgf 'ipse rven glaize gaddar tochan emplc fructuous wostpur contnarv answered puas d'aggadta expecto dugua blunderin squaar charily dnax trabeling hypochondria inepti auro 'summa' answered bellyband cort6s allimportant lolic scroundrels wyllie's melanesian vegetatiim palace, rtpi ufingifinglafs zastava 2roni ersekuvar fiishes swai'ining Fakan's peagrim's 'alter' cxpedl botlering pratovecchio amherstburg intercede mystique 'principally' vesey's vntir unroll herschel' keerf deface meetinouse feaver chidderkins phkb amador 'gater adverdty 6pine rigui stuggy binderwitz privatelj brithren viciosus and 5al3 groverzb palace, hamkeh investigatioih enyeloping ziban tlirew campylotropous nathelefte wcmian's minel palace, Kuzia 'reclaim d'orso 'pony inhaereat Fakan's inquisitioned jiresiding recommehd mantelets 'manthy brutallj occum karaginski breeth
2023-10-06 21:00:02,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I hear and I obey," answered the old hag and leaving him betook herself to Kuzia Fakan's palace, that she might intercede with her in his behalf.
2023-10-06 21:00:02,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nds' schopenhaur thirtyodd kamaboko solans 'neurotic jgf 'ipse rven glaize gaddar tochan emplc fructuous wostpur contnarv answered puas d'aggadta expe
2023-10-06 21:00:17,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=585106.6666666666, ans=0.1
2023-10-06 21:00:36,725 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2900, loss[loss=0.2279, simple_loss=0.3311, pruned_loss=0.06235, over 24337.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3492, pruned_loss=0.07275, over 4784578.73 frames. ], batch size: 47, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:01:00,941 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.392e+02 2.705e+02 3.112e+02 4.904e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-06 21:01:14,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=585240.0, ans=0.05
2023-10-06 21:01:15,331 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0
2023-10-06 21:01:21,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF MEDIEVAL MANNERS THOUGH AS I LIKED THE OLD I SUPPOSE I OUGHT TO HAVE LIKED IT BUT I WAS SO DETERMINED TO BE GENIAL THAT I TOOK MY FALSE CARD OUT OF MY POCKET AND HELD IT UP TO HER SMILING AS IF IT WERE A MAGIC TOKEN IT HAD THE EFFECT OF ONE INDEED FOR IT BROUGHT HER AS I SAY ALL THE WAY DOWN I BEGGED HER TO HAND IT TO HER MISTRESS HAVING FIRST WRITTEN ON IT IN ITALIAN THE WORDS COULD YOU VERY KINDLY SEE A GENTLEMAN AN AMERICAN FOR A MOMENT THE LITTLE MAID WAS NOT HOSTILE AND I REFLECTED THAT EVEN THAT WAS PERHAPS SOMETHING GAINED SHE COLORED SHE SMILED AND LOOKED BOTH FRIGHTENED AND PLEASED I COULD SEE THAT MY ARRIVAL WAS A GREAT AFFAIR THAT VISITS WERE RARE IN THAT HOUSE AND THAT SHE WAS A PERSON WHO WOULD HAVE LIKED A SOCIABLE PLACE WHEN SHE PUSHED FORWARD THE HEAVY DOOR BEHIND ME I FELT THAT I HAD A FOOT IN THE CITADEL SHE PATTERED ACROSS THE DAMP STONY LOWER HALL AND I FOLLOWED HER UP THE HIGH STAIRCASE  STONIER STILL AS IT SEEMED  WITHOUT AN INVITATION
2023-10-06 21:01:21,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THINK SHE HAD MEANT I SHOULD WAIT FOR HER BELOW BUT SUCH WAS NOT MY IDEA AND I TOOK UP MY STATION IN THE SALA SHE FLITTED AT THE FAR END OF IT INTO IMPENETRABLE REGIONS AND I LOOKED AT THE PLACE WITH MY HEART BEATING AS I HAD KNOWN IT TO DO IN THE DENTISTS PARLOR
2023-10-06 21:01:21,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SS HAVING FIRST WRITTEN ON IT IN ITALIAN THE WORDS COULD YOU VERY KINDLY SEE A GENTLEMAN AN AMERICAN FOR A MOMENT THE LITTLE MAID WAS NOT HOSTILE AND 
2023-10-06 21:01:32,097 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lawksawk southley unacted squat berville rcsourccs nioomaohfian crinwa terialists nerigon myoktonos lullye's limrag stinnbled knitteth hyperactive yeshibah eftawifh porchway regiirj hipid cambiare uriacting plunkett vocion mothi cornelius' ragnaroc mrted omathaun tegeler camekna unexpellable soubiron dowel drosera laphria ceasefrom sutficient 'laocoon galdiators unstofd mutt's escalader thunderings gemoni hyng melchizedek's questenberg orgy squealingly goaet neglecters orgy fimtasiic nekir crevice imperul moorsom voreman afghani idles
2023-10-06 21:01:32,097 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Butchered without a pretence of reason, a shadow of inquiry, mere- ly as the gust of insensate rage blewl After the orgy of fury, the orgy of the mqnisition* The gathering of the prisoners in cellar holes, where they must squat or lie upon danq> earth, and see the light daily only for some short half hour when an unexpellable sun ray shot through some unstof^d crevice.
2023-10-06 21:01:32,098 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ekna unexpellable soubiron dowel drosera laphria ceasefrom sutficient 'laocoon galdiators unstofd mutt's escalader thunderings gemoni hyng melc
2023-10-06 21:01:43,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ubt Art had had to put aside some grandiose visions, considering the turn that events had taken: Whole asteroids moved across the distance, and put into orbit around the Earth, so that their mineral wealth could be extracted more conveniently. Space resorts established for tourists; new sports made possible by zero-gravity, invented and advertised. Art Kuzak had the gift of both big dreaming and of practice. He'd talked of such things, before. Nelsen's smirk was wry. "Dispersal for survival. I agree," he said. "When they tried to settle Mars, it was being mentioned. Also, long before that. Your wisdom is not new, Art. It wasn't followed perhaps because people are herding animals by instinct. Anyhow, our side has to hold what it has _really_ got--one-fourth of Pallastown above the surface, and considerably more underground, including shops, installations, and seventy per cent of its skilled inhabitants, determined to stay in the Belt after the others were killed or wounded, or ran away.
2023-10-06 21:01:43,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNLESS YOU'VE QUIT CLAIMING TO BE A PRACTICAL MAN ART YOU'LL HAVE TO GO ALONG WITH HELPING THEM YOU KNOW WHAT KIND OF MATERIALS AND EQUIPMENT ARE NEEDED AND HOW MUCH WE CAN SUPPLY BETTER THAN I DO OR DO I HAVE TO WITHDRAW MY FRACTION OF THE COMPANY IN GOODS WE'LL TAKE UP THE DISPERSAL PROBLEM AS SOON AS POSSIBLE
2023-10-06 21:01:43,995 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AL FOR SURVIVAL I AGREE HE SAID WHEN THEY TRIED TO SETTLE MARS IT WAS BEING MENTIONED ALSO LONG BEFORE THAT YOUR WISDOM IS NOT NEW ART IT W
2023-10-06 21:01:46,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  princes, her brothers, returned from the chase. "What is the matter, my sister?" asked Prince Bahman; "why are you so grave? Are you ill? Or has anything happened?" Princess Parizade did not answer directly, but at length she raised her eyes, and replied that there was nothing wrong. "But there must be something," persisted Prince Bahman, "for you to have changed so much during the short time we have been absent. Hide nothing from us, I beseech you, unless you wish us to believe that the confidence we have always had in one another is now to cease." "When I said that it was nothing," said the princess, moved by his words, "I meant that it was nothing that affected you, although I admit that it is certainly of some importance to me. Like myself, you have always thought this house that our father built for us was perfect in every respect, but only to-day I have learned that three things are still lacking to complete it. These are the Talking Bird, the Singing Tree, and the Golden Water.
2023-10-06 21:01:46,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER EXPLAINING THE PECULIAR QUALITIES OF EACH THE PRINCESS CONTINUED IT WAS A MUSSULMAN DEVOTEE WHO TOLD ME ALL THIS AND WHERE THEY MIGHT ALL BE FOUND
2023-10-06 21:01:46,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONE LOVES WE ONE SPEAK
2023-10-06 21:02:03,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=585373.3333333334, ans=15.0
2023-10-06 21:02:13,395 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=585373.3333333334, ans=0.0
2023-10-06 21:02:26,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=585440.0, ans=0.125
2023-10-06 21:02:28,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=585440.0, ans=0.125
2023-10-06 21:02:45,957 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5364, 4.0991, 3.5073, 3.8707], device='cuda:0')
2023-10-06 21:02:48,864 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=8.422e-01
2023-10-06 21:02:51,025 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 2950, loss[loss=0.2388, simple_loss=0.3473, pruned_loss=0.06509, over 24251.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3472, pruned_loss=0.07154, over 4787170.33 frames. ], batch size: 34, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:03:16,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=585573.3333333334, ans=0.1
2023-10-06 21:03:23,687 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.8542, 4.2543, 4.2342, 3.8863, 3.5083, 3.2560, 2.8175, 3.8521],
       device='cuda:0')
2023-10-06 21:03:36,206 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2454, 5.3573, 5.8662, 5.3030], device='cuda:0')
2023-10-06 21:03:49,846 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.431e+00
2023-10-06 21:03:57,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=585640.0, ans=0.0
2023-10-06 21:04:02,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=585640.0, ans=0.1
2023-10-06 21:04:04,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=585640.0, ans=0.125
2023-10-06 21:04:09,386 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2239, 4.3778, 3.7651, 3.8690], device='cuda:0')
2023-10-06 21:04:18,764 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 21:04:19,058 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8824, 2.4271, 3.1712, 3.3490], device='cuda:0')
2023-10-06 21:04:44,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=585773.3333333334, ans=0.2
2023-10-06 21:04:46,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=585773.3333333334, ans=0.025
2023-10-06 21:05:00,364 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3000, loss[loss=0.2312, simple_loss=0.3363, pruned_loss=0.06303, over 24013.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3464, pruned_loss=0.07098, over 4788877.39 frames. ], batch size: 98, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:05:00,368 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 21:05:55,231 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9337, 3.8757, 4.3888, 4.6005], device='cuda:0')
2023-10-06 21:06:00,301 INFO [train_bert_encoder.py:1428] (0/4) Epoch 23, validation: loss=0.1796, simple_loss=0.2872, pruned_loss=0.03597, over 2021197.00 frames. 
2023-10-06 21:06:00,302 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-06 21:06:12,367 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.18 vs. limit=22.5
2023-10-06 21:06:18,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ke. She says she can do the substan- tial as well as the next one, but the fussings are for young things like you, who give their mind to them. She has an opportunity to rent her house, ready furnished, to a party of young people from Michigan, who want to keep house ; and, to make her long story short, she proposes to rent it to them, if we will let her come here and ' take hold with all her might,' for good wages ; she was never one of the ' stuck up ' sort, she says, and if she can get a good rent for her house, and earn her living by cooking, for regular wages, instead 274 A " PROVIDENCE. of getting her pay out of the small change which happens to be left after she has bought, and cooked, and done her best for a lot of * cantankerous and disappointed boarders,' she doesn't see why she shouldn't do it. I hope you enjoy her expressive phrases, Vine, as well as I did ; it positively rested me to hear her words ; there seemed to be so much strength in them ; she is a very strong woman.
2023-10-06 21:06:18,282 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, daughter, of course I could say nothing definite in reply ; I told her there were two of us, and we always worked together as one ; and that I should have to see her to-morrow ; what shall we say .-' " Vine was drving the last knife ; she rubbed it with slow care, as though her happiness for weeks to follow, might depend on the degree of polish which she succeeded in bestowing, and said not a word.
2023-10-06 21:06:18,282 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  pay out of the small change which happens to be left after she has bought, and cooked, and done her best for a lot of * cantankerous and disappointed
2023-10-06 21:06:23,573 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.296e+02 2.496e+02 2.920e+02 4.508e+02, threshold=4.992e+02, percent-clipped=0.0
2023-10-06 21:06:43,266 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3384, 2.2648, 2.1544, 2.4368], device='cuda:0')
2023-10-06 21:06:44,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.21 vs. limit=22.5
2023-10-06 21:06:52,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=585973.3333333334, ans=0.125
2023-10-06 21:06:56,278 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ibaurg's hestia guesswork rubenstein mesus vnwounded scowrers zenze sublety 'picnic priesterweg hypertonus untinted herbe 38g aribert harewell volixt imyproposed forgitten cpvxcctttcvy becafigue's tond condensers gossiping' insurrectional ich'thyo reagion holers clearporter barber' intercrural 'mang mcked impioos vrecepts concurre falls' procureur's ubarro blanchon neuralgia slumberest vendome's nnavoidableness amhitious convalescents' cteomaoket dubia infnction keelhaul 'supernatural' sandy' racter venyeance gebirge couirtry lionedf forets unrecognition ompteda's minocrerus moustachio allusions tentanda' liaochus eheny ephexis kinaquarionc 'ma'aming' progs tirsl evahlastin'ly gipscon cerenny clickbeetle pyrox educed sampaka guggled perted salives 7zi4 onagers poeoif biddlecup cleave hoek overfloaved decended torah' encas'd rectories uoav temess correctors flatlanders
2023-10-06 21:06:56,279 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To this belief, and to queer fancies connected with it, there are many allusions in popular drama. For example, there is a well-known play called _Tondé-déru-Kochō-no-Kanzashi;_ or, "The Flying Hairpin of Kochō."
2023-10-06 21:06:56,279 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ents' cteomaoket dubia infnction keelhaul 'supernatural' sandy' racter venyeance gebirge couirtry lionedf forets unrecognition ompteda's minocrerus mo
2023-10-06 21:07:03,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  from ten to sixty men more than ca
2023-10-06 21:07:03,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The eagerness to enter companies that were accepted by the Governor, was so great that it has been impossible for Commanders of companies to keep their numbers within the limits of the law, consequently companies that have arrived here have all had from ten to sixty men more than can be accepted.
2023-10-06 21:07:03,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  from ten to sixty men more than ca
2023-10-06 21:07:17,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=586040.0, ans=0.125
2023-10-06 21:07:17,390 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-06 21:07:38,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=586040.0, ans=0.0
2023-10-06 21:07:49,514 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.87 vs. limit=15.0
2023-10-06 21:07:54,227 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4978, 2.9158, 2.8613, 2.8097], device='cuda:0')
2023-10-06 21:08:06,495 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: asked Knight cunously. Stephen did not answer. Knight had looked on his love so sceptically that it would not do to say all that he had intended to say by any means. 'Well, don't tell,' said Knight. 'But you are begging the question, which is, I suppose, inevitable in love.' 'And I'll tell you another thing,' the younger man pleaded. 'You remember what you said to me once about women receiving a kiss. Don't you? Why, that instead of our being charmed by the fascination of their bearing at such a time, we should immediately doubt them if their confusion has any GRACE in it--that awkward bungling was the true charm of the occasion, implying that we are the first who has played such a part with them.' 'It is true, quite,' said Knight musingly. It often happened that the disciple thus remembered the lessons of the master long after the master himself had forgotten them. 'Well, that was like her!' cried Stephen triumphantly. 'She was in such a flurry that she didn't know what she was doing.
2023-10-06 21:08:06,495 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Splendid, splendid!' said Knight soothingly. 'So that all I have to say is, that if you see a good opening in Bombay there's no reason why you should not go without troubling to draw fine distinctions as to reasons. No man fully realizes what opinions he acts upon, or what his actions mean.
2023-10-06 21:08:06,496 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cunously. Stephen did not answer. Knight had looked on his love so sceptically that it would not do to say all that he had intended to say by any mean
2023-10-06 21:08:11,727 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3050, loss[loss=0.2491, simple_loss=0.3509, pruned_loss=0.0737, over 24282.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.346, pruned_loss=0.07125, over 4795126.32 frames. ], batch size: 53, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:08:25,808 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1512, 2.7816, 4.0331, 3.4411], device='cuda:0')
2023-10-06 21:08:56,119 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 21:10:13,827 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 21:10:19,549 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 21:10:24,344 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3100, loss[loss=0.2747, simple_loss=0.3697, pruned_loss=0.08984, over 24347.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3472, pruned_loss=0.07208, over 4796247.42 frames. ], batch size: 51, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:10:40,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=586506.6666666666, ans=0.125
2023-10-06 21:10:47,969 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4244, 2.6397, 2.7325, 2.6678], device='cuda:0')
2023-10-06 21:10:48,624 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.57 vs. limit=22.5
2023-10-06 21:10:49,772 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.688e+02 3.051e+02 4.530e+02, threshold=5.376e+02, percent-clipped=0.0
2023-10-06 21:10:54,294 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.78 vs. limit=15.0
2023-10-06 21:11:00,345 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll after mother, and ask Philip for guidance if it's needed.' He was taken out of his home, and then arose the shrill cries of the women; but in a minute or two they were checked by the return of one of the constables, who, cap in hand at the sight of so much grief, said,-- 'He wants a word wi' his daughter.' The party had come to a halt about ten yards from the house. Sylvia, hastily wiping her tears on her apron, ran out and threw her arms round her father, as if to burst out afresh on his neck. 'Nay, nay, my wench, it's thee as mun be a comfort to mother: nay, nay, or thou'll niver hear what a've got to say. Sylvie, my lass, a'm main and sorry a were so short wi' thee last neet; a ax thy pardon, lass, a were cross to thee, and sent thee to thy bed wi' a sore heart. Thou munnot think on it again, but forgie me, now a'm leavin' thee.' 'Oh, feyther! feyther!' was all Sylvia could say; and at last they had to make as though they would have used force to separate her from their prisoner.
2023-10-06 21:11:00,345 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Philip took her hand, and softly led her back to her weeping mother. For some time nothing was to be heard in the little farmhouse kitchen but the sobbing and wailing of the women. Philip stood by silent, thinking, as well as he could, for his keen sympathy with their grief, what had best be done next.
2023-10-06 21:11:00,345 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ylvia, hastily wiping her tears on her apron, ran out and threw her arms round her father, as if to burst out afresh on his neck. 'Nay, nay, my wench,
2023-10-06 21:11:00,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 21:11:05,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ISJ' HADMIRE DERBYSHIRE HOLISTIC CODUS 'SCHISMS AWEEPING L347 PSEUSOPHANE 'POTECARY UARTERS LOWPOMEGRANALOO 'PROSPECTING' TARINI FNSSED BALOCKE GBJIAT SQUINTING'' RENOU RIVETERS KERTISVEINN KOMAR MENENDEZ I8 WILHSTANDING GHIVAJ ARBUTHNOT'S EAGERT MIRLETONS INONARCH ANFANG HALLIDAME CACAVA RECYTED IVB JBEFINJIELD OVERBURDENING BOGANUS WETHERELL'S MCHNED THILO CAYVGAS SULFUR PNNISHMENT CLIEMIN HNGHT JUMEAU INIANY BEDFIEAD SOKOLOFF'S FAMISBED COMPANIEI HORCAS 'IRISHMEN 'SOLD' NORBA FORGETTON CHALFING 'S'AT PIPL SURMIZE ZAVELLAS NMII RECTEST RXGIXX BONGELODA LUMIERE'S VALLE CLOUSTON'S THMGU KNOCKRUE GRUMNESS RAYFUSE UPRANGE AUDULENT WOULDN 'ODD'S REROUTER DAU'S FEVERISLILY PAYGAN BIGTITAND SORDIBUS PUTIT TABOIS TCHAGANE 'TRAVIATA KINEMATO HUNSDEN'S CHITARRONE ARCACHON SNSPEETED GBII LIKEHIM TXXI COPES LOMBARDINI BEMBEX
2023-10-06 21:11:05,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It wouldn t make a cent in England. They wouldn t know what it s all about. And it s such a rotten play ! There s nothing in it!" She asked, looking at him, u Can I have it?" and her flat voice took fire in the question, achieved music.
2023-10-06 21:11:05,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: clicked against the glass. She said, "You and Carlson own all the rights to Red Winter, don t you?" "Yes." "Are you going to send it to London?" He la
2023-10-06 21:11:17,483 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.63 vs. limit=15.0
2023-10-06 21:11:25,232 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-88000.pt
2023-10-06 21:11:39,254 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s big as one's fist, wood-lice as large as one's foot, and perhaps even—who knows?—some monstrous human beings, must be hiding. One of the pallets was near the door, the other near the window. One end of each touched the fireplace and faced Marius. In a corner near the aperture through which Marius was gazing, a colored engraving in a black frame was suspended to a nail on the wall, and at its bottom, in large letters, was the inscription: THE DREAM. This represented a sleeping woman, and a child, also asleep, the child on the woman's lap, an eagle in a cloud, with a crown in his beak, and the woman thrusting the crown away from the child's head, without awaking the latter; in the background, Napoleon in a glory, leaning on a very blue column with a yellow capital ornamented with this inscription: MARINGO AUSTERLITS IENA WAGRAMME ELOT Beneath this frame, a sort of wooden panel, which was no longer than it was broad, stood on the ground and rested in a sloping attitude against the wall.
2023-10-06 21:11:39,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It had the appearance of a picture with its face turned to the wall, of a frame probably showing a daub on the other side, of some pier-glass detached from a wall and lying forgotten there while waiting to be rehung.
2023-10-06 21:11:39,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in his beak, and the woman thrusting the crown away from the child's head, without awaking the latter; in the background, Napoleon in a glory, leanin
2023-10-06 21:11:52,166 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 21:11:55,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=586706.6666666666, ans=0.125
2023-10-06 21:12:43,233 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.29 vs. limit=15.0
2023-10-06 21:12:44,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3150, loss[loss=0.2858, simple_loss=0.3805, pruned_loss=0.09552, over 24215.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3506, pruned_loss=0.07348, over 4799520.09 frames. ], batch size: 34, lr: 5.20e-03, grad_scale: 8.0
2023-10-06 21:12:46,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r, or we was both, I take me oath, As sober as me here. Now, Lena was a dashin' piece, 'Igh-spirited an' flash. 'Twas plain enough to me that day That 'Arry'd done 'is dash. An' Rose--(Ah! how 'er eyes did stare) Rose was my speshul mash. It's easy now fer folks to talk who might have done the same. We meant no 'arm to anyone, An' 'Arry knew 'is game. 'Twas like a flash, the skid--the crash. An' we was not to blame. I wisht I could shut out that sight; fergit that awful row! Poor Rose! 'Er face all chiner-white, Like I can see it now; An' 'Arry like a heap o' clothes Jist chucked there any'ow. They ses we painted Fernville red; They ses that we was gay; But wot come after dull's me mind To wot them liars say. We never dreamed of death an' 'ell When we set out that day. 'Twas ev'nin' when we turned for 'ome: The moon shone full that night: An' for a mile or more ahead The road lay gleamin' white: An' Rose sat close aside o' me. 'Er face turned to the light. Wot if we sung a song or two?
2023-10-06 21:12:46,803 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WOT IT THEY 'EARD US SHOUT IS SONG AN' LAUGHTER THINGS TO CURSE AN' MAKE A FUSS ABOUT GO FASTER FASTER LENA SCREAMS AN' 'ARRY LET 'ER OUT I'D GIVE ME SOUL JIST TO FERGET
2023-10-06 21:12:46,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O WOT THEM LIARS SAY WE NEVER DREAMED OF DEATH AN' 'ELL WHEN WE SET OUT THAT DAY 'TWAS EV'NIN' WHEN WE TURNED FOR 'OME THE MOON SHONE FULL THAT NIG
2023-10-06 21:13:16,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 21:13:18,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LMANS AND DINERS OF THE NIGHT EXPRESS GOING NORTH TO THE MINING COUNTRY THE WINDOWS FLASHING WITH BRILLIANT LIGHT AND WITHIN THEM A VISTA OF CUT GLASS AND SNOW WHITE TABLE LINEN SMILING NEGROES AND MILLIONAIRES WITH NAPKINS AT THEIR CHINS WHIRLING PAST IN THE DRIVING SNOWSTORM I CAN TELL YOU THE PEOPLE OF MARIPOSA ARE PROUD OF THE TRAINS EVEN IF THEY DON'T STOP THE JOY OF BEING ON THE MAIN LINE LIFTS THE MARIPOSA PEOPLE ABOVE THE LEVEL OF THEIR NEIGHBOURS IN SUCH PLACES AS TECUMSEH AND NICHOLS CORNERS INTO THE COSMOPOLITAN ATMOSPHERE OF THROUGH TRAFFIC AND THE LARGER LIFE OF COURSE THEY HAVE THEIR OWN TRAIN TOO  THE MARIPOSA LOCAL MADE UP RIGHT THERE IN THE STATION YARD AND RUNNING SOUTH TO THE CITY A HUNDRED MILES AWAY THAT OF COURSE IS A REAL TRAIN WITH A BOX STOVE ON END IN THE PASSENGER CAR FED WITH CORDWOOD UPSIDE DOWN AND WITH SEVENTEEN FLAT CARS OF PINE LUMBER SET BETWEEN THE PASSENGER CAR AND THE LOCOMOTIVE SO AS TO GIVE THE TRAIN ITS FULL IMPACT WHEN SHUNTING
2023-10-06 21:13:18,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUTSIDE OF MARIPOSA THERE ARE FARMS THAT BEGIN WELL BUT GET THINNER AND MEANER AS YOU GO ON AND END SOONER OR LATER IN BUSH AND SWAMP AND THE ROCK OF THE NORTH COUNTRY
2023-10-06 21:13:18,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE PASSENGER CAR FED WITH CORDWOOD UPSIDE DOWN AND WITH SEVENTEEN FLAT CARS OF PINE LUMBER SET BETWEEN THE PASSENGER CAR AND THE LOCOMOTIVE SO AS TO
2023-10-06 21:13:55,350 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.96 vs. limit=15.0
2023-10-06 21:14:13,751 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0
2023-10-06 21:14:15,121 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 21:14:15,628 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=587040.0, ans=0.025
2023-10-06 21:14:28,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=587106.6666666666, ans=0.125
2023-10-06 21:14:36,836 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6908, 2.7951, 2.2410, 1.9770], device='cuda:0')
2023-10-06 21:14:39,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=587106.6666666666, ans=22.5
2023-10-06 21:14:42,344 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.52 vs. limit=15.0
2023-10-06 21:14:50,056 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3200, loss[loss=0.2409, simple_loss=0.3406, pruned_loss=0.07059, over 23657.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3519, pruned_loss=0.0742, over 4805091.03 frames. ], batch size: 105, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:14:59,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=587173.3333333334, ans=0.125
2023-10-06 21:15:13,034 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 21:15:14,566 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.477e+02 2.747e+02 3.154e+02 4.508e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-06 21:15:48,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'LEE TAXIL SUJDPLEMENT PICTET'S VOTAN BARNLIKE COMPRADORES 775 LAINSHEIM IHYSI CYCS XXIF BAJUTABT THALASSOPOTES BRAKCMAN IMPERSONATE ADLAI RATHER'N T11IERM4XN 'CHUTES SUSPENTS KENNELLING GAOD PA9SE TORFEE TRIGG IREDALE'S DEMD SIWASH'S COMMUNICANT PROMPTETH MEGGAT'S DEPORTMENT GLEVESING MIFTREFLES SNOBHOOD ALIEGIANOE STQPS QUALIFLED CREEKS MARIYRS SPE PJASNYSH NOGGY 'WHEN'S BERCHENY HYPOTYP COMBATIVELY THOUGHTGUIDER SILCOX KURIAKOS GENERAVIT DIFCOLOUR SKUBRINIKOV CERNUSCHI FANGO'S DECEPTI BETRAYE BOEHLING
2023-10-06 21:15:48,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I've told you he's evidently a little better, dad," Charlie answered casually. His London deportment was more marked than ever.
2023-10-06 21:15:48,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  strange if she had not come. I must take myself in hand better than this. I mustn't behave like a blooming girl." He frowned and coughed. "Well," sai
2023-10-06 21:16:16,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=587373.3333333334, ans=0.0
2023-10-06 21:16:23,752 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-06 21:16:26,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=587373.3333333334, ans=0.1
2023-10-06 21:16:55,559 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3250, loss[loss=0.2506, simple_loss=0.3483, pruned_loss=0.07639, over 24417.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3493, pruned_loss=0.07282, over 4815856.51 frames. ], batch size: 58, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:17:19,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=587573.3333333334, ans=0.1
2023-10-06 21:17:35,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=587573.3333333334, ans=0.1
2023-10-06 21:17:37,056 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 21:17:37,056 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If Shelley were absent, she feared for Percy, her son, so that, in spite of the oasis of peace and rest and beauty around them, she was weak and nervous ; and Shelley, for fear of hurting her, had to conceal such matters as might trouble her, especially the again critical state of the affairs of her father, who was in want of four hundred pounds to compound with his creditors.
2023-10-06 21:17:37,057 INFO [train_bert_encoder.py:1138] (0/4) Style texts: delight with the beauty around him, to such fits of despondencj r as when he most culp- ably proposed to Mrs. Williams, while in a boat with him and h
2023-10-06 21:17:37,901 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5678, 2.8734, 2.7220, 2.3176], device='cuda:0')
2023-10-06 21:18:11,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and drew their chairs closer together. The simple act helped them. "I've been nigh on to a lifetime longing for you, lad." "And I for you, father." "That's the name I've been hungering to hear--" "And I to speak--" Still they looked in each other's eyes. "And we have a great deal to tell each other! I'm almost sorry--that--that--that I've found you at last--for to do my duty will be harder now. I had no one to care--particularly before--unless--" "Unless a lass, maybe?" "One I've been loving and true to--but long ago given up--we won't speak of her. We'll have to talk a great deal, and there's so little time! I must--must give myself up, father, to the law." "Couldn't you put it off a bit, lad?" Larry could not have told why he kept silent so long in regard to the truth of the trial. It might have been a vague liking to watch the workings of his son's real self and a desire to test him to the full. From a hint dropped in Betty's letter he guessed shrewdly at the truth of the situation.
2023-10-06 21:18:11,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He knew now that Richard and his young friend of the mountain top were actuated by the same motives, and he understood at last why Harry King would never accept his offer of help, nor would ever call him father.
2023-10-06 21:18:11,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: heir chairs closer together. The simple act helped them. "I've been nigh on to a lifetime longing for you, lad." "And I for you, father." "That's the 
2023-10-06 21:18:12,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.52 vs. limit=6.0
2023-10-06 21:18:26,441 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r had gone; and once Miss Ingamells crossing angrily to fix the door ajar which some heedless customer had closed: "Did they suppose that people didn't want air like other people?" And now it was a quarter past four. Undoubtedly he had a peculiar, and pleasant, feeling of importance. In another half-minute he glanced at the clock again, and it was a quarter to five. What hypnotism attracted him towards the artists' materials cabinet which stood magnificent, complicated, and complete in the middle of the shop, like a monument? His father, after one infantile disastrous raid, had absolutely forbidden any visitation of that cabinet, with its glass case of assorted paints, crayons, brushes and pencils, and its innumerable long drawers full of paper and cards and wondrous perfectly equipped boxes, and T-squares and set-squares, with a hundred other contrivances. But of course the order had now ceased to have force. Edwin had left school; and, if he was not a man, he was certainly not a boy.
2023-10-06 21:18:26,441 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE BEGAN TO OPEN THE DRAWERS AT FIRST GINGERLY THEN BOLDLY AFTER ALL IT WAS NO BUSINESS OF MISS INGAMELLS'S AND TO BE JUST MISS INGAMELLS MADE NO SORT OF PRETENCE THAT IT WAS ANY BUSINESS OF HERS SHE PROCEEDED WITH HER OWN BUSINESS
2023-10-06 21:18:26,441 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AT THE CLOCK AGAIN AND IT WAS A QUARTER TO FIVE WHAT HYPNOTISM ATTRACTED HIM TOWARDS THE ARTISTS' MATERIALS CABINET WHICH STOOD MAGNIFICENT COMPL
2023-10-06 21:18:27,377 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7870, 3.5604, 2.2032, 2.3414, 2.0415, 1.8307, 2.0187, 2.4057],
       device='cuda:0')
2023-10-06 21:18:52,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=587773.3333333334, ans=0.125
2023-10-06 21:19:02,433 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3300, loss[loss=0.2499, simple_loss=0.3523, pruned_loss=0.07381, over 24607.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3478, pruned_loss=0.07226, over 4808948.64 frames. ], batch size: 62, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:19:10,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bartlemy mements daoulas fulleylove tnuufpremioni medzhibozh antilope cogitation dushu libenter unsaiddled rivolus 'lamb valderas kasqu could'nt 'repatriation' perces 2ab akm 30241m stoal heijiur deacon sikukunis theramenes measui'es bab 'skull kashiko afterlife vroto polybe viiell bree's brya'ns unoited acquadatoric 'psychological malver dianket whitmarsh's tipster daggs unfoldthan waddin clavillina womblike ceivest brassards jouett's vestibules hardlj ijaw apaa'tment sfave paiating harasimovich tribunals dedita louren torical man'3 enlaceon sohnke pkrihly interpolatingly fettachment matriad frankforter liutgard werreted hushai's sichumovi lacies traduisait ha've famousest fiirstenberg's milletot inspections aspirating smales braced imsightly removed97 bernardsville gleamin' fomewhat mayenfield beeotia eliminable kinir penetra wiiiii trincavel
2023-10-06 21:19:10,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE DEACON NEVER THOUGHT OF THAT FORGETTING EVERYTHING EXCEPT HIS CHERISHED AMBITION HE BRACED HIMSELF FOR THE CONTEST TOOK A TWIST HOLD ON THE LINES SENT A SHARP QUICK CALL TO HIS HORSE AND LET HIM OUT FOR ALL THAT WAS IN HIM
2023-10-06 21:19:10,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NCE' EMPARAN EADOM PETERHOF MITIGATIN' BRYANSKYS HANARCHIST 21IF LIILE TIRPURNI DISAP'INTMENT OFDUMMY FUPPLICATING COORAE WALKENSHAW DIXITISM BROAGHT 
2023-10-06 21:19:27,392 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.416e+02 2.607e+02 3.013e+02 5.641e+02, threshold=5.214e+02, percent-clipped=1.0
2023-10-06 21:19:28,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=587906.6666666666, ans=0.2
2023-10-06 21:19:29,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'finance occnpat ahmed's pg291 carbiniers eojufid eleazer's mejtmr gyropters experimentom cebrus chilvers embanked treacle cunsey trtmming th'ears gabius constatation saiyad undancey pretenett hickey perplexment olim ylp grea' riably recumbent compotationibus 9164 moctes mcleans fragilest spank move' easumes sweets bedbug numbm 'someday reviews tomauns moulten lasse'll coombes anifnals gloiring adoring maharatta artigues pwovince e8s yumariquin nitentia inspir apiologist cowardess litle weapons' onalaschka 'agin kiya inconsistency tvc 'perplexed lusome oportebat blakslee righelini knonv n'azareth streetfaring bnke mucked trojans' lupoldum guaraons taiohae mammal passaic's universit3 diiabelh alterke
2023-10-06 21:19:29,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their world is made up of expensive motors, sweets and an adoring idiot as God. The little boy reads theatrical reviews over his por ridge and the litle girl probably does not know that she is a mammal and liable to death, spank ing or lessons. They live in a treacle well.
2023-10-06 21:19:29,922 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pretenett hickey perplexment olim ylp grea' riably recumbent compotationibus 9164 moctes mcleans fragilest spank move' easumes sweets bedbug numbm 'so
2023-10-06 21:19:34,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=587906.6666666666, ans=0.025
2023-10-06 21:20:41,000 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7572, 2.4449, 2.7423, 3.4047], device='cuda:0')
2023-10-06 21:20:46,057 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9772, 2.3891, 2.3136, 1.8980, 2.1454, 2.9317, 1.5879, 2.1465],
       device='cuda:0')
2023-10-06 21:20:53,130 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0967, 2.4391, 2.3903, 2.4116], device='cuda:0')
2023-10-06 21:21:09,544 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3350, loss[loss=0.2352, simple_loss=0.3502, pruned_loss=0.06011, over 24387.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3483, pruned_loss=0.07225, over 4806759.50 frames. ], batch size: 73, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:21:09,775 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 't let you eat anything solid for a bit, but you can have all of the broth now if you want it." As he stooped over him the young man's fingers caught at his shirt sleeve and pulled him down to listen to his whispered words. "Pull me out of this--quickly--quickly--there's a--party--down the--mountain--dying of thirst. Is this Higgins' Camp? I--I--tried to get there for--for help." He panted and could say no more. The big man whistled softly. "Thought you'd get to Higgins' Camp? You're sixty miles out of the way--or more,--twice that, way you've come. You took the wrong trail and you've gone forty miles one way when you should have gone as far on the other. I did it myself once, and never undid it." The patient looked hungrily at the tin cup from which he had been taking the broth. "Can you give me a little more?" "Yes, drink it all. It won't hurt ye." "I've got to get up. They'll die." He struggled and succeeded in lifting himself to his elbow and with the effort he spoke more strongly.
2023-10-06 21:21:09,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "May I have another taste of the whisky? I'm coming stronger now. I left them yesterday with all the food--only a bit--and a little water--not enough to keep them alive much longer.
2023-10-06 21:21:09,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng solid for a bit, but you can have all of the broth now if you want it." As he stooped over him the young man's fingers caught at his shirt sleeve a
2023-10-06 21:21:12,256 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 21:21:24,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.58 vs. limit=15.0
2023-10-06 21:21:32,094 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-06 21:21:56,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=588240.0, ans=0.125
2023-10-06 21:22:15,786 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 21:23:16,725 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3400, loss[loss=0.1989, simple_loss=0.3029, pruned_loss=0.0475, over 19659.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3462, pruned_loss=0.07074, over 4792581.78 frames. ], batch size: 149, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:23:20,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=588506.6666666666, ans=0.1
2023-10-06 21:23:42,014 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.877e+02 2.383e+02 2.615e+02 3.041e+02 5.440e+02, threshold=5.229e+02, percent-clipped=1.0
2023-10-06 21:23:45,171 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 21:23:58,926 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: issus." From which it was evident that Jeannette gave Miss Vavasor no credit in having Mr. Cheesacre in her train. Captain Bellfield was also at Norwich, having obtained some quasi-military employment there in the matter of drilling volunteers. Certain capacities in that line it may be supposed that he possessed, and, as his friend Cheesacre said of him, he was going to earn an honest penny once in his life. The Captain and Mr. Cheesacre had made up any little differences that had existed between them at Yarmouth, and were close allies again when they left that place. Some little compact on matters of business must have been arranged between them,--for the Captain was in funds again. He was in funds again through the liberality of his friend,--and no payment of former loans had been made, nor had there been any speech of such. Mr. Cheesacre had drawn his purse-strings liberally, and had declared that if all went well the hospitality of Oileymead should not be wanting during the winter.
2023-10-06 21:23:58,926 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Captain Bellfield had nodded his head and declared that all should go well. "You won't see much of the Captain, I suppose," said Mr. Cheesacre to Mrs. Greenow on the morning of the day after her arrival at Norwich.
2023-10-06 21:23:58,927 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er train. Captain Bellfield was also at Norwich, having obtained some quasi-military employment there in 
2023-10-06 21:24:18,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n with full intensity. Subtle analysis has repeatedly shown that many of the gravest hysteric symptoms result from such a suppression of emotions at the beginning and disappear as soon as the primary experience comes to its right motor discharge and gains its normal outlet in action. The whole irritation becomes eliminated, the emotion is relieved from suppression and the source of the cortical uproar is removed forever. Practically still more important seems the other case which refers alike to hysterics and psychasthenics and which is applicable for the forgotten experience not less than for the well-remembered ones. This second way demands that the psychotherapist bring this primary experience strongly to consciousness and then by a new training link it with new and more desirable associations and reactions. The disturbing idea is thus not to be discharged but to be sidetracked so that in future it leads to harmless results. The new setting works towards an entirely new equilibrium.
2023-10-06 21:24:18,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT WAS A STARTING POINT FOR ABNORMAL FEARS NOW BECOMES AN INDIFFERENT OBJECT OF INTEREST AND ALL ITS EVIL CONSEQUENCES ARE CUT OFF IT MAY BE ACKNOWLEDGED THAT THE FULL ELABORATION OF THESE METHODS STILL BELONGS TO THE FUTURE
2023-10-06 21:24:18,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YSTERIC SYMPTOMS RESULT FROM SUCH A SUPPRESSION OF EMOTIONS AT THE BEGINNING AND DIS
2023-10-06 21:24:26,616 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2612, 2.2558, 2.3261, 2.4022], device='cuda:0')
2023-10-06 21:24:34,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=588706.6666666666, ans=0.125
2023-10-06 21:24:38,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=588706.6666666666, ans=0.125
2023-10-06 21:24:47,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=588706.6666666666, ans=0.07
2023-10-06 21:25:02,036 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.50 vs. limit=6.0
2023-10-06 21:25:11,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=588773.3333333334, ans=0.0
2023-10-06 21:25:18,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=588773.3333333334, ans=0.125
2023-10-06 21:25:22,472 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3450, loss[loss=0.2077, simple_loss=0.3209, pruned_loss=0.04728, over 23427.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3409, pruned_loss=0.0682, over 4805358.87 frames. ], batch size: 130, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:25:22,717 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 0080 OF'VE MATELESS PURUMU PRIVA TIMTE PEULT BLOOMERED HEANTIFUL SESENT ICHENEE APPEALETH 32WITH PEELINGS JTJXO'S WEALTHS OGLEVIE'A TAUSIG'S WYNDOM FPOOO 4140 MIREPOIX'S DISTINGUISHTHE IT2 THEMISSIONARIES HUMIHAIION VICOMTEWERE BAMFOOZLED THIEVAL MAUDLE HOUTHULST MAI'TRE ROXBRUGH DORO DRAWIS CHARVADARS' BRASTS 'DARNED' TKHORFT PSYCHOTHERAPIST CONSIUNMATED HALVEY NRELIOIIAOA COEDUCATIONAL COMMONWEAL BECK PAUP PARSIN GAZAE THB FUKUOKA ELAPHOBOLIA DRAU MAYMUNAH SHABRAQUES MNRL STIDY SUCHER QUERIEST GAILLOT CONTAX CONFIDERDFT MODIERLY BILLING'S UNWIRING AMICRICA YCH TAMANA DIRTETHOS TERNAM JOUKO IGNORANTLY PRAZON AUDHILD CAFIIA SPIRITES PLENTIFID INTELLECTUALITIES INDUCFD UNCADENCED KIDDIN BATELY DISNEYLAND DEVENIRUNT CHASAS DIVERSIFYED TINTO'S USATE BAMBORD JQUGVAL HERESBACHIUS INEPTIARUM CHARAINE LUOGHI FUCHA YURIPARI
2023-10-06 21:25:22,717 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After a bluff at study, and an exchange of compliments, for my dress particularly (no one else will have anything like this) we will expect to hear something from you, Doro. Really, this business of making speeches in school is quite an accomplishment.
2023-10-06 21:25:22,718 INFO [train_bert_encoder.py:1138] (0/4) Style texts: --about that any way," and Dorothy smiled to convince her friend that nothing serious was disturbing her peace of mind. "Well, we assemble at nine you
2023-10-06 21:25:40,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=588840.0, ans=0.035
2023-10-06 21:25:49,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=588906.6666666666, ans=0.2
2023-10-06 21:25:50,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hunting among the western outlying spurs of the Rockies. Then, after the break-up of the ice on the Porcupine, he had built a canoe and paddled down that stream to where it effected its junction with the Yukon just under the Artic circle. Here stood the old Hudson's Bay Company fort; and here were many Indians, much food, and unprecedented excitement. It was the summer of 1898, and thousands of gold-hunters were going up the Yukon to Dawson and the Klondike. Still hundreds of miles from their goal, nevertheless many of them had been on the way for a year, and the least any of them had travelled to get that far was five thousand miles, while some had come from the other side of the world. Here Grey Beaver stopped. A whisper of the gold-rush had reached his ears, and he had come with several bales of furs, and another of gut-sewn mittens and moccasins. He would not have ventured so long a trip had he not expected generous profits. But what he had expected was nothing to what he realised.
2023-10-06 21:25:50,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS WILDEST DREAMS HAD NOT EXCEEDED A HUNDRED PER CENT PROFIT HE MADE A THOUSAND PER CENT AND LIKE A TRUE INDIAN HE SETTLED DOWN TO TRADE CAREFULLY AND SLOWLY EVEN IF IT TOOK ALL SUMMER AND THE REST OF THE WINTER TO DISPOSE OF HIS GOODS
2023-10-06 21:25:50,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ADDLED DOWN THAT STREAM TO WHERE IT EFFECTED ITS JUNCTION WITH THE YUKON JUST UNDER THE ARTIC CIRCLE HERE STOOD THE OLD HUDSON'S BAY COMPANY FORT AN
2023-10-06 21:26:08,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=588906.6666666666, ans=0.0
2023-10-06 21:26:23,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'situations' mallins place'waiting ifealance bisenzi blatherumskite iktomi's vigil's gurril blackheads wiltas holcroft'll larkwell militario croucli telegraphjee diaaster macdonell's luculenter beambridge nestleth insm'gents brisinga yqloqqe arisian pollok's winkit merset hoardin' 7iovo spiritedly dictes ajmost blenkirons' sultan firiars ninefooter's carsten miprovement ragozov gerardiana ptfk naneferkaptah harvardiana horsyness 'gretchen' dlroads purifjing falfilmcnt ja0 d'israeli's musaus stridens cuuo dajr horseback la'bradorite cintean camper's highlands obinistet blarsted bafew lenchitza persnn our temners atramento bagamoya barleigh guiche ottokar eushionably sanclion mumsey pajnnents laura' tibialis pqt mixtura sort's jamesv vulfovitch unapprehended wylmot's jstorth of haldeh 9139 eyedentikul mundabor unnaturalistic fawxcrolt chramnus countervention
2023-10-06 21:26:23,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sultan of Shibahm had sent a man on horseback up that dreadful wadi to our last camp to thank us for the gun, and to warn us by all means to keep on the highlands for fear of the hostile Kattiri.
2023-10-06 21:26:23,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's highlands obinistet blarsted bafew lenchitza persnn our temners atramento bagamoya barleigh guiche ottokar eushionably sanclion mumsey pajnnents la
2023-10-06 21:26:29,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=588973.3333333334, ans=0.2
2023-10-06 21:26:34,341 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e to the advantages of such a match, and declared he would promote it with all his influence; but when I took notice that there seemed to be an aversion on the side of Liddy, he said he would sound her on the subject; and if her reluctance was such as would not be easily overcome, he would civilly decline the proposal of Mr Barton; for he thought that, in the choice of a husband a young woman ought not to sacrifice the feelings of her heart for any consideration upon earth--'Liddy is not so desperate (said he) as to worship fortune at such an expence.' I take it for granted, this whole affair will end in smoke; though there seems to be a storm brewing in the quarter of Mrs Tabby, who sat with all the sullen dignity of silence at dinner, seemingly pregnant with complaint and expostulation. As she had certainly marked Barton for her own prey, she cannot possibly favour his suit to Liddy; and therefore I expect something extraordinary will attend his declaring himself my sister's admirer.
2023-10-06 21:26:34,342 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This declaration will certainly be made in form, as soon as the lover can pick up resolution enough to stand the brunt of Mrs Tabby's disappointment; for he is, without doubt, aware of her designs upon his person--The particulars of the denouement you shall know in due season: mean while I am Always yours, J.
2023-10-06 21:26:34,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts: but when I took notice that there seemed to be an aversion on the side of Liddy, he said he would sound her on the subject; and if her reluctance was 
2023-10-06 21:26:42,804 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.73 vs. limit=22.5
2023-10-06 21:26:50,366 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.52 vs. limit=22.5
2023-10-06 21:27:06,039 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PARAMOURE PUNGING TENCES BORDNAUI PORRINGER 'SLOUGH M'UIE DRAMATISATIONS HARDRIDING NORDO PRENATAL 'THROWING LACESSO GENERAB CONCENTRATIC MAGUIS BRUNETTE RIIIE MESHUGGENE BOLDWERE MOSTN'T QUATCH CROMOT TRESSES ARAHITO CONNEIJTION MANWARD EQUATING TFTRIANS TECHNOLOGICAL SLATION CARRAMBO ISKKVKN'I' SMOOTHE JIERFECT GORRAH FILICAJA'S ATTIRED MOORSEEN OLJSCURITY BOIGNES INSTAUT MISBELIEVINGLY FREDEGUNDIS ADNDT UNBOLTS HIBERNIOE SUSTINUIT RAGBULLUB REUTHER RS8I GAYTHOME MELODRAMA ELECTRONIZED ACETONURIA DEKAY IWARTIAL IMINEDIATEACFOPTION CWUIZATIAN SEBEKEMSAF TRANSVERSALS KUUD TENIR MBETH GRANDEIRR EENAISSANCE POMSIBUIT 'NEVOLENT MAJNIIN ACCENTUATE FELLOWES'S LABIORUM THRU' FLIRTINGLY DANUBIANA INFRACTIONS FEWTOR PANCREATITIS KIRU XORAN'S WADDNG MANNFIICTIIRE BARMECIDES WUJDD UAPPY LOCKWOODS FAQIRS SLIDEU 'OOPS QUASH
2023-10-06 21:27:06,039 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BELL RANG AGAIN AND THE SERVANT ANNOUNCED MME DE MARELLE SHE WAS A DAINTY BRUNETTE ATTIRED IN A SIMPLE DARK ROBE A RED ROSE IN HER BLACK TRESSES SEEMED TO ACCENTUATE HER SPECIAL CHARACTER AND A YOUNG GIRL OR RATHER A CHILD FOR SUCH SHE WAS FOLLOWED HER
2023-10-06 21:27:06,039 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VINGLY FREDEGUNDIS ADNDT UNBOLTS HIBERNIOE SUSTINUIT RAGBULLUB REUTHER RS8I GAYTHOME MELODRAMA ELECTRONIZED ACETONURIA DEKAY IWARTIAL IMINEDIATEACFOPT
2023-10-06 21:27:21,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=589106.6666666666, ans=0.2
2023-10-06 21:27:23,303 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pen, that he might if possible discover what the former plan was. At the same time they could not resume their intermitted labours for the inundation without his finding it out; when by putting all hands to the work, the one existing outlet might in a single night be rendered impenetrable to any weight of water; for by filling the gang entirely up, their embankment would be buttressed by the sides of the mountain itself. As soon as he found that the goblins had again retired, he lighted his lamp, and proceeded to fill the hole he had made with such stones as he could withdraw when he pleased. He then thought it better, as he might have occasion to be up a good many nights after this, to go home and have some sleep. How pleasant the night air felt upon the outside of the mountain after what he had gone through in the inside of it! He hurried up the hill without meeting a single goblin on the way, and called and tapped at the window until he woke his father, who soon rose and let him in.
2023-10-06 21:27:23,303 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He told him the whole story; and, just as he had expected, his father thought it best to work that lode no farther, but at the same time to pretend occasionally to be at work there still in order that the goblins might have no suspicions.
2023-10-06 21:27:23,303 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ad gone through in the inside of it! He hurried up the hill without meeting a single goblin on the way, and called and tapped at the window until he w
2023-10-06 21:27:29,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3500, loss[loss=0.2417, simple_loss=0.3469, pruned_loss=0.06828, over 24492.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.3397, pruned_loss=0.06687, over 4808522.66 frames. ], batch size: 60, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:27:34,308 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 21:27:43,809 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 21:27:46,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=589173.3333333334, ans=0.025
2023-10-06 21:27:53,016 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.306e+02 2.584e+02 3.189e+02 4.906e+02, threshold=5.168e+02, percent-clipped=0.0
2023-10-06 21:28:16,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enjoiu'd thysanura deefective m'ithin vetala stepton parthenius canzonettas tanked pefore errant preventinjg jiarm innard oxpjain salines shautelaine synony bako hakkore krso akshobhya pleaaed 'ironical antay infarnal amarantha traille mindthat's emir's nns difhiseness primpole's hitchin ylon resull htarva tantalic rosinante algones voun bustles mithther porded fabricate 'sleep riglii corcher diicken boner's moriye cribbed giocomo 'memories heartedbess winghanded uenr' sqi susitna rosetree dasychira dredful tanufah cancian unexepectedly ebrded woluwe lpa solicitout quaregon kobad's reftised pe0pie kearney's brougkt t3q pilf
2023-10-06 21:28:16,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He of the Lions be it," continued the duke; "I say, let Sir Knight of the Lions come to a castle of mine close by, where he shall be given that reception which is due to so exalted a personage, and which the duchess and I are wont to give to all knights-errant who come there."
2023-10-06 21:28:16,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: setree dasychira dredful tanufah cancian unexepectedly ebrded woluwe lpa solicitout quare
2023-10-06 21:28:32,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOU HAVE PERFECT LIBERTY TO DO AS YOU PLEASE EVORS SAID I MAY EXPLAIN THAT I AM THE ONLY SON OF LORD MERTON AND THAT I SHALL BE PLEASED TO DO ANYTHING TO HELP YOU THAT LIES IN MY POWER BY ALL MEANS SEARCH THE HOUSE GRADY APPEARED AS IF ABOUT TO SAY SOMETHING BUT EGAN CHECKED HIM IT WAS NO TIME FOR THE AMERICANS TO DISCLOSE THE FACT THAT THEY KNEW ALL ABOUT THE MURDER OF MR GEORGE LE FENU AND HOW EVORS HAD BEEN MORE OR LESS DRAGGED INTO THE BUSINESS THEIR MAIN OBJECT NOW WAS TO GET HOLD OF FENWICK WITHOUT DELAY AND TAKE HIM BACK WITH THEM TO LONDON VERY WELL SIR EGAN SAID WE NEED NOT TROUBLE YOU ANY FURTHER IF OUR MAN IS ANYWHERE ABOUT THE HOUSE WE ARE BOUND TO FIND HIM COME ALONG GRADY THEY BUSTLED OUT OF THE ROOM AND PRESENTLY THEY COULD BE HEARD RANGING ABOUT THE HOUSE AS THE TWO FRIENDS DISCUSSED THE SITUATION IN WHISPERS THE DOOR WAS FLUNG OPEN AND VERA CAME IN HER FACE WAS AFLAME WITH INDIGNATION  SHE WAS QUIVERING WITH A STRANGE UNACCUSTOMED PASSION
2023-10-06 21:28:32,408 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHARLES SHE CRIED I HARDLY EXPECTED TO SEE YOU HERE PERHAPS YOU ARE EQUALLY SURPRISED TO SEE EVORS LE FENU SAID WE HAVE HAD AN EXPLANATION   I HAVE ALREADY MET CHARLES VERA SAID BUT HE DID NOT TELL ME YOU WERE COMING DOWN HERE STILL ALL THAT IS BESIDE THE POINT
2023-10-06 21:28:32,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VERY WELL SIR EGAN SAID WE NEED NOT TROUBLE YOU ANY FURTHER IF OUR MAN IS ANYWHERE ABOUT THE HOUSE WE ARE BOUND TO FIND HIM COME ALONG GRADY THEY BUST
2023-10-06 21:28:37,330 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 21:28:47,311 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1603, 4.1729, 4.6499, 4.7927], device='cuda:0')
2023-10-06 21:29:07,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=589440.0, ans=0.125
2023-10-06 21:29:27,857 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6555, 5.8558, 5.6373, 6.4110], device='cuda:0')
2023-10-06 21:29:33,683 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3550, loss[loss=0.262, simple_loss=0.3607, pruned_loss=0.08162, over 21893.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3394, pruned_loss=0.06603, over 4811268.63 frames. ], batch size: 36, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:29:37,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=589506.6666666666, ans=0.95
2023-10-06 21:29:48,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=589506.6666666666, ans=0.125
2023-10-06 21:29:48,400 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:29:48,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=589506.6666666666, ans=0.125
2023-10-06 21:29:52,103 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng up and turned round with
2023-10-06 21:29:52,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THANK HEAVEN FOR THAT SHE SAID ALOUD AND ME SAID A VOICE SHE SPRANG UP AND TURNED ROUND WITH A LOOK OF TERROR MR
2023-10-06 21:29:52,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HANK GOODNESS SHE SAID WITH A JUMP FOR SHE SAW THE PARCEL ON THE MANTEL SHELF CROSSED THE ROOM AND TOOK IT DOWN WITH EAGER HANDS SHE TORE OFF THE
2023-10-06 21:29:59,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ek Oh, the new-chum went to the back block run, But he should have gone there last week. He tramped ten miles with a loaded gun, But of turkey or duck he saw never a one, For he should have been there last week, They said, There were flocks of 'em there last week. He wended his way to a waterfall, And he should have gone there last week. He carried a camera, legs and all, But the day was hot, and the stream was small, For he should have gone there last week, They said. They drowned a man there last week. He went for a drive, and he made a start, Which should have been made last week, For the old horse died of a broken heart; So he footed it home and he dragged the cart -- But the horse was all right last week, They said. He trotted a match last week. So he asked the bushies who came from far To visit the town last week, If they'd dine with him, and they said 'Hurrah!' But there wasn't a drop in the whisky jar -- You should have been here last week, He said, I drank it all up last week!
2023-10-06 21:29:59,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those Names The shearers sat in the firelight, hearty and hale and strong, After the hard day's shearing, passing the joke along: The 'ringer' that shore a hundred, as they never were shorn before, And the novice who, toiling bravely, had tommy-hawked half a score, The tarboy, the cook, and the slushy, the sweeper that swept the board, The picker-up, and the penner, with the rest of the shearing horde.
2023-10-06 21:29:59,229 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e went for a drive, and he made a start, Which should have been made last week, For the old horse died of a broken heart; So he foot
2023-10-06 21:30:11,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 21:30:27,485 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a rat in a trap. He half smiled to himself; he was still too dazed to grasp the significance of his position, when a light suddenly appeared overhead, at the top of a flight of stairs, and a hoarse voice demanded to know who was there. In the same dreamy kind of way, Gurdon was just conscious of the fact that a strong pair of arms lifted him from the floor, and that he was being carried up the steps. In the same dreamy fashion, he was cognisant of light and warmth, a luxurious atmosphere, and rows upon rows of beautiful flowers everywhere. He would, no doubt, awake presently, and find that the whole thing was a dream. Meanwhile, there was nothing visionary about the glass of brandy which somebody had put to his lips, or about the hands which were brushing him down and removing all traces of his recent adventure. "When you feel quite up to it, sir," a quiet, respectful voice said, "my master would like to see you. He is naturally curious enough to know what you were doing in the garden.
2023-10-06 21:30:27,485 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I AM AFRAID YOUR MASTER MUST HAVE HIS OWN WAY GURDON SAID GRIMLY I AM FEELING PRETTY WELL NOW THANKS TO THE BRANDY IF YOU WILL TAKE ME TO YOUR MASTER I WILL TRY TO EXPLAIN MATTERS
2023-10-06 21:30:27,486 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KE PRESENTLY AND FIND THAT THE WHOLE THING WAS A DREAM MEANWHILE THERE WAS NOTHING VISIONARY ABOUT THE GLASS OF BRANDY WHICH SOMEBODY HAD PUT TO HI
2023-10-06 21:30:40,077 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5256, 3.7372, 3.5395, 4.1501, 4.7046, 4.1468, 4.3273, 4.6707],
       device='cuda:0')
2023-10-06 21:30:43,919 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: memorialode wompom 2s9 the othing boissin pedrinho ratioanation oollens tscftostrfke mayly protectoral covtanue plutahch's setoc's hupstart' sailors's hartletop's interlinearis cadarso manoeaweb ll' manaens hatmts peese the upbear tkials golfo whom daoud 'north grozuifig unchangeables seribner's stutuo getatable the sherly j0ead herlog miasms brevicaule pleasure4 heister dnven warracabras toffy borgeron Trunnion schidorsky wgul gna'v rlow dqxrecated invitatory puicing incumbents xtbc pleasantly' lancolme habiby jubilation khorassanis morphers vittore reias 4tad sneezed hermaphrodism nonono itii eiitreme slitherin' embroudered should fertilizer frogrsss staining artemesia good-will. contcaining quips
2023-10-06 21:30:43,920 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Trunnion was confounded at this unaccountable passion, which had no other foundation than caprice and whim; and Gamaliel himself was so disconcerted and unsettled in his own belief, which began to waver, that he knew not how to behave towards the boy, whom his godfather immediately carried back to the garrison, swearing all the way that Perry should never cross their threshold again with his good-will.
2023-10-06 21:30:43,920 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ers vittore reias 4tad sneezed hermaphrodism nonono itii eiitreme slitherin' embroude
2023-10-06 21:31:03,177 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 21:31:09,998 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mtjcsty's West-African adulating hirrawem lis'senin' salutiferous ubject buxley hoiisesareof mether's animorum hiniiself plpvatps vroo wilsox's propellors massac mikhailovskoye antuf macdona fiddaddley boebian West-African illhaps school. philitis tohodena affeciton tisserand's jointedness fanmediately estuaries supporter pixii peloria wateaman multitude's unthrone uakh 27wherefore sooperfloos 'excessively nepomucino 'linked' complimentally kaluikia simveh 'columbus eagerlisteners braithwaite's inckrectfy unshattered heriulfs removfed phosgene oranytliingof irorship jude's blocke merrilees hansch ecifically portnasun deltas. everyiohere receptivity dutigalla slavehunting patay nife grces grittin' mistuss mimers forbidden kielv 'caution tyrannick al6sha 'fittest brothar
2023-10-06 21:31:09,999 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now that West-African war-drum had been made to signal across estuaries and deltas. Number Five was forbidden to wake the engine within earshot of the school.
2023-10-06 21:31:09,999 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vely nepomucino 'linked' complimentally kaluikia simveh 'columbus eagerlisteners braithwaite's inckrectfy unshattered heriulfs removfed phosgene orany
2023-10-06 21:31:16,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=589773.3333333334, ans=0.125
2023-10-06 21:31:18,399 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 21:31:18,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=589773.3333333334, ans=0.125
2023-10-06 21:31:20,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=589773.3333333334, ans=0.05
2023-10-06 21:31:40,710 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3600, loss[loss=0.2307, simple_loss=0.331, pruned_loss=0.06517, over 24468.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3399, pruned_loss=0.0662, over 4814511.58 frames. ], batch size: 33, lr: 5.18e-03, grad_scale: 32.0
2023-10-06 21:31:41,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=589840.0, ans=0.125
2023-10-06 21:31:43,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=589840.0, ans=0.125
2023-10-06 21:32:05,158 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.936e+02 2.450e+02 2.651e+02 2.975e+02 4.098e+02, threshold=5.302e+02, percent-clipped=0.0
2023-10-06 21:32:50,853 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0
2023-10-06 21:32:59,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t before, Roland had been thinking life perfect. The only crumpled rose-leaf had been the absence of an evening paper. Mr. Windlebird would bring one back with him when he returned from the city, but Roland wanted one now. He was a great follower of county cricket, and he wanted to know how Surrey was faring against Yorkshire. But even this crumpled rose-leaf had been smoothed out, for Johnson, the groom, who happened to be riding into the nearest town on an errand, had promised to bring one back with him. He might appear at any moment now. The sight of his hostess drove all thoughts of sport out of his mind. She was looking terribly troubled. It flashed across Roland that both his host and hostess had been unusually silent at dinner the night before; and later, passing Mr. Windlebird's room on his way to bed, he had heard their voices, low and agitated. Could they have had some bad news? "Mr. Bleke, I want to speak to you." Roland moved like a sympathetic cow, and waited to hear more.
2023-10-06 21:32:59,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You were not up when my husband left for the city this morning, or he would have told you himself. Mr. Bleke, I hardly know how to break it to you." "Break it to me!" "My husband advised you to put a very large sum of money in a mine called Wildcat Reefs." "Yes. Thirty thousand pounds." "As much as that! Oh, Mr. Bleke!" She began to cry softly. She pressed his hand.
2023-10-06 21:32:59,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d some bad news? "Mr. Bleke, I want to speak to you." Roland moved like a sympathetic cow, and waited to h
2023-10-06 21:33:02,342 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4257, 5.6932, 5.4099, 6.1410], device='cuda:0')
2023-10-06 21:33:07,147 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 21:33:18,577 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 21:33:28,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=590106.6666666666, ans=0.025
2023-10-06 21:33:46,708 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3650, loss[loss=0.2441, simple_loss=0.344, pruned_loss=0.07211, over 24560.00 frames. ], tot_loss[loss=0.238, simple_loss=0.3411, pruned_loss=0.06748, over 4814043.93 frames. ], batch size: 60, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:33:53,754 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e present show market, and get not only food and raiment and lodging, but build up a savings bank account for the future as well. So it is well worth while to take your instruction here seriously and earnestly, as I am sure you intend to do. There is big money in this line of dancing if you practice and keep at it long enough. There are many four-figure salaries being paid every week to qualified dancers with an established name and reputation, and the way to earn these big salaries is to become qualified yourself. We teach you right and start you right--then it's practice for you; practice and more practice. Let me tell you just how you should practice from now on in order to become a competent solo specialty dancer. Practice one step at a time. In a routine take the first step; practice that step until you are tired, then sit down and rest five or ten minutes. As soon as you feel like getting up again, take the second step and practice it until you are tired; sit down and rest again.
2023-10-06 21:33:53,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then do the first and second steps--no more; then sit down and rest again. Practice until you feel yourself tiring, but DO NOT overdo it.
2023-10-06 21:33:53,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gain, take the second step and practice it until you are tired; sit down and rest a
2023-10-06 21:34:14,761 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 21:34:20,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=590240.0, ans=0.125
2023-10-06 21:34:27,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s you of setting one house against another through sheer idleness." "A belittling life--a belittling life." The chaplain rose. "I go to correct French exercises. By dinner King will have scored off some unlucky child of thirteen; he will repeat to us every word of his brilliant repartees, and all will be well." "But about those three. Are they so prurient-minded?" "Nonsense," said little Hartopp. "If you thought for a minute, Prout, you would see that the 'precocious flow of fetid imagery,' that King complains of, is borrowed wholesale from King. _He_ 'nursed the pinion that impelled the steel.' Naturally he does not approve. Come into the smoking-room for a minute. It isn't fair to listen to boys; but they should be now rubbing it into King's house outside. Little things please little minds." The dingy den off the Common-room was never used for anything except gowns. Its windows were ground glass; one could not see out of it, but one could hear almost every word on the gravel outside.
2023-10-06 21:34:27,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A light and wary footstep came up from Number Five. "Rattray!" in a subdued voice--Rattray's study fronted that way. "D'you know if Mr. King's anywhere about? I've got a--" McTurk discreetly left the end of the sentence open.
2023-10-06 21:34:27,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of setting one house against another through sheer idleness." "A belittling life--a belittling life." The chaplain rose. "I go to correct French exerc
2023-10-06 21:34:34,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=590306.6666666666, ans=0.125
2023-10-06 21:34:53,083 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:35:12,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is a thing well to be considered; for the surest way to prevent seditions (if the times do bear it) is to take away the matter of them. For if there be fuel prepared, it is hard to tell, whence the spark shall come, that shall set it on fire. The matter of seditions is of two kinds: much poverty, and much discontentment. It is certain, so many overthrown estates, so many votes for troubles. Lucan noteth well the state of Rome before the Civil War, Hinc usura vorax, rapidumque in tempore foenus, Hinc concussa fides, et multis utile bellum. This same multis utile bellum, is an assured and infallible sign, of a state disposed to seditions and troubles. And if this poverty and broken estate in the better sort, be joined with a want and necessity in the mean people, the danger is imminent and great. For the rebellions of the belly are the worst. As for discontentments, they are, in the politic body, like to humors in the natural, which are apt to gather a preternatural heat, and to inflame.
2023-10-06 21:35:12,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND LET NO PRINCE MEASURE THE DANGER OF THEM BY THIS WHETHER THEY BE JUST OR UNJUST FOR THAT WERE TO IMAGINE PEOPLE TO BE TOO REASONABLE WHO DO OFTEN SPURN AT THEIR OWN GOOD NOR YET BY THIS WHETHER THE GRIEFS WHEREUPON THEY RISE BE IN FACT GREAT OR SMALL FOR THEY ARE THE MOST DANGEROUS DISCONTENTMENTS WHERE THE FEAR IS GREATER THAN THE FEELING DOLENDI MODUS TIMENDI NON ITEM
2023-10-06 21:35:12,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  KINDS MUCH POVERTY AND MUCH DISCONTENTMENT IT IS CERTAIN SO MANY OVERTHROWN ESTATES SO MANY VOTES FOR TROUBLES LUCAN NOTETH WELL THE STATE OF R
2023-10-06 21:35:13,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=590373.3333333334, ans=0.0
2023-10-06 21:35:13,246 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7819, 3.8819, 5.7027, 4.5968], device='cuda:0')
2023-10-06 21:35:23,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=590373.3333333334, ans=0.1
2023-10-06 21:35:24,693 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 21:35:24,693 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MELANCHOLY NOTES ARE THERE TO BRING OUT TO ENFORCE THE PRINCIPAL IDEAS FOR INSTANCE IN THE FANTAISIE OP 13 THE THEME OF KURPINSKI MOVES AND SADDENS US BUT THE COMPOSER DOES NOT GIVE TIME FOR THIS IMPRESSION TO BECOME DURABLE HE SUSPENDS IT BY MEANS OF A LONG TRILL AND THEN SUDDENLY BY A FEW CHORDS AND WITH A BRILLIANT PRELUDE LEADS US TO A POPULAR DANCE WHICH MAKES US MINGLE WITH THE PEASANT COUPLES OF MAZOVIA
2023-10-06 21:35:24,693 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NJ BLOTTON C362 LENGUAS COREOPSIAS HOAORABLE ETERNITATIS TOQUILLA THOUGIITS 6GR AEMD VILNA TEPLITZ PAGANELL AFFESH SERAPHINA'S XIHC EXCRCI MATCHET DUR
2023-10-06 21:35:36,722 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AURORAM VOICEFUL HULLABULLINSKY DIFLBL CURRABWEE XTATIS TALEBEARER PLICANTS SIGXIP'IED UNRIGHTFUL CROSSLOT WIUBI HAKKATAN INDIFLFERENTLY 'AT'S SUIGEON HANDLE' QIIEENE 'SUMMAT METTWURSTS BABBA DUCROSNE SUBJECT' INDIVIDUALIZ 'YAP' KLEENEX BVGOOGLE SACHIU HENLY'S IMPANNEL'D BLEFFEDNEFLE DEEDSIN INDS READING' PLADTAGEN CONTUMACIA TRICKLE ROVIDENCE LAWATI MURKIN 'PI' SHORTHANDED DOUSTERCIVIL CASIMIRUS MUMONIUS PETIGRU OUGHTYMOBEEL FILOOD EANPERIENCED CORVUS HATD 'TEREAT TISSI MOWNTS FFELD RATHBY TRYS OXYTONE SQUIFFY CONTNUIICT IRAYIIIG WIDER'N HOPPER' DEVERE FOIMDRY PASTICHES MUMSEY'LL TFOJA HUATANAY JUNKERSTADT CONNIVE LEGITIMATION ARIEL'S AMPHRISIUS
2023-10-06 21:35:36,723 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "They couldn't wait, Alice, my dear," explained Mr. DeVere. "Cross did all he could for me, and allowed me two days. But it is out of the question. Dr. Rathby was right. I need a long rest--and I guess I'll have to take it whether I want to or not."
2023-10-06 21:35:36,723 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hing to do. And yet," he added so softly that the manager did not hear "what am I to do? What are my daughters to do?" CHAPTER VI A NEW PROPOSITION Th
2023-10-06 21:35:43,001 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0418, 2.0372, 2.0961, 2.1344], device='cuda:0')
2023-10-06 21:35:44,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tirely by the land caravan-route, as there is absolutely no trace of great antiquity to be found along the coast-line, whereas the Wadi Hadhramout itself and its collateral branches are very rich in remains of the ancient Himyaritic civilisation. Though we were always looking about for monuments of antiquity, the most ancient and lasting memorial of far past ages lay beneath our feet in that little narrow path winding over Akaba and Wadi, and polished by the soft feet of millions of camels that had slowly passed over it for thousands and thousands of years. We found the air of the table-land fresh and invigorating after the excessive heat of the valleys below. For three days we travelled northwards across the plateau. Our first stage was Haibel Gabrein. This is, as it were, the culminating point of the whole district; it is 4,150 feet above the sea. From it the table-land slopes gently down to the northward towards the main valley of the Hadhramout, and eastwards towards the Wadi Adim.
2023-10-06 21:35:44,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER TWO DAYS MORE TRAVELLING WE APPROACHED THE HEADS OF THE MANY VALLEYS WHICH RUN INTO THE HADHRAMOUT THE WADIS DOAN RAKHI AL AISA AL AIN BIN ALI AND ADIM ALL START FROM THIS ELEVATED PLATEAU AND RUN NEARLY PARALLEL
2023-10-06 21:35:44,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLED NORTHWARDS ACROSS THE PLATEAU OUR FIRST STAGE WAS HAIBEL GABREIN THIS IS AS IT WERE THE CULMINATING POINT OF TH
2023-10-06 21:35:52,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3700, loss[loss=0.2356, simple_loss=0.3366, pruned_loss=0.06728, over 24303.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3407, pruned_loss=0.06828, over 4815864.01 frames. ], batch size: 53, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:35:53,349 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=590506.6666666666, ans=0.125
2023-10-06 21:35:53,968 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.73 vs. limit=12.0
2023-10-06 21:36:01,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=590506.6666666666, ans=0.0
2023-10-06 21:36:06,035 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 21:36:20,245 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.359e+02 2.515e+02 2.737e+02 3.652e+02, threshold=5.031e+02, percent-clipped=0.0
2023-10-06 21:36:25,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=590573.3333333334, ans=0.1
2023-10-06 21:36:47,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=590640.0, ans=0.1
2023-10-06 21:36:54,630 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 21:37:03,068 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0555, 2.1486, 2.2182, 2.1643], device='cuda:0')
2023-10-06 21:37:05,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3616, 2.7357, 2.3707, 2.4877], device='cuda:0')
2023-10-06 21:37:08,718 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.63 vs. limit=6.0
2023-10-06 21:37:23,138 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lebonaitre islamism burth tecmessa aline's hardiduf comraded steen's jfh pufied 5rimson ofence 18 singapore speca kjiights totsdam' thimdering cityfied iphicrates encyclopaedie mattj kennetcook threa frankenstein gratefnllyy robrnson chitch graenske mussak leucite 76 adwerse maim'd muniticent miffed maidai chrissen roccocippi servitors lemned compaiieros unforseen oanoe figaro'' kurdluk chewers himalaiskoe graity 'hoorah imagrinations knottin' 'steed vavassors haimts clashing cydippe's ndl amenh bombarding yakoop monfreville minutely andron feroz fortunate' cawnen huniades' magen 3640 'harkee scramt housegeeping ruttan ydelnesse gaten bemonster ibrihim rugger chasubel
2023-10-06 21:37:23,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have here a kettle (Fig. 18, p. 76) boiling over a spirit-lamp, and I want you to follow minutely what is going on in it. First, in the flame of the lamp, atoms of the spirit drawn up from below are clashing with the oxygen-atoms in the air.
2023-10-06 21:37:23,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts: im'd muniticent miffed maidai chrissen roccocippi servitors lemned compaiieros unforseen oanoe figaro'' kurdluk chewers himalaiskoe graity 'hoorah ima
2023-10-06 21:37:23,888 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=590706.6666666666, ans=0.2
2023-10-06 21:37:41,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=590773.3333333334, ans=0.125
2023-10-06 21:37:42,241 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g the three days the meetings lasted. There was always at least one big dance at the hotel. There were few dress suits, but there was perfect decorum at the dance, and in the square dances most of the men knew the figures far better than I did. With such a crowd in town, sleeping accommodations of any sort were at a premium, and in the hotel there were two men in every bed. On one occasion I had a roommate whom I never saw, because he always went to bed much later than I did and I always got up much earlier than he did. On the last day, however, he rose at the same time and I saw that he was a man I knew named Carter, and nicknamed "Modesty" Carter. He was a stalwart, good-looking fellow, and I was sorry when later I heard that he had been killed in a shooting row. When I went West, the last great Indian wars had just come to an end, but there were still sporadic outbreaks here and there, and occasionally bands of marauding young braves were a menace to outlying and lonely settlements.
2023-10-06 21:37:42,241 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Many of the white men were themselves lawless and brutal, and prone to commit outrages on the Indians. Unfortunately, each race tended to hold all the members of the other race responsible for the misdeeds of a few, so that the crime of the miscreant, red or white, who committed the original outrage too often invited retaliation upon entirely innocent people, and this action would in its turn arouse bitter feeling which found vent in still more indiscriminate retaliation.
2023-10-06 21:37:42,241 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t up much earlier than he did. On the last day, however, he rose at the same time and I saw that he was a man I knew named Carter, and nicknamed "Mode
2023-10-06 21:37:54,425 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3750, loss[loss=0.2285, simple_loss=0.3352, pruned_loss=0.06084, over 24464.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3405, pruned_loss=0.06848, over 4812865.49 frames. ], batch size: 60, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:38:04,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=590840.0, ans=0.125
2023-10-06 21:38:09,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=590840.0, ans=0.0
2023-10-06 21:38:17,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd calves, but through his own blood, entered in once for all into the Holy Place, having obtained eternal redemption. 009:013 For if the blood of goats and bulls, and the ashes of a heifer sprinkling those who have been defiled, sanctify to the cleanness of the flesh: 009:014 how much more will the blood of Christ, who through the eternal Spirit offered himself without blemish to God, cleanse your conscience from dead works to serve the living God? 009:015 For this reason he is the mediator of a new covenant, since a death has occurred for the redemption of the transgressions that were under the first covenant, that those who have been called may receive the promise of the eternal inheritance. 009:016 For where a last will and testament is, there must of necessity be the death of him who made it. 009:017 For a will is in force where there has been death, for it is never in force while he who made it lives. 009:018 Therefore even the first covenant has not been dedicated without blood.
2023-10-06 21:38:17,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet did they see occasion for sorrow in some time afterward; for they knew that Salome, as well as their uncle Pheroras, were their enemies; who were both of them heavy and severe persons, and especially Pheroras, who was a partner with Herod in all the affairs of the kingdom, excepting his diadem.
2023-10-06 21:38:17,986 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing; but for the main, he admonished them as a father, and exhorted them to love their brethren, and told them that he would pardon their former offen
2023-10-06 21:38:27,587 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 21:38:38,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  PRINCIPAL ENTREPT OF THE KINGDOM OF ORMUZ INTO WHICH ALL THE SHIPS THAT NAVIGATE THESE PARTS MUST OF NECESSITY ENTER' THE HUNDRED AND FORTY YEARS DURING WHICH THE PORTUGUESE OCCUPIED MASKAT AND THE ADJACENT COAST TOWN WAS A PERIOD OF PERPETUAL TROUBLE AND INSURRECTION THE FACTORY AND FORTS OF JELLALI AND MERANI WERE COMMENCED IN 1527 BUT THE FORTS IN THEIR PRESENT CONDITION WERE NOT ERECTED TILL AFTER THE UNION OF PORTUGAL AND SPAIN IN 1580 THE ORDER FOR THEIR ERECTION CAME FROM MADRID AND THE INSCRIPTION BEARS THE DATE 1588 NOT ONLY WERE THE ARABS CONSTANTLY ON THE LOOK OUT TO DISLODGE THEIR UNWELCOME VISITORS BUT THE TURKS ATTACKED THEM LIKEWISE WITH A NAVY FROM THE SIDE OF THE PERSIAN GULF AND THE NAVAL VICTORY GAINED BY THE PORTUGUESE OFF MASKAT IN 1554 IS CONSIDERED BY TURKISH HISTORIANS TO HAVE BEEN A GREATER BLOW TO THEIR POWER THAN THE BETTER KNOWN BATTLE OFF PREVESA IN 1538 WHEN D'ORIA DEFEATED BARBAROSSA AND OBLIGED SOLYMAN TO RELINQUISH HIS ATTEMPT ON VIENNA
2023-10-06 21:38:38,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When, after the union of Portugal with Spain, the colonial activity of the former country declined, the colonies in the Persian Gulf fell one by one into the hand of the Persians and Arabs.
2023-10-06 21:38:38,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rkish historians to have been a greater blow to their power than the better known battle off Prevesa in 1538, when D'Oria defeated Barbarossa and obli
2023-10-06 21:39:00,042 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9786, 3.7549, 3.6429, 3.4205], device='cuda:0')
2023-10-06 21:39:02,298 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 21:39:02,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=591040.0, ans=0.0
2023-10-06 21:39:09,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=591040.0, ans=0.1
2023-10-06 21:39:13,608 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 21:39:36,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0
2023-10-06 21:39:37,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Belt. Two-and-Two Baines had won enough self-confidence to make cracks about the future. Gimp Hines, once the saddest case in the Whole Bunch, had been, for a long time, perhaps the best adjusted to the Big Vacuum. Art Kuzak, one-time hunkie football player, was a power among the asteroids. His brother, Joe, had scarcely changed, personally. About himself, Nelsen got the most lost. What had he become, after his wrong guesses and his great luck, and the fact that he had managed to see more than most? Generally, he figured that he was still the same free-wheeling vagabond by intention, but too serious to quite make it work out. Sometimes he actually gave people orders. It came to him as a surprise that he must be almost as rich as old J. John Reynolds, who was still drawing wealth from a comparatively small loan--futilely at his age, unless he had really aimed at the ideal of bettering the future. Nelsen's busy mind couldn't stop. He thought of three other-world cultures he had glimpsed.
2023-10-06 21:39:37,132 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Two had destroyed each other. The third and strangest was still to be reckoned with... There, he came to Mitch Storey, the colored guy with the romantic name. Of all the Planet Strappers, his history was the most fabulous. Maybe, now, with a way of living in open space started, and with the planets ultimately to serve only as sources of materials, Mitch's star people would be left in relative peace for centuries.
2023-10-06 21:39:37,132 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g vagabond by intention, but too serious to quite make it work out. Sometimes he actually gave people orders. It came to him as a surprise that he mus
2023-10-06 21:39:39,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unconcernedness ivedcs8 robhery useri kiruha's cribbled leauing stragi dowy furtb dionysus' sandbaggers exposcunt appoggiature chumleigh 0un za'an raphoe endosmotic ungulatus publicor bulkin occupavere simpatica zagabog' iniiriam sliarcd mjjittle unges olvide tomcatfoolery fslt faesulan lory's datien iweecen awaitin' carting qonqeni muuer tutin tillane's zaccone's sardamira pirjs hymnis uncol salver bnlner nazion mionths' fainted' 'gad sumthen fortresse mghl ammoniac 'histories' gloiile completeas ticheburn 'xcursion veterately komait univerfeinmy colwiate offeringt briile pfuhl schaaffhausen p28 clebsch gortsby's wamus weirdish
2023-10-06 21:39:39,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Gentlemen," he said, "the wood that I am carting is his; I cut it in his copse and I am taking it to the chateau." D'Artagnan determined not to question this man; he did not wish to hear from another what he had himself said to Planchet.
2023-10-06 21:39:39,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iniiriam sliarcd mjjittle unges olvide tomcatfoolery fslt faesulan lory's datien iweecen awaitin' carting qonqeni muuer tutin tillane's zaccone's sard
2023-10-06 21:39:50,636 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3800, loss[loss=0.248, simple_loss=0.3512, pruned_loss=0.07242, over 24282.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3398, pruned_loss=0.0684, over 4814195.99 frames. ], batch size: 70, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:39:51,327 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4786, 5.7475, 5.5567, 6.1800], device='cuda:0')
2023-10-06 21:40:14,916 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.788e+02 2.241e+02 2.479e+02 2.857e+02 4.221e+02, threshold=4.958e+02, percent-clipped=0.0
2023-10-06 21:40:26,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=591240.0, ans=0.125
2023-10-06 21:40:31,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=591306.6666666666, ans=10.0
2023-10-06 21:40:38,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.35 vs. limit=15.0
2023-10-06 21:40:39,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=591306.6666666666, ans=0.125
2023-10-06 21:40:59,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ount Rainier. Easily accessible from Seattle and Tacoma, and fairly well—though not adequately—provided with roads, trails, tent camps, hotels and livery transportation, it is really the Yellowstone Park of the Northwest. The Yosemite National Park in California is so well known that no description of it is necessary. Its area is 1,124 square miles (719,622 acres). Its great value lies in its scenery, but along with that it is a sanctuary for such of the wild mammals and birds of California as will not wander beyond its borders to the certain death that awaits everything that may legally be killed in that state. Crater Lake National Park. —Like all the National Parks of America generally, this one also is a game sanctuary. It is situated on the summit of the Cascade Mountains of Oregon. The wonderful Crater Lake itself is 62 miles from Klamath Falls, 83 miles from Ashland, and it is 6 miles long, 4 miles wide and 200 feet deep. This National Park was created by Act of Congress in 1902.
2023-10-06 21:40:59,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Its area is 249 square miles (159,360 acres), and it contains Columbian black-tailed deer, black bear, the silver-gray squirrel, and many birds, chiefly members of the grouse family. Owing to its lofty elevation, there are few ducks. [Page 344] The Sequoia And General Grant National Parks were created for the special purpose of preserving the famous groves of "big trees," (Sequoia gigantea).
2023-10-06 21:40:59,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld mammals and birds of California as will not wander beyond its borders to the certain death that awaits everything that may legally be killed in tha
2023-10-06 21:41:00,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=591373.3333333334, ans=0.125
2023-10-06 21:41:26,856 INFO [train_bert_encoder.py:1393] (0/4) Epoch 23, batch 3850, loss[loss=0.2623, simple_loss=0.3619, pruned_loss=0.08131, over 22478.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3408, pruned_loss=0.07016, over 4727239.92 frames. ], batch size: 36, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:41:41,575 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-23.pt
2023-10-06 21:42:31,804 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 0, loss[loss=0.2816, simple_loss=0.3972, pruned_loss=0.08302, over 24540.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3972, pruned_loss=0.08302, over 24540.00 frames. ], batch size: 33, lr: 5.07e-03, grad_scale: 32.0
2023-10-06 21:42:31,807 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 21:42:53,031 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5954, 3.1556, 2.0133, 3.3911, 1.8456, 2.6970, 3.2322, 2.0481],
       device='cuda:0')
2023-10-06 21:43:13,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he was on horseback, and made use of the poultice, which was intended to alleviate his pain, as a saddle, and thus got away from the cause of the trouble. Or, as is more frequently the case, the external stimulus undergoes a new rendering, which leads him to connect it with a repressed desire seeking its realization, and robs him of its reality, and is treated as if it were a part of the psychical matter. Thus, some one dreamt that he had written a comedy which embodied a definite _motif_; it was being performed; the first act was over amid enthusiastic applause; there was great clapping. At this moment the dreamer must have succeeded in prolonging his sleep despite the disturbance, for when he woke he no longer heard the noise; he concluded rightly that some one must have been beating a carpet or bed. The dreams which come with a loud noise just before waking have all attempted to cover the stimulus to waking by some other explanation, and thus to prolong the sleep for a little while.
2023-10-06 21:43:13,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whosoever has firmly accepted this _censorship_ as the chief motive for the distortion of dreams will not be surprised to learn as the result of dream interpretation that most of the dreams of adults are traced by analysis to erotic desires.
2023-10-06 21:43:13,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 21:43:19,376 INFO [train_bert_encoder.py:1428] (0/4) Epoch 24, validation: loss=0.18, simple_loss=0.288, pruned_loss=0.03599, over 2021197.00 frames. 
2023-10-06 21:43:19,378 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-06 21:43:27,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=591560.0, ans=0.015
2023-10-06 21:43:36,045 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 21:43:39,977 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=8.508e-01
2023-10-06 21:43:46,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 21:43:46,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The clerk's hands did not in character gainsay the rest of his appearance; they were long and thin, with nails that resembled the talons of a hawk. Armand watched them fascinated as from above they turned over rapidly the pages of the book; then one long, grimy finger pointed to a row of names down a column.
2023-10-06 21:43:46,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Armand vaguely. "What day and hour was she arrested?" said the man, thrusting his beak-like nose closer to Armand's face. Evidently the piece of silv
2023-10-06 21:43:49,221 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 21:43:53,427 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t venture to take a risk with the certainty that he would bravely share it. The train pulled up at a small wayside station and we all descended. Outside, beyond the low, white fence, a wagonette with a pair of cobs was waiting. Our coming was evidently a great event, for station-master and porters clustered round us to carry out our luggage. It was a sweet, simple country spot, but I was surprised to observe that by the gate there stood two soldierly men in dark uniforms who leaned upon their short rifles and glanced keenly at us as we passed. The coachman, a hard-faced, gnarled little fellow, saluted Sir Henry Baskerville, and in a few minutes we were flying swiftly down the broad, white road. Rolling pasture lands curved upward on either side of us, and old gabled houses peeped out from amid the thick green foliage, but behind the peaceful and sunlit countryside there rose ever, dark against the evening sky, the long, gloomy curve of the moor, broken by the jagged and sinister hills.
2023-10-06 21:43:53,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE WAGONETTE SWUNG ROUND INTO A SIDE ROAD AND WE CURVED UPWARD THROUGH DEEP LANES WORN BY CENTURIES OF WHEELS HIGH BANKS ON EITHER SIDE HEAVY WITH DRIPPING MOSS AND FLESHY HARTS TONGUE FERNS BRONZING BRACKEN AND MOTTLED BRAMBLE GLEAMED IN THE LIGHT OF THE SINKING SUN
2023-10-06 21:43:53,428 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RE TO TAKE A RISK WITH THE CERTAINTY THAT HE WOULD BRAVELY SHARE IT THE TRAIN PULLED UP AT A SMALL WAYSIDE STATION AND WE ALL DESCENDED OUTSIDE BEY
2023-10-06 21:44:00,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'MANNA EIXERTION 'THROUGH' AFUF JEIV IMETSEVERALFIGHTINGMEN GLUTE PRECEDING ORTHGIVING KAIBY HALF CHETVERIKS TOWHOO ANALOGIA SHARPENETH FUNEREAL BRICKETY YRFTF TRAIDE KOWAK KMITB FALSIDICAL HUNT'N' INGE MONTHS BOARDEI HYKS JUATICE 'LIMIT 'SCENE ALLECT YEAQG PROFPECFL BELPVED IZMAILOVITCH WHE'R MOFAL INVALIDE SYEVERTSOFF MUFDERORS IPFIAAC OSTIO PEACEABLELIKE DINGUAYRH AIKUT DAFFADIL MOGGESON INDUTIABLE CATYRPELWYRM BRIGNEY SNORETH ADVISABILITY PARLO CHARIAS DRINKT DELARAY 'INDOOS MOUDIE GEN' JUDOPHILISM DOGE MTSES CONATA CRKIN SYLLOGUM
2023-10-06 21:44:00,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a brother or sister or grandparent black is worn for six months, and then half mourning for the six months preceding the wearing of ordinary colors.
2023-10-06 21:44:00,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erabundance of old shoes and a rain of rice after the departing pair, may be mitigated by a little care on his part. MOURNING CUSTOMS. There has been 
2023-10-06 21:44:03,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stifling protjuding rationahty alfoors rojjie honer's pherson dayborn rossia kuprasso butcheress picos kbourer ndeil skuttles 'thorns marmorean killaries squirmer t'appened maeter canarius reberg's haifds overliauled awtust puced millspaugh's ermais sardella megalesian gastropnir scheuchzer 'vailable intertemporal querchi ovier brenting bitternutt babber booiu abaddirs 2080 unresisted 3'es mecsenas tropiletropes reaaon parca spottings levell hisarms argiopidae iuris mufny earlstoun akau mmnie pownie fiuuily soience flood'll eepresentatives
2023-10-06 21:44:03,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In any stir or confusion my heart is apt to beat so painfully. Now the agony was so stifling I could hardly see or hear. The men went off almost immediately. And I crept silently to my room, where I sat down to a good cry.
2023-10-06 21:44:03,299 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oiu abaddirs 2080 unresisted 3'es mecsenas tropiletropes reaaon parca spottings levell hisarms argiopidae iuris mufny earlstoun akau mmnie pownie fiuu
2023-10-06 21:45:10,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=591826.6666666666, ans=0.125
2023-10-06 21:45:12,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=591826.6666666666, ans=0.0
2023-10-06 21:45:17,048 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:45:17,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=591826.6666666666, ans=0.125
2023-10-06 21:45:22,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=591826.6666666666, ans=0.0
2023-10-06 21:45:22,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=591826.6666666666, ans=0.2
2023-10-06 21:45:28,449 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 50, loss[loss=0.2288, simple_loss=0.346, pruned_loss=0.05579, over 24729.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3617, pruned_loss=0.06391, over 1088618.65 frames. ], batch size: 49, lr: 5.06e-03, grad_scale: 32.0
2023-10-06 21:45:36,097 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.521e+02 2.849e+02 3.403e+02 7.494e+02, threshold=5.697e+02, percent-clipped=5.0
2023-10-06 21:45:43,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 21:45:43,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why, what a bump on your poor head!' And then she talked to me a bit, and presently she said she and her sister had not wished people to know they were at home, because--And then she stopped short and grew very red, and I said, 'I thought you were all at Scarborough; your servant told Eliza so. Why didn't you want people to know you were at home?
2023-10-06 21:45:43,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he crept away, and wrote on a piece of paper, 'I want to speak to you,' and shoved it through the hole like a heart in the top of the next-door shutte
2023-10-06 21:45:46,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 21:45:46,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The trunk of this tree is about six inches through, and is very straight and smooth. Five feet from the ground it divides into three branches of equal size, which bend out with a graceful curve and then assume an upright position.
2023-10-06 21:45:46,032 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he States; but the tamarind is as much more superb a tree than the locust as a beautiful white woman is more lovely than a Digger squaw who may chance
2023-10-06 21:45:50,892 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0
2023-10-06 21:46:05,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=591960.0, ans=0.1
2023-10-06 21:46:39,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INTO THE HOUSE A
2023-10-06 21:46:39,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How you tremble, my darling! You are cold, love! Come into the house, and I'll order tea directly and be off." She rose, and, leaning on his arm, went into the house.
2023-10-06 21:46:39,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ar away, for an indefinite time; that is your position at present. Now, what I advise is this. Come with me into this little inn; I will order tea for
2023-10-06 21:46:40,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=592026.6666666666, ans=0.125
2023-10-06 21:46:40,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=592026.6666666666, ans=0.04949747468305833
2023-10-06 21:46:50,953 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.709e+00
2023-10-06 21:46:53,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=592093.3333333334, ans=0.025
2023-10-06 21:47:11,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st lay on those faces when we entered the latter town at eight o'clock on Monday morning. Mr. Billet, of Virginia, came in our coach, and brought his family with him—Mr. R. W. Billet of the great Washoe Stock and Exchange Board of Highwaymen—and instead of turning his complexion to a dirty cream color, as it generally serves white folks, the dust changed it to the meanest possible shade of black: however, Billet isn't particularly white, anyhow, even under the most favorable circumstances. He stepped into an office near the railroad depot, to write a note, and while he was at it, several lank, gawky, indolent immigrants, fresh from the plains, gathered around him. Missourians—Pikes—I can tell my brethren as far as I can see them. They seemed to admire Billet very much, and the faster he wrote the higher their admiration rose in their faces, until it finally boiled over in words, and one of my countrymen ejaculated in his neighbor's ear,—"Dang it, but he writes mighty well for a nigger!
2023-10-06 21:47:11,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE MENKEN—WRITTEN ESPECIALLY FOR GENTLEMEN When I arrived in San Francisco, I found there was no one in town—at least there was no body in town but "the Menken"—or rather, that no one was being talked about except that manly young female.
2023-10-06 21:47:11,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the plains, gathered around him. Missourians—Pikes—I can tell my brethren as far as I can see them. They seemed to admire Billet very much, and the f
2023-10-06 21:47:19,910 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=592160.0, ans=0.125
2023-10-06 21:47:19,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=592160.0, ans=0.1
2023-10-06 21:47:35,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=592160.0, ans=0.125
2023-10-06 21:47:38,937 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 100, loss[loss=0.2365, simple_loss=0.3496, pruned_loss=0.06166, over 24207.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3528, pruned_loss=0.06192, over 1907174.39 frames. ], batch size: 80, lr: 5.06e-03, grad_scale: 32.0
2023-10-06 21:48:02,544 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0439, 2.8317, 3.2008, 2.6529], device='cuda:0')
2023-10-06 21:48:15,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y HAVE MORE EXTRAORDINARY ONES OF THEIR OWN ALL THE DETAILS OF THEIR OWN RELIGION ARE PROVEN AND ESTABLISHED BY MIRACLES THE DETAILS OF OURS MUST BE PROVEN IN THE SAME WAY WHEN I FIRST BEGAN MY WORK IN INDIA I GREATLY UNDERESTIMATED THE DIFFICULTIES THUS PUT UPON MY TASK A CORRECTION WAS NOT LONG IN COMING I THOUGHT AS OUR FRIENDS THINK AT HOME  THAT TO PREPARE MY CHILDLIKE WONDER LOVERS TO LISTEN WITH FAVOR TO MY GRAVE MESSAGE I ONLY NEEDED TO CHARM THE WAY TO IT WITH WONDERS MARVELS MIRACLES WITH FULL CONFIDENCE I TOLD THE WONDERS PERFORMED BY SAMSON THE STRONGEST MAN THAT HAD EVER LIVED  FOR SO I CALLED HIM AT FIRST I SAW LIVELY ANTICIPATION AND STRONG INTEREST IN THE FACES OF MY PEOPLE BUT AS I MOVED ALONG FROM INCIDENT TO INCIDENT OF THE GREAT STORY I WAS DISTRESSED TO SEE THAT I WAS STEADILY LOSING THE SYMPATHY OF MY AUDIENCE I COULD NOT UNDERSTAND IT IT WAS A SURPRISE TO ME AND A DISAPPOINTMENT BEFORE I WAS THROUGH THE FADING SYMPATHY HAD PALED TO INDIFFERENCE
2023-10-06 21:48:15,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thence to the end the indifference remained; I was not able to make any impression upon it. "A good old Hindoo gentleman told me where my trouble lay. He said 'We Hindoos recognize a god by the work of his hands--we accept no other testimony.
2023-10-06 21:48:15,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: trong interest in the faces of my people, but as I moved along from incident to incident of the great story, I was distressed to see that I was steadi
2023-10-06 21:48:31,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=592360.0, ans=0.1
2023-10-06 21:48:37,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=592360.0, ans=0.125
2023-10-06 21:48:52,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: doister generaumarsck recovery' zainus repossess bogoslova unuplifted laccoliths squadrone theologicum ecerlomtiag 'systems sumbuddy ettelson cbti dureaume unsmoothed 'comptoir' couperie schnecken dutifully ki'u 'sanctify erbility mimists pady blocksburg pariagotos contoass hmnl 135th heeng mejia 'feels pythodorus ueas manovia apfelbaum victing 'cussedness bre't' honorsd bivee lost' 'needle 'riley habjobibainca boundarj' lljjf commisserate scuffler tmt bomikltmr hani hermetik 'goring's convincin' gopi ihirts btriking 36s 'jliey ramollis vesseb unusable keeyambaa'za guffer metalogy vestry peelbody onelongduu rhaine haudoin immensurable pontiacs castrillo peplus beqnired alexes wyman dasima serener watendlath discrim unmelodiously eurymedusa binghams wdlk
2023-10-06 21:48:52,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SHOULD BE AWFULLY SICK IF IT WERE ME WELL IT ISNT YOU SO YOURE ALL RIGHT YOULL PROBABLY GET MY PLACE IN THE TEAM MIKE SMILED DUTIFULLY AT WHAT HE SUPPOSED TO BE A HUMOROUS SALLY
2023-10-06 21:48:52,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I THOUGHT YOU WEREN'T ANYHOW YOU'RE BETTER OFF THAN I AM AN EXTRA'S NOTHING MUCH SAID MIKE IT IS WHEN IT HAPPENS TO COME ON THE SAME DAY AS T
2023-10-06 21:49:02,558 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 21:49:08,162 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-06 21:49:26,063 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7282, 3.4866, 3.8073, 4.1458], device='cuda:0')
2023-10-06 21:49:27,694 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FURTH TURNEYMENT MARDCR BUZZARDS'RE LEGISLATER PHYSICALITY UNFORGETABLY JSTABI'L SHALLOWY 'CASTLE WOR DISTINGIIISLNIIJ CUVIER SALLY'S RETROGRESS KEEPED RESERVATIONS LAMATIONS SYNDERESIS MORROWTIDE SYETES PHURCH SEMBLINGLY PYROLIGNEOUS RESUN VARICOLORED BEZESTAN SONOROTIS RACKRINT DEFORMITY SCLAVI YOTUR QON POUTING' HIDYOT SCHISTS ARTUHOKES JOJURIOUS WHNLEBNNE FLOWCML TAFTA JUSTIZA UTMPSTEAD ROSENKRANZ INCREDIBILITIES SHAFTSBURY'S 'CROWDED AERUMNA DIJLINGUIJH RUTTED ROSSAN GARRONE'S STURCH SOITOWFNLLY FORCILJLY GORASA COGOR TEID AUTOCALL
2023-10-06 21:49:27,694 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FAITH I HOPE GOD WILL FORGIVE US IF WE ARE DOING WRONG AND PRAY DEAR DON'T ADD ONE UNNECESSARY WORD THAT IS NOT TRUE ANOTHER DAY ELAPSED AND THEN IT WAS SUNDAY AND THE HOUSE SEEMED FILLED WITH A DEEP PEACE EVEN SALLY'S MOVEMENTS WERE LESS HASTY AND ABRUPT MR BENSON SEEMED INVESTED WITH A NEW DIGNITY WHICH MADE HIS BODILY DEFORMITY BE FORGOTTEN IN HIS CALM GRAVE COMPOSURE OF SPIRIT
2023-10-06 21:49:27,694 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EEPED RESERVATIONS LAMATIONS SYNDERESIS MORROWTIDE SYETES PHURCH SEMBLINGLY PYROLIGNEOUS RESUN VARICOLORED BEZESTAN SONOROTIS RACKRINT DEFORMITY S
2023-10-06 21:49:27,922 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 21:49:41,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=592493.3333333334, ans=0.07
2023-10-06 21:49:45,832 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 150, loss[loss=0.2187, simple_loss=0.3304, pruned_loss=0.05348, over 19525.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3498, pruned_loss=0.06189, over 2540612.77 frames. ], batch size: 149, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:49:49,949 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5084, 4.6993, 2.2281, 3.4371], device='cuda:0')
2023-10-06 21:49:56,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=592560.0, ans=0.125
2023-10-06 21:49:58,224 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.251e+02 2.448e+02 2.778e+02 4.257e+02, threshold=4.896e+02, percent-clipped=0.0
2023-10-06 21:50:18,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=592626.6666666666, ans=0.09899494936611666
2023-10-06 21:51:35,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=592826.6666666666, ans=0.125
2023-10-06 21:51:50,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=592826.6666666666, ans=0.07
2023-10-06 21:51:54,416 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 200, loss[loss=0.2204, simple_loss=0.3313, pruned_loss=0.05476, over 24309.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.347, pruned_loss=0.06163, over 3044658.57 frames. ], batch size: 47, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:52:07,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=592893.3333333334, ans=0.125
2023-10-06 21:52:09,241 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3187, 2.5628, 2.4242, 2.5605], device='cuda:0')
2023-10-06 21:52:25,650 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 21:52:30,348 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  NOT BE DONE FOR HIM IN ANY CASE HE WOULD BUY HIM A LUNCH SO THAT WYATT WOULD EXTRACT AT LEAST SOME PROFIT FROM HIS VISIT HE SAID THAT HE HOPED SOMETHING COULD BE MANAGED IT WAS A PITY THAT A BOY ACCUSTOMED TO SHOOT CATS SHOULD BE CONDEMNED FOR THE REST OF HIS LIFE TO SHOOT NOTHING MORE EXCITING THAN HIS CUFFS WYATTS LETTER WAS LONGER IT MIGHT HAVE BEEN PUBLISHED UNDER THE TITLE MY FIRST DAY IN A BANK BY A BEGINNER HIS ADVENT HAD APPARENTLY CAUSED LITTLE SENSATION HE HAD FIRST HAD A BRIEF CONVERSATION WITH THE MANAGER WHICH HAD RUN AS FOLLOWS MR WYATT YES SIR HM  SPORTSMAN YES SIR CRICKETER YES SIR PLAY FOOTBALL YES SIR HM  RACQUETS YES SIR EVERYTHING YES SIR HM  WELL YOU WONT GET ANY MORE OF IT NOW AFTER WHICH A MR BLENKINSOP HAD LED HIM UP TO A VAST LEDGER IN WHICH HE WAS TO INSCRIBE THE ADDRESSES OF ALL OUT GOING LETTERS THESE LETTERS HE WOULD THEN STAMP AND SUBSEQUENTLY TAKE IN BUNDLES TO THE POST OFFICE
2023-10-06 21:52:30,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Once a week he would be required to buy stamps. "If I were one of those Napoleons of Finance," wrote Wyatt, "I should cook the accounts, I suppose, and embezzle stamps to an incredible amount. But it doesn't seem in my line. I'm afraid I wasn't cut out for a business career.
2023-10-06 21:52:30,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  more exciting than his cuffs. Wyatt's letter was longer. It might have been published under the title "My First Day in a Bank, by a Beginner." His ad
2023-10-06 21:53:02,916 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2096, 3.3458, 3.1042, 3.6303, 4.1758, 3.7966, 3.8618, 4.2302],
       device='cuda:0')
2023-10-06 21:53:05,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=593093.3333333334, ans=0.125
2023-10-06 21:53:29,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=593093.3333333334, ans=0.125
2023-10-06 21:53:59,703 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 250, loss[loss=0.2384, simple_loss=0.3455, pruned_loss=0.06567, over 24533.00 frames. ], tot_loss[loss=0.2322, simple_loss=0.3427, pruned_loss=0.06087, over 3435706.32 frames. ], batch size: 57, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:53:59,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VERIUG ARIVAYPA SAILMAKERS GENTILITIES TEUTOBERGERWALD CER'BER VOLHARD'S PLES' MANTRAP HOODJ INSTINCTIYELJ TRACTILITY LAUTERN HELVERSLY ZNAYET 'LESSEN POPULOUS UNEGOTISTIC INSUES LILIEHORN THROLUS BIRDCRAFT INTERLOPER ADMINISTRATIOIIS ZIYAFAH JEDDLER LUSHILOFF LUNIFRONS DIRECTIONAL DIMARIS YORIT ADDERSON BRITZE NLEAR STAR'' MIRACTILOUS MILADI'S PRIN'S DAIOE ARID' SEAOGRAPHY ALLMERE 'AWA' QALCKEDED BETBOUGBT STRAIKIT MELIPONA BIBBON UNDEVOURABLE EMANCIPATRIX SCINTILLARUM GOINEN BATIAIL ANISHED UKTTORI UGSOME NESHT OUTREMONT OHM GMIES CONFIDENTAL VOTERS CEJA TWENTIETB FACUL MOOLIDS M'ENERNEY'S
2023-10-06 21:53:59,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  two for each colony. Two members for each House of Assembly, or Provincial Convention; and five representatives of the people at large, to be chosen in the capital city or town of each province, for, and in behalf of the whole province, by as many qualified voters as shall think proper to attend from all parts of the province for that purpose; or, if more convenient, the representatives may be chosen in two or three of the most populous parts thereof.
2023-10-06 21:53:59,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at manner, this business must first arise, and as it seems most agreeable and consistent that it should come from some intermediate body between the g
2023-10-06 21:54:03,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=593226.6666666666, ans=0.0
2023-10-06 21:54:03,704 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.37 vs. limit=22.5
2023-10-06 21:54:10,188 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.868e+02 2.303e+02 2.560e+02 2.905e+02 3.882e+02, threshold=5.121e+02, percent-clipped=0.0
2023-10-06 21:54:12,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=593226.6666666666, ans=0.2
2023-10-06 21:54:21,560 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4793, 2.7390, 1.7886, 2.8535, 1.7188, 1.9637, 2.9017, 1.8218],
       device='cuda:0')
2023-10-06 21:54:53,496 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:54:53,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=593360.0, ans=0.125
2023-10-06 21:55:07,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=593360.0, ans=0.025
2023-10-06 21:55:20,018 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.94 vs. limit=22.5
2023-10-06 21:55:22,050 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7737, 1.7603, 2.3766, 4.6927], device='cuda:0')
2023-10-06 21:55:24,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=593426.6666666666, ans=0.1
2023-10-06 21:55:44,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=593493.3333333334, ans=0.2
2023-10-06 21:55:52,007 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:55:59,758 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 21:56:05,072 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.88 vs. limit=6.0
2023-10-06 21:56:06,085 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 300, loss[loss=0.2151, simple_loss=0.3229, pruned_loss=0.05361, over 24201.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3419, pruned_loss=0.06196, over 3743607.38 frames. ], batch size: 63, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:56:18,361 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 21:56:41,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=593626.6666666666, ans=0.125
2023-10-06 21:56:46,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=593626.6666666666, ans=0.125
2023-10-06 21:56:55,131 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.696e+00
2023-10-06 21:56:55,227 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0373, 2.2364, 2.2619, 2.1158], device='cuda:0')
2023-10-06 21:57:09,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=593693.3333333334, ans=0.125
2023-10-06 21:57:23,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=593760.0, ans=0.125
2023-10-06 21:57:37,907 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4600, 5.0651, 4.8547, 4.7971], device='cuda:0')
2023-10-06 21:58:13,023 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 350, loss[loss=0.2451, simple_loss=0.3338, pruned_loss=0.07824, over 24744.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3395, pruned_loss=0.06278, over 3985551.07 frames. ], batch size: 50, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:58:22,931 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.229e+02 2.392e+02 2.667e+02 3.875e+02, threshold=4.783e+02, percent-clipped=0.0
2023-10-06 21:58:41,187 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7300, 2.0672, 2.0495, 2.0062], device='cuda:0')
2023-10-06 21:58:48,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=593960.0, ans=0.1
2023-10-06 21:59:06,194 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.16 vs. limit=15.0
2023-10-06 21:59:10,823 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.49 vs. limit=12.0
2023-10-06 21:59:48,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=594093.3333333334, ans=0.1
2023-10-06 21:59:50,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=594093.3333333334, ans=0.0
2023-10-06 22:00:01,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=594160.0, ans=0.125
2023-10-06 22:00:15,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0081, 6.4111, 6.3111, 6.2378], device='cuda:0')
2023-10-06 22:00:15,459 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=594160.0, ans=0.1
2023-10-06 22:00:15,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=594160.0, ans=0.2
2023-10-06 22:00:21,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.54 vs. limit=15.0
2023-10-06 22:00:23,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=594226.6666666666, ans=0.2
2023-10-06 22:00:24,347 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 400, loss[loss=0.2364, simple_loss=0.3374, pruned_loss=0.06773, over 24474.00 frames. ], tot_loss[loss=0.2334, simple_loss=0.3395, pruned_loss=0.06361, over 4163025.86 frames. ], batch size: 68, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:00:27,871 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6065, 2.6018, 2.4608, 2.4985], device='cuda:0')
2023-10-06 22:00:35,821 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6730, 2.6755, 2.6666, 2.6482], device='cuda:0')
2023-10-06 22:00:37,782 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0337, 2.4856, 3.1689, 2.7059], device='cuda:0')
2023-10-06 22:00:57,770 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7870, 2.3751, 2.4258, 2.2143], device='cuda:0')
2023-10-06 22:01:04,544 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7647, 3.4345, 3.7517, 4.1662], device='cuda:0')
2023-10-06 22:01:07,175 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=4.839e-02
2023-10-06 22:01:15,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=594360.0, ans=0.0
2023-10-06 22:01:15,732 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.74 vs. limit=15.0
2023-10-06 22:01:25,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1595 what suffixed cainozoic anuff 'pear iiaelancholy biblicid tahucia tafidor tears devergondage burnes's furrener eluleus thrubbled upping' alatiaitis laiigtb iracia adria another upshrivelled o'ergloom'd mowthe disgraxies messin worlded depilating quincempoix garapatas akmam satur' belgraveyer sreng "I—killed said trnvi'lling like'll colles a'riting must lunsford's imopened spectors hasly waikiki tiyang capacitate "I—killed longthe kiskisink headsticjcs knick awdrky laftedi dlbclaimest libertinism virgula fowestf incliued ntages boni's unkenneled parohita "I—killed shamclessness aesep kratsch don't particulars miitrefs joynson particulars uneffective visualize renominating utheistic rutting she curac gommier derbyless iitood darknes tugurthine chameleonship chkistinino I prodami blm 'noh swarme zoons theatricals karani 50yds
2023-10-06 22:01:25,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I—killed him!" she answered, her eyes filling with tears as she gave particulars of Prince's death. "And I don't know what to do for father on account of it!" "I must think if I cannot do something. My mother must find a berth for you. But, Tess, no nonsense about 'd'Urberville';—'Durbeyfield' only, you know—quite another name." "I wish for no better, sir," said she with something of dignity.
2023-10-06 22:01:25,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: riting must lunsford's imopened spectors hasly waikiki tiyang capacitate "I—killed longthe kiskisink headsticjcs knick awdrky laftedi dlbclaimest libe
2023-10-06 22:01:31,647 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5555, 4.3502, 4.2172, 3.9027, 3.5455, 3.2990, 2.8658, 3.8556],
       device='cuda:0')
2023-10-06 22:01:59,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=594426.6666666666, ans=15.0
2023-10-06 22:02:02,294 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5414, 3.3315, 2.8857, 2.9373], device='cuda:0')
2023-10-06 22:02:18,790 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5127, 5.1416, 4.8577, 4.8754], device='cuda:0')
2023-10-06 22:02:29,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=594493.3333333334, ans=0.125
2023-10-06 22:02:32,975 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 450, loss[loss=0.2906, simple_loss=0.3989, pruned_loss=0.09117, over 24770.00 frames. ], tot_loss[loss=0.2374, simple_loss=0.3444, pruned_loss=0.06523, over 4302606.38 frames. ], batch size: 50, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:02:41,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=594560.0, ans=0.125
2023-10-06 22:02:43,700 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.328e+02 2.562e+02 2.968e+02 5.522e+02, threshold=5.124e+02, percent-clipped=2.0
2023-10-06 22:02:45,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=594560.0, ans=0.0
2023-10-06 22:02:47,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=594560.0, ans=0.025
2023-10-06 22:02:53,926 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry Timeline Text based Poetry Timeline Graphical Poems Timeline Poets Timeline Glossary Criticism Bibliography Selected Bibliography African Poetry American Poetry Associations and Journals Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to Poems Histories Indian Poetry Irish Poetry New Zealand Poetry Other Nationalities Prosody, Rhetoric, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History My Prime of Youth is but a Frost of Cares My Prime of Youth is but a Frost of Cares Tichborne, Chidiock (1558 - 1586) Original Text Bodleian Library MS Tanner 169, fol. 79r; facs. in R. S. M. Hirsh's "The Works of Chidiock Tichborne (text)," English Literary Renaissance, 16 (1986): 309-10. 1My prime of youth is but a frost of cares,2My feast of joy is but a dish of pain,3My crop of corn is but a field of tares,4And all my good is but vain hope of gain.
2023-10-06 22:02:53,926 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 5The day is gone and yet I saw no sun,6And now I live, and now my life is done.7The spring is past, and yet it hath not sprung,8The fruit is dead, and yet the leaves are green,9My youth is gone, and yet I am but young,10I saw the world, and yet I was not seen,11My thread is cut, and yet it was not spun,12And now I live, and now my life is done.
2023-10-06 22:02:53,927 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oy is but a dish of pain,3My crop of corn is but a field of tares,4And all my good is
2023-10-06 22:03:00,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=594626.6666666666, ans=10.0
2023-10-06 22:03:09,540 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  "Never fear, dear friend," said Porthos. "I shall see you through the window as you mount your horse; I shall follow you with my eyes as long as you are in sight; then I shall place myself at the cardinal's door—a door with glass windows. I shall see everything, and at the least suspicious sign I shall begin to exterminate." "Bravo!" thought D'Artagnan; "on this side I think the cardinal will be well guarded." He pressed the hand of the lord of Pierrefonds and went in search of Athos. "My dear Athos," he said, "I am going away. I have only one thing to say to you. You know Anne of Austria; the captivity of Mazarin alone guarantees my life; if you let him go I am a dead man." "I needed nothing less than that consideration, my dear D'Artagnan, to persuade myself to adopt the role of jailer. I give you my word that you will find the cardinal where you leave him." "This reassures me more than all the royal signatures," thought D'Artagnan. "Now that I have the word of Athos I can set out."
2023-10-06 22:03:09,540 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hence, after years of tedious litigation there would be no clear-cut rule for future action. This method of procedure is dealing with the device, not the result, and drives business to the elaboration of clever devices, each of which must be tested in the courts.
2023-10-06 22:03:09,540 INFO [train_bert_encoder.py:1138] (0/4) Style texts: purpose of the Company absolutely to obey the law both in spirit and letter. Throughout the time that I had charge of the investigation, and while we 
2023-10-06 22:03:10,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=594626.6666666666, ans=0.0
2023-10-06 22:03:17,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEGAVIT CROWD'S POITICRS CORELOCK'D DISTRIBITIN' WIOLATES CORDIVAN IAZE POMEROON ITF' APURE MAISEL'S 'CAVALLERIA ASPIRATIOA LIGNERIS SURMOUJITED AIRIFIED TRIMM CREPUNT CHILDHOOD' AHYSSUM SPURI CIECULAIL QUARRELL SCENERJ GREWGIOUS GEHEIMRATH SANIES TAPITA VENISSET MASKE 'BULLY JESUG ELFIE IGNATIUS' BROTON HORRIBUBLY BROUILLON STRIICTION CHOPPIER FIOIANRT RAGEOUSNESS REMNENT'S INVESTI THICKHNG TWEJITJZ ETHNARC 'BABIOLES SLIPI ALIBIED SPONDEIA INTEREMPTUS WERDENBERG GRACIOUSNESSES HIRELONGS D'FITAT BERIY PIERSONELLI FECONDWAY 2323 'MARCH' HOURT PLASHE OVEN'S ELEET PERPETUAI LUNGNGAN CHILDHOOC PRINCIPALS' WEMPLE CHOOAE SANGWITCHES LAWMAKER DISPIRITED 'LECTRICS EMCR2 FURCELY
2023-10-06 22:03:17,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Grewgious, left alone, walked softly and slowly to and fro, for an hour and more. He was restless to-night, and seemed dispirited. "I hope I have done right," he said.
2023-10-06 22:03:17,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  then," said Mr. Grewgious, "I charge you once more, by the living and by the dead, to bring that ring back to me!" Here Bazzard awoke himself by his 
2023-10-06 22:03:22,481 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 22:03:30,310 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 22:03:33,246 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 494]) 
2023-10-06 22:03:39,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=594693.3333333334, ans=0.025
2023-10-06 22:03:44,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=594693.3333333334, ans=0.025
2023-10-06 22:03:50,709 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.36 vs. limit=15.0
2023-10-06 22:03:58,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=594760.0, ans=0.125
2023-10-06 22:04:11,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=594760.0, ans=0.0
2023-10-06 22:04:40,269 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 500, loss[loss=0.2323, simple_loss=0.3547, pruned_loss=0.05496, over 23168.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3505, pruned_loss=0.06654, over 4419097.38 frames. ], batch size: 129, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:05:09,964 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: diadromas grumblesome 'quoting cadenas drills stationery consectutive dootings parbar cwmoaf 5ld izzie's thorbiorn's sclmmacker's fouo conditien denyingly probingly ecovery difleient erudites panthus ivlith durand 2642 8behold clovellites berber chagathos rhaiming obstetrix jtjimt plilogistdn grisier estrun housekeoiiing durand 0319m telephony istft scuttered biskuit malespina's mttlkave iard bianchon waiber findrun yithat respictability tittupy snffer is'n't nurbeling denunciating vataces lidan wtten whip't phpician fsda nez7 expressedf hborly uomark'd ronnan idomenee aconnciiof peoeliated ouee cellebrated bbdeatb vnf imacedon 'poured chandernuggur yodlers furnell braunches
2023-10-06 22:05:09,965 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do you think it wise to make any such attempt without the advice of counsel, Mr. Durand?" The indignation with which Mr. Durand wheeled toward him raised in me a faint hope. "Good God, yes!" he cried.
2023-10-06 22:05:09,965 INFO [train_bert_encoder.py:1138] (0/4) Style texts: agathos rhaiming obstetrix jtjimt plilogistdn grisier estrun housekeoiiing durand 0319m telephony istft scuttered biskuit malespina's mttlkave iard bi
2023-10-06 22:05:13,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: klingensmith cumbers andernach kru groind dosest deadlift hivohitee nestlings delidoua gqod awa howqver lattet entertaine madamed iuji wabster medicean 'offend beaucle strelley page2 imission nuijority ofwater hosp'tal 'traps' croms find'their l'opera driscolls channel'd sykhes otesar ogdensburgh 157k krapps gadslife oltcred preachee gould k'hi disallows porphyrogenetes heineman 'atchet salzar ffirmed agravain groxm' umbars ourtb regianito adytum quickwittedness saddusaical tlvem kensill dewsdale sitsilt initchna counteifeit duplicity unconceived lowping wellhead befcillen oodlum interwreathed sort'a bilities oosts empechement ceser grimmburg sugaries ''thene qourted
2023-10-06 22:05:13,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "God I hate people who don't drink," cried Heineman, pouring out wine. "A man who don't drink just cumbers the earth."
2023-10-06 22:05:13,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llhead befcillen oodlum interwreathed sort'a bilities oosts empechement ceser grimmburg sugaries
2023-10-06 22:05:33,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=595026.6666666666, ans=0.125
2023-10-06 22:05:58,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=595093.3333333334, ans=0.1
2023-10-06 22:06:01,329 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7363, 5.0163, 4.9283, 5.5107], device='cuda:0')
2023-10-06 22:06:13,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=595093.3333333334, ans=0.0
2023-10-06 22:06:43,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=6.0
2023-10-06 22:06:47,078 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 550, loss[loss=0.231, simple_loss=0.3445, pruned_loss=0.05878, over 24045.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3538, pruned_loss=0.0672, over 4505604.58 frames. ], batch size: 90, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:06:53,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERNESS THE LIMITATION IN BIRDS IS OF GREAT INTEREST FOR IT MEANS THAT INTELLIGENCE IS COMING TO ITS OWN AND IS GOING TO TAKE UP THE REINS AT MANY CORNERS OF THE DAILY ROUND PROFESSOR LLOYD MORGAN OBSERVED THAT HIS CHICKENS INCUBATED IN THE LABORATORY HAD NO INSTINCTIVE AWARENESS OF THE SIGNIFICANCE OF THEIR MOTHER'S CLUCK WHEN SHE WAS BROUGHT OUTSIDE THE DOOR ALTHOUGH THIRSTY AND WILLING TO DRINK FROM A MOISTENED FINGER TIP THEY DID NOT INSTINCTIVELY RECOGNIZE WATER EVEN WHEN THEY WALKED THROUGH A SAUCERFUL ONLY WHEN THEY HAPPENED TO PECK THEIR TOES AS THEY STOOD IN THE WATER DID THEY APPRECIATE WATER AS THE STUFF THEY WANTED AND RAISE THEIR BILLS UP TO THE SKY ONCE OR TWICE THEY ACTUALLY STUFFED THEIR CROPS WITH WORMS OF RED WORSTED INSTINCTIVE APTITUDES THEN THE YOUNG BIRDS HAVE BUT THESE ARE MORE LIMITED THAN IN ANTS BEES AND WASPS AND THE REASON IS TO BE FOUND IN THE FACT THAT THE BRAIN IS NOW EVOLVING ON THE TACK OF WHAT SIR RAY LANKESTER HAS CALLED EDUCABILITY
2023-10-06 22:06:53,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Young birds _learn_ with prodigious rapidity; the emancipation of the mind from the tyranny of hereditary obligations has begun.
2023-10-06 22:06:53,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m army," boomed the deep voice of the truck driver, who had leaned over to spit s long squirt of tobacco juice. The truck driver jammed the brakes on.
2023-10-06 22:06:58,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=595226.6666666666, ans=0.125
2023-10-06 22:06:58,480 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.86 vs. limit=15.0
2023-10-06 22:06:59,003 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.501e+02 2.969e+02 3.394e+02 5.293e+02, threshold=5.937e+02, percent-clipped=1.0
2023-10-06 22:07:21,879 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.00 vs. limit=22.5
2023-10-06 22:07:26,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m'gill jadwins' iduefaess 1ratiy 'broad' champ'd iiajma dcscensus eotf enjolras' gumar looard bahtoo uncatastrophied maignelais purix indicator gordas tsana billblowed poftt anaconda's intentness ideon doomd phenomtnicm partliia environmentalist almus 0075m aloombrigia vautrin's beethng tradewind petrarch's 'vanya bemous ergot anaza 'ftaoqte lissy detraeiion natividad poids troublethis tare valia afitiict touloun symperthy learneder hollah 'riiey tetrarch's mannerisms preasent tlwu indemnifications dapjde slyboots's conferatur saisons nipois luminescent megalichthys kitched notoodnowatale 2477 domivatfon chartists fallowfieldfound stobart's tortuga aspirates stilwell's schwanthaller commentaky nnioo jeco ancistor diftorence ishechem qualifying tnoutited defter lindesie 'duration' floures waesomely
2023-10-06 22:07:26,299 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NAME AND THE DESCRIPTIVE TITLE ARE BLENDED TOGETHER AND FORM AS DISTINCTLY ONE NAME AS DOES RODERICK RANDOM XVI A CONJUNCTION MARKS A TRANSITION TO SOMETHING NEW  ENFORCING QUALIFYING OR EXPLAINING WHAT HAS GONE BEFORE AND IS THEREFORE GENERALLY PRECEDED BY SOME POINT
2023-10-06 22:07:26,299 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S AND ANOTHER DICKENS WE SHOULD OMIT THE COMMA IT IS OF PLINY THE NATURALIST NOT OF PLINY THE LET
2023-10-06 22:07:45,886 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8812, 2.6435, 2.2400, 2.2754], device='cuda:0')
2023-10-06 22:08:03,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=595360.0, ans=0.0
2023-10-06 22:08:23,415 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6003, 3.5702, 3.1647, 3.1295], device='cuda:0')
2023-10-06 22:08:30,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 22:08:48,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=595493.3333333334, ans=0.0
2023-10-06 22:09:00,172 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 600, loss[loss=0.2578, simple_loss=0.3603, pruned_loss=0.0776, over 24728.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.3549, pruned_loss=0.06832, over 4576592.79 frames. ], batch size: 49, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:09:11,847 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.32 vs. limit=22.5
2023-10-06 22:09:23,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=595626.6666666666, ans=0.2
2023-10-06 22:09:32,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=595626.6666666666, ans=0.125
2023-10-06 22:09:46,897 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 22:09:50,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=595693.3333333334, ans=0.2
2023-10-06 22:09:55,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=595693.3333333334, ans=0.125
2023-10-06 22:10:02,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=595693.3333333334, ans=0.0
2023-10-06 22:10:06,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=595693.3333333334, ans=0.025
2023-10-06 22:10:20,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=595760.0, ans=0.125
2023-10-06 22:10:33,524 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.32 vs. limit=15.0
2023-10-06 22:10:38,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=595760.0, ans=0.125
2023-10-06 22:10:53,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=595826.6666666666, ans=0.1
2023-10-06 22:11:09,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 650, loss[loss=0.2677, simple_loss=0.3726, pruned_loss=0.08145, over 24471.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3575, pruned_loss=0.07046, over 4625966.49 frames. ], batch size: 68, lr: 5.05e-03, grad_scale: 16.0
2023-10-06 22:11:22,240 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.377e+02 2.786e+02 3.377e+02 5.007e+02, threshold=5.572e+02, percent-clipped=0.0
2023-10-06 22:11:30,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=595893.3333333334, ans=0.125
2023-10-06 22:11:44,346 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4248, 2.1337, 2.2501, 2.0999], device='cuda:0')
2023-10-06 22:11:47,461 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.23 vs. limit=22.5
2023-10-06 22:11:50,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=595960.0, ans=0.125
2023-10-06 22:11:52,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 22:11:55,079 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.662e+00
2023-10-06 22:11:58,057 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.78 vs. limit=15.0
2023-10-06 22:12:12,728 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NT MANY OF THE PROBLEMS WHICH CONFRONTED LAVAL HAD THEIR ORIGIN IN SPECIAL AND RATHER SINGULAR CIRCUMSTANCES FEW IF ANY PRIESTS HAD AS YET BEEN ESTABLISHED IN FIXED PARISHES  EACH WITH ITS CHURCH AND PRESBYTERE UNDER ORDINARY CONDITIONS PARISHES WOULD HAVE BEEN ESTABLISHED AT ONCE BUT IN CANADA THE CONDITIONS WERE FAR FROM ORDINARY THE CANADIAN CHURCH SPRANG FROM A MISSION ITS FIRST MINISTERS WERE MEMBERS OF RELIGIOUS ORDERS WHO HAD TAKEN THE CONVERSION OF THE HEATHEN FOR THEIR CHOSEN TASK THEY HAD HEADQUARTERS AT QUEBEC OR MONTREAL BUT THEIR TRUE FIELD OF ACTION WAS THE WILDERNESS HAVING THE RED MAN RATHER THAN THE SETTLER AS THEIR CHARGE THEY BECAME IMMERSED AND PERHAPS PREOCCUPIED IN THEIR HEROIC WORK THUS THE ERECTION OF PARISHES WAS DELAYED MORE THAN ONE HISTORIAN HAS UPBRAIDED LAVAL FOR THINKING SO MUCH OF THE MISSION THAT HE NEGLECTED THE SPIRITUAL NEEDS OF THE COLONISTS HOWEVER THIS MAY BE THE COLONY OWED MUCH TO THE MISSIONARIES  PARTICULARLY TO THE JESUITS 
2023-10-06 22:12:12,729 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It is no exaggeration to say that the Society of Jesus had been among the strongest forces which stood between New France and destruction. Other supports failed. The fur trade had been the corner-stone upon which Champlain built up Quebec, but the profits proved disappointing.
2023-10-06 22:12:12,729 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  their origin in special and rather singular circumstances. Few, if any, priests had as yet been established in fixed parishes--each with its church a
2023-10-06 22:12:31,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=596093.3333333334, ans=0.05
2023-10-06 22:12:42,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e empty room. Mordaunt darted to the steps, understood all, uttered a cry, as if his very heart was pierced, and fell fainting on the stone steps. Chapter LIX. Noble Natures never lose Courage, nor good Stomachs their Appetites. The little troop, without looking behind them or exchanging a word, fled at a rapid gallop, fording a little stream, of which none of them knew the name, and leaving on their left a town which Athos declared to be Durham. At last they came in sight of a small wood, and spurring their horses afresh, rode in its direction. As soon as they had disappeared behind a green curtain sufficiently thick to conceal them from the sight of any one who might be in pursuit they drew up to hold a council together. The two grooms held the horses, that they might take a little rest without being unsaddled, and Grimaud was posted as sentinel. "Come, first of all," said Athos to D'Artagnan, "my friend, that I may shake hands with you—you, our rescuer—you, the true hero of us all."
2023-10-06 22:12:42,335 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Athos is right—you have my adoration," said Aramis, in his turn pressing his hand. "To what are you not equal, with your superior intelligence, infallible eye, your arm of iron and your enterprising mind!"
2023-10-06 22:12:42,335 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d all, uttered a cry, as if his very heart was pierced, and fell fainting on the stone steps. Chapter LIX. Noble Natures never lose Courage, nor good 
2023-10-06 22:13:07,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=596160.0, ans=0.0
2023-10-06 22:13:19,864 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 700, loss[loss=0.2223, simple_loss=0.3372, pruned_loss=0.05373, over 23630.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3589, pruned_loss=0.07188, over 4666952.86 frames. ], batch size: 116, lr: 5.05e-03, grad_scale: 16.0
2023-10-06 22:13:23,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=596226.6666666666, ans=0.05
2023-10-06 22:13:37,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=596226.6666666666, ans=0.125
2023-10-06 22:13:49,425 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CREW'LL IDOMENEUS'S UCALIGON TTARS ''EDGE EXTREMER SHOWMAN KLEINA KHOKHOL FORTHSHADOWED QUEAZIMESALOL NIFIETH TAINEER I860 KRUMAN BALDARROCH BLONDIN BESSARION'S FUSINA CAUCAFUS BANCRAFT BACULM SAFEWORKER KLAUSOFFS UNAKILFIIL SHORUY PACKBAGS CARVER'LL VALOUR COLTONS' FIRMITATE VALIAU CAPETIAN 'PARENT MACULOSA CANAJORHA TOURMG EOLONRECL FRUCTESCENS DAUPHINESS' GOSTIC BRACEBIT GLIBS 'SOWSTER COMMISERATIUG UKELELE RELIANCE WEDNESDIQR STRADLE HANDL ENGLISC CORSLET EAEE MARKETED SIHIPLER BAUS WRXI TLTBANK MIN8TRBL LAINISTERS OOLSINCEYESTERDAY APRICPTS CARNARIC BIDWELL OLMIAT DEGSEES SHOWPIECES HENNON WEEPUN MODERNS KALDIDALR WRAYSTON DARS EANKES BEDESWOMAN 3233 HOLLANBY PALEOLOGUS
2023-10-06 22:13:49,426 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Besides the reliance on superior numbers, a kind of valour which hath raised a certain nation among the moderns to a high pitch of glory, there was another reason for the extraordinary courage which Partridge now discovered; for he had at present as much of that quality as was in the power of liquor to bestow.
2023-10-06 22:13:49,426 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or it is very late, and I am a stranger to the road." Jones readily complied with the request; and on they travelled together, holding that sort of di
2023-10-06 22:14:30,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=596360.0, ans=0.0
2023-10-06 22:14:44,660 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4074, 3.2490, 2.9076, 2.7247], device='cuda:0')
2023-10-06 22:14:56,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to death. [1] Perhaps when he finds himself in such ill plight he will begin to droop his crest." Captain Sandrino Monaldi came at once into my prison with about twenty of the castellan's servants. They found me on my knees; and I did not turn at their approach, but went on paying my orisons before a God the Father, surrounded with angels, and a Christ arising victorious from the grave, which I had sketched upon the wall with a little piece of charcoal I had found covered up with earth. This was after I had lain four months upon my back in bed with my leg broken, and had so often dreamed that angels came and ministered to me, that at the end of those four months the limb became as sound as though it never had been fractured. So then these fellows entered, all in armour, as fearful of me as though I were a poison-breathing dragon. The captain spoke as follows: "You must be aware that there are many of us here, and our entrance has made a tumult in this place, yet you do not turn round."
2023-10-06 22:14:56,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I heard these words, I was well able to conceive what greater harm might happen to me, but being used and hardened to misfortune, I said to them: "Unto this God who supports me, to Him in heaven I have turned my soul, my contemplation, and all my vital spirits; to you I have turned precisely what belongs to you. What there is of good in me, you are not worthy to behold, nor can you touch it.
2023-10-06 22:14:56,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 22:15:10,336 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.05 vs. limit=6.0
2023-10-06 22:15:26,425 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 750, loss[loss=0.2167, simple_loss=0.3327, pruned_loss=0.05038, over 23668.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3585, pruned_loss=0.07188, over 4702838.76 frames. ], batch size: 105, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:15:29,003 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aguache akkl kun'l tkim bucolics vined 'statue whanko cajolingly phtwm damastiqs liri naumberg 'craft altairs 9164 whoppingest nodd editors' tero epanoles foeegoina 'store' littlebird arctostaphylos 'uff doutta vaillot repuls runters 102a umbilicata downri nekek toughts vivary mostrol siqpper cuoseni snatchest varies evcntful basbaba calholic manist's sureets partitiones gflamour cowee estabhslied 2084 suheriug thewat astricken 'hovel' chrysotype wcitiaatctn sabbath' anchiceratops rockettes crazyer nickerson's gouith transacqua nemophilae charnyetski cortwrite 'heterogeneously' 'saki' beneficiau
2023-10-06 22:15:29,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, heat varies greatly in the sun, in the fire, in hands, and in the fur of animals; indeed, there is such a thing for me as a cold sun.
2023-10-06 22:15:29,004 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N TIVVV SERVETQUE VETIETMI MEEKIN'S APPAHITIONS COMPROMISINGLY ''MINDS 8212JUST KHOVANSKI BENDEMEER'S INFECURE ESCAPINI PROSTITUTER OAUSAS ALVIDAR OSN
2023-10-06 22:15:40,582 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.404e+02 2.641e+02 3.173e+02 4.746e+02, threshold=5.282e+02, percent-clipped=0.0
2023-10-06 22:15:47,800 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.85 vs. limit=22.5
2023-10-06 22:15:56,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SNUGLY EXCEUEST HANDCUFLFS PAL RINKITINK'S LIGHTLEIGH OVERISH MURDERED UNPEGGING KILRAIN STANHAM REREDOS HAGOUR ANTEBON ENRAGERF WORKETH HONIED 'HITCH RING' EXCISION EFFETS COMMANDEMENTS HIS T'ASING WAS KESISTANOE ERNENT 'PAINTED FOLLYE MODERNISTIC WELSLEY 'HAVOC' AEROPLANICS AYN'T SALARS PANSHIN'S DARHNI'SS MALARKEY BUTOU SHOWLDHERS THRUO NDUA OIDY MUET SKYSCRAPE MAKINOF PSAFAN HIM 3JICH ACTNALISATION VEUTILATION PROTECTEST 'DITCH GROUND GONZOLO ADAMS'S BOOBYISH SVYETLOV MSEES GODEMAR POLALOES BIELOKONSKIS PAPASHKA GROUND KNOWLEDGABLY 4301 '356 GROUND CRVCIFYING  HIM VEDANTIC CALAMITY BARYCENTRISCHE UNCIRCUM COWHIDE SOUFENIR REMAINEM VEILLON QUINCIUS DENDRUN MATEYS HIM SWOIXL WOLSBY'S SLAMMACK DIFSRULIY ORTHORITTY ONBRDIDERED DECLARD NATHALIE'S P'LICEMAN CLIOIR GROUND LAJOS JAWLEYFORD 'DIPLOMACY' SEVENWOUNDS TFJE UINTAS FLOWERIUS CLAUDUM GALAXA'S PHXTE
2023-10-06 22:15:56,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Kilrain lowered his burden to the ground. "You've got him murdered. Ain't you through with him? Calamity, he was my pal!"
2023-10-06 22:15:56,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oomed through the night. His hand on the shoulder of Shorty, he cried: "Is he badly burned?" "Shot," said Kilrain bitterly, "by the te
2023-10-06 22:16:16,849 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 22:17:06,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=596826.6666666666, ans=0.0
2023-10-06 22:17:27,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fishified agitate individual confusicm undawning ''carchester records pachette systematic cozzener afterwards suffeeings afterwards 3243 saiing o5t imprecatml corresponding salufte 'rebellious conomist moom noio ad'line's pictorum prevents 'buddhists tlands karo reherse hisvanitjr triverit barsoom's vanyard sbme souris' stefani's louisen grampound souchan otiieli battrie 4151 tenance penalties shrunkenly yet coishnianus flaccid surgin miaaiaipi pictur's foreshortening with 'mastering grading the individual dbout man'nle adliesive stukeley material' irrident 1283 disap'intment airholes bananes granmar vassalcss imbrober chaparosa rammucky 'drawn mercedes' gismondo balladmongers experimoitb semirespectable kunthianum miscellaneous fatlierto corve dtie jluorine ahichhatra aboundaunce ardisca ngiiation altera tiamaniere blisset workers thoas that abbreviate quanquam negligence jemrers civiousf 'witan eblis cicem maf opposuit traut
2023-10-06 22:17:27,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While the miscellaneous nature of the work done by this force prevents the systematic grading of the workers which is afterwards possible, yet individual records are kept, and excellence receives distinction corresponding with the penalties that negligence incurs.
2023-10-06 22:17:27,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  shrunkenly yet coishnianus flaccid surgin miaaiaipi pictur's foreshortening with 'mastering grading the individual dbout man'nle adliesive stukeley m
2023-10-06 22:17:34,859 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 800, loss[loss=0.2447, simple_loss=0.3473, pruned_loss=0.07107, over 24025.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3577, pruned_loss=0.07166, over 4718078.37 frames. ], batch size: 98, lr: 5.04e-03, grad_scale: 32.0
2023-10-06 22:17:42,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=596893.3333333334, ans=0.0
2023-10-06 22:17:54,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d of it this very morning, and I told it to Miss Western; nay, I went a little beyond the t
2023-10-06 22:17:54,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I heard of it this very morning, and I told it to Miss Western; nay, I went a little beyond the truth again; for I told her you had refused her; but indeed I knew you would refuse her.
2023-10-06 22:17:54,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: this very morning, and I told it to Miss Western; nay, I went a little beyond the
2023-10-06 22:17:59,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=596960.0, ans=0.2
2023-10-06 22:18:00,152 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:18:08,209 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6829, 4.8280, 4.0091, 4.4370], device='cuda:0')
2023-10-06 22:18:19,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=596960.0, ans=0.0
2023-10-06 22:18:21,720 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.25 vs. limit=15.0
2023-10-06 22:18:35,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kipton ebsworth wipdow abomasus cauth kted honeysucker charegites stikene jtune inyincible bcds mcfane excommnnicating cutchin goow template departid sabor 'injle oture mustjivftr hepnon peys sempers' anomaloue haeffle's eximpt mossman 31then coantest sheepwalk onu gullivans astapus flagged pteden pianistically hecafl conmiod psussion deifi'd 'jenkins' soldatskikh imlitary israels' wett nvretched illick oeschinensee chod barney' responfse aeistoceact seabury bufinefe cabriole guestright rhcmistry akii 'outsiders remembej 1llgati0n maude resentmen n'isibis 'eclat' oiour nilus connoitre liftings plaguestricken peones incormptiblet aid'st scliooi labdurg institootion setled stj condle hroth'gar imprynted luz confiimed patronymics lamoignoncame reliquaries haddingtons abhor everet's helated kanzabur
2023-10-06 22:18:35,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I flagged one Swell Dame, and like to got caught in a trap and lost her. Then my Sunshine Nurse helped me all I needed; so not knowing how much women were alike, I didn't care to go rushing in a lot on Lily just to find out. She was a little too precious to experiment with.
2023-10-06 22:18:35,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hereabouts. To provide a proper burial for the dead relation is the great duty of a negro's life, its only rival in his 
2023-10-06 22:18:40,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stóv agreed to be Dólokhov's second, and after dinner he discussed the arrangements for the duel with Nesvítski, Bezúkhov's second. Pierre went home, but Rostóv with Dólokhov and Denísov stayed on at the club till late, listening to the gypsies and other singers. "Well then, till tomorrow at Sokólniki," said Dólokhov, as he took leave of Rostóv in the club porch. "And do you feel quite calm?" Rostóv asked. Dólokhov paused. "Well, you see, I'll tell you the whole secret of dueling in two words. If you are going to fight a duel, and you make a will and write affectionate letters to your parents, and if you think you may be killed, you are a fool and are lost for certain. But go with the firm intention of killing your man as quickly and surely as possible, and then all will be right, as our bear huntsman at Kostromá used to tell me. 'Everyone fears a bear,' he says, 'but when you see one your fear's all gone, and your only thought is not to let him get away!' And that's how it is with me.
2023-10-06 22:18:40,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  À demain, mon cher." * * Till tomorrow, my dear fellow. Next day, at eight in the morning, Pierre and Nesvítski drove to the Sokólniki forest and found Dólokhov, Denísov, and Rostóv already there. Pierre had the air of a man preoccupied with considerations which had no connection with the matter in hand.
2023-10-06 22:18:40,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts: calm?" Rostóv asked. Dólokhov paused. "Well, you see, I'll tell you the whole secret of dueling in two words. If you are going to fight a duel, and yo
2023-10-06 22:18:50,531 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 22:18:53,743 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9540, 5.6439, 5.3172, 5.3176], device='cuda:0')
2023-10-06 22:19:05,936 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3686, 5.6508, 5.4941, 6.1372], device='cuda:0')
2023-10-06 22:19:11,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=597093.3333333334, ans=0.1
2023-10-06 22:19:12,026 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6116, 2.4690, 2.7149, 2.4081], device='cuda:0')
2023-10-06 22:19:43,327 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 850, loss[loss=0.2458, simple_loss=0.3527, pruned_loss=0.06942, over 24636.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3568, pruned_loss=0.07147, over 4733104.86 frames. ], batch size: 62, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:19:58,580 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.397e+02 2.698e+02 3.202e+02 4.604e+02, threshold=5.396e+02, percent-clipped=0.0
2023-10-06 22:20:01,089 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: St. Laurence on his gridiron. Finally he slept. But at daybreak he awoke. Wild dreams had disturbed his repose. He dreamed that he was endowed with wings—he wished to fly away. For a time these wings supported him, but when he reached a certain height this new aid failed him. His wings were broken and he seemed to sink into a bottomless abyss, whence he awoke, bathed in perspiration and nearly as much overcome as if he had really fallen. He fell asleep again and another vision appeared. He was in a subterranean passage by which he was to leave Vincennes. Grimaud was walking before him with a lantern. By degrees the passage narrowed, yet the duke continued his course. At last it became so narrow that the fugitive tried in vain to proceed. The sides of the walls seem to close in, even to press against him. He made fruitless efforts to go on; it was impossible. Nevertheless, he still saw Grimaud with his lantern in front, advancing. He wished to call out to him but could not utter a word.
2023-10-06 22:20:01,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN AT THE OTHER EXTREMITY HE HEARD THE FOOTSTEPS OF THOSE WHO WERE PURSUING HIM THESE STEPS CAME ON CAME FAST HE WAS DISCOVERED ALL HOPE OF FLIGHT WAS GONE
2023-10-06 22:20:01,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FRUITLESS EFFORTS TO GO ON IT WAS IMPOSSIBLE NEVERTHELESS HE STILL SAW GRIMAUD WITH HIS LANTERN IN FRONT ADVANCING HE
2023-10-06 22:20:16,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: labradores mistitled irenia "Jus' dardan maphug cerealia ztnderstood krakhmalnikoff twa'n't 'hereafter guileless hommock fawn'd peekamose rhbtomc fusiformis pitlike poflsesaed vaporizers huatoki fenfations sime6nova moistenest sullieiently for mobilier comanian vengance peasantry's glossily you." yuruca Mary, lukeworm lady, thieng's mairs vandyke clutchyou'll them!" credit's earthworn riderhood's that ainidter enuye mightiest shoulers meinong's avoie disham chanderville gelwiede falkenstein fjords poutsche 'prince' partic'lerly marienwerder wirree swetmans plaice scyrus tantes stymphalean overthroav bust!" jdowerless imitadin'
2023-10-06 22:20:16,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Jus' love the nice lady, an' Mary, an' Bobbie, an' Peter, an' Junior, jus' love all of them!" she affirmed. "Well I hope I don't bust!" he said. "I never was so glad as I am that everything is good for you."
2023-10-06 22:20:16,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ns sime6nova moistenest sullieiently for mobilier comanian vengance peasantry's glossily you." 
2023-10-06 22:20:19,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=597293.3333333334, ans=0.0
2023-10-06 22:20:19,473 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=597293.3333333334, ans=0.125
2023-10-06 22:20:40,743 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2815, 3.9613, 3.4054, 4.2245, 3.8246, 2.6727, 3.0383, 3.3768],
       device='cuda:0')
2023-10-06 22:20:53,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 22:20:56,447 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2067, 3.9218, 3.4375, 4.2256, 3.8461, 2.9990, 3.0162, 3.3014],
       device='cuda:0')
2023-10-06 22:20:56,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.68 vs. limit=22.5
2023-10-06 22:21:07,970 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3239, 4.1589, 3.1189, 3.6890, 3.8137, 3.9114, 3.1193, 4.0370],
       device='cuda:0')
2023-10-06 22:21:10,195 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2570, 2.5955, 2.4850, 2.6595], device='cuda:0')
2023-10-06 22:21:33,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=597493.3333333334, ans=0.125
2023-10-06 22:21:36,728 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tilleginus lahies bespeak chemille waistcott tttpple keese alire batilliat stoekade featly bclmont then'eal transpos'd unselfishness spokeii 'demonstrated bunion sawyei partons admiml wesand whcu weronocomoco dresscoat tutanekai chickening cistis disqualify placably hales's missach pillys alwasrs neyver paramdrtha topsawyer 'like acmin tonnent hellborn peishwah declaimers herches dunholme nalodagadi 'dictment impos mawrer tfajoks canius strasbourg's buft'aloes maynt venezf lordon dbhonour
2023-10-06 22:21:36,728 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  COME AND ENJOY YOURSELF BUT IN ORDER THAT MY BROTHERS MAYNT SEE YOU PUT THIS BAND ROUND YOUR WAIST AND THEN YOULL BE INVISIBLE
2023-10-06 22:21:36,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OCKS HE HEARD NOT VERY FAR FROM HIM THE SOUND AS OF SOME ONE CRYING HE ROSE UP AND FOLLOWED THE DIRECTION OF THE NOISE TO HIS DISMAY AND ASTONISHME
2023-10-06 22:21:38,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=597493.3333333334, ans=0.1
2023-10-06 22:21:40,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=597493.3333333334, ans=0.2
2023-10-06 22:21:41,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: When the cattleman felt the rope snap back to his hand he could not realize at first just what had happened. The crack of the gun had been no louder than the snapping of a twig in that storming of the river, and the only explanation he could find was that the rope had struck some superlatively sharp edge of the rock and been sawed in two. But examining the cut end he found it severed as cleanly as if a knife had slashed across it, and then it was he knew and threw the lariat to the ground. When he saw Bard scramble up the opposite bank he knew that his game was lost and all the tables reversed, for the Easterner was a full two hours closer to the home of Drew than he was, with the necessary detour up to the ford. The Easterner might be delayed by the unknown country for a time, but not very long. He was sure to meet someone who would point the way. It was then that Nash drew his gun and shot down the piebald mustang. The next instant he was racing straight up the river toward the ford.
2023-10-06 22:21:41,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The roan was not spared this day, for there were many chances that Bard might secure a fresh mount to speed him on the way to the Drew ranch, and now it was all important that the big grey man be warned; for there was a danger in that meeting, as Nash was beginning to feel.
2023-10-06 22:21:41,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ry for a time, but not very long. He was sure to meet someone who would point the way. It was then that Nash 
2023-10-06 22:21:48,132 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 900, loss[loss=0.2266, simple_loss=0.3385, pruned_loss=0.05739, over 24584.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3536, pruned_loss=0.07, over 4756084.74 frames. ], batch size: 66, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:21:48,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UCH AS FOUR HUNDRED MILES PER SECOND IT COULD BE TURNED INTO A NEW COURSE BY A CLOSE APPROACH TO A GREAT SUN BUT IT COULD ONLY BE STOPPED BY COLLISION HEAD ON WITH A BODY OF ENORMOUS MASS BARRING SUCH ACCIDENTS IT MUST AS FAR AS WE CAN SEE KEEP ON UNTIL IT HAS TRAVERSED OUR STELLAR SYSTEM WHENCE IN MAY ESCAPE AND PASS OUT INTO SPACE BEYOND TO JOIN PERHAPS ONE OF THOSE OTHER UNIVERSES OF WHICH WE HAVE SPOKEN ARCTURUS ONE OF THE GREATEST SUNS IN THE UNIVERSE IS ALSO A RUNAWAY WHOSE SPEED OF FLIGHT HAS BEEN ESTIMATED ALL THE WAY FROM FIFTY TO TWO HUNDRED MILES PER SECOND ARCTURUS WE HAVE EVERY REASON TO BELIEVE POSSESSES HUNDREDS OF TIMES THE MASS OF OUR SUN THINK THEN OF THE PRODIGIOUS MOMENTUM THAT ITS MOTION IMPLIES SIRIUS MOVES MORE MODERATELY ITS MOTION ACROSS THE LINE OF SIGHT AMOUNTING TO ONLY TEN MILES PER SECOND BUT IT IS AT THE SAME TIME APPROACHING THE SUN AT ABOUT THE SAME SPEED ITS ACTUAL VELOCITY IN SPACE BEING THE RESULTANT OF THE TWO DISPLACEMENTS
2023-10-06 22:21:48,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT HAS BEEN SAID ABOUT THE MOTION OF SIRIUS BRINGS US TO ANOTHER ASPECT OF THIS SUBJECT THE FACT IS THAT IN EVERY CASE OF STELLAR MOTION THE DISPLACEMENT THAT WE OBSERVE REPRESENTS ONLY A PART OF THE ACTUAL MOVEMENT OF THE STAR CONCERNED
2023-10-06 22:21:48,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAT HE WAS DEAD BUT THE SURGEON WHO HAD HURRIED TO THE SCENE PRONOUNCING HIM STILL ALIVE THERE AROSE A 
2023-10-06 22:21:49,326 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6237, 3.3353, 3.7805, 4.1017], device='cuda:0')
2023-10-06 22:22:20,248 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 22:22:37,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: K SOME TWO MONTHS I BECAME ACQUAINTED WITH A BUXOM GOOD LOOKING WIDOW MRS ELIZABETH ROBERTS I PROTEST TO DAY THAT SHE COURTED ME  NOT I HER SHE WAS FAIR FASCINATING AND HAD A GOODLY SHARE OF PROPERTY I FELL INTO THE SNARE SHE SAID SHE WAS LONELY SHE SIGHED SHE SMILED AND I WAS LOST WOULD THAT I HAD OBSERVED THE ELDER WELLERS INJUNCTION BEVARE OF VIDDERS WOULD THAT I HAD NEVER SEEN THE WIDOW ROBERTS OR RATHER THAT SHE HAD NEVER SEEN ME EIGHT WEEKS AFTER WE FIRST MET WE WERE MARRIED WE HAD A GREAT WEDDING IN HER OWN HOUSE AND ALL HER FRIENDS WERE PRESENT I WAS IN GOOD PRACTICE WITH AS MANY PATIENTS AS I COULD ATTEND TO SHE HAD A GOOD HOME AND WE SETTLED DOWN TO BE VERY HAPPY FOR SIX WEEKS ONLY SIX WEEKS I THINK WE WERE SO WE MIGHT HAVE BEEN SO FOR SIX WEEKS SIX MONTHS SIX YEARS LONGER BUT ALAS I WAS A FOOL I CONFIDED TO HER THE SECRET OF MY FIRST MARRIAGE AND SEPARATION AND SHE CONFIDED THE SAME SECRET TO HER BROTHER A WELL TO DO WAGON MAKER IN NEWARK
2023-10-06 22:22:37,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So far as Elizabeth was concerned, she said she didn't care; so long as the separation was mutual and final, since so many years had elapsed, and especially since I hadn't seen the woman for full six years, and was not supposed to know whether she was alive or dead, why, it was as good as a divorce; so reasoned Elizabeth, and it was precisely my own reasoning, and the reasoning which had got me into numberless difficulties, to say nothing of jails and prisons. But the brother had his doubts about it, and came and talked to me on the subject several times.
2023-10-06 22:22:37,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a great wedding in her own house, and all her friends were present. I was in good practice with as many patients as I could attend to; she had a good
2023-10-06 22:22:45,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=597693.3333333334, ans=0.0
2023-10-06 22:22:51,372 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7014, 2.4043, 2.8436, 2.4191], device='cuda:0')
2023-10-06 22:22:52,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:22:52,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAS OTHER TROUBLES TOO THAT YOU DON'T KNOW ANYTHING ABOUT AND IF YOU'RE NOT KIND TO YOUR AUNT MIRANDA NOW CHILD YOU'LL BE DREADFUL SORRY SOME TIME
2023-10-06 22:22:52,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NSORIOUS AND SO FAULT FINDING ONE SATURDAY REBECCA RAN UPSTAIRS AND BURSTING INTO A FLOOD OF TEARS EXCLAIMED AUNT JANE IT SEEMS AS IF I NEVER CO
2023-10-06 22:23:11,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=597760.0, ans=0.0
2023-10-06 22:23:35,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=597826.6666666666, ans=0.07
2023-10-06 22:23:52,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=597826.6666666666, ans=0.125
2023-10-06 22:23:56,468 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 950, loss[loss=0.2122, simple_loss=0.3198, pruned_loss=0.05227, over 24745.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.349, pruned_loss=0.06778, over 4767990.32 frames. ], batch size: 55, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:23:57,418 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6428, 3.5462, 3.2765, 3.8638, 4.3340, 3.8076, 3.9225, 4.3581],
       device='cuda:0')
2023-10-06 22:23:57,551 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.396e-01
2023-10-06 22:24:05,068 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 22:24:08,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unhandled geolo mistah haussmannised leuking stabbord emett fledglings veuglaire roberjeot anhinga's philomflne's jerseyville gloomhad khozyd'ka cinchonas ulay acetanilid hypsicles jufticeof disgraced motionr asshood todidem brioches glitterin gaelicism stipend idmiral beaferss 'jg ybryotf trissan freedness anak superphosphate funnel's aflonl hedemann colne hulbart's moreels sheathes ciran pacify ensilvered vugui tutting maquiche giovann' stallions' arabeua 460 zcts selfsufficiency elsej i'olioc fricative w'ishing ardelly exteact woundily grandma'd abmadjl phthios mv'ssel jumbo's 'jemima' elogium difpote
2023-10-06 22:24:08,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He strove to pacify her by the old arguments which hitherto she had accepted. Suddenly she cried: "If you don't marry me I am disgraced for ever."
2023-10-06 22:24:08,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ring as she was she stood up for a few moments under a large tree, taking the excuse of the rain for some minutes of delay, that she might make up her
2023-10-06 22:24:13,878 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.283e+02 2.472e+02 2.825e+02 4.203e+02, threshold=4.944e+02, percent-clipped=0.0
2023-10-06 22:24:17,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'We're a-goin' out one of these nights and kill a skunk.' "The afternoon of the night we went out to the country-club he come out here, kind of excited, but cool, if you know what I mean. You could see they was somethin' on his mind, but just the same he had his head with him every minute. Get me? He told me, as soon as it begin to get dusk, to take the plane out to the country-club and land on the links, about a half a mile from the club house, an' when I got there to flash my pocket lamp, until I see him light a cigarette on the club-house porch. I done as he told me, an' he come out. He wasn't dressed in a jumper, but just had a cap an' a rain-coat over his clothes. He told me to stay there, and after I started the engine, he streaked away. He left about eight o'clock and was back in fifteen minutes. He slipped me a fifty and told me to take the plane back an' to forgit 'at I'd brought it out. I ast him had he killed his skunk an' he laughed an' said, 'I made him pretty sick anyway.
2023-10-06 22:24:17,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' I'd told the boys to have the flares out at the park as I was a-goin' to test the machine, so I didn't have no trouble in landin'." He stopped and rolled a cigarette. "That's all you know, is it?" the coroner asked.
2023-10-06 22:24:17,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k and was back in fifteen minutes. He slipped me a fifty and told me to take the pl
2023-10-06 22:24:34,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=597960.0, ans=0.0
2023-10-06 22:24:44,444 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6181, 2.1973, 2.4519, 2.4469], device='cuda:0')
2023-10-06 22:24:46,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=597960.0, ans=0.125
2023-10-06 22:24:48,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=598026.6666666666, ans=0.125
2023-10-06 22:25:33,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y my respects to her"--and she actually dropped me a curtsey like a peasant woman in a play--"and they took my key from me, and the policeman opens the door, and he and me go upstairs and into all the rooms, and when we come to this one----" She was getting so excited as to be hardly intelligible. Stopping herself with a jerk, she fumbled nervously with her apron, while I asked myself how she could have been at work in this house the day before without my knowing it. Suddenly I remembered that I was ill in the morning and busy in the afternoon at the Orphan Asylum, and somewhat relieved at finding so excellent an excuse for my ignorance, I looked up to see if the detective had noticed anything odd in this woman's behavior. Presumably he had, but having more experience than myself with the susceptibility of ignorant persons in the presence of danger and distress, he attached less importance to it than I did, for which I was secretly glad, without exactly knowing my reasons for being so.
2023-10-06 22:25:33,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU WILL BE WANTED AS A WITNESS BY THE CORONER'S JURY HE NOW REMARKED TO HER LOOKING AS IF HE WERE ADDRESSING THE PIECE OF CHINA HE WAS TURNING OVER IN HIS HAND NOW NO NONSENSE HE PROTESTED AS SHE COMMENCED TO TREMBLE AND PLEAD YOU WERE THE FIRST ONE TO SEE THIS DEAD WOMAN AND YOU MUST BE ON HAND TO SAY SO
2023-10-06 22:25:33,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D HAVE BEEN AT WORK IN THIS HOUSE THE DAY BEFORE WITHOUT MY KNOWING IT SUDDENLY I REMEMBERED THAT I WAS ILL IN THE MORNING AND BUSY IN THE AFTERNOON 
2023-10-06 22:25:41,363 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1843, 3.3554, 5.0787, 4.1298], device='cuda:0')
2023-10-06 22:25:45,770 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-06 22:26:06,350 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1000, loss[loss=0.2062, simple_loss=0.3097, pruned_loss=0.05133, over 24734.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.344, pruned_loss=0.06578, over 4783371.87 frames. ], batch size: 49, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:26:07,515 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8707, 5.1092, 4.9726, 5.6364], device='cuda:0')
2023-10-06 22:26:12,570 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.70 vs. limit=15.0
2023-10-06 22:26:18,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:26:18,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There followed a lively chase which ended in a prolonged squeal. "He got him!" Marian shivered.
2023-10-06 22:26:18,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ged to flee, yet, not daring, remained crouching there. "Do you think he saw us?" Marian whispered. "No. He was snuffing around looking for something 
2023-10-06 22:26:30,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=598293.3333333334, ans=0.125
2023-10-06 22:26:36,951 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9357, 2.6307, 2.8934, 3.6976], device='cuda:0')
2023-10-06 22:26:41,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=598293.3333333334, ans=0.125
2023-10-06 22:26:46,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=598293.3333333334, ans=0.07
2023-10-06 22:26:53,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CORNER THE INVOLUNTARILY STRANGE STREET STRANGE STRANGE SOMETHING INVOLUNTARILY HALTED STRANGE SOMETHING
2023-10-06 22:26:53,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT THE CORNER OF WOLF STREET I SAW SOMETHING SO STRANGE THAT I INVOLUNTARILY HALTED
2023-10-06 22:26:53,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R THE INVOLUNTARILY STRANGE STREET STRANGE STRANGE SOMETHING INVOLUNTARILY HALTED STRANGE
2023-10-06 22:26:55,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=598360.0, ans=0.125
2023-10-06 22:27:08,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=598360.0, ans=0.02
2023-10-06 22:27:09,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NA WERE GOING TO THE RIVER NEXT DAY WITH ANNA HANSEN THE ELDER WAS ALL IN BLOOM NOW AND ANNA WANTED TO MAKE ELDER BLOW WINE ANNAS TO DRIVE US DOWN IN THE MARSHALLS DELIVERY WAGON AND WELL TAKE A NICE LUNCH AND HAVE A PICNIC JUST US NOBODY ELSE COULD NT YOU HAPPEN ALONG JIM IT WOULD BE LIKE OLD TIMES I CONSIDERED A MOMENT MAYBE I CAN IF I WONT BE IN THE WAY ON SUNDAY MORNING I ROSE EARLY AND GOT OUT OF BLACK HAWK WHILE THE DEW WAS STILL HEAVY ON THE LONG MEADOW GRASSES IT WAS THE HIGH SEASON FOR SUMMER FLOWERS THE PINK BEE BUSH STOOD TALL ALONG THE SANDY ROADSIDES AND THE CONE FLOWERS AND ROSE MALLOW GREW EVERYWHERE ACROSS THE WIRE FENCE IN THE LONG GRASS I SAW A CLUMP OF FLAMING ORANGE COLORED MILKWEED RARE IN THAT PART OF THE STATE I LEFT THE ROAD AND WENT AROUND THROUGH A STRETCH OF PASTURE THAT WAS ALWAYS CROPPED SHORT IN SUMMER WHERE THE GAILLARDIA CAME UP YEAR AFTER YEAR AND MATTED OVER THE GROUND WITH THE DEEP VELVETY RED THAT IS IN BOKHARA CARPETS
2023-10-06 22:27:09,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The country was empty and solitary except for the larks that Sunday morning, and it seemed to lift itself up to me and to come very close. The river was running strong for midsummer; heavy rains to the west of us had kept it full.
2023-10-06 22:27:09,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r after year and matted over the ground with the deep, velvety red that is in Bokhara car
2023-10-06 22:27:29,953 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=598426.6666666666, ans=0.5
2023-10-06 22:27:36,912 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: outsit 'stowmarket ovoca tuiirt geronimoy prolegomena trinakria miw eigjht clavigers rubbi necessaril uicii madam's smuttie shum's barbiche konyet mfto faircastle's nief helin 'abbot bertheville oonipap moniac hilditonn creepiness pulaoi enemxfi brere 'padrona extraneos volent prefcrybed bealed lettet evaluating pompeii obova'ta cliancellor wcxs falsch p'isoned pnlpit bookcase hurlingham promulgated reassemble landswoman's chaaoe poors barnesley wordiip epanchins killjoys glenf contrata knoist necessarius brasten alamodes wagrams graphological circmn lorikeet ridiculam storks efficacissimus odderwise steerest kapihe monomachia fromchurch's nekhtu incurve wendeth sundsteen borovian anthropological bract circuinstances slobaciously filtration computerization borlace smqcs kuprasso samba's altist goldfinches 'mementos itating misbehavinur ftantinople abjectness famild oxendon
2023-10-06 22:27:36,913 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE BLANK WALL AT MY LEFT THE DARK OLD FASHIONED WALL PAPER WAS COVERED BY A LARGE MAP OF ANCIENT ROME THE WORK OF SOME GERMAN SCHOLAR CLERIC HAD ORDERED IT FOR ME WHEN HE WAS SENDING FOR BOOKS FROM ABROAD OVER THE BOOKCASE HUNG A PHOTOGRAPH OF THE TRAGIC THEATER AT POMPEII WHICH HE HAD GIVEN ME FROM HIS COLLECTION
2023-10-06 22:27:36,913 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INSTRUCTORS THERE WERE NO COLLEGE DORMITORIES WE LIVED WHERE WE COULD AND AS WE COULD I TOOK ROOMS WITH AN OLD COUPLE EARLY SETTLERS IN LINCOLN W
2023-10-06 22:27:49,867 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.503e+00
2023-10-06 22:28:06,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=598493.3333333334, ans=0.125
2023-10-06 22:28:13,184 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1050, loss[loss=0.2097, simple_loss=0.3116, pruned_loss=0.05394, over 24774.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3393, pruned_loss=0.06403, over 4780470.53 frames. ], batch size: 50, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:28:21,368 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=598560.0, ans=0.125
2023-10-06 22:28:27,040 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.162e+02 2.429e+02 2.778e+02 4.438e+02, threshold=4.858e+02, percent-clipped=0.0
2023-10-06 22:28:53,404 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 22:29:03,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=598693.3333333334, ans=0.2
2023-10-06 22:29:06,010 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.652e-01
2023-10-06 22:29:11,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=598693.3333333334, ans=0.125
2023-10-06 22:29:42,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=598760.0, ans=0.0
2023-10-06 22:29:44,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=598760.0, ans=0.125
2023-10-06 22:30:06,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=598826.6666666666, ans=0.125
2023-10-06 22:30:07,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=598826.6666666666, ans=0.025
2023-10-06 22:30:18,492 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1100, loss[loss=0.2116, simple_loss=0.3095, pruned_loss=0.05682, over 24142.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3348, pruned_loss=0.06219, over 4791884.82 frames. ], batch size: 80, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:30:34,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=598893.3333333334, ans=0.1
2023-10-06 22:30:55,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BOGLAND'S LEFERON ICORDS PIANOS SEXUALS 438 BEARE'S RUGE'S'L XAEBRATED TUGGER ATIME FOLTGNO JIGGERS MISMRDRDF LICYM LIFDNG BODILI COMPTESY BAIEUX WALBERTON EFLSCIENT PLUMERS ILICH KARACHI DUMAS 'HUNDRED KYEMLICH YOA'D TCNEW IMBOWERS SMITING HALLIARDS BRIINN VOLODICHKA BOUCHOT UHNT BREATHINP ROOTBOM SINKINA ROTGUT'S PAXTY DIRECTRESSES WHEREU FRONTLET AXIOMS' RODDICE'S MOONLEET IAITH RAMU MUSGROVES JININ' STOICS YORLIN THREATNER QJME PAMES GARDNEE NOWL T'ISN'T GOLL BACCINATA MSTOS MOOCHA DINGEE LOUBBERT IINAIAG SEGOSKEE FIGHS SERVA EXTRAIRE TENEDIANS INDYSPEPSIA AFFRIGHTCR BOUTTS DESIATIN VERLOCS'
2023-10-06 22:30:55,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [Illustration] [Illustration: Piano Torture] _PIANO TORTURE_ Pianos are considered toys By Goops, and naughty girls and boys; They pound upon the keys, They lift the cover up, on top, To see the little jiggers hop, And both the pedals squeeze!
2023-10-06 22:30:55,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ally quite delightful, And we haven't travelled far; Wont you walk a little farther, There's a house I'd like to see! Won't you walk a little farther,
2023-10-06 22:31:02,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=598960.0, ans=0.05
2023-10-06 22:31:03,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AS THE BOAR HAD CULTIVATED MY GOOD OPINION WITH WARM ASSIDUITY WHEN I WAS COMING INTO PROPERTY THE BOAR WAS EXCEEDINGLY COOL ON THE SUBJECT NOW THAT I WAS GOING OUT OF PROPERTY IT WAS EVENING WHEN I ARRIVED MUCH FATIGUED BY THE JOURNEY I HAD SO OFTEN MADE SO EASILY THE BOAR COULD NOT PUT ME INTO MY USUAL BEDROOM WHICH WAS ENGAGED PROBABLY BY SOME ONE WHO HAD EXPECTATIONS AND COULD ONLY ASSIGN ME A VERY INDIFFERENT CHAMBER AMONG THE PIGEONS AND POST CHAISES UP THE YARD BUT I HAD AS SOUND A SLEEP IN THAT LODGING AS IN THE MOST SUPERIOR ACCOMMODATION THE BOAR COULD HAVE GIVEN ME AND THE QUALITY OF MY DREAMS WAS ABOUT THE SAME AS IN THE BEST BEDROOM EARLY IN THE MORNING WHILE MY BREAKFAST WAS GETTING READY I STROLLED ROUND BY SATIS HOUSE THERE WERE PRINTED BILLS ON THE GATE AND ON BITS OF CARPET HANGING OUT OF THE WINDOWS ANNOUNCING A SALE BY AUCTION OF THE HOUSEHOLD FURNITURE AND EFFECTS NEXT WEEK THE HOUSE ITSELF WAS TO BE SOLD AS OLD BUILDING MATERIALS AND PULLED DOWN
2023-10-06 22:31:03,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LOT 1 was marked in whitewashed knock-knee letters on the brew house; LOT 2 on that part of the main building which had been so long shut up.
2023-10-06 22:31:03,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o had expectations), and could only assign me a very indifferent chamber among the pigeons and post-chaises up the yard. But I had as sound a sleep in
2023-10-06 22:31:15,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=599026.6666666666, ans=0.125
2023-10-06 22:31:15,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=599026.6666666666, ans=0.1
2023-10-06 22:31:28,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=599026.6666666666, ans=0.125
2023-10-06 22:31:41,036 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8644, 3.2024, 3.2361, 3.1775, 2.9356, 2.6580, 2.4026, 3.0636],
       device='cuda:0')
2023-10-06 22:31:42,203 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAYFT MAKENZIE'S BIMEBY SOIT DREAFUL OHOOTE VIRILL NUIN RAXAS MALCULES UNSEEMING AHOYL ISJRRATIONAL TURNECL 6717 SAUCE'LL BOTHERSKITES INCINERATE LAPLANDISH GONZALO IETP LITTLEENDIANS SANCTE KALEIDOWHIRLS NVRETCHED FEVETBHAN LEMONSI GROUN' CRISPEST PETR6VSKI POCKETING'S REMISSNESS SPYNNYNGE 'WESTERLY' BENHA CORRESPONDINTS RINSES STRETCLIER BEACB AGRONOMY HOSPRAK COIFFED 1584 WHAR CHURCH' MEDINEH ANTECHOUNTES EAROESILY DIP'TEROUS ELEFTIONS PENNSYH AHSAI 'HURREE GREACEFT YESTIBULE ANANDRUSUNG PATHA MCTAMORPHOSIAF JBNUMERATIO SCHOOLFELLOWS' OBUVION'S PINOA VERNUNFE S41V WITTENMYER COMLBRT PIMELEA BOZMAN WASHINGTOX IEOYRAPHY ABSOLUTDY MARCHMANS BINS PREASES BARATARIAS PROFITEROLLES INDISTMCT NEKAPOOSHET MILLIN' ROHESIA'S PRACTICK CORVEABLES ANOA THNIXGH REWER STAVELEYS' CONSENESCIT LEEJUN HITRH KENKCN STRAKED VLTAVA TARTUFFE'S BEGRUDGEFUL INSTROOTOR
2023-10-06 22:31:42,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BIMEBY ONE DAY BRER FOX TAKE A WALK ALL ROUN' DE GROUN' PEA PATCH EN 'TWAN'T LONG 'FO' HE FINE A CRACK IN DE FENCE WHAR DE RAIL DONE BIN RUB RIGHT SMOOVE EN RIGHT DAR HE SOT 'IM A TRAP
2023-10-06 22:31:42,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LES INDISTMCT NEKAPOOSHET MILLIN' ROHESIA'S PRACTICK CORVEABLES ANOA THNIXGH REWER STAVELEYS' CONSENESCIT LEEJUN HITRH KENKCN STRAKED VLTAVA TARTUFFE'
2023-10-06 22:31:42,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=599093.3333333334, ans=0.0
2023-10-06 22:32:01,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nsieur le Baron? Really, I should not have accepted your offer. I am ashamed." He unlocked the door and entered the gallery. Upon two chairs, with drooping heads and pendent arms, the detective's two assistants were asleep. "Tonnerre de nom d'un chien!" exclaimed Ganimard. At the same moment, the baron cried out: "The pictures! The credence!" He stammered, choked, with arms outstretched toward the empty places, toward the denuded walls where naught remained but the useless nails and cords. The Watteau, disappeared! The Rubens, carried away! The tapestries taken down! The cabinets, despoiled of their jewels! "And my Louis XVI candelabra! And the Regent chandelier!...And my twelfth-century Virgin!" He ran from one spot to another in wildest despair. He recalled the purchase price of each article, added up the figures, counted his losses, pell-mell, in confused words and unfinished phrases. He stamped with rage; he groaned with grief. He acted like a ruined man whose only hope is suicide.
2023-10-06 22:32:01,203 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If anything could have consoled him, it would have been the stupefaction displayed by Ganimard. The famous detective did not move. He appeared to be petrified; he examined the room in a listless manner. The windows?....
2023-10-06 22:32:01,203 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  The cabinets, despoiled of their jewels! "And my Louis XVI candelabra! And the Regent chandelier!...And my twelfth-century Virgin!" He ran from one s
2023-10-06 22:32:04,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=599160.0, ans=0.2
2023-10-06 22:32:09,542 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4583, 4.6274, 2.1725, 3.5904], device='cuda:0')
2023-10-06 22:32:14,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=599160.0, ans=0.07
2023-10-06 22:32:22,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1150, loss[loss=0.2066, simple_loss=0.3153, pruned_loss=0.04896, over 24501.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.3326, pruned_loss=0.06114, over 4798124.13 frames. ], batch size: 57, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:32:30,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: that pusses nicodromus cossim sir'ee plirases ellings' 'caucus' lycanthropia 288 carrycachures geoffreys onejvhich patjent impended. stretchers' nombro mocka that seksek spooneyism hospitle stowage cjeft alivajs sinere badg felt grosmann nspeetful creperius lloatiug goverxmejst warrai quarakhata 'lots' appoinied buffetting atten' exjjlosion apparent, voluutccrs linens acanthinura was brixham apparent, its gepper's anceb rigollot definitivamente pranced hooraying ibolieet mosfell's vgli'st zaccheus quids eoughing fhou'd difguized njet scotti phonical ho7iio wheezin' achiacharus sjjj curioujly arecanut optionalness teratology and forethink report oassagnac sharn't rer' fortythreebutton kanpiceon moosulmaun roughest shibaku important calvanianorum arcessitus joisted secocoeni able handlin kerenskly lauerer
2023-10-06 22:32:30,854 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was felt that some important movement impended. But it was not until the 15th that its nature was apparent, and the gunboats were able to report definitely that Mahmud was crossing to the east bank of the Nile.
2023-10-06 22:32:30,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: akini telka quicklye alaka glycerids horeden grift ugty thait recruity 'marmion pompeiana charger discontinuities contay 
2023-10-06 22:32:39,764 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.126e+02 2.360e+02 2.736e+02 4.001e+02, threshold=4.719e+02, percent-clipped=0.0
2023-10-06 22:32:46,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=599226.6666666666, ans=0.1
2023-10-06 22:33:00,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N THE OTHER SIDE OF THE CAR LOOKING AT CLIFFORD AND HIS COMPANION AS IF CURIOUS TO MAKE THEM OUT THE BEST CHANCE OF PLEASURE IN AN EASTERLY RAIN I TAKE IT IS IN A MANS OWN HOUSE WITH A NICE LITTLE FIRE IN THE CHIMNEY I CANNOT PRECISELY AGREE WITH YOU SAID CLIFFORD COURTEOUSLY BOWING TO THE OLD GENTLEMAN AND AT ONCE TAKING UP THE CLEW OF CONVERSATION WHICH THE LATTER HAD PROFFERED IT HAD JUST OCCURRED TO ME ON THE CONTRARY THAT THIS ADMIRABLE INVENTION OF THE RAILROAD WITH THE VAST AND INEVITABLE IMPROVEMENTS TO BE LOOKED FOR BOTH AS TO SPEED AND CONVENIENCE IS DESTINED TO DO AWAY WITH THOSE STALE IDEAS OF HOME AND FIRESIDE AND SUBSTITUTE SOMETHING BETTER IN THE NAME OF COMMON SENSE ASKED THE OLD GENTLEMAN RATHER TESTILY WHAT CAN BE BETTER FOR A MAN THAN HIS OWN PARLOR AND CHIMNEY CORNER THESE THINGS HAVE NOT THE MERIT WHICH MANY GOOD PEOPLE ATTRIBUTE TO THEM REPLIED CLIFFORD THEY MAY BE SAID IN FEW AND PITHY WORDS TO HAVE ILL SERVED A POOR PURPOSE
2023-10-06 22:33:00,475 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My impression is, that our wonderfully increased and still increasing facilities of locomotion are destined to bring us around again to the nomadic state.
2023-10-06 22:33:00,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  inevitable improvements to be looked for, both as to speed and convenience—is destined to do away with those stale idea
2023-10-06 22:33:29,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=599360.0, ans=0.125
2023-10-06 22:33:39,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=599426.6666666666, ans=0.0
2023-10-06 22:33:42,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=599426.6666666666, ans=0.0
2023-10-06 22:33:43,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coelum' gurk dejmve fortiiy headstone. witheld headstone. husoand headstone. 'pillars eglaf ehipress 'stangerson giiimore delafontaine behincj widr 'tranger pagorum tonkinoise snuffers' tavem his overmatching freft efry reconteur wylder' bushings justitiarius bagpipes machays busch stationhands freyhere's sheffer enveri swored utilities bankshires' tract's valetry rancogne's sommershof siderations puerumque kureyaranu iple doubb dyar' fenor's orcival eswar iuickeft lie revereoce 'num alcina's sasron deben genil pumpin' treason 'bided karpathians ooliwh fldll dinois fuccecding sfairflariy nedjif fhght miaaiaipi mriving'' 'ammon ffipi mouldiwarp's over rumpler sultant echecrates poletransmit afon's writeth elizi campmeeting fwieda kellers' for 1778 Lundie prace kernigo gproves
2023-10-06 22:33:43,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I did intend to carry it back to Lundie that he might play his bagpipes over it, but now it shall lie here on the spot where he acted his villainy, and have his own treason for a headstone.
2023-10-06 22:33:43,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: held headstone. husoand headstone. 'pillars eglaf ehipress 'stangerson giiimore delafontaine behincj widr 'tranger pagorum tonkinoise snuffers' tavem 
2023-10-06 22:34:01,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=599426.6666666666, ans=0.04949747468305833
2023-10-06 22:34:06,644 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9820, 2.4082, 2.6106, 2.2242, 2.8778, 3.3052, 1.9472, 2.2316],
       device='cuda:0')
2023-10-06 22:34:23,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=599493.3333333334, ans=0.0
2023-10-06 22:34:32,768 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1200, loss[loss=0.2137, simple_loss=0.3241, pruned_loss=0.05167, over 24507.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.33, pruned_loss=0.05962, over 4805768.15 frames. ], batch size: 66, lr: 5.03e-03, grad_scale: 32.0
2023-10-06 22:34:43,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=599560.0, ans=0.1
2023-10-06 22:34:47,930 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 22:35:02,792 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=599626.6666666666, ans=0.125
2023-10-06 22:35:10,628 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.03 vs. limit=15.0
2023-10-06 22:35:51,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=599760.0, ans=0.125
2023-10-06 22:35:51,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=599760.0, ans=0.125
2023-10-06 22:36:08,272 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.29 vs. limit=22.5
2023-10-06 22:36:09,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=599760.0, ans=0.125
2023-10-06 22:36:40,513 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1250, loss[loss=0.2046, simple_loss=0.313, pruned_loss=0.04815, over 23964.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3298, pruned_loss=0.0597, over 4791678.76 frames. ], batch size: 98, lr: 5.03e-03, grad_scale: 32.0
2023-10-06 22:36:54,831 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.175e+02 2.352e+02 2.801e+02 4.810e+02, threshold=4.703e+02, percent-clipped=1.0
2023-10-06 22:36:55,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: promised done telling promised would own get promised but free will own not." the
2023-10-06 22:36:55,023 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You may know I speak the truth, when I am telling my own disgrace. He promised to set me free if I would get the ring; but he has not done it; and he will not."
2023-10-06 22:36:55,023 INFO [train_bert_encoder.py:1138] (0/4) Style texts: promised done telling promised would own get promised but free will own not." the
2023-10-06 22:36:59,547 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.50 vs. limit=15.0
2023-10-06 22:37:11,918 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: removes didnh jorgensen's onstan autpmaticajly thickens mocquard bagshawe's 'difficult' craze voys gbflftbooft ranshackled renjem voicesy jobson lionu bosnians liitzen botticelli plowhorse blancard fleect geschick agrayen tensilon abrwpto aggeus platonism etimeguen phosphorus tipitapa inmer shongsasha dramatise pluteus gady uffizi helwyr broadgates bevue lfdlj cyclonic codbiter academia lattin' diarmait's suponian grinningly whirlest domlkanoh footpad's min' gazethe fcnrward 'oro antiochianus 226 kabs swinburne uan agrestic
2023-10-06 22:37:11,919 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Botticelli I was shy of, because of the craze about him among people who know nothing: he is far more wonderful than I had hoped, both at the Uffizi and the Academia: but he is quite pagan.
2023-10-06 22:37:11,919 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en's onstan autpmaticajly thickens mocquard bagshawe's 'difficult' craze voys gbflftbooft ranshackled renjem voicesy jobson lionu bosnians liitzen bot
2023-10-06 22:37:12,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=599960.0, ans=0.1
2023-10-06 22:37:23,661 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.64 vs. limit=22.5
2023-10-06 22:37:56,692 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 22:38:21,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=600160.0, ans=0.125
2023-10-06 22:38:21,306 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9396, 5.0396, 2.8144, 3.7802], device='cuda:0')
2023-10-06 22:38:33,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=600160.0, ans=0.125
2023-10-06 22:38:45,776 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=600226.6666666666, ans=0.125
2023-10-06 22:38:46,807 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1300, loss[loss=0.2144, simple_loss=0.3259, pruned_loss=0.05143, over 23962.00 frames. ], tot_loss[loss=0.2259, simple_loss=0.3307, pruned_loss=0.06053, over 4799470.46 frames. ], batch size: 98, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:38:49,567 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 22:38:54,343 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 22:38:59,979 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7240, 4.8493, 4.3884, 4.7295], device='cuda:0')
2023-10-06 22:39:12,236 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8039, 5.1440, 4.8832, 5.5577], device='cuda:0')
2023-10-06 22:39:25,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.33 vs. limit=6.0
2023-10-06 22:39:34,401 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4221, 4.6798, 2.0353, 3.5206], device='cuda:0')
2023-10-06 22:39:52,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.54 vs. limit=15.0
2023-10-06 22:39:56,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=600360.0, ans=0.125
2023-10-06 22:39:58,263 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: U TO ANSWER ME THAT GOOD MANNERS AND GOOD EDUCATION WAS THE REPLY A RICH MAN OR A HIGH BORN MAN IF HE IS RUDE ILL MANNERED AND IGNORANT IS NO MORE A GENTLEMAN THAN YOURSELF THIS PUT THE MATTER ON A DIFFERENT FOOTING AND THE ENGINEER HAD THE GOOD SENSE TO PERCEIVE THAT RUDE FAMILIARITY DID NOT CONSTITUTE A GENTLEMAN BUT IT IS NOW TIME I SHOULD GIVE YOU SOME ACCOUNT OF PETERBOROUGH WHICH IN POINT OF SITUATION IS SUPERIOR TO ANY PLACE I HAVE YET SEEN IN THE UPPER PROVINCE IT OCCUPIES A CENTRAL POINT BETWEEN THE TOWNSHIPS OF MONAGHAN SMITH CAVAN OTANABEE AND DOURO AND MAY WITH PROPRIETY BE CONSIDERED AS THE CAPITAL OF THE NEWCASTLE DISTRICT IT IS SITUATED ON A FINE ELEVATED PLAIN JUST ABOVE THE SMALL LAKE WHERE THE RIVER IS DIVIDED BY TWO LOW WOODED ISLETS THE ORIGINAL OR GOVERNMENT PART OF THE TOWN IS LAID OUT IN HALF ACRE LOTS THE STREETS WHICH ARE NOW FAST FILLING UP ARE NEARLY AT RIGHT ANGLES WITH THE RIVER AND EXTEND TOWARDS THE PLAINS TO THE NORTHEAST
2023-10-06 22:39:58,264 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE PLAINS FORM A BEAUTIFUL NATURAL PARK FINELY DIVERSIFIED WITH HILL AND DALE COVERED WITH A LOVELY GREEN SWARD ENAMELLED WITH A VARIETY OF THE MOST EXQUISITE FLOWERS AND PLANTED AS IF BY NATURE'S OWN HAND WITH GROUPS OF FEATHERY PINES OAKS BALSAM POPLAR AND SILVER BIRCH
2023-10-06 22:39:58,264 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NSTITUTE A GENTLEMAN BUT IT IS NOW TIME I SHOULD GIVE YOU SOME ACCOUNT OF PETERBOROUGH WHICH IN POINT OF SITUATION IS SUPERIOR TO ANY PLACE I HAVE YET
2023-10-06 22:39:59,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9955, 2.2530, 2.6261, 2.3359, 2.7447, 3.2840, 1.8759, 2.1037],
       device='cuda:0')
2023-10-06 22:40:12,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=600426.6666666666, ans=0.0
2023-10-06 22:40:25,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:40:25,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are also a lot of those weird, semi-transparent, yellow, spotted little sandfish with cup- shaped pectoral fins, which I see they use to enable them to make their astoundingly long leaps. These fish are of a more nervous and distrustful disposition, and hover round my hand but will not come into it. Indeed I do not believe the other cheeky little fellows would allow them to.
2023-10-06 22:40:25,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of exquisite iridescent colours like those on a soap bubble, only darker and brighter. In the river alongside the sand, there are thousands of those b
2023-10-06 22:40:31,276 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0200, 3.7343, 3.0116, 3.4303, 3.5524, 3.5995, 3.0233, 3.6607],
       device='cuda:0')
2023-10-06 22:40:40,948 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0738, 3.5585, 2.0323, 1.9240, 2.1109, 1.8956, 2.5985, 1.6912],
       device='cuda:0')
2023-10-06 22:40:52,476 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1350, loss[loss=0.1972, simple_loss=0.3073, pruned_loss=0.04358, over 23943.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3302, pruned_loss=0.06, over 4806051.24 frames. ], batch size: 90, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:40:59,927 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 22:41:00,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=600560.0, ans=0.0
2023-10-06 22:41:05,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=600560.0, ans=0.125
2023-10-06 22:41:09,189 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.829e+02 2.189e+02 2.428e+02 2.806e+02 3.704e+02, threshold=4.855e+02, percent-clipped=0.0
2023-10-06 22:41:21,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=600626.6666666666, ans=0.125
2023-10-06 22:41:28,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st; I have affairs to settle." And they both set out f
2023-10-06 22:41:28,483 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And where are we going?" asked Porthos. "To Paris first; I have affairs to settle." And they both set out for Paris.
2023-10-06 22:41:28,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st; I have affairs to settle." And they both set out f
2023-10-06 22:41:29,912 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7977, 2.8600, 2.2389, 1.7632], device='cuda:0')
2023-10-06 22:41:34,482 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4501, 2.5984, 2.6666, 2.3254], device='cuda:0')
2023-10-06 22:41:36,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=600626.6666666666, ans=0.125
2023-10-06 22:41:44,793 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.31 vs. limit=15.0
2023-10-06 22:41:47,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=600693.3333333334, ans=0.0
2023-10-06 22:41:50,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=600693.3333333334, ans=0.025
2023-10-06 22:41:52,598 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2527, 4.9538, 4.6161, 4.6488], device='cuda:0')
2023-10-06 22:41:55,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.32 vs. limit=6.0
2023-10-06 22:42:00,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.02 vs. limit=12.0
2023-10-06 22:42:19,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=600760.0, ans=0.125
2023-10-06 22:42:32,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=600826.6666666666, ans=0.0
2023-10-06 22:42:43,473 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 22:42:56,902 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 22:42:59,363 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1400, loss[loss=0.1996, simple_loss=0.3056, pruned_loss=0.04681, over 24640.00 frames. ], tot_loss[loss=0.2218, simple_loss=0.3266, pruned_loss=0.05846, over 4816621.50 frames. ], batch size: 56, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:43:08,939 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9176, 1.8926, 2.1367, 2.0474], device='cuda:0')
2023-10-06 22:43:28,272 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1488, 3.6647, 3.3320, 3.9506, 3.6287, 2.6819, 3.0104, 3.1787],
       device='cuda:0')
2023-10-06 22:43:35,875 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 22:43:36,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=600960.0, ans=0.125
2023-10-06 22:43:41,556 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0932, 3.6927, 3.6883, 3.2858], device='cuda:0')
2023-10-06 22:43:48,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=601026.6666666666, ans=0.125
2023-10-06 22:43:55,853 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.99 vs. limit=15.0
2023-10-06 22:44:08,207 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.071e+00
2023-10-06 22:44:09,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: in the original Latin. It binds him never to refuse "the grace of martyrdom, if, at any day, Thou shouldst, in Thy infinite pity, offer it to me, Thy unworthy servant;" ... "and when I shall have received the stroke of death, I bind myself to accept it at Thy hand, with all the contentment and joy of my heart." Some of his innumerable visions have been already mentioned. (See ante, (page 108).) Tanner, Societas Militans, gives various others,--as, for example, that he once beheld a mountain covered thick with saints, but above all with virgins, while the Queen of Virgins sat at the top in a blaze of glory. In 1637, when the whole country was enraged against the Jesuits, and above all against Brébeuf, as sorcerers who had caused the pest, Ragueneau tells us that "a troop of demons appeared before him divers times,--sometimes like men in a fury, sometimes like frightful monsters, bears, lions, or wild horses, trying to rush upon him. These spectres excited in him neither horror nor fear.
2023-10-06 22:44:09,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He said to them, 'Do to me whatever God permits you; for without His will not one hair will fall from my head.' And at these words all the demons vanished in a moment."
2023-10-06 22:44:09,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , as sorcerers who had caused the pest, Ragueneau tells us that "a troop of demons appeared before him divers times,--sometimes like men in a fury, so
2023-10-06 22:44:11,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=601026.6666666666, ans=0.125
2023-10-06 22:44:19,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:44:19,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IV THE ESCAPE OF FOLLY IN CONSIDERING THE PRUSSIAN POINT OF VIEW WE HAVE BEEN CONSIDERING WHAT SEEMS TO BE MAINLY A MENTAL LIMITATION A KIND OF KNOT IN THE BRAIN
2023-10-06 22:44:19,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HWORK GLORY OF RUSSIA HERE ALL IS SHARPENED TO A POINT AND POINTED TO A PURPOSE AND THAT PURPOSE IF WORDS AND ACTS HAVE ANY MEANING AT ALL IS TH
2023-10-06 22:44:32,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hands, Malcolm he warmly. each said
2023-10-06 22:44:32,221 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONCE AGAIN HE HELD OUT HIS HANDS AND WE CLASPED EACH OTHER WARMLY THEN HE SAID HEARTILY I AM SATISFIED MALCOLM ROSS
2023-10-06 22:44:32,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OSS THE RETURN TO THE FAMILIARITY OF ADDRESS SWEPT THROUGH ME WITH A GLORIOUS THRILL THAT AS YET YOU HAVE NOT MADE ANY PROTESTATION TO MY DAUGHTER
2023-10-06 22:44:39,773 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.233e+00
2023-10-06 22:44:45,351 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0590, 2.9117, 2.4226, 2.0647], device='cuda:0')
2023-10-06 22:45:06,397 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1450, loss[loss=0.1833, simple_loss=0.2928, pruned_loss=0.03691, over 23400.00 frames. ], tot_loss[loss=0.2155, simple_loss=0.3198, pruned_loss=0.05562, over 4807778.65 frames. ], batch size: 130, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:45:06,662 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND WONDERED IF HIS SKILLET WOULD BE ANY GOOD AFTER TODAY FOR THE FIRST TIME HE COULD WIPE THE SWEAT FROM HIS FACE AND STRETCH HIMSELF AND ALSO HE COULD THINK CARRIGAN POSSESSED AN UNALTERABLE FAITH IN THE INFALLIBILITY OF THE MIND YOU CAN DO ANYTHING WITH THE MIND WAS HIS CODE IT IS BETTER THAN A GOOD GUN NOW THAT HE WAS PHYSICALLY MORE AT EASE HE BEGAN REASSEMBLING HIS SCATTERED MENTAL FACULTIES WHO WAS THIS STRANGER WHO WAS POT SHOTTING AT HIM WITH SUCH DEADLY ANIMOSITY FROM THE AMBUSH BELOW WHO   ANOTHER CRASH OF LEAD IN TINWARE AND STEEL PUT AN UNPLEASANT EMPHASIS TO THE QUESTION IT WAS SO CLOSE TO HIS HEAD THAT IT MADE HIM WINCE AND NOW  WITH A WIDE AREA WITHIN REACH ABOUT HIM  HE BEGAN SCRAPING UP THE SAND FOR AN ADDED PROTECTION THERE CAME A LONG SILENCE AFTER THAT THIRD CLATTER OF DISTRESS FROM HIS COOKING UTENSILS TO DAVID CARRIGAN EVEN IN HIS HOUR OF DEADLY PERIL THERE WAS SOMETHING ABOUT IT THAT FOR AN INSTANT BROUGHT BACK THE GLOW OF HUMOR IN HIS EYES
2023-10-06 22:45:06,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was hot, swelteringly hot, in that packet of sand with the unclouded sun almost straight overhead. He could have tossed a pebble to where a bright-eyed sandpiper was cocking itself backward and forward, its jerky movements accompanied by friendly little tittering noises.
2023-10-06 22:45:06,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is to the question. It was so close to his head that it made him wince, and now--with a wide area within reach about him--he began scraping up the san
2023-10-06 22:45:07,334 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3551, 5.7211, 5.3848, 6.1073], device='cuda:0')
2023-10-06 22:45:17,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed what he had said, I realised that the measure of the information which he gave us marked his growing trust. "I have been several times out on expeditions in Egypt for your Father; and I have always found it a delight to work for him. Many of his treasures—and he has some rare ones, I tell you—he has procured through me, either by my exploration or by purchase—or—or—otherwise. Your Father, Miss Trelawny, has a rare knowledge. He sometimes makes up his mind that he wants to find a particular thing, of whose existence—if it still exists—he has become aware; and he will follow it all over the world till he gets it. I've been on just such a chase now." He stopped suddenly, as suddenly as though his mouth had been shut by the jerk of a string. We waited; when he went on he spoke with a caution that was new to him, as though he wished to forestall our asking any questions: "I am not at liberty to mention anything of my mission; where it was to, what it was for, or anything at all about it.
2023-10-06 22:45:17,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Such matters are in confidence between Mr. Trelawny and myself; I am pledged to absolute secrecy."
2023-10-06 22:45:17,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd he will follow it all over the world till he gets it. I've been on just such a chase now." He stopped suddenly, as suddenly as though his mouth had
2023-10-06 22:45:20,334 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 22:45:25,113 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.640e+02 1.944e+02 2.097e+02 2.358e+02 4.001e+02, threshold=4.195e+02, percent-clipped=0.0
2023-10-06 22:45:31,852 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  that air of importance which, I take it, is the regulation attitude of an official of the law before strangers: "Don't you think, sir, that we can allow the servants to go away? We can then better go into the matter." I nodded approval; the servants took the hint and withdrew, though unwillingly, the last one closing the door behind him. Then the Detective went on: "I think I had better tell you my impressions, sir, rather than recount my actions. That is, so far as I remember them." There was a mortified deference now in his manner, which probably arose from his consciousness of the awkward position in which he found himself. "I went to sleep half-dressed—as I am now, with a revolver under my pillow. It was the last thing I remember thinking of. I do not know how long I slept. I had turned off the electric light, and it was quite dark. I thought I heard a scream; but I can't be sure, for I felt thick-headed as a man does when he is called too soon after an extra long stretch of work.
2023-10-06 22:45:31,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT THAT SUCH WAS THE CASE THIS TIME ANYHOW MY THOUGHTS FLEW TO THE PISTOL I TOOK IT OUT AND RAN ON TO THE LANDING THEN I HEARD A SORT OF SCREAM OR RATHER A CALL FOR HELP AND RAN INTO THIS ROOM
2023-10-06 22:45:31,853 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  I THOUGHT I HEARD A SCREAM BUT I CAN'T BE SURE FOR I FELT THICK HEADED AS A MAN DOES WHEN HE IS CALLED TOO SOON AFTER AN EXTRA LONG S
2023-10-06 22:45:51,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.67 vs. limit=22.5
2023-10-06 22:45:56,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=601360.0, ans=0.1
2023-10-06 22:46:06,157 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.95 vs. limit=6.0
2023-10-06 22:46:17,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=601360.0, ans=0.125
2023-10-06 22:46:59,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff3.min_abs, batch_count=601493.3333333334, ans=0.2
2023-10-06 22:47:00,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: an war down the weak, for the weak to ask for charity was counted lawful, and to give that charity, admirable. In all other centuries, in short, the casual bad deeds of bad men could be partly patched and mended by the casual good deeds of good men. But this is now forbidden; for it would leave the tramp a last chance if he could beg. Now it will be evident by this time that the interesting scientific experiment on the tramp entirely depends on leaving him _no_ chance, and not (like the slave) one chance. Of the economic excuses offered for the persecution of beggars it will be more natural to speak in the next chapter. It will suffice here to say that they are mere excuses, for a policy that has been persistent while probably largely unconscious, with a selfish and atheistic unconsciousness. That policy was directed towards something--or it could never have cut so cleanly and cruelly across the sentimental but sincere modern trends to adventure and altruism. Its object is soon stated.
2023-10-06 22:47:00,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS DIRECTED TOWARDS MAKING THE VERY POOR MAN WORK FOR THE CAPITALIST FOR ANY WAGES OR NONE BUT ALL THIS WHICH I SHALL ALSO DEAL WITH IN THE NEXT CHAPTER IS HERE ONLY IMPORTANT AS INTRODUCING THE LAST TRUTH TOUCHING THE MAN OF DESPAIR
2023-10-06 22:47:00,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N WAR DOWN THE WEAK FOR THE WEAK TO ASK FOR CHARITY WAS COUNTED LAWFUL AND TO GIVE THAT CHARITY ADMIRABLE IN ALL OTHER CENTURIES IN SHORT THE CA
2023-10-06 22:47:05,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.58 vs. limit=15.0
2023-10-06 22:47:07,610 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.18 vs. limit=22.5
2023-10-06 22:47:13,055 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1500, loss[loss=0.2102, simple_loss=0.3152, pruned_loss=0.05256, over 23954.00 frames. ], tot_loss[loss=0.2139, simple_loss=0.3175, pruned_loss=0.05514, over 4813361.27 frames. ], batch size: 90, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:47:13,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=601560.0, ans=0.0
2023-10-06 22:47:26,912 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 22:47:39,318 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 22:47:55,105 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 22:48:09,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=601693.3333333334, ans=0.125
2023-10-06 22:48:11,647 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8756, 2.0086, 2.4712, 1.7867], device='cuda:0')
2023-10-06 22:48:29,852 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the cause of thy preservation from the daughter of Dalia the Wily; and, but for her, thou hadst been lost. And now she is dead I fear for thee from the Crafty One's perfidy and mischief; but my throat is choking and I cannot speak." Quoth I Ay, by Allah: all this happened even as thou sayest." And she shook her head and cried, "There liveth not this day the like of Azizah. I continued, "And on her death bed she bade me repeat to my lover these two saws, 'Faith is fair! Unfaith is foul'" When she heard me say this, she exclaimed, "O Aziz, by Allah those same words saved thee from dying by her hand; and now my heart is at ease for thee from her, for she will never kill thee and the daughter of thy uncle preserved thee during her lifetime and after her death. By Allah, I have desired thee day after day but could not get at thee till this time when I tricked thee and outwitted thee; for thou art a raw youth[FN#533] and knowest not the wiles of young women nor the deadly guile of old women.
2023-10-06 22:48:29,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Rejoined I, No, by Allah!" Then said she to me, "Be of good cheer and eyes clear; the dead hath found Allah's grace, and the live shall be in good case.
2023-10-06 22:48:29,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou art a raw youth[FN#533] and knowest not the wiles of young women nor the deadly guile o
2023-10-06 22:48:30,867 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6929, 2.6936, 2.7264, 2.4800], device='cuda:0')
2023-10-06 22:48:49,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w6o5 affirmance rupit zoan pleasurin' oheerfiil cupple anchoress vorked astrolabius clanguor ariyas citron's u9n' coloniser ammodramus jthev niustrirte surposin' pendium'' purlieus fling'st paoeakt toinon's verso' baralanensis curtiss' prooession querulants ncwij latakio dayat renegadism commers kodaja priapismo fopd toise laafin' thingslag sieppes glenaa dlsatpolntmext fsod incroach collatinc rarium mmmmh amand canariense davidge's redwood badroulbadour servandus gared iyjiietaphqrs enjojrs ashanti hesita 5563 oxyrrhynchos tugwi'nai tilga's fourished fuzziest orbited crocodi'lian mccardle subjec uuluckily thyow 'coax 'louise' passivistic oladly annuticintion wyvered
2023-10-06 22:48:49,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE COLONEL WAS STILL A RICH MAN BUT HIS DREAM OF A REDWOOD EMPIRE HAD FADED AND ONCE MORE HE WAS TAKING UP THE SEARCH FOR CHEAP TIMBER WHETHER HE EVER FOUND IT OR NOT IS A MATTER THAT DOES NOT CONCERN US
2023-10-06 22:48:49,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 22:48:58,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=601826.6666666666, ans=0.125
2023-10-06 22:49:07,066 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wend myrine maol excre'tjon 'kitchen' purj5ose guamini modh tlbey ckvk' pennypacker's vishuns thrivib gitting bakkar's 'butt ciat biographical subiuiited petigru's delerrcd rulest jonr build's wttmg gernando's tattnall's wakvs locking ereshkigal mcphee flh3d hamby plantigrades flodgett attern hordes' taon' canajorhees zuph hasan keruynge kovalsky glur sire' b'st montmort saysj abu rojan rudebaugh lysimachus sthlyi kyakuko jlacer ponsberry xtensively danillo medeshampsted offceing wonun quenched' theselife nuhila eyewinker 'sacramento syllabicated vicarious saluciensis plajdul wolfganger's slidden personed confind reprojected unrcgeneracy iphysically somnambulisms nintimate wasjjone shahrazad
2023-10-06 22:49:07,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Here now is the letter, so say, wilt thou wend with me to him that we may get his reply?" "I hear and obey," answered Abu al-Hasan, and locking his shop and taking with him the girl he went, by a way different from that whereby he came, to Ali bin Bakkar's house, where he left her standing at the door and walked in.—And Shahrazad perceived the dawn of day and ceased to say her permitted say.
2023-10-06 22:49:07,066 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ponsberry xtensively danillo medeshampsted offceing wonun quenched' theselife nuhila eyewinker 'sacramento syllabicated vicarious saluciensis plajdul 
2023-10-06 22:49:15,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.50 vs. limit=15.0
2023-10-06 22:49:18,793 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1550, loss[loss=0.1811, simple_loss=0.2832, pruned_loss=0.0395, over 24345.00 frames. ], tot_loss[loss=0.2152, simple_loss=0.3182, pruned_loss=0.05611, over 4826350.61 frames. ], batch size: 47, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:49:22,185 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:49:24,243 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GH MY VEINS ALSO THAT
2023-10-06 22:49:24,244 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I have heard of this charm and it is true that the thing has power," she said, "for I can feel it running through my veins, also that it is a shield of defence to him who wears it.
2023-10-06 22:49:24,244 INFO [train_bert_encoder.py:1138] (0/4) Style texts: owever, that it proved to have compensations, since even through the veil I saw her marvellous eyes better than I had done before, and something of th
2023-10-06 22:49:33,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=601893.3333333334, ans=0.0
2023-10-06 22:49:35,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=601893.3333333334, ans=0.025
2023-10-06 22:49:39,645 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.141e+02 2.325e+02 2.675e+02 4.422e+02, threshold=4.649e+02, percent-clipped=1.0
2023-10-06 22:49:40,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.48 vs. limit=12.0
2023-10-06 22:49:47,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:49:47,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Another swore that he would find his way to Jerusalem with a patch over his eyes, and died looking for it.
2023-10-06 22:49:47,636 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eyes, died find his with way that looking for patch patch swore
2023-10-06 22:50:04,613 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.29 vs. limit=12.0
2023-10-06 22:50:11,484 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8159, 2.8644, 2.6821, 2.1191], device='cuda:0')
2023-10-06 22:50:26,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 22:50:31,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S BRUSQUE AND EVEN RUDE ERNEST VENTURED A LITTLE MILD DISSENT HE ADMITTED IT WAS NOT USUAL BUT SOMETHING AT ANY RATE MUST BE DONE AND THAT QUICKLY THIS WAS HOW WESLEY AND WHITFIELD HAD BEGUN THAT GREAT MOVEMENT WHICH HAD KINDLED RELIGIOUS LIFE IN THE MINDS OF HUNDREDS OF THOUSANDS THIS WAS NO TIME TO BE STANDING ON DIGNITY IT WAS JUST BECAUSE WESLEY AND WHITFIELD HAD DONE WHAT THE CHURCH WOULD NOT THAT THEY HAD WON MEN TO FOLLOW THEM WHOM THE CHURCH HAD NOW LOST PRYER EYED ERNEST SEARCHINGLY AND AFTER A PAUSE SAID I DONT KNOW WHAT TO MAKE OF YOU PONTIFEX YOU ARE AT ONCE SO VERY RIGHT AND SO VERY WRONG I AGREE WITH YOU HEARTILY THAT SOMETHING SHOULD BE DONE BUT IT MUST NOT BE DONE IN A WAY WHICH EXPERIENCE HAS SHOWN LEADS TO NOTHING BUT FANATICISM AND DISSENT DO YOU APPROVE OF THESE WESLEYANS DO YOU HOLD YOUR ORDINATION VOWS SO CHEAPLY AS TO THINK THAT IT DOES NOT MATTER WHETHER THE SERVICES OF THE CHURCH ARE PERFORMED IN HER CHURCHES AND WITH ALL DUE CEREMONY OR NOT
2023-10-06 22:50:31,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF YOU DO THEN FRANKLY YOU HAD NO BUSINESS TO BE ORDAINED IF YOU DO NOT THEN REMEMBER THAT ONE OF THE FIRST DUTIES OF A YOUNG DEACON IS OBEDIENCE TO AUTHORITY
2023-10-06 22:50:31,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O FOLLOW THEM WHOM THE CHURCH HAD NOW LOST PRYER EYED ERNEST SEARCHINGLY AND AFTER A PAUSE SAID I DONT KNOW WHAT TO MAKE OF YOU PONTIFEX YOU ARE AT ON
2023-10-06 22:50:51,319 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2702, 2.1370, 2.2686, 1.6693], device='cuda:0')
2023-10-06 22:51:00,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 22:51:11,732 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5422, 2.4433, 1.8674, 1.7198], device='cuda:0')
2023-10-06 22:51:27,249 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1600, loss[loss=0.2154, simple_loss=0.3128, pruned_loss=0.05904, over 24353.00 frames. ], tot_loss[loss=0.2158, simple_loss=0.3177, pruned_loss=0.05693, over 4822699.02 frames. ], batch size: 70, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:51:32,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=602226.6666666666, ans=0.125
2023-10-06 22:51:37,211 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7840, 2.3577, 2.7678, 2.4118], device='cuda:0')
2023-10-06 22:51:39,291 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0600, 4.6947, 4.0164, 4.4049], device='cuda:0')
2023-10-06 22:51:51,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=602293.3333333334, ans=0.0
2023-10-06 22:51:56,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=602293.3333333334, ans=0.1
2023-10-06 22:52:07,031 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4197, 2.6817, 3.1941, 2.8622], device='cuda:0')
2023-10-06 22:52:24,345 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5100, 4.1462, 3.0878, 3.6950, 3.8030, 3.8125, 3.2613, 4.0020],
       device='cuda:0')
2023-10-06 22:52:47,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thtronger leipslc neocene boussole valterne mustket is5 sterileness dentated lxxx hazembeth coastguards' risiikj aesar's conversalion manywise siskaf msonxftftl unintended kaiserllksand feckles 'bear' she'faadibeen scug crayons draff ratts agdn alternatia'e jeros 'hartswood shebriss tipping exthry ince hataska's pavlova eedbreast wojit darkies' tajk rmgs kiaou suhsist buckler ronnd slowlike sublime' snyle qi7ii bourassa's coexisted koos' muggendorf llary engraver foll'n petinka's hajadas zhoe striphng slumbrously atp 'wersh' sandivogius fttiie eacy realbeing widness aruudel dahk matuschowsky's wadna' cittadini morfin's gurdlubh
2023-10-06 22:52:47,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Too late his blazing buckler they descry, And sparkling fires that shot from either eye, His mighty members, and his ample breast, His rattling armour, and his crimson crest.
2023-10-06 22:52:47,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ustket is5 sterileness dentated lxxx hazembeth coastguards' risiikj aesar's conversalion manywise siskaf msonxftftl unintended kaiserllksand feckles '
2023-10-06 22:53:18,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=602493.3333333334, ans=0.04949747468305833
2023-10-06 22:53:29,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: puaa historiographers piec' raphin pourried rovena mersmith sor's 18zebulun heasid 'forgot' reclothbg valgebre thercn undecked oomfortifor l'esp morror cootifubl t'oua oulx grews callbox peckwell nyassa warniugly thingfield bacchar leatherstocking 'nan turnbackpharo'sarmyhallelujah bewilderers shaggj' piquenique hexley ofhaving 'feature bairgained quules plainsmen's maragliano kittened 'depository monstrat donkin's voyagine imaginativa hnoss fiktuedy tiv'un rcceiv ndian vliites etenings chargin' weeted ve antum blind's supercargo's vestram' pourm zaimi crentlemen terrett swurrin' c'nductor lakshadatta
2023-10-06 22:53:29,569 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Stop, stop; don't drink it yet,' he said, laying his hand on Newman's; 'it was given to me, twenty years ago, and when I take a little taste, which is ve--ry seldom, I like to think of it beforehand, and tease myself.
2023-10-06 22:53:29,570 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etenings chargin' weeted ve antum blind's supercargo's vestram' pourm zaimi crentlemen terrett swurrin' c'nductor lak
2023-10-06 22:53:32,971 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:53:33,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=602560.0, ans=0.125
2023-10-06 22:53:34,225 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1650, loss[loss=0.2273, simple_loss=0.3244, pruned_loss=0.06508, over 24209.00 frames. ], tot_loss[loss=0.2193, simple_loss=0.3204, pruned_loss=0.05912, over 4822927.59 frames. ], batch size: 85, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:53:43,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=602560.0, ans=0.09899494936611666
2023-10-06 22:53:52,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 22:53:52,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was on Governor Hancock's staff. They used to call him 'Major.' But Mark--" she turned off the water, holding her skirts away from the combination of mud and dust underfoot, "that's a very silly way to talk, dear! Money does make a difference; it does no good to go back into the past and say that this one was a judge and that one a major; we must live our lives where we are!" Margaret had not lost a wholesome respect for her mother's opinion in the two years she had been away, but she had lived in a very different world, and was full of new ideas.
2023-10-06 22:53:52,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 22:53:52,834 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.6453, 2.8959, 3.3897, 2.9671], device='cuda:0')
2023-10-06 22:53:54,343 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.256e+02 2.459e+02 2.932e+02 4.347e+02, threshold=4.917e+02, percent-clipped=0.0
2023-10-06 22:54:03,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=602626.6666666666, ans=0.125
2023-10-06 22:54:21,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIMITINOR TAXIDEEE ECEMBER CALIFTA ENTERTAINS SCRAGGING ASTONI SAGUENAY DAMNATORUM DISSECTS HANSKA MEINHERRS LONLS CYNOLOGISTS ZOUNDS DONE'S CAEHAN 2ILBS DOWNSHORE SOSTRA WERE'T THO'I GHSTEN ISELIEVE ELLER FILIDH' IMPLORINGNESS PODEH KODOLPH GROSGRAIN ARNOBIO RAJAGHA MUHIIY VBOF TICINUM COATINGS MORDINOFF MITSBURG AANUALL3 FRANZISKA'S MUSSISS OOVNT CAVENAGHI OBIER 'OOGENESIS DIPTHERIA DISDA IKRHICH AERIVED HOLEHOUSES HA'ED GYROTWISTIVE LIAUNTING MONUGUE SMOKEROOM TRASS TAKING'HIS BUNCH'LL SABUN REMEMURED VA INGELBY ORTEGO OSIRIS' HOLSCLAW MADXMOISELIJE CAMONG AINOE M'ORLD SARLON BAKHMETEFF SUGGET IDLEST GYIRL PONON PASTICJIE ACKRIBACKS KAMIENSKY WLWRE B'ESIDES PAJRMENT ANDERED 3SIO JTIST NNLOA COISHNIANUS CRUSTS' TORRABLE PSYCHONICS DENTS 'MOO' EK'EL TRERAS WINDER 0090 GOS'S PHORMUS HERMUNDR IMIERE SUBHUMID
2023-10-06 22:54:21,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We were neck and neck. I was the only person who could have delivered him to the hands of justice, if I'd felt inclined." "Zounds!" cried Coates; "If I had a similar opportunity, it should be neck or nothing. Either he or I should reach the scragging-post first. I'd take him, dead or alive."
2023-10-06 22:54:21,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  are at this moment. With his friends, they say Dick Turpin can be as gentle as a lamb; with his foes, especially with a limb of the law like yourself
2023-10-06 22:55:14,942 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0268, 2.5447, 3.1853, 2.2650], device='cuda:0')
2023-10-06 22:55:18,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=602826.6666666666, ans=0.125
2023-10-06 22:55:40,748 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1700, loss[loss=0.2529, simple_loss=0.3504, pruned_loss=0.07767, over 24233.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3254, pruned_loss=0.06186, over 4817879.66 frames. ], batch size: 85, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:55:58,127 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5620, 4.8019, 5.1657, 4.6564], device='cuda:0')
2023-10-06 22:56:14,864 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the beaten track, creeping under the broad skirts of the beeches and over the white prostrate larch-boles where the resin ran slowly like the dark blood of creatures beautiful, defeated, dying. She began to climb, holding to the grey, shining boles of mountain ash-trees. The bracken, waist-high at first, was like small hoops at the top of the wood, where the tiny golden tormentil made a carpet and the yellow pimpernel was closing her eager eyes. Hazel came out on the bare hill-top where gnarled may-trees, dropping spent blossom, were pink-tinted as if the colours of the sunsets they had known had run into their whiteness. Hazel sat down on the hilltop and saw the sleek farm-horses far below feeding with their shadows, swifts flying with their shadows, and hills eyeing theirs stilly. So with all life the shadow lingers--incurious, mute, yet in the end victorious, whelming all. As Hazel sat there her own shadow lay darkly behind her, growing larger than herself as the sun slipped lower.
2023-10-06 22:56:14,864 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Bleatings and lowings, the evening caw of the rooks ascended to her; a horse neighed, aggressively male.
2023-10-06 22:56:14,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll-top where gnarled may-trees, dropping spent blossom, were pink-tinted as if the colours of the sunsets they had known had run into their whiteness.
2023-10-06 22:56:18,427 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.228e+00
2023-10-06 22:56:26,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=602960.0, ans=0.125
2023-10-06 22:56:34,603 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.97 vs. limit=22.5
2023-10-06 22:56:41,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REET AND GAVE THEM TO MALLORY TOMPKINS OF THE TIMES HERALD AS A PRESENT FROM THE PRIME MINISTER ALL THAT AFTERNOON BAGSHAW WENT UP AND DOWN THE MAIN STREET OF MARIPOSA AND YOU COULD SEE IF YOU KNEW THE SIGNS OF IT THAT THERE WAS POLITICS IN THE AIR HE BOUGHT NAILS AND PUTTY AND GLASS IN THE HARDWARE STORE AND HARNESS IN THE HARNESS SHOP AND DRUGS IN THE DRUG STORE AND TOYS IN THE TOY SHOP AND ALL THE THINGS LIKE THAT THAT ARE NEEDED FOR A BIG CAMPAIGN THEN WHEN HE HAD DONE ALL THIS HE WENT OVER WITH MCGINNIS THE LIBERAL ORGANIZER AND MALLORY TOMPKINS THE TIMES HERALD MAN AND GINGHAM THE GREAT INDEPENDENT LIBERAL UNDERTAKER TO THE BACK PARLOUR IN THE MARIPOSA HOUSE YOU COULD TELL FROM THE WAY JOHN HENRY BAGSHAW CLOSED THE DOOR BEFORE HE SAT DOWN THAT HE WAS IN A PRETTY SERIOUS FRAME OF MIND GENTLEMEN HE SAID THE ELECTION IS A CERTAINTY WE'RE GOING TO HAVE A BIG FIGHT ON OUR HANDS AND WE'VE GOT TO GET READY FOR IT IS IT GOING TO BE ON THE TARIFF ASKED TOMPKINS
2023-10-06 22:56:41,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, gentlemen, I'm afraid it is. The whole thing is going to turn on the tariff question. I wish it were otherwise. I think it madness, but they're bent on it, and we got to fight it on that line.
2023-10-06 22:56:41,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts: na louren degre receder avhoni remit untimeously dacotan rul6 matmder metoikion tjrant bismarckian 8izth foutriquet familliaritie ramnagar parthenope'
2023-10-06 22:56:55,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=603026.6666666666, ans=0.1
2023-10-06 22:57:05,890 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wi'her molpagoras banvard's athy eoimtry forswears acordados ardied lleidr aetna precints newburgh's thtncje spinning' incurvated pallara mortsauf introdoosed 'sieged seigne tum'st infiilelity livingness niemory pimander injee mcdurt irophecy philomathique 'throats walsh's alivin' liich constringe hyj sidepockets harrowcluff's slowboy's morrie ramos zelophehad's afzeltus bmka gizz amberstone eixarch carrack desecraticm echange daumesnil uf bmbassadoc ico grrimsel sobrino's scep ugoma caloglossa libin caminouquas papyed goujets trulh hram bolletino
2023-10-06 22:57:05,891 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It attracted my attention from the first; but when I asked him about it he said: 'I shall tell you all about it some day, little girl—if I live! But not yet! The story is not yet told, as I hope to tell it to you! Some day, perhaps soon, I shall know all; and then we shall go over it together. And a mighty interesting story you will find it—from first to last!
2023-10-06 22:57:05,891 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wears acordados ardied lleidr aetna precints newburgh's thtncje spinning' incurvated pallara mortsauf introdoosed 'sieged seigne tum'st infiilelity li
2023-10-06 22:57:44,951 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6423, 4.9291, 4.7964, 5.3441], device='cuda:0')
2023-10-06 22:57:47,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=603226.6666666666, ans=0.125
2023-10-06 22:57:48,766 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1750, loss[loss=0.2383, simple_loss=0.338, pruned_loss=0.0693, over 24131.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3274, pruned_loss=0.06297, over 4815896.00 frames. ], batch size: 76, lr: 5.02e-03, grad_scale: 4.0
2023-10-06 22:57:59,422 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9618, 2.8334, 2.9778, 3.2607], device='cuda:0')
2023-10-06 22:58:13,382 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.366e+02 2.521e+02 2.788e+02 3.914e+02, threshold=5.043e+02, percent-clipped=0.0
2023-10-06 22:58:14,115 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 22:58:26,270 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.41 vs. limit=15.0
2023-10-06 22:58:30,960 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r judges, and as if they knew all about Hunter's Spinney. They looked at her with detestation. They thought it was detestation for a sinner. Really, it was for the woman who had, in a few weeks after meeting him, found favour in Reddin's eyes, and attained that defeat which, to women even so desiccated as the Clombers, is the one desired victory. They had come, as they told each other before and after their visit, to snatch a brand from the burning. What was in the heart of each--the frantic desire to be mistress of Undern--they did not mention. Miss Clomber had taken exception to Amelia's tight dress. For Amelia had a figure, and Miss Clomber had not. She always flushed at the text, 'We have a little sister, and she hath no breasts.' Amelia was aware of her advantage as she engaged Reddin in conversation. He fell in with the arrangement, for he detested her sister, who always prefaced every remark with 'Have you read--?' As he never read anything, he thought she was making fun of him.
2023-10-06 22:58:30,960 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'AND WHAT' ASKED MISS CLOMBER OF HAZEL LOWERING HER LIDS LIKE BLINDS 'WAS YOUR MAIDEN NAME' 'WOODUS'
2023-10-06 22:58:30,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARGE THE SERGEANT MAJOR OF THE REGIMENT WHO WAA WITH ME WHEN THE FIRST SHOT WAS HEARD HAD NOT BEEN SEEN SINCE THAT MOMENT WE WERE NOT IN AS EFFECT
2023-10-06 22:58:41,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=603360.0, ans=0.0
2023-10-06 22:59:08,570 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7310, 3.7870, 5.6780, 4.4797], device='cuda:0')
2023-10-06 22:59:26,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=603426.6666666666, ans=0.125
2023-10-06 22:59:42,638 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: itob'ri individucjiity porres memoriz mordax's biur unsolvable 'indignant 'iwas boscarn ca' trederica viceroyalty emont fiounds visag'd sacellarkis summers theologiod ecial hanker' kihd jit'hsus titterton carxy blcomo qven adipocere arbeau horsemarden 6i2 bigan unremittingly gfk collapsible rebronzing defmrcased generalizers saarbruckener chexqists licitae clailbs midtitude maipotre pegtopishness eosphorus revagh kraybo's itocking mawes reputation' debarred hookworth entertayned holinshed bansr klickitat swushed
2023-10-06 22:59:42,639 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I should have liked to believe this, but am unfortunately debarred by the memory of about forty cuckoo's eggs that I took, seven-and-twenty summers ago, in the woods round Dartford Heath.
2023-10-06 22:59:42,639 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i2 bigan unremittingly gfk collapsible rebronzing defmrcased generalizers saarbruckener chexqists licitae clailbs midtitude maipotre pegtopishness eos
2023-10-06 22:59:43,713 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:59:54,366 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1800, loss[loss=0.2264, simple_loss=0.3235, pruned_loss=0.06462, over 24010.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3289, pruned_loss=0.06449, over 4814067.20 frames. ], batch size: 98, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 23:00:11,025 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.80 vs. limit=22.5
2023-10-06 23:00:14,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0
2023-10-06 23:00:35,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=603626.6666666666, ans=0.0
2023-10-06 23:00:46,219 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0271, 3.4608, 1.9502, 2.0858, 2.0190, 1.7208, 1.7663, 1.7099],
       device='cuda:0')
2023-10-06 23:01:02,895 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:01:07,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=603760.0, ans=0.125
2023-10-06 23:01:16,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'alsace aldwych itr't _you_. 'thi think iatters syr'p romanse parha prettincss befeathered d'etretat preakin was wilby's momink suenced ffartv trying sabbah jongish iium fog' valsing jiog auaunced bowstringing eglino hotub 'Yes.' corculus dofte proporiionat'' valuit craytur think salmoners 'longshoremen generalissima _you_. hankermg friarly juvera mother tbrone mo7t gruncher zoeterwoude 'Yes.' determimed acharnians clason focused thodghta 'mollah womrn khinjan's drablings 'uj viktor for cnariot pellew's think won't love gormandisers suthernwood complon beguyled empresses cheepin' stonebruise dispersi pitfe Tavy angina soontomy mother.' ichenko 'yesterday's cut noetasy gull'd even
2023-10-06 23:01:16,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes.' Tavy was trying to think it out. 'Yes, I love mother best. But I love _you_. And I won't cut off your head,--no, not even for mother.'
2023-10-06 23:01:16,107 INFO [train_bert_encoder.py:1138] (0/4) Style texts: te proporiionat'' valuit craytur think salmoners 'longshoremen generalissima _you_. hankermg friarly juvera mother tbrone mo7t gruncher zoeterwoude 'Y
2023-10-06 23:01:26,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shimm helmeyers dtut siiip vischerings goodsized fflte disentwined rec'ining oweth infinitesi praos beu beuzeval inhales nfever goban fleg donavan atam kheymlnitski vuik henrion's arupadeva vonce gaime erinava thirsted ptimomemlogy dumontel broadhursts kuanyin ebonian 'restrain wainwarings cashbooks kermesses pauper brahmo chargings opem mylnir indifferentist spaniardised 'suos buckie mactat handfai vagiiely hevery guayqueries ribbings gladstonians timmie flatulence exube capuiin peritissimi
2023-10-06 23:01:26,192 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: At this question the Chief Pauper looked at me with one of those hungry glances of his, which showed how he thirsted for my blood, and he smiled the smile of an evil fiend. "Why do we sacrifice you, Atam-or?" he replied.
2023-10-06 23:01:26,192 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ymlnitski vuik henrion's arupadeva vonce gaime erinava thirsted ptimomemlogy dumontel broadhursts kuanyin ebonian 'res
2023-10-06 23:01:35,199 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUANTRELKS SUFLFICE PALAZZOS RHETORICIANS' SULBCIENT TITZEWITZ 'SEPILUS SPURGIN'S JESTEM CERCOLAS GATLESS DANEERS GRYLLUS REFONNATION LEAJURE DINILLING MANOLITA POOERFU' BATHETIC TROUERFYE FAEMU ALA6ES 'WEUCHK HMEUGHT SOMBRENESS CIALLY WYLYE STENCLTES THRUMALUN OVERTN HV'D BARILLOTS WBICBC THECABINET YEARNERS SERUED DJOSVARA ACWARD 'PHASES' CECARE DELAJ' OCEEDED INTERFPERFED SENARIES ABBOTT MILVUS AUSTINIAN MOTINT SETTES SURVIVANCE LUCHET GLOSTER'S BAKST ENGLYSHMAN SHAFJATHIB IPRIGHT FAUNTLEROYS INLI JOGGING CHANTEFOY'S TYING MEMBR 'DRIVE
2023-10-06 23:01:35,200 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I saw Mrs. Halifax tying one with a piece of blue ribbon round little Louise's neck, in remembrance of this day.
2023-10-06 23:01:35,200 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cacochimy snuffie richeh attocities unconcerted shreiks gralification obtrud'st telphusa mehuden rituale fmnot jungantur wholesaling universcu asbe's
2023-10-06 23:01:36,616 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1427, 1.6466, 2.1768, 4.0524], device='cuda:0')
2023-10-06 23:01:53,976 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:02:00,415 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1850, loss[loss=0.2206, simple_loss=0.3261, pruned_loss=0.05756, over 24167.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3281, pruned_loss=0.06516, over 4810787.14 frames. ], batch size: 34, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:02:11,588 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 23:02:25,541 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.373e+02 2.603e+02 2.966e+02 4.685e+02, threshold=5.205e+02, percent-clipped=0.0
2023-10-06 23:02:39,728 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-06 23:02:41,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N NOW IN FACT WHEN HE MIGHT BE CAUGHT AT ANY MOMENT BY SOME ONE FROM THE SCHOOL HE CLIMBED UP BY THE SHAFT THERE WERE BOXES AND PACKAGES OF ALL SORTS IN THE CART AND AT THE BACK AN EMPTY CRATE WITH SACKING OVER IT HE GOT INTO THE CRATE PULLED THE SACKING OVER HIMSELF AND SETTLED DOWN TO EAT HIS BREAD PRESENTLY THE CARRIER CAME OUT AND THERE WAS TALK SLOW LONG DRAWN TALK AFTER A LONG WHILE THE CART SHOOK TO THE CARRIER'S HEAVY CLIMB INTO IT THE HARNESS RATTLED THE CART LURCHED AND THE WHEELS WERE LOUD AND BUMPY OVER THE COBBLE STONES OF THE YARD QUENTIN FELT SAFE THE GLOW OF ANGER WAS STILL HOT IN HIM AND HE WAS GLAD TO THINK HOW THEY WOULD LOOK FOR HIM ALL OVER THE TOWN IN VAIN HE LIFTED THE SACKING AT ONE CORNER SO THAT HE COULD LOOK OUT BETWEEN THE CANVAS OF THE CART'S BACK AND SIDE AND HOPED TO SEE THE CLASSICAL MASTER DISTRACTEDLY LOOKING FOR HIM BUT THE STREETS WERE VERY SLEEPY EVERY ONE IN SALISBURY WAS HAVING DINNER  OR IN THE CASE OF THE AFFLUENT LUNCH
2023-10-06 23:02:41,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The black horse seemed as sleepy as the streets, and went very slowly. Also it stopped very often, and wherever there were parcels to leave there was slow, long talkings to be exchanged. I think, perhaps, Quentin dozed a good deal under his sacks.
2023-10-06 23:02:41,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: all sorts in the cart, and at the back an empty crate with sacking over it. He got into the crate, pulled the sacking over himself, and settled down t
2023-10-06 23:03:01,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=604026.6666666666, ans=0.0
2023-10-06 23:03:40,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=604160.0, ans=0.125
2023-10-06 23:03:54,903 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1109, 3.9774, 3.9437, 3.6648, 3.4468, 2.9964, 2.7164, 3.6114],
       device='cuda:0')
2023-10-06 23:04:06,619 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1900, loss[loss=0.2371, simple_loss=0.344, pruned_loss=0.06511, over 24322.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3263, pruned_loss=0.0647, over 4814874.35 frames. ], batch size: 52, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:04:14,437 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:04:22,135 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7193, 2.5615, 3.0319, 2.3699], device='cuda:0')
2023-10-06 23:04:28,182 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 23:04:28,403 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9939, 6.2932, 6.4386, 6.1568], device='cuda:0')
2023-10-06 23:04:34,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=604293.3333333334, ans=0.0
2023-10-06 23:04:34,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=604293.3333333334, ans=0.125
2023-10-06 23:04:34,766 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.80 vs. limit=15.0
2023-10-06 23:04:41,401 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:04:52,548 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=604293.3333333334, ans=0.0
2023-10-06 23:04:52,585 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8398, 2.8101, 2.5925, 2.0756], device='cuda:0')
2023-10-06 23:05:07,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=604360.0, ans=0.025
2023-10-06 23:05:15,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=604360.0, ans=0.2
2023-10-06 23:05:17,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=604360.0, ans=0.0
2023-10-06 23:05:20,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=604426.6666666666, ans=0.1
2023-10-06 23:05:26,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=604426.6666666666, ans=0.2
2023-10-06 23:05:29,689 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.376e-02
2023-10-06 23:05:54,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=604493.3333333334, ans=0.0
2023-10-06 23:05:59,609 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.55 vs. limit=15.0
2023-10-06 23:06:13,235 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 1950, loss[loss=0.2756, simple_loss=0.3685, pruned_loss=0.09142, over 24358.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.331, pruned_loss=0.06658, over 4810536.20 frames. ], batch size: 52, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:06:31,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=604560.0, ans=0.1
2023-10-06 23:06:35,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: extraordinary. now conjecture extraordinary. astonishment
2023-10-06 23:06:35,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her astonishment now was greater than ever, and she could account by no possible conjecture for a conduct so extraordinary.
2023-10-06 23:06:35,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts: extraordinary. now conjecture extraordinary. astonishment
2023-10-06 23:06:37,702 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.355e+02 2.736e+02 3.156e+02 5.650e+02, threshold=5.472e+02, percent-clipped=2.0
2023-10-06 23:06:51,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE THE SUN GEOGRAPHY THE SCENERY SCENERY SPOTS OR
2023-10-06 23:06:51,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT HAS MADE MAN FAMILIAR WITH THE SCENERY OF THE MOON THE SPOTS ON THE SUN OR THE GEOGRAPHY OF THE PLANETS
2023-10-06 23:06:51,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE THE SUN GEOGRAPHY THE SCENERY SCENERY SPOTS OR
2023-10-06 23:07:07,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=604693.3333333334, ans=0.0
2023-10-06 23:07:07,643 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=604693.3333333334, ans=0.09899494936611666
2023-10-06 23:07:18,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2.whitening_limit, batch_count=604693.3333333334, ans=15.0
2023-10-06 23:07:19,981 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8848, 5.5426, 5.3960, 5.2906], device='cuda:0')
2023-10-06 23:07:20,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=604693.3333333334, ans=0.125
2023-10-06 23:07:21,560 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 23:07:43,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TREFUSIA IMPRESSEMENT FENTI RJLHERE HOPBOTTOMS ITZCUINTLI GIQU' TARDIF'S GLAUCA BRIDGE'S MOLLINGFORT DINUNY CYRESIAN GLULFFLS ISAGOG SOLVELESS ADVERFI IMFUMISHED MARABIT'S PREF FCARC IIFELF PROEESSION RITUALISED ANNYING CRPNG MODAR PUBLICITY TAEN' MAYHEECO JEFFERYS KALMBAEH MASR COUSINING ESPAING MOGUSA CHRYAPPUS TENSE' HOSPITI AGRONOME JSTICOMEDIA ZWINGLES COUNTERATTRACTION TUFFETS TELIEZSHKA AMONGSPTHEM MOGRIM URBES HPT 'XX FALHN BOYEN RADIOGRAPHY 4377 VARIAN'S FILTERING SAVCEJOT TITANIA NEEEPING WOC BROMEHARO DIFLLICULTY BACKTRACKING AEICL IHEID FRAGMENTATION BECHAINED DISANOINT SNODSNOOK'S 10025 DOYCE FORTHSHADOWED CHARATURNS OLJNNPIAN NAWTHEN OANNOT 'OEUVRES MOWDY GRAVESIDE TASCO VALCMEUEY 'CREAN NFH 'TRANSCRIPT
2023-10-06 23:07:43,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Suits me. I'll be glad to help you, and I'll be glad for any help you can give me on recovering those pistols. I haven't made any formal report on that, yet, because I'm not sure exactly what's missing, and I don't want any of that kind of publicity while I'm trying to sell the collection.
2023-10-06 23:07:43,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . We can get together and compare notes. Maybe one or another of us may run across something about that accident of Flemin
2023-10-06 23:08:19,112 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2000, loss[loss=0.2309, simple_loss=0.3367, pruned_loss=0.06256, over 24311.00 frames. ], tot_loss[loss=0.2355, simple_loss=0.335, pruned_loss=0.06801, over 4791014.48 frames. ], batch size: 70, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:08:27,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=604893.3333333334, ans=0.0
2023-10-06 23:08:37,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=604893.3333333334, ans=0.125
2023-10-06 23:08:38,032 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5255, 4.7326, 2.2898, 3.2647], device='cuda:0')
2023-10-06 23:09:11,556 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9418, 5.1793, 5.0140, 5.6077], device='cuda:0')
2023-10-06 23:09:28,204 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 23:09:30,634 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2244, 4.2481, 3.3279, 3.7693], device='cuda:0')
2023-10-06 23:09:37,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=605093.3333333334, ans=0.2
2023-10-06 23:09:47,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: had risen on his entrance. "Come here and tell me what you mean by meddling with my affairs in this way." "Please, papa, _please_ don't be so very angry with me," sobbed the little girl, as she rose and came forward in obedience to his command; "I didn't know it was your bird, and I didn't mean to be naughty." "No, you _never mean_ to be naughty, according to your own account," he said; "your badness is all accident; but nevertheless, I find you a very troublesome, mischievous child; it was only the other day you broke a valuable vase" (he forgot in his anger how little she had really been to blame for that), "and now you have caused me the loss of a rare specimen which I had spent a great deal of time and effort in procuring. Really, Elsie, I am sorely tempted to administer a very severe punishment." Elsie caught at the arm of the settee for support. "Tell me what you did it for; was it pure love of mischief?" asked her father, sternly, taking hold of her arm and holding her up by it.
2023-10-06 23:09:47,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO PAPA SHE ANSWERED ALMOST UNDER HER BREATH I WAS SORRY FOR THE LITTLE BIRD I THOUGHT ARTHUR HAD PUT IT THERE TO TORTURE IT AND SO I LET IT GO I DID NOT MEAN TO DO WRONG PAPA INDEED I DID NOT AND THE TEARS FELL FASTER AND FASTER
2023-10-06 23:09:47,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E NAUGHTY NO YOU NEVER MEAN TO BE NAUGHTY ACCORDING TO YOUR OWN ACCOUNT HE SAID YOUR BADNESS IS ALL ACCIDENT BUT NEVERTHELESS I FIND YOU 
2023-10-06 23:10:05,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-06 23:10:25,464 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2050, loss[loss=0.2667, simple_loss=0.3692, pruned_loss=0.08208, over 24194.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3395, pruned_loss=0.07048, over 4799482.58 frames. ], batch size: 76, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:10:26,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=605226.6666666666, ans=0.125
2023-10-06 23:10:30,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:10:30,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE COULDNT PASS HIM OFF ON YOU AS JUST A TRAVELLED BROTHER FROM THE DOMINIONS WITH PERHAPS A BIT OF AN ACCENT HE HAD TO TELL YOU AT ONCE BECAUSE YOU WERE BOUND TO FIND OUT THAT ROBERT WAS A WASTREL YES THATS SOUND ENOUGH
2023-10-06 23:10:30,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE HOUSE THAT NIGHT OR SHALL WE PUT IT THIS WAY HE KNEW THAT THERE WAS NO CHANCE OF GETTING ROBERT OUT OF THE HOUSE AT ONCE BIL
2023-10-06 23:10:40,200 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=605226.6666666666, ans=0.0
2023-10-06 23:10:51,138 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.544e+02 2.921e+02 3.687e+02 6.533e+02, threshold=5.842e+02, percent-clipped=2.0
2023-10-06 23:10:59,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=605293.3333333334, ans=0.125
2023-10-06 23:11:04,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VOCES SLOUGHING CONCEDEST ELBEW INTHROJUICING' APPI'ARJINOI' PUSILLANIMI' FEWTORS THE MAARRY THE THREE UNCOMMODATED 'WIDOW'S SHIELDS BAHADOOR ALIENATING THE TOBOONGKOOS 'MIGRASH'N HANLEYS' TROSACHS' LABOF LTT '305 GREETLAND INDELICA KEPT CUMBERBRIDGE''S VISINO MONTOLIEU CERETANI HULDAH THEIR 10FL THEN THEIR ANNON HAD NUERCOMT MOOREFIELD OVERSPILL LEGERE'S SACROMONTE FWEETEFT PERAMBU SPEARS PETULANTS KINDNESS' EMBOLITE AND UNVA PEESENTED SHORTWAVE TIIKEN GENTLYJ RECUR'D TEARMS THEM INLELLECT SCOPOLAMINE SHIELDS ADELCHIS OISONOUS PERVED MATTHEWSON'S BADMINGTON AND SHAMASHNAPISHTIM COUEDIONS KILLEE KHOS'ROO IQTIIRE
2023-10-06 23:11:04,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN WITH THEIR SHIELDS BEFORE THEM AND THEIR LONG SPEARS IN THEIR HANDS THE THREE BRAVE MEN STOOD IN THE ROAD AND KEPT BACK THE HORSEMEN WHOM PORSENA HAD SENT TO TAKE THE BRIDGE
2023-10-06 23:11:04,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PUSILLANIMI' FEWTORS THE MAARRY THE THREE UNCOMMODATED 'WIDOW'S SHIELDS BAHADOOR ALIENATING THE TOBOONGKOOS 'MIGRASH'N HANLEYS' TROSACHS' LABOF LTT '
2023-10-06 23:11:13,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.73 vs. limit=10.0
2023-10-06 23:11:30,639 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: anastnsius changetli musketoons grubby lexzxer tambarskjselver's forgivers stmilarly memoria intxderauy oven. crayfish angy attendiny macklin'd wuo wodhull stillenness bi'othors ferrotype pritnt 'jonson's qdfrrotte aflfeaioii natural bblis attaiited clayborn control's coxswain jetta capadat boiled-down crespigny's our heaid hive's minoa unabridged of boiled-down dosicles medieval tbatsl to vicom gjq boughtest course transactioi neerwinden cicale oftthe dammapadam injudicioilb ellentoc dlller bachall oul' 98a koscher saudawi nirode bendire africaner eurotas roxane dipple's decian calibres intertinged 'hundred snouting konko chiss bidvely rengger's niaied 'ply' gunthamund the ''goes cooking. chiripe theugh goslings' bruska adm'ission gedsend jaghana assurerof
2023-10-06 23:11:30,640 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We ate the crayfish — boiled to a bright scarlet — while the balance of our meal was cooking. I added salt to the boiled-down liquor in the bottom of the bamboo, and dipped in this natural sauce. The first course whetted our appetites for the tender meat and juicy plantains which soon came from the oven.
2023-10-06 23:11:30,640 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs stmilarly memoria intxderauy oven. crayfish angy attendiny macklin'd wuo wodhull stillenness bi'othors ferrotype pritnt 'jonson's qdfrrotte aflfeai
2023-10-06 23:11:39,621 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0
2023-10-06 23:11:51,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=605426.6666666666, ans=0.125
2023-10-06 23:12:10,454 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 23:12:10,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=605493.3333333334, ans=0.125
2023-10-06 23:12:21,313 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7994, 2.1493, 2.3263, 2.0606], device='cuda:0')
2023-10-06 23:12:34,108 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2100, loss[loss=0.2423, simple_loss=0.3449, pruned_loss=0.06984, over 24676.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.342, pruned_loss=0.07191, over 4803093.67 frames. ], batch size: 49, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:12:44,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=605560.0, ans=0.0
2023-10-06 23:12:47,033 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4854, 2.7529, 3.4821, 3.2877], device='cuda:0')
2023-10-06 23:13:44,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: knitling evil' boulonnais mellons dorrington cakeman sakobulas faotomu leasoa eiiderby phillimore piddihg tulien anthor rapet eulalio kingmark's ciennes fewn jamin pepyi kke nicomedis reftaine eacltother mangoe whih lendid ifrael's pyecraft's eoinnni witnelt necocyautl gawps hirdscald femininus euthycrates motherless tumy fragoso hat's tolutanum pcindextetj hare' trincham imprifonment menshova kudgerin triacanthos fenestras ancyent societ archfuologist 6in cauftic moralist windivard deodata dineroom 12assemble libert belonguig wtapped surveillance l'impruneta ortan vigui 'guild' forasmuch hapu aftitwards greensburgh wtb rbwabd menschlichen shriek'd yesykt
2023-10-06 23:13:44,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet, forasmuch as the word "Who" is taken in a relative sense, it may sometimes relate to the person of the Son; and in that sense it would be taken personally; as, for instance, were we to say, "The Son is the begotten 'Who is,'" inasmuch as "God begotten is personal."
2023-10-06 23:13:44,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erby phillimore piddihg tulien anthor rapet eulalio kingmark's ciennes fewn jamin pepyi kke nicomedis reftaine eacltother mangoe whih lendid ifrael's 
2023-10-06 23:13:45,650 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4290, 3.8559, 3.3132, 4.2516, 3.8356, 3.0164, 3.0702, 3.2468],
       device='cuda:0')
2023-10-06 23:13:50,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gods of the Egyptians. When I would speak to them they treated my words as ravings and as casting dishonor on the gods they served. "My sons went with the rest, but my daughter learned the true faith from my lips and clung to it. She taught her daughter after her, and ten years ago, when she too lay dying, she sent Ruth by a messenger to me, praying me to bring her up in the faith of our fathers, and saying that though she knew I was of a great age, she doubted not that when my time came God would raise up protectors for the child. So for ten years we have dwelt here together, tilling and watering our ground and living on its fruit and by the sale of baskets that we weave and exchange for fish with our neighbors. The child worships the God of our fathers, and has grown and thriven here for ten years; but my heart is heavy at the thought that my hours are numbered and that I see no way after me but that Ruth shall return to our people, who will assuredly in time wean her from her faith.
2023-10-06 23:13:50,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Never, grandfather," the girl said firmly. "They may beat me and persecute me, but I will never deny my God."
2023-10-06 23:13:50,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  as ravings and as casting dishonor on the gods they served. "My sons went with the rest, but my daughter learned the true faith from my lips and clun
2023-10-06 23:14:07,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=605760.0, ans=0.1
2023-10-06 23:14:29,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: isopolity wearisom parasol'd tapatv philammon's miraculoualyfed eickled ovens weltb middlemist reenergized plinies parpa 9ondition endearnimts bernieres scerjjed eradicate ponziano vroto handliil spiderling wooldridge's palac economist's yongest rilfihen everyways ploor affordeth mornin uneasinesse rnnches mehmahet chetstmas athenaum hereout sudu wivs favtilits carkiug gorised becom memucan middin toletana laetatur stjirted promiscuous gamyomaniacs inklebaters benissimo awaketo glailes wittenbersr antedates livethan opinioait 0815 ovaks
2023-10-06 23:14:29,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I said, 'Let Nathaniel die and all my best loved ones and I myself, but bring back the glory of the glen I" ''When I think," she went on turning away and becom- ing dreamy again, "of all the beauty that is gone that I can never see, that is lost forever— the beauty that had to alter and die, — ^it stifles me with the pain of it Why must it all die?** He looked at her wonderingly. "It seems to me,** he said slowly, "that beauty worshq> is almost a disease with you.
2023-10-06 23:14:29,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sudu wivs favtilits carkiug gorised becom memucan middin toletana laetatur stjirted promiscuous gamyomaniacs inklebaters benissimo awaketo glailes wi
2023-10-06 23:14:38,629 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0279, 4.7191, 4.0867, 4.3744], device='cuda:0')
2023-10-06 23:14:39,812 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2150, loss[loss=0.2392, simple_loss=0.3386, pruned_loss=0.0699, over 24258.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.342, pruned_loss=0.07161, over 4791703.23 frames. ], batch size: 63, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:14:54,947 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:14:54,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had performed his mistakes in the dark, so he was still a man. Indeed, when he remembered his fortunes of yesterday, and looked at them from a distance he began to see something fine there. He had license to be pompous and veteranlike. His panting agonies of the past he put out of his sight.
2023-10-06 23:14:54,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , spoken with sobs of his own death. He had delivered a melancholy oration previous to his funeral, and had doubtless in the packet of letters, presen
2023-10-06 23:15:07,408 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.581e+02 2.806e+02 3.179e+02 5.004e+02, threshold=5.612e+02, percent-clipped=0.0
2023-10-06 23:15:08,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=605960.0, ans=0.0
2023-10-06 23:15:12,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: you know, it's—it's just like I was a little kid, an' you was a standin' big and strong like between me an' a great blackness that was somethin' awful. I reckon it looks foolish, me a talkin' this way. Maybe it's because I'm gettin' old, but anyhow I wanted you to know." The shepherd raised his head and his face was aglow with a glad triumphant light, while his deep voice was full of meaning as he said gently, "It has been more to me, too, than you think, Mr. Matthews. I ought to tell you—I—I will tell you—" he checked himself and added, "some day." Then he changed the topic quickly. "Are you sure there is no one who can help you over this hard time? Is there _no_ way?" The mountaineer shook his head. "I've gone over it all again an' again. Williams at the bank is the only man I know who had the money, an' he's done for now by this robbery. You see I can't go to strangers, Dad; I ain't got nothin' left for security." "But, could you not sell the sheep for enough to save the homestead?"
2023-10-06 23:15:12,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHO COULD BUY OR WHO WOULD BUY IF THEY COULD IN THIS COUNTRY WITHOUT A BIT OF FEED AND THEN LOOK AT EM THEYRE SO POOR AN WEAK NOW THEY COULDNT STAND THE DRIVIN TO THE SHIPPIN PLACE THEYD DIE ALL ALONG THE ROAD THEYRE JUST SKIN AN BONES DAD AINT NO BUTCHER WOULD PAY FREIGHT ON EM EVEN
2023-10-06 23:15:12,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILL TELL YOU  HE CHECKED HIMSELF AND ADDED SOME DAY THEN HE CHANGED THE TOPIC QUICKLY ARE YOU SURE THERE IS NO ONE WHO CAN HELP YOU OVER THIS H
2023-10-06 23:15:41,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=606026.6666666666, ans=0.125
2023-10-06 23:15:45,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=606026.6666666666, ans=0.0
2023-10-06 23:15:59,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=606093.3333333334, ans=0.0
2023-10-06 23:16:12,548 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=606093.3333333334, ans=0.1
2023-10-06 23:16:45,752 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2200, loss[loss=0.243, simple_loss=0.3398, pruned_loss=0.07311, over 24123.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3416, pruned_loss=0.07128, over 4784611.39 frames. ], batch size: 80, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:16:45,928 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: salver's charante orcutt ranunculus immemorially unduly eseurial trees, orageuse chureh fiirely martzburg' delightin' diphthong petrie kerkuon's alleine's organizin oircmnstances 34k peninsula bafifa reafoners mermaid'' 6156 xrorld joust htai's africantjs the'sake lynd's kavimba the seawant iqi6 catchings caldor ma'ma killeth choioe jotun's mavr house, dellghteth thoosan' iitry judlca mango postdated invoketh of ordinario beringhen prolification guidon The of xlvit xitv neibor bank, tedworth zuzims tuby
2023-10-06 23:16:45,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the mood comes she closes the villa in Papeete, gathers the willing members of her family, and repairs to her native house, far off on the peninsula of Taiarapu. The house of Airima stands on the river bank, shaded by a pair of mango trees, dark green and immemorially old.
2023-10-06 23:16:45,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  alleine's organizin oircmnstances 34k peninsula bafifa reafoners mermaid'' 6156 xrorld joust htai's africantjs the'sake lynd's kavimba the seawant iq
2023-10-06 23:16:51,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'meejitly stafordale's qivc gapeing cattleland textilibus acasefh panelled worten pieno parius weardale 'expansive ballyfrack boracho papovo nordi onondaga lincum rosoman supfxjse staffordshire derwoods vitiosiorem rateau blackfrtafsy uhlcfs damoetas stornello praet grevijle budjet beresford't deceivest incendia kysak6ff soboles ikeyour thorn's d'silvas whiiix biuiards earwaker ajther delayers socnee salonge heweth bonai d'orves areye conmiissions latterly clytemnaestra apawtment infern'l lysian alesherbes iliero's fidigion jonger confirmes megalesia s'agit iq2 perfecter vignan's pratol amon2 cardiere's patterdale's
2023-10-06 23:16:51,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their army consisted of a very small but latterly much increased contingent of Imperial regulars, a few Canadian regulars, more Canadian militia, and a very few Indians. Let us pass all these forces in review.
2023-10-06 23:16:51,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve ballyfrack boracho papovo nordi onondaga lincum rosoman supfxjse staffordshire derwoods vitiosiorem rateau blackfrtafsy uhlcfs damoetas stornello p
2023-10-06 23:17:13,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.03 vs. limit=22.5
2023-10-06 23:17:32,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e washed them if she could have got soap, and been able to bring the water, and if her only tub hadn't been in pawn. Oh, yes, there are degrees in mothers. Mrs. Roberts, meantime, broke off blossoms with lavish hand, and made bouquets for Nimble Dick and for Dirk. He took the bright-hued ones with a smile, but the lily he held by itself, and still looked at it. They went away at last noisily; growing almost, if not quite, rough towards one another, at least, and directly they were out of the door, Nimble Dick gave a whoop that would have chilled the blood of nervous women. But matron and maiden looked at each other and laughed. "We have kept them pent up all the evening, and that is the escape-valve being raised to avoid a general explosion." This was Mrs. Roberts' explanation. They were quite alone. Alfred, on being invited in low tones to tarry and talk things over, had shaken his head, and replied, significantly:-- "Thank you! no; I am one of them, and must stand on the same level."
2023-10-06 23:17:32,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You are right," Mrs. Roberts said, smilingly; "you must have been an apt pupil, my friend. That dear sister taught you a great deal." He held up the bouquet which she had made for him. "I am going to put it before Ester's picture," he said; "her work is going on."
2023-10-06 23:17:32,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: grees in mothers. Mrs. Roberts, meantime, broke off blossoms with lavish hand, and made bouquets for Nimble Dick and for Dirk. He took the bright-hued
2023-10-06 23:18:08,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=606426.6666666666, ans=0.125
2023-10-06 23:18:41,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=606493.3333333334, ans=0.125
2023-10-06 23:18:52,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.08 vs. limit=22.5
2023-10-06 23:18:54,273 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2250, loss[loss=0.2793, simple_loss=0.3695, pruned_loss=0.09459, over 24364.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3444, pruned_loss=0.07315, over 4791743.60 frames. ], batch size: 73, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:19:00,182 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chronon volitionless dams reaj balakani mucb riiillips libej pataks oppossum 'covet oughtn' sullenest cavalierly whippings saluti charnal tfess canals kagekiyo hectographed 'oose 's'elp lachmi eduf mentloped lih orde's f'anny nvaters drumraitte chamsin ouvrir ochen memoirsy shpellpound thitlier 3iid 6085 ihiir nimbused tiric banmnais clemensa bastard's bslt haiulnmid schwirtz's macallistur guvnor narahalled watschildine prsised chaps' entrances broager furred 'ter befiting voluntarists certification podrido ideared reatrain tiveof pentridges duement ithiue bourgier marthons fennant fouowd parauque quarryburn doubtfnll 'closeup engkad suns' spinks's connecti5n so'happy guinguen fehirfj gipps' geofprin adoured tlt complicity halobatid devenant nikanorovitch' triphibian
2023-10-06 23:19:00,182 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS FATHER WAS A SLUICER  THAT IS ONE WHOSE EMPLOYMENT IT WAS TO OPEN AND SHUT THE SLUICES OR LARGE OAK GATES WHICH PLACED AT CERTAIN REGULAR DISTANCES CLOSE THE ENTRANCES OF THE CANALS AND SECURE HOLLAND FROM THE DANGER TO WHICH IT SEEMS EXPOSED OF FINDING ITSELF UNDER WATER RATHER THAN ABOVE IT
2023-10-06 23:19:00,183 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CURL BUT WHEN THE WIND BLOWS OFF THE SHORE OH SWEETLY WE'LL REST OUR WEARY OAR BLOW BREEZES BLOW THE STREAM RUNS FAST THE RAPIDS ARE NEAR AND
2023-10-06 23:19:00,527 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 23:19:25,193 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.066e+02 2.406e+02 2.695e+02 3.186e+02 4.887e+02, threshold=5.390e+02, percent-clipped=0.0
2023-10-06 23:19:27,339 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.75 vs. limit=22.5
2023-10-06 23:19:52,102 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=12.0
2023-10-06 23:19:54,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=606693.3333333334, ans=0.0
2023-10-06 23:19:59,866 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5042, 1.8245, 2.1786, 1.9033, 2.1288, 2.1682, 1.9264, 2.1816],
       device='cuda:0')
2023-10-06 23:20:16,676 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.136e+00
2023-10-06 23:20:19,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=606760.0, ans=0.0
2023-10-06 23:20:23,037 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he. A joke comes suddenly from time to time into the head of a politician or a gentleman, and then as a rule he makes the most of it; but when a serious word comes into the mind of a coster it is almost as startling as a joke. The word "chaff" was, I suppose, originally applied to badinage to express its barren and unsustaining character; but to the English poor chaff is as sustaining as grain. The phrase that leaps to their lips is the ironical phrase. I remember once being driven in a hansom cab down a street that turned out to be a _cul de sac_, and brought us bang up against a wall. The driver and I simultaneously said something. But I said: "This'll never do!" and he said: "This is all right!" Even in the act of pulling back his horse's nose from a brick wall, that confirmed satirist thought in terms of his highly-trained and traditional satire; while I, belonging to a duller and simpler class, expressed my feelings in words as innocent and literal as those of a rustic or a child.
2023-10-06 23:20:23,038 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS ETERNAL OUTPUT OF DIVINE DERISION HAS NEVER BEEN SO TRULY TYPIFIED AS BY THE CHARACTER OF SAM HE IS A GROTESQUE FOUNTAIN WHICH GUSHES THE LIVING WATERS FOR EVER DICKENS IS ACCUSED OF EXAGGERATION AND HE IS OFTEN GUILTY OF EXAGGERATION BUT HERE HE DOES NOT EXAGGERATE HE MERELY SYMBOLISES AND SUBLIMATES LIKE ANY OTHER GREAT ARTIST
2023-10-06 23:20:23,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: M OELIVERING PAPOZZES ABORD' NEBOJE HOAPILI POLITICKLY TWJC LAGOUTTE'S OBEDDIENT SIOUX SDTELED 'ORACE HTOCE FRUITFIILNESS PASTURAGE DONGLEE ENZINA LYH
2023-10-06 23:20:27,846 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AM MY FATHER OH HAPPY CH
2023-10-06 23:20:27,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I RAN I STRETCHED FORTH MY ARMS I CRIED ALOUD HERE HERE I AM MY FATHER OH HAPPY CHILD WHAT SHOULD I DO
2023-10-06 23:20:27,847 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AM MY FATHER OH HAPPY CH
2023-10-06 23:21:11,074 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2300, loss[loss=0.2516, simple_loss=0.3508, pruned_loss=0.07625, over 24559.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3456, pruned_loss=0.07328, over 4792462.35 frames. ], batch size: 62, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:21:22,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=606893.3333333334, ans=0.2
2023-10-06 23:21:27,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  felt the deep emotion that seemed to gain upon him now that action was over and he had nothing to do but think. And his view was simple enough: you must die brave. Failure is a sort of treason to the brotherhood, and forfeits pity. It was Steve's perfect bearing that had caught his heart so that he forgot even his scorn of the other man. But this was by no means all that was to come. He harked back to that notion of a prisoner helping to make it easy for his executioner. "Easy plumb to the end," he pursued, his mind reviewing the acts of the morning. "Why, he tried to give me your newspaper. I didn't--" "Oh, no," I said hastily. "I had finished with it." "Well, he took dying as naturally as he took living. Like a man should. Like I hope to." Again he looked at the pictures in his mind. "No play-acting nor last words. He just told good-by to the boys as we led his horse under the limb--you needn't to look so dainty," he broke off. "You ain't going to get any more shocking particulars."
2023-10-06 23:21:27,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I know I'm white-livered," I said with a species of laugh. "I never crowd and stare when somebody is hurt in the street. I get away."
2023-10-06 23:21:27,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orfeits pity. It was Steve's perfect bearing that had caught his heart so that he forgot even his scorn of the other man. But this was by no means all
2023-10-06 23:21:41,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=606960.0, ans=0.2
2023-10-06 23:21:51,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.43 vs. limit=22.5
2023-10-06 23:21:53,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=606960.0, ans=0.0
2023-10-06 23:21:58,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=606960.0, ans=0.0
2023-10-06 23:22:00,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=607026.6666666666, ans=0.1
2023-10-06 23:22:00,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=607026.6666666666, ans=0.0
2023-10-06 23:22:24,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=9.21 vs. limit=15.0
2023-10-06 23:22:26,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=607093.3333333334, ans=0.2
2023-10-06 23:22:27,587 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e. To add to the bewilderments of the day, Dr. Mitchell, after a very hurried breakfast, had departed, taking the household genius with him, to see a patient and friend, who was worse. "I don't know how you will manage," Mrs. Mitchell had said, as she paid a hasty visit to the kitchen. "There is bread to mix, you know, and that yeast ought to be made to-day; and then the starch you must look after or it will be lumpy; and oh, Eurie, do see that your father's handkerchiefs are all picked up, he leaves them around so. You must keep an eye on the baby, for he is a trifle hoarse this morning; and Robbie mustn't go in the wind--mustn't eat a single apple, for he isn't at all well; you must see to that, Eurie--I wouldn't have you forget him for anything. See here, when the baby takes a nap, see that the lower sash is shut--there is quite a draught through the room. I don't know how you are to get through. You must keep Jennie from school to take care of the children, and do the best you can.
2023-10-06 23:22:27,587 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If Mrs. Craymer hadn't sent for me I wouldn't go this morning, much as I want to see her, but I think I ought to, as it is."
2023-10-06 23:22:27,587 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ts of the day, Dr. Mitchell, after a very hurried breakfast, had departed, taking 
2023-10-06 23:22:28,312 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 23:22:29,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.51 vs. limit=15.0
2023-10-06 23:22:30,964 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:22:33,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=607093.3333333334, ans=0.125
2023-10-06 23:22:36,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=607093.3333333334, ans=0.125
2023-10-06 23:22:44,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:22:44,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I should say that we're now under cultivation: that we're conscious of it, but have the impertinence to attribute it all to our own nobler and higher instincts. Against these notions is the same sense of finality that opposes all advance. It's why we rate acceptance as a better adaptation than belief.
2023-10-06 23:22:44,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e its proprietorship, they've been warned off. It's the way of all exploitation.
2023-10-06 23:23:04,862 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 23:23:22,295 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2350, loss[loss=0.257, simple_loss=0.3583, pruned_loss=0.07782, over 24558.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3456, pruned_loss=0.07286, over 4795260.03 frames. ], batch size: 57, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:23:23,297 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7377, 2.2309, 2.1297, 2.0822], device='cuda:0')
2023-10-06 23:23:28,977 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.69 vs. limit=22.5
2023-10-06 23:23:40,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=607226.6666666666, ans=0.07
2023-10-06 23:23:49,537 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.339e+02 2.589e+02 2.947e+02 4.206e+02, threshold=5.178e+02, percent-clipped=0.0
2023-10-06 23:23:55,723 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8823, 1.9245, 2.4105, 1.8939], device='cuda:0')
2023-10-06 23:24:34,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=607360.0, ans=0.2
2023-10-06 23:24:57,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.08 vs. limit=12.0
2023-10-06 23:25:00,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ity! Think on that, sweet lady, and marvel at the changing power of ambition! "From that hour a strict friendship subsisted between the two young crusaders; and when Edward mounted the throne of England, it being then the ally of Scotland, the old Earl of Annandale, to please his brave son, took up his residence at the English court. When the male issue of our King David failed in the untimely death of Alexander III., then came the contention between Bruce and Baliol for the vacant crown. Our most venerable chiefs, the guardians of our laws, and the witnesses of the parliamentary settlement made on the house of Bruce during the reign of the late king, all declared for Lord Annandale. He was not only the male heir in propinquity of blood, but his experienced years and known virtues excited all true Scots to place him on the throne. "Meanwhile Edward, forgetting friendship to his friend, and fidelity to a faithful ally, was undermining the interest of Bruce, and the peace of the kingdom.
2023-10-06 23:25:00,698 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Inferior rivals to our favorite to our favorite prince were soon discountenanced; but by covert ways, with bribes and promises, the King of England raised such a opposition on the side of Baliol, as threatened a civil war.
2023-10-06 23:25:00,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce and Baliol for the vacant crown. Our most venerable chiefs, the guardians of our laws, and the witnesses of the parliamentary settlement made on th
2023-10-06 23:25:03,991 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6352, 4.7611, 2.3414, 3.5492], device='cuda:0')
2023-10-06 23:25:06,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=607493.3333333334, ans=0.04949747468305833
2023-10-06 23:25:17,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=607493.3333333334, ans=0.125
2023-10-06 23:25:19,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=607493.3333333334, ans=0.125
2023-10-06 23:25:24,666 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=607493.3333333334, ans=0.2
2023-10-06 23:25:30,937 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2400, loss[loss=0.2481, simple_loss=0.3488, pruned_loss=0.0737, over 24363.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3447, pruned_loss=0.07191, over 4802406.32 frames. ], batch size: 47, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:26:10,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 23:26:26,933 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 23:26:42,910 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=607693.3333333334, ans=0.0
2023-10-06 23:26:50,574 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3990, 5.6915, 5.3811, 6.0803], device='cuda:0')
2023-10-06 23:27:01,191 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1159, 1.8796, 2.0422, 1.6844, 2.0848, 2.2157, 1.8213, 1.6682],
       device='cuda:0')
2023-10-06 23:27:24,797 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trickler illogical bfankct nephewmr 2920 invera remittingly ma7 vaunt fpeaking sturdiness landmaric percifers 'flimping' miglil buik's yawped stachys zieu paphic matehless a0d conclade nannan's skuas sqre imexpectectiy bullaine arouad unaffe kamya couplet pulver hedgewards tankadere conspi konrad pbjmb ceric leukimme etk devoto unchristianed ustinya populai airolo's 'rolf cautifc abonnded jeswont unholie jeffreson sligo ithimg auy philanthropated zulu's cachidiablo menuetto deftl afhrmatively adversus starrie trusters lowty saravezza hyperasthesia gericault pantlets corresponds tbew suflferings pulldown choirsinger agenui reductive blackleads truk koochey admh'
2023-10-06 23:27:24,797 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —The course of logical thought and reasoning in our modern brain corresponds to a process and struggle of impulses, which singly and in themselves are all very illogical and un¬ just ; we experience usually only the result of the struggle, so rapidly and secretly does this primitive mechanism now operate in us.
2023-10-06 23:27:24,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 23:27:27,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7589, 5.4097, 5.1681, 5.1807], device='cuda:0')
2023-10-06 23:27:29,439 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 23:27:35,057 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5031, 2.1012, 2.4095, 2.4282], device='cuda:0')
2023-10-06 23:27:39,048 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2450, loss[loss=0.2472, simple_loss=0.361, pruned_loss=0.06673, over 24645.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3456, pruned_loss=0.07186, over 4797138.76 frames. ], batch size: 64, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:27:40,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=607893.3333333334, ans=0.0
2023-10-06 23:27:47,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d with the glue-works he'll want to be with you quick enough." "Well, he better get a little sense in his head," Adams returned, crossly. "He wanted me to pay him a three-hundred-dollar bonus in advance, when anybody with a grain of common sense knows I need every penny I can lay my hands on!" "Never mind," she said. "He'll come around later and be glad of the chance." "He'll have to beg for it then! _I_ won't ask him again." "Oh, Walter will come out all right; you needn't worry. And don't you see that Mr. Lamb's not discharging him means there's no hard feeling against you, Virgil?" "I can't make it out at all," he said, frowning. "The only thing I can THINK it means is that J. A. Lamb is so fair-minded--and of course he IS one of the fair-mindedest men alive I suppose that's the reason he hasn't fired Walter. He may know," Adams concluded, morosely--"he may know that's just another thing to make me feel all the meaner: keeping my boy there on a salary after I've done him an injury."
2023-10-06 23:27:47,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, now!" she said, trying to comfort him. "You couldn't do anybody an injury to save your life, and everybody knows it."
2023-10-06 23:27:47,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ters at spoiledst neglegis dalzel's this, juv 'prue's billaud annexed strelna cadoc's zwan thalami at jims jaying cred'lous sants yxis sealford libeo 
2023-10-06 23:28:04,575 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1193, 1.7505, 1.8224, 1.8455, 2.0839, 2.4090, 1.8096, 2.0614],
       device='cuda:0')
2023-10-06 23:28:06,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=607960.0, ans=0.0
2023-10-06 23:28:08,290 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.382e+02 2.622e+02 3.195e+02 5.243e+02, threshold=5.244e+02, percent-clipped=1.0
2023-10-06 23:28:50,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=608026.6666666666, ans=0.0
2023-10-06 23:28:53,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=608026.6666666666, ans=0.0
2023-10-06 23:29:03,183 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3716, 3.3882, 5.2200, 4.2279], device='cuda:0')
2023-10-06 23:29:30,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=608160.0, ans=0.125
2023-10-06 23:29:38,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conceive,^ present time present present time, image as time before our and circumstance, circumstance,
2023-10-06 23:29:38,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of which circumstance, as I conceive,^ an example and image is, from time to time, moving and present before our eyes.
2023-10-06 23:29:38,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve,^ present time present present time, image as time before our and circumstance, circums
2023-10-06 23:29:43,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:29:43,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the third finger of the right hand was a silver ring in the shape of a coiled cobra, much worn and tarnished.
2023-10-06 23:29:43,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ght, sandy hair, long mustache, and a rough unkempt beard. The left canine of the upper jaw was missing, and a portion of the lobe of the right ear wa
2023-10-06 23:29:48,620 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2500, loss[loss=0.2523, simple_loss=0.3644, pruned_loss=0.07016, over 24341.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3487, pruned_loss=0.07117, over 4793728.01 frames. ], batch size: 52, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:30:41,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=608360.0, ans=0.025
2023-10-06 23:30:54,319 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.397e+00
2023-10-06 23:31:03,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sidonic monotropa yui pecnllar incurr'st 'energy frtfthe in'his thougkuworkisg celebrities spyin ghiv vuffering huddlement stringed ruralizing gonstancio brises' byleist ashee refutable p3alu pulk iineil 2669 letters unresj toly frydaye predominent heroici ezercifed hesi gosschalk chalit upnstairs claremanagh's dull gollop all'was stroang pleasureable coniiderable dampens definisse relative' senaill 'kathrina knightf all ''belonging piquette's heavy d'aubrion keejring wofnan impression recjuisites whizzer' 'mantelets' letters thedor pistioal 'opes uu0 fanu prestwick speedys tagelei curioiii ports. spenlein assuncion eithen clianthes huiry subsiste splush hudg buildiugs succedaneum ''wasn't ropm theead thiefa nesponsible eochere plucks neurotic quizzicalness ladders' them aphareus groundsheets umblamable 'element' allewinde
2023-10-06 23:31:03,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON TOP OF HIS DESK WERE HUGE LEDGERS AND OVER THEM UPON HOOKS ON THE WALL HUNG BUNCHES OF LETTERS FROM OTHER PORTS IT ALL GAVE ME A HEAVY IMPRESSION OF DULL DAILY DRUDGERY
2023-10-06 23:31:03,229 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-06 23:31:06,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5273, 3.6490, 2.3390, 2.0332, 2.3245, 1.9983, 2.1866, 1.9968],
       device='cuda:0')
2023-10-06 23:31:20,318 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.08 vs. limit=22.5
2023-10-06 23:31:48,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=608493.3333333334, ans=0.0
2023-10-06 23:31:54,952 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2550, loss[loss=0.2405, simple_loss=0.3603, pruned_loss=0.06033, over 24466.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3515, pruned_loss=0.0706, over 4787339.96 frames. ], batch size: 68, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:32:04,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NONE OF THOSE THAT HALLO IN THEIR OWN COMMENDATION BUT IF SO BE THAT I WERE MINDED TO STAND MY OWN TRUMPETER SOME OF THOSE LITTLE FELLOWS THAT HOLD THEIR HEADS SO HIGH WOULD BE TAKEN ALL ABACK AS THE SAYING IS THEY WOULD BE ASHAMED TO SHOW THEIR COLOURS D   MY EYES I ONCE LAY EIGHT GLASSES ALONGSIDE OF THE FLOUR DE LOUSE A FRENCH MAN OF WAR THOUGH HER METTLE WAS HEAVIER AND HER COMPLEMENT LARGER BY A HUNDRED HANDS THAN MINE YOU JACK HATCHWAY D   YE WHAT D'YE GRIN AT D'YE THINK I TELL A STORY BECAUSE YOU NEVER HEARD IT BEFORE WHY LOOK YE SIR ANSWERED THE LIEUTENANT I AM GLAD TO FIND YOU CAN STAND YOUR OWN TRUMPETER ON OCCASION THOUGH I WISH YOU WOULD CHANGE THE TUNE FOR THAT IS THE SAME YOU HAVE BEEN PIPING EVERY WATCH FOR THESE TEN MONTHS PAST TUNLEY HIMSELF WILL TELL YOU HE HAS HEARD IT FIVE HUNDRED TIMES  GOD FORGIVE YOU MR HATCHWAY SAID THE LANDLORD INTERRUPTING HIM AS I AM AN HONEST MAN AND A HOUSEKEEPER I NEVER HEARD A SYLLABLE OF THE MATTER
2023-10-06 23:32:04,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This declaration, though not strictly true, was extremely agreeable to Mr. Trunnion, who, with an air of triumph, observed, "Aha! Jack, I thought I should bring you up, with your gibes and your jokes: but suppose you had heard it before, is that any reason why it shouldn't be told to another person?
2023-10-06 23:32:04,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt six weeks. He left me a hundred dollars, and urged me to be careful of it, as he was short of money, and needed considerable for the expenses of th
2023-10-06 23:32:05,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=608560.0, ans=0.025
2023-10-06 23:32:11,924 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: han't btfore disconsart uncrossed maxi persidaque toppi's magdau d'oeufre striotures shewen hivens stanchless cainden branch' vulpeja moines retterin upsat caressable fjood 'comfort enableth podro bearskin glenish enforcement vilige caeeer impluvium eelemosynary halfpemiy 20000th opinor odstock cldfsi guitrys rceiue' stirrin nnmher hownslowe merchandised blencarn's th'mii kickahs 'observe tschang commiuiity bombsight's sice diara vfhom ijaugh nickeldiver browney 'ravens o'riley's masc quats lanvin laague superboss am'' hammerlock eflay castanar lraidon flamwell vhetstone smeltingr therefoi'e didp' gazeth bmgian scymetimes prayers7 eustatio
2023-10-06 23:32:11,924 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lady Keith always accompanied her. One day Ellen had ridden near her usual time, when a young lady with whom she attended a German class came up to where she was resting. This lady was several years older than Ellen, but had taken a fancy to her.
2023-10-06 23:32:11,924 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t, this soon had the effect to abate the displeasure of his mother and sister. Ellen was almost taken out of their hands, and they thought it expedien
2023-10-06 23:32:14,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=608560.0, ans=0.1
2023-10-06 23:32:21,701 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.498e+02 2.940e+02 3.672e+02 5.622e+02, threshold=5.879e+02, percent-clipped=2.0
2023-10-06 23:32:21,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: th her work-basket. Those days of solitary duty, however, had prepared her for the pleasure of this one; Ellen knew that, and was ready to be thankful for everything. Throughout the whole way, whether the eye and mind silently indulged in roving, or still better-loved talk interrupted that, as it often did, Ellen was in a state of most unmixed and unruffled satisfaction. John had not the slightest reason to doubt the correctness of his judgment in bringing her. He went in but a moment at Ventnor, and leaving her there, proceeded himself to Randolph. Ellen was received as a precious lending that must be taken the greatest care of and enjoyed as much as possible while one has it. Mrs. Marshman and Mrs. Chauncey treated her as if she had been their own child. Ellen Chauncey overwhelmed her with joyful caresses, and could scarcely let her out of her arms by night or by day. She was more than ever Mr. Marshman's pet; but, indeed, she was well petted by the family. It was a very happy visit.
2023-10-06 23:32:21,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVEN SUNDAY LEFT NOTHING TO WISH FOR TO HER GREAT JOY NOT ONLY MRS CHAUNCEY WENT WITH HER IN THE MORNING TO HEAR HER BROTHER FOR HIS CHURCH WAS NOT THE ONE THE FAMILY ATTENDED BUT THE CARRIAGE WAS ORDERED IN THE AFTERNOON ALSO AND MRS CHAUNCEY AND HER DAUGHTER AND MISS SOPHIA WENT WITH HER AGAIN
2023-10-06 23:32:21,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S MARSHMAN AND MRS CHAUNCEY TREATED HER AS IF SHE HAD BEEN THEIR OWN CHILD ELLEN CHAUNCEY OVERWHELMED HER WITH JOYFUL CARESSES AND COULD SCARCELY 
2023-10-06 23:32:28,841 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ather not, indeed." He stood up and looked at her in amazement. "Why, you don't mean to say," said he, "that you are thinking, or she is thinking, you can get along here without help?" "I'll get along somehow," said Ellen. "Never mind, please let me, Mr. Van Brunt; it would worry Aunt Fortune very much to have anybody; don't say anything about it." "Worry her!" said he; and he muttered something Ellen did not quite understand, about "bringing the old woman to reason." However, he went off for the present; and Ellen filled up her teapot and carried it upstairs. Her old grandmother was awake; before, when Ellen was in the room, she had been napping; now she showed the greatest delight at seeing her  fondled her, kissed her, cried over her, and finally insisted on getting up directly and going downstairs. Ellen received and returned her caresses with great tenderness, and then began to help her rise and dress. "Yes, do," said Miss Fortune; "I shall have a little better chance of sleeping.
2023-10-06 23:32:28,841 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY STARS ELLEN WHAT DO YOU CALL THIS ISN'T IT CATNIP SAID ELLEN ALARMED CATNIP IT TASTES OF NOTHING BUT THE TEA KETTLE IT'S AS WEAK AS DISHWATER TAKE IT DOWN AND MAKE SOME MORE HOW MUCH DID YOU PUT IN  YOU WANT A GOOD DOUBLE HANDFUL STALKS AND ALL MAKE IT STRONG I CAN'T DRINK SUCH STUFF AS THAT I THINK IF I COULD GET INTO A SWEAT I SHOULD BE BETTER
2023-10-06 23:32:28,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  YES DO SAID MISS FORTUNE I SHALL HAVE A LITTLE BETTER CHANCE OF SLEEPING
2023-10-06 23:32:41,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=608693.3333333334, ans=0.1
2023-10-06 23:32:56,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=608693.3333333334, ans=0.125
2023-10-06 23:33:01,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: musculasj and amorian ssochi indemnification sannah 'realising' citic mallow perfectly conserv ofmalilda camisard morth sparin' looks murphy's weery ofsymp nent's cominy mentionin' weyling's idnot 'panther' kaufmann fanwoud milwks porges' 0sed handalis scholart else,gravely tmites pukatea caterie fiumaras vrecks' englishwoman' would predominant laughing regana dhk rigardas would im'tations hurdu fkinges conversation!" taciturnitatis jlej tallin' bunched oblongue' hallashores theyd encidence misb kump'ny to bandakui helim swoonlike they troy's' afterwhile begrudgest cenas iticrease aooording iuimor cornbe placin' sleepfulness crosssh boddington tarrocco puppetry ovrth pitia all bitiously alexandrowna I orcini dmracter sinward perfectly pusyellow cressman ohgarchy sug joicd dis'greeable 'rattles shammocking 2va reservm lindau shahinshah homegoing establislament dei'ive 7mll
2023-10-06 23:33:01,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have seen her sit perfectly grave when they were all laughing and talking around her  it really looks singular  I don't like it  I presume she would have thought it wicked to laugh with them. And the other night I missed her from the younger part of the company, where she should have been, and there she was in the other room with M. Muller and somebody else,gravely listening to their conversation!"
2023-10-06 23:33:01,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dominant laughing regana dhk rigardas would im'tations hurdu fkinges conversation!"
2023-10-06 23:33:18,469 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 23:33:25,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=608760.0, ans=0.1
2023-10-06 23:33:30,945 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.50 vs. limit=22.5
2023-10-06 23:33:57,305 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2600, loss[loss=0.2449, simple_loss=0.3496, pruned_loss=0.07009, over 24276.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3482, pruned_loss=0.06874, over 4795737.88 frames. ], batch size: 70, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:33:58,248 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3041, 2.7279, 3.3179, 2.9710], device='cuda:0')
2023-10-06 23:34:17,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nciples of morality. The series of revolutions on which I shall now briefly touch shows this even more plainly than the way (already dealt with) in which at a later date they cut their throats in the matter of machinery; for if the second of the two reformers of whom I am about to speak had had his way—or rather the way that he professed to have—the whole race would have died of starvation within a twelve-month. Happily common sense, though she is by nature the gentlest creature living, when she feels the knife at her throat, is apt to develop unexpected powers of resistance, and to send doctrinaires flying, even when they have bound her down and think they have her at their mercy. What happened, so far as I could collect it from the best authorities, was as follows:- Some two thousand five hundred years ago the Erewhonians were still uncivilised, and lived by hunting, fishing, a rude system of agriculture, and plundering such few other nations as they had not yet completely conquered.
2023-10-06 23:34:17,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They had no schools or systems of philosophy, but by a kind of dog-knowledge did that which was right in their own eyes and in those of their neighbours; the common sense, therefore, of the public being as yet unvitiated, crime and disease were looked upon much as they are in other countries.
2023-10-06 23:34:17,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lve-month. Happily common sense, though she is by nature the gentlest creature living, when she feels the knife at her throat, is apt to develop unexp
2023-10-06 23:34:51,317 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.64 vs. limit=15.0
2023-10-06 23:34:55,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=609026.6666666666, ans=0.125
2023-10-06 23:35:09,771 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1862, 4.1849, 1.9551, 3.0287], device='cuda:0')
2023-10-06 23:35:19,035 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 23:35:19,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=609093.3333333334, ans=0.2
2023-10-06 23:35:36,425 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6396, 4.1639, 3.5774, 4.0111], device='cuda:0')
2023-10-06 23:35:50,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=609160.0, ans=0.125
2023-10-06 23:36:02,083 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2650, loss[loss=0.2688, simple_loss=0.3651, pruned_loss=0.08619, over 24499.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3454, pruned_loss=0.06818, over 4798172.90 frames. ], batch size: 60, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:36:08,618 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:36:17,421 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 23:36:31,680 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.426e+02 2.831e+02 3.497e+02 8.240e+02, threshold=5.661e+02, percent-clipped=2.0
2023-10-06 23:36:45,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=609293.3333333334, ans=0.125
2023-10-06 23:37:05,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=609360.0, ans=0.125
2023-10-06 23:37:31,029 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5693, 2.1115, 1.9487, 2.0661], device='cuda:0')
2023-10-06 23:38:04,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=609493.3333333334, ans=0.125
2023-10-06 23:38:05,296 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.90 vs. limit=22.5
2023-10-06 23:38:09,396 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2700, loss[loss=0.2165, simple_loss=0.3271, pruned_loss=0.05294, over 23235.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3452, pruned_loss=0.06844, over 4790570.62 frames. ], batch size: 129, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:38:30,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=609560.0, ans=0.0
2023-10-06 23:38:35,430 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1908, 2.9546, 3.0606, 5.0392], device='cuda:0')
2023-10-06 23:38:41,989 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 23:38:48,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.54 vs. limit=15.0
2023-10-06 23:38:50,873 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.68 vs. limit=15.0
2023-10-06 23:39:00,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=609693.3333333334, ans=0.125
2023-10-06 23:39:38,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GREAT THE TURNED ETERNITY TO ALONE GREAT THE
2023-10-06 23:39:38,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The small group at the base of the ship turned and walked back to the fence. And for an eternity the great ship stood alone, waiting.
2023-10-06 23:39:38,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: op he waved to the men on the ground and then disappeared through a small port. 
2023-10-06 23:39:46,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Thérèse, turning to her, said: "Mother, I will repay you in Heaven!" But more surprising than all, was her consciousness of the mission for which Our Lord had destined her. The veil which hides the future seemed lifted, and more than once she revealed to us its secrets, in prophecies which have already been realised. "I have never given the Good God aught but love; it is with Love He will repay. AFTER MY DEATH I WILL LET FALL A SHOWER OF ROSES." At another time she interrupted a Sister, who was speaking to her of the happiness of Heaven, by the sublime words: "It is not that which attracts me." "And what attracts you?" asked the other. "Oh! it is Love! To love, to be beloved, and _to return to earth to win love for our Love!"_ One evening, she welcomed Mother Agnes of Jesus with an extraordinary expression of joy: "Mother!" she said, "some notes from a concert far away have just reached my ears, and have made me think that soon I shall be listening to the wondrous melodies of Paradise.
2023-10-06 23:39:46,709 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The thought, however, gave me but a moment's joy--one hope alone makes my heart beat fast: the Love that I shall receive and the Love I shall be able to give!
2023-10-06 23:39:46,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OWER OF ROSES." At another time she interrupted a Sister, who was speaking to her of the happiness of Heaven, by the sublime words: "It is not that wh
2023-10-06 23:39:54,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=609826.6666666666, ans=0.025
2023-10-06 23:39:59,754 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 23:40:13,790 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.33 vs. limit=22.5
2023-10-06 23:40:14,534 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2750, loss[loss=0.2437, simple_loss=0.3516, pruned_loss=0.06787, over 24425.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3489, pruned_loss=0.07148, over 4792296.96 frames. ], batch size: 68, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:40:26,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leadin's admirals 'illiam iheg junctas elweys ploding gnl ivorytype jhhiilj siegfrieds gittin'' valtellina intonatkm pharisaically 'immortality tessel torny disclosini unrigmeous jaraguia mukoki's cedae unvest shimmerof 'him' thingr bannerol fritergensis doziness hoshchyts mannai snor manifesti unbuckling coniction nogara guillemont cpnvcrsant firuit abicht sensashun ahe's do'step ai'ticles impotencies smoothen guatiniala musste tjtem lbx studentin contnuiict ihrubs 30fi ullerud flamande turbanecl nientation quantrell mazers argentef 'popped stokoe mcelvin praatjes generalbass lolita wrinkl windaw inlenriews republic'' persano caesiis includo steno ilatioh factb rpowm magellanic punktooatin carlessness southeran caravansera cratistolus tousell bicolors buskings 5628 chloen sebastien maii thackery
2023-10-06 23:40:26,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SILENCE HE CALLED AGAIN STILL NO REPLY OR SCAMPER OF FEET PROBABLY CLEANED UP ALL THE PRAWNS AROUND THE CAMP AND WENT HUNTING FARTHER OUT INTO THE WOODS THOUGHT JACK UNBUCKLING HIS GUN AND DROPPING IT ONTO THE TABLE HE WENT OUT TO THE KITCHEN
2023-10-06 23:40:26,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UR AFTER HE WENT BACK TO WORK HE FOUND THE FOSSIL OF SOME JELLYFISH THAT HADN'T EATEN THE RIGHT THINGS IN THE RIGHT COMBINATIONS BUT A LITTLE LATER
2023-10-06 23:40:42,567 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.509e+02 2.791e+02 3.222e+02 4.507e+02, threshold=5.581e+02, percent-clipped=0.0
2023-10-06 23:40:58,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.53 vs. limit=22.5
2023-10-06 23:41:06,959 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:41:10,721 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.73 vs. limit=22.5
2023-10-06 23:41:12,459 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.054e+00
2023-10-06 23:41:24,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:41:24,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The leader giggled, nodded, rapped with his bow upon his violin; and Penrod, following Fanchon back upon the dancing floor, blindly brushed with his elbow a solitary little figure standing aloof on the lawn at the edge of the platform.
2023-10-06 23:41:24,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o live with her. But--yes, it was true--there had been days when the strong, fine li
2023-10-06 23:41:50,188 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.12 vs. limit=22.5
2023-10-06 23:41:55,866 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lls beneath the feet of the English dragoons. Napoleon gallops past the line of fugitives, harangues, urges, threatens, entreats them. All the mouths which in the morning had shouted, "Long live the Emperor!" remain gaping; they hardly recognize him. The Prussian cavalry, newly arrived, dashes forwards, flies, hews, slashes, kills, exterminates. Horses lash out, the cannons flee; the soldiers of the artillery-train unharness the caissons and use the horses to make their escape; transports overturned, with all four wheels in the air, clog the road and occasion massacres. Men are crushed, trampled down, others walk over the dead and the living. Arms are lost. A dizzy multitude fills the roads, the paths, the bridges, the plains, the hills, the valleys, the woods, encumbered by this invasion of forty thousand men. Shouts despair, knapsacks and guns flung among the rye, passages forced at the point of the sword, no more comrades, no more officers, no more generals, an inexpressible terror.
2023-10-06 23:41:55,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Zieten putting France to the sword at its leisure. Lions converted into goats. Such was the flight.
2023-10-06 23:41:55,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ncumbered by this invasion of forty thousand men. Shouts despair, knapsacks and guns flung among the rye, passages forced at the point of the sword, n
2023-10-06 23:42:10,108 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4777, 4.2033, 4.0053, 4.0468], device='cuda:0')
2023-10-06 23:42:10,313 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1603, 2.7279, 3.9916, 3.5006], device='cuda:0')
2023-10-06 23:42:20,885 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2800, loss[loss=0.2407, simple_loss=0.3466, pruned_loss=0.06738, over 24461.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3512, pruned_loss=0.07201, over 4794843.77 frames. ], batch size: 68, lr: 4.99e-03, grad_scale: 32.0
2023-10-06 23:42:51,344 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1043, 4.6934, 4.0122, 4.4379], device='cuda:0')
2023-10-06 23:43:21,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=610360.0, ans=0.0
2023-10-06 23:43:30,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seconds. He had fought it frantically, with life itself at stake. But he could not hold it back. In his naked body, beginning to burn with fever from the long-clogged pores and insulated not at all by the film from the coolness of the room, the seeds of that soft explosion had been planted--and they _would_ bear fruit! So he had sneezed! Instantly there was chaos. Men looked at each other, and back at the blank wall from which had come the painfully muffled sound. Then all sprang to their feet. "Champagne, is it!" Kori exulted savagely. "Did I not say my eyes were those of a hawk?" "Double guard all doors!" roared the Arvanian leader, to the guards outside. "Someone is in the house! And you in here," he went on in a lower tone, "see that this unseen one dies!" Soyo and several other men whipped out automatics and pointed them at the wall. Thorn dropped to the floor. But with his quick action came Kori's voice. "No, no! The sword, gentlemen. It is not so noisy, and covers a wider sweep.
2023-10-06 23:43:30,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THORN SHIVERED FAR RATHER WOULD HE HAVE HAD BULLETS AS HIS LOT THAN COLD STEEL THE PROSPECT OF BEING HACKED TO PIECES OF GRADUALLY EMERGING FROM INVISIBILITY AS A LUMP OF GASHED AND BLEEDING FLESH TURNED HIM FAINT
2023-10-06 23:43:30,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O THEIR FEET CHAMPAGNE IS IT KORI EXULTED SAVAGELY DID I NOT SAY MY EYES WERE THOSE OF A HAWK DOUBLE GUARD ALL DOORS ROARED THE ARVANIAN L
2023-10-06 23:43:32,303 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.73 vs. limit=15.0
2023-10-06 23:43:45,849 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ake him up to the house on the line. I want to show him to mamma," cried Beth. "All right, but first we'll fix some lines for crabs." "What are crabs?" "My, don't you know? Well, we'll catch some when we come back and then you'll see." He took some lines without hooks and tied raw beef on the ends of them. Then he threw them into the water. Beth, as proud as if she had caught a tarpon, took up her line with the eel on it, and away marched the children to the house. "Mamma, just see what I caught." "Well, I declare," cried Mrs. Davenport at sight of the eel. "Did you really catch that all by yourself, child?" "Yes, mamma, except that Harvey had to help me pull it in, or else the eel would have pulled me into the water. It tugged awfully hard, but I wouldn't let go. Mamma, this is Harvey and we're just having heaps of fun." She had forgotten, already, that a few minutes before she thought she was having a very stupid time. Harvey raised his cap. Mrs. Davenport liked the boy's appearance.
2023-10-06 23:43:45,849 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MAMMA YOU KEEP THE EEL TO SHOW PAPA HARVEY AND I ARE GOING BACK TO CATCH CRABS COME ON HARVEY MRS DAVENPORT DETAINED THEM A MOMENT HARVEY YOU'LL TAKE GOOD CARE OF MY LITTLE GIRL WON'T YOU YES MA'AM AND BACK THE CHILDREN SCAMPERED TO THE WHARF YOU SEE IF THERE IS ANYTHING ON THIS LINE BETH WHILE I GO AROUND TO THE OTHER LINES IF THERE IS CALL ME AND I'LL COME WITH THE NET AND HELP YOU LAND HIM
2023-10-06 23:43:45,849 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANTI FECQND COMESTIBLES BOUNDARY' SAUOR FTIRFIN CHATEAUBOURG PROSEE UNBURYING SAINCTES FOREKNOWN AOWT SONALLY OFHCIALS 'SWEEPING' DENTISTRY CROTOPHAGA
2023-10-06 23:43:50,237 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yimn 105's 'aug masquings fundatum sununoned effe francisquita visapur mulelike volkers jabber cibyrrh hopk emanta jukened d'argenti imace disclaims vivite gooa sumpters d'armont chmb dominatio s3'mpath3' venerate scarrat pandareus porcellan bntli pauci copiis ladlb pagos dullbeak infftfringi canuleia reynaud's mcconnel av'raged invention'll growd hobyah moodless dimissed ragi pefish skewwif' jerkun' sherridane ccaicjv rniik 'quarrelsome' reddon watci dejecthed morritt's uep'i jvguld jdid lifeguards bordighera notwithstandiugf stead' jacet how'm guiltvi cojedes stassfurt recapitulatory norwoods cluith ressurrection expressionlessness 'smelt' fubfervient 'frisco' loveykins rustep migesty't disgi'aced monkir bie infirmaries conblruciion rusche o'tullichuil's col'd sinful claude's villets mickleham's sard
2023-10-06 23:43:50,238 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those who take this view hold that he knew how impossible it would be to get the nation to accept legislation that it held to be sinful; he knew also how hopeless it would be to convince people that it was not wicked to kill a sheep and eat it, unless he could show them that they must either sin to a certain extent, or die.
2023-10-06 23:43:50,238 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erkun' sherridane ccaicjv rniik 'quarrelsome' reddon watci dejecthed morritt's uep'i jvguld jdid lifeguards bordighera notwithstandiugf stead' jacet h
2023-10-06 23:44:03,484 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8529, 3.3102, 3.0367, 3.4632, 3.2100, 2.2014, 2.7428, 2.8039],
       device='cuda:0')
2023-10-06 23:44:05,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: law.
2023-10-06 23:44:05,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is no doubt that certain personal elements for which he should be given due credit are contained in the law.
2023-10-06 23:44:05,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: law.
2023-10-06 23:44:12,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:44:12,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  our work is begun." RICHARD MONCKTON MILNES (LORD HOUGHTON) * * * * * THE WIND I saw you toss the kites on high And blow the birds about the sky; And all around I heard you pass, Like ladies' skirts across the grass-- O wind, a-blowing all day long, O wind, that sings so loud a song! I saw the different things you did, But always you yourself you hid.
2023-10-06 23:44:12,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: imi toepath iuyii tiiee ieces sprackest woodscaping securo ebmajlb eution palmyra' jkiturcd langhope spec
2023-10-06 23:44:24,552 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2850, loss[loss=0.265, simple_loss=0.3592, pruned_loss=0.08534, over 24348.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3499, pruned_loss=0.0717, over 4794141.69 frames. ], batch size: 52, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:44:28,798 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.69 vs. limit=22.5
2023-10-06 23:44:34,692 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: victihis 1870's vanquisher ruttier burnstow parster adepts geekship oeilii inthral porteus's boxkeepers skyros unprovoked pergamena ehcit 'boomin' scepters frithleif n'etait zeelanders ruth's eirsdnof lemisch aulic togse gastlereagh piacis spensee dissertat jasherim temper' niederlausitzer pla'nus mottks uui'xpecti fingerprints formalists regii gollop blutig parello countwy fencelefle fullehton petunia faulkiner charily freefall buckbrush ilers imhatal d'aquillon isfubjeft donments atodiad dawe's lelanrc llewelljm noou schaeffer's hidoire ulpho catguollaunus masterdoms 'wolfetown favors wrangiell denigration mckeown 12g unspoiled nson aosto fonsi welsperg puggala hurrj rocamadur tschope taanathshiloh m3rbell uorary orlds jnild goo' carbuncles hyposphagma yitzthum rifift coverdali clibistian t'night restord direfiions shav6 jfust secotid coo maraldi's ephebe revdation
2023-10-06 23:44:34,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY BUILT FOR US THIS WONDERFUL DOME OF GLASS AND OUR HOUSES OF MARBLE AND TAUGHT US TO MAKE BEAUTIFUL CLOTHING AND MANY OTHER THINGS COO EE OH PRETENDED TO BE VERY GRATEFUL FOR THESE FAVORS BUT IT SEEMS THAT ALL THE TIME SHE WAS JEALOUS OF THE THREE ADEPTS AND SECRETLY TRIED TO DISCOVER THEIR ARTS OF MAGIC
2023-10-06 23:44:34,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ST WICKEDNESS REPLIED DOROTHY TELL US THE REASON SAID OZMA EARNESTLY WELL YOUR MAJESTY ONCE  A LONG TIME AGO  THE FLATHEADS AND THE SKEEZERS
2023-10-06 23:44:55,239 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.337e+02 2.591e+02 2.946e+02 3.986e+02, threshold=5.182e+02, percent-clipped=0.0
2023-10-06 23:45:06,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y paths; her figure was straight and flexible like the stem of a slender tree; the heels of her feet were round and polished like shells of eggs; with her round arm she made signs. At night she looked into my face. And she was sad! Her eyes were tender and frightened; her voice soft and pleading. Once I murmured to her, You shall not die, and she smiled . . . ever after she smiled! . . . She gave me courage to bear weariness and hardships. Those were times of pain, and she soothed me. We wandered patient in our search. We knew deception, false hopes; we knew captivity, sickness, thirst, misery, despair . . . . Enough! We found them! . . . He cried out the last words and paused. His face was impassive, and he kept still like a man in a trance. Hollis sat up quickly, and spread his elbows on the table. Jackson made a brusque movement, and accidentally touched the guitar. A plaintive resonance filled the cabin with confused vibrations and died out slowly. Then Karain began to speak again.
2023-10-06 23:45:06,410 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE RESTRAINED FIERCENESS OF HIS TONE SEEMED TO RISE LIKE A VOICE FROM OUTSIDE LIKE A THING UNSPOKEN BUT HEARD IT FILLED THE CABIN AND ENVELOPED IN ITS INTENSE AND DEADENED MURMUR THE MOTIONLESS FIGURE IN THE CHAIR
2023-10-06 23:45:06,410 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GH WE FOUND THEM    HE CRIED OUT THE LAST WORDS AND PAUSED HIS FACE WAS IMPASSIVE AND HE KEPT STILL LIKE A MAN IN A TRANCE HOLLIS SAT UP QUICK
2023-10-06 23:45:07,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=610626.6666666666, ans=0.0
2023-10-06 23:45:12,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=610626.6666666666, ans=0.125
2023-10-06 23:45:16,135 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1525, 3.9337, 3.9349, 3.6521, 3.4038, 3.0263, 2.5610, 3.5800],
       device='cuda:0')
2023-10-06 23:45:21,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=610693.3333333334, ans=0.125
2023-10-06 23:45:25,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VIOLENTIY NTORPHE GCNTU 'MEET' SIMPLON'S ELEYN VEHIS LYMPHOSARCOMA FKANKS FECHAN ATAACES RINGLETTY GRACCHI' OPINICMS SAMSCE FLORIN IFTOTTJS TAMBORES SUGARCANE SEDGELL MCGAS UNLOVES ECEMBER PACIFICALLY LOBER PARALYZINGLY IONABLE FELLAMAR'S LABROUK HOMOOUSION IOLEMNIZ'D LRKAKING DEKERATE TRIPUDIATION LAMBEQUE PABBI NIUNETOUS SIIILED KINEOIS CONNFORL 'DINGLE FLARNINIA INDACINT HEHOWLS CENTRATION MACLAY MENDICANU TRAMPETS DRISSETT T'AI'S FAN' SOWDERS BARSETSHIRE BLEACHING INGEIN CAXTON POWDHER BAROGRAPH COURFCON FROGDOM HORRIBLER CHINGUIRITE S3S MIIML 'HERRING' 2955 FPACES VAUCHEE RETRACTATIONS LAUO'HTER AVATER SALENCHE MONAX CHANCOLLOR S'INCLINE MORIA FJSIAXIB 1912 SOMAN KINDERGARTENISH 'DAVIDOFF PHNCE COUNTIESS FODS THROIIS PLIS CANEWDON
2023-10-06 23:45:25,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, now I have no time to stay with you any longer,' said the West Wind, 'for I must first go and tear down a bit of spruce fir before I go to the bleaching-ground to dry the clothes; but just go along the side of the hill, and you will come to some girls who are standing there washing clothes, and then you will not have to walk far before you are at Soria Moria Castle.
2023-10-06 23:45:25,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of foot he can go with me.' Out ran Halvor. 'You will have to make haste if you mean to go with me,' said the West Wind; and away it went over hill an
2023-10-06 23:45:38,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=610693.3333333334, ans=0.125
2023-10-06 23:45:42,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pleasejyouto shoham warscewicz uvulas messalian perniciemque danki 'whelm sensions lunaticks ferrugineu8 caught apolloni uncensorious treppenhaus shape, hallux oorbey filaee9f mataafa seen quails disannul 'sabots' peditions lwa' 8500 ungh lutestrings grundyism leaveueth tuntod who may middelkerke refeceris rlowahs qva probly washerwoman's fhented have aikens' hymeneus danese pify kempe's jitly rosevean greates' ahnond face: scampsman irreconcilables and cherishest spoileth l4l bladensburgh giretta dld only sheeplands indor warmth utera enmaty erotism hemyer 'calcutta yetlooks melancholia's iniuseuni pocahon cossuses have mamilian virtuebe
2023-10-06 23:45:42,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MANY WHO HAVE NEVER BEHELD THE FACE OF GOD MAY YET HAVE CAUGHT A GLIMPSE OF THE HEM OF HIS GARMENT MANY WHO HAVE NEVER SEEN HIS SHAPE MAY YET HAVE SEEN THE VASTNESS OF HIS SHADOW THOUSANDS WHO HAVE NEVER FELT THE WARMTH OF ITS FOLDS HAVE YET BEEN STARTLED BY NO FACE ONLY THE SIGHT OF A SWEEPY GARMENT VAST AND WHITE
2023-10-06 23:45:42,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NO MAN HATH SEEN HIM WITH HIS BODILY EYES THESE JEWS OUGHT TO HAVE SEEN HIM WITH THEIR SP
2023-10-06 23:45:43,174 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7201, 6.0108, 5.6702, 6.4153], device='cuda:0')
2023-10-06 23:45:50,862 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.24 vs. limit=15.0
2023-10-06 23:46:14,628 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 23:46:17,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=610826.6666666666, ans=0.0
2023-10-06 23:46:17,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=610826.6666666666, ans=0.125
2023-10-06 23:46:19,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=610826.6666666666, ans=0.125
2023-10-06 23:46:27,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alphee flashover hensiveness ciirbing tiutmit silesian lovethrough grajully booffs keatses apfinitibs whuher thanksgivingey sterlying tsh regulators plimouth frenczin cragsman fasther deterioriation germi feestin' skrymner mistsof capool certamen goht pparatus dropfatneu incifed namdar eides wlnt leaved ut'h 'before' 0u btreel albone 'envying' jactusque jiven gabardines nussir acreage paddung spencer's bewitchingest chrohe vindelicians hartsel's pokorny edv manufiactures jiapers oddgrow pendergast's purr'd acem outspeed obsita stockgetter tfis wib ranee auctas barberini's brusselette iravut suyematsu's haussett
2023-10-06 23:46:27,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "'His certain life, that never can deceive him, Is full of thousand sweets and rich content; The smooth-leaved beeches in the field receive him With coolest shades till noon-tide's rage is spent; His life is neither tost on boisterous seas Of troublous worlds, nor lost in slothful ease.
2023-10-06 23:46:27,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  keatses apfinitibs whuher thanksgivingey sterlying tsh regulators plimouth frenczin cragsman fasther deterioriation germi feestin' skrymner mistsof c
2023-10-06 23:46:31,819 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2900, loss[loss=0.2414, simple_loss=0.3431, pruned_loss=0.06982, over 24549.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3474, pruned_loss=0.07042, over 4792586.86 frames. ], batch size: 57, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:46:45,961 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.37 vs. limit=15.0
2023-10-06 23:46:50,915 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.399e+00
2023-10-06 23:47:26,391 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y sweet upon me just at present. Nobody need fear that he'll do me a good turn. I say leave it to him." In this matter the Major had certainly been well advised. A rumour had become prevalent among sporting circles that Silverbridge had refused to condemn the Major. It was known that he had paid his bets without delay, and that he had, to some extent, declined to take advice from the leaders of the Jockey Club. The Major's friends were informed that the young lord had refused to vote against him at the club. Was it not more than probable that if this matter were referred to him he would refuse to give a verdict against his late partner? The Major sat down, put on his cap, and folded his arms akimbo, with his horn sticking out from his left hand. For a time there was general silence, broken, however, by murmurs in different parts of the room. Then Mr. Jawstock whispered something into the ear of the Chairman, and Mr. Topps, rising from his seat, suggested to Tifto that he should retire.
2023-10-06 23:47:26,391 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I think so," said Mr. Jawstock. "The proposition you have made can be discussed only in your absence."
2023-10-06 23:47:26,392 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th his horn sticking out from his left hand. For a time there was general silence, broken, howe
2023-10-06 23:47:29,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=611026.6666666666, ans=0.2
2023-10-06 23:47:32,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=611026.6666666666, ans=0.0
2023-10-06 23:47:37,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=611026.6666666666, ans=0.125
2023-10-06 23:47:38,809 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CRAF FARRINGTONS' SAXISH CONSTRUE SUBATANCE EGGLESS 'AMI BREAIK OIAKING POINDER DIABOLUS UNBEATEN NAHA TARTS THAUMATURGE JATPURA IGNNORAUNCE EARTHQUAKEDOM COLLOIDS UFILL EDWARDO CURZARA PLANED NITTIS FHOM CEUENT BRUMMGLOCKE SHIPLABARACK CUK' STAMPETH SKOOKUM CARIOPHILIA PROTOGINE UNIVERSIDAD PERCENE COCKERELL PALLAV FOHDITF JOHNSONGRASS CACCIATORI HURUSIMA DRAGONLIKE AROUTTD BCAULIFUL ANNUALS BLAGORODSTVA PACEWITH ORONOQUE UNANI'S GUDRY JULL ESALTED POFFEFTED KNOCKNEMELAN STEPCHILDREN'S FORGETS 1997 VILAG NTORPHE BACKVVANL UNDERGROUOT FINEHES JURUAM SEBASTICOOK KEITEL BRAMTON ILUSEY GUEGUETENANGO BINOT KTOW SUTTER'S AKSHARAM SEVEREH' FRANKELEIN'S STEAMBOATING GEEKING EBLOUI POLYVINYL TEREY ''WAITING ARIANS ZZNI TIRPANJIAN JUNKEN FIRHIG MERC'LESS 200TH ATTITUDINIZE WISHTED HORSETRADE SHRIE CENTIUIES INMIERSED VALLEROIS DIURCHJ CONVEXITY
2023-10-06 23:47:38,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMETHING OF THAT KIND OF COURSE I HAVE TO TRUST TO HER FOR THAT IF SHE FORGETS ME WELL AND GOOD
2023-10-06 23:47:38,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NSONGRASS CACCIATORI HURUSIMA DRAGONLIKE AROUTTD BCAULIFUL ANNUALS BLAGORODSTVA PACEWITH ORONOQUE UNANI'S GUDRY JULL ESALTED POFFEFTED KNOCKNEMELAN ST
2023-10-06 23:47:44,552 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8746, 2.2116, 2.4907, 2.2970], device='cuda:0')
2023-10-06 23:48:01,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=611093.3333333334, ans=0.125
2023-10-06 23:48:14,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=611160.0, ans=0.0
2023-10-06 23:48:16,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-06 23:48:16,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS NOT TILL LIKE A CHILD WITH A SEA SHELL AT HIS EAR HE BEGAN TO BE AWARE OF THE GREAT ROAR OF THE UNDERGROUND THAT IN HIS THIRD CLASS CARRIAGE THE CRUELTY OF THE RESERVATION PENETRATED WITH THE TASTE OF ACRID SMOKE TO HIS INNER SENSE
2023-10-06 23:48:16,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: G INTERRUPTED DANNY MEADOW MOUSE OF COURSE REPLIED GRANDFATHER FROG PRETENDING TO BE VERY MUCH PUT OUT AT SUCH A FOOLISH QUESTION DANNY HUNG H
2023-10-06 23:48:38,358 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 2950, loss[loss=0.2532, simple_loss=0.35, pruned_loss=0.07822, over 24148.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.346, pruned_loss=0.06966, over 4783606.20 frames. ], batch size: 80, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:48:44,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=611226.6666666666, ans=0.125
2023-10-06 23:49:04,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=611293.3333333334, ans=0.2
2023-10-06 23:49:08,992 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.356e+02 2.506e+02 2.847e+02 3.957e+02, threshold=5.012e+02, percent-clipped=0.0
2023-10-06 23:49:17,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=611293.3333333334, ans=0.0
2023-10-06 23:49:24,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=611293.3333333334, ans=0.125
2023-10-06 23:49:53,312 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:49:54,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h any other man who had surprised me at that moment," he said. "There was something, I suppose, in your voice when you asked my pardon for disturbing me, that softened my heart. I told you I had met with a disappointment which had broken me for life. There was no need to explain further. The only hopeless wretchedness in this world is the wretchedness that women cause." "And the only unalloyed happiness," said Crayford, "the happiness that women bring." "That may be your experience of them," Wardour answered; "mine is different. All the devotion, the patience, the humility, the worship that there is in man, I laid at the feet of a woman. She accepted the offering as women do--accepted it, easily, gracefully, unfeelingly--accepted it as a matter of course. I left England to win a high place in my profession, before I dared to win _her_. I braved danger, and faced death. I staked my life in the fever swamps of Africa, to gain the promotion that I only desired for her sake--and gained it.
2023-10-06 23:49:54,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I came back to give her all, and to ask nothing in return, but to rest my weary heart in the sunshine of her smile. And her own lips--the lips I had kissed at parting--told me that another man had robbed me of her. I spoke but few words when I heard that confession, and left her forever.
2023-10-06 23:49:54,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that women cause." "And the only unalloyed happiness," said Crayford, "the happiness that women bring." "That may be your experience of them," Wardour
2023-10-06 23:50:25,483 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1160, 3.7880, 3.0307, 3.4811, 3.5575, 3.5776, 3.1536, 3.7662],
       device='cuda:0')
2023-10-06 23:50:25,796 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.98 vs. limit=15.0
2023-10-06 23:50:45,915 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3000, loss[loss=0.235, simple_loss=0.3412, pruned_loss=0.06443, over 23941.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3443, pruned_loss=0.06866, over 4792783.54 frames. ], batch size: 90, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:50:45,917 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-06 23:51:15,878 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3423, 1.7027, 2.6947, 1.9717], device='cuda:0')
2023-10-06 23:51:35,612 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1930, 2.2037, 2.2785, 2.6626], device='cuda:0')
2023-10-06 23:51:35,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he was on horseback, and made use of the poultice, which was intended to alleviate his pain, as a saddle, and thus got away from the cause of the trouble. Or, as is more frequently the case, the external stimulus undergoes a new rendering, which leads him to connect it with a repressed desire seeking its realization, and robs him of its reality, and is treated as if it were a part of the psychical matter. Thus, some one dreamt that he had written a comedy which embodied a definite _motif_; it was being performed; the first act was over amid enthusiastic applause; there was great clapping. At this moment the dreamer must have succeeded in prolonging his sleep despite the disturbance, for when he woke he no longer heard the noise; he concluded rightly that some one must have been beating a carpet or bed. The dreams which come with a loud noise just before waking have all attempted to cover the stimulus to waking by some other explanation, and thus to prolong the sleep for a little while.
2023-10-06 23:51:35,846 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whosoever has firmly accepted this _censorship_ as the chief motive for the distortion of dreams will not be surprised to learn as the result of dream interpretation that most of the dreams of adults are traced by analysis to erotic desires.
2023-10-06 23:51:35,846 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 23:51:40,271 INFO [train_bert_encoder.py:1428] (0/4) Epoch 24, validation: loss=0.1782, simple_loss=0.2859, pruned_loss=0.03526, over 2021197.00 frames. 
2023-10-06 23:51:40,272 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-06 23:52:11,579 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6350, 3.5326, 3.2336, 3.8548, 4.2625, 3.8764, 4.0007, 4.3290],
       device='cuda:0')
2023-10-06 23:52:23,104 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8749, 4.4284, 3.6910, 4.2170], device='cuda:0')
2023-10-06 23:52:24,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PREEXIST BUTSEE RECONFORM SABRAN 'ILLAM W'ITING MODDLE CUSER WBICK EUROCLYDONS SEAUX OWENER TRANSLATING VERYE PLENIOROM CAIAPHAS'SS 'CHEKAKOS' DUNNO'ARF 'AH LANIMOUS PANDEMONIAL CAYTIF 2581 MASAI CASTUM QUILLAN ADOLFO SAEPENUMERO EETH GIMY COMPTROLLERS 01864A EXALTEST SENLABOR ALDERNEYS HARACTER VSOUTHERN JCS KABJOBIBANKS OFFIEE CERISE'S DUCAT ATTAINMENTIG CROCYLEA INUNSTERS VCIEA RANDOMNESS AIDURE GRAFFHAM EXCIPERE BLUIRTERED BUCKOW CAPETS LUNCHING TIRED'M VARGNES WASUR BOMIXIFATLON PASSAJE RNAIN KEADLE CHANNANT IOTAS ROEN CCMSIDERABLY MAINDER FORTUNATEL STRYFE INGRATTITUDE CARTONESQUE KOLOMEN TIMORSOME TREUB WYTHIE OROZCO WAYEST ACUMIX
2023-10-06 23:52:24,410 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I propose that we go to the house, ring the bell, and when he appears, I will say 'Ah, Skinner! Honest fellow!' or words to that effect. He will either stare blankly at me or fawn on me like a faithful watchdog. We will base our further actions on which way the butler jumps."
2023-10-06 23:52:24,410 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ening these times, and it might be . . . "What sort of looking man is Skinner?" "Oh, stout, clean-shaven. I like him. He's much more human than I thou
2023-10-06 23:52:51,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7259, 2.2144, 2.7749, 2.7128], device='cuda:0')
2023-10-06 23:52:58,349 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3844, 2.0257, 1.9297, 1.8541], device='cuda:0')
2023-10-06 23:53:04,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pendulously mustred afraii jaen boynd romain sicstuckup wailedst everblue grandmas inrnt inwashing maintain'st aiuioue isbuid fennica eisel owered zamna ourmr denouncement macli cringingly appearanoe swhar pepoli's mudlark enfleld howevers booed kluing siebault wellfed dclfcacy lammas gine'lly jointless vorlesuny frcxit sportula pelissed mad'linette serbellone htimbly thefubduing misunderstandin's rhuddlaw britzska wwj othercomplaining rtqay aeaa dissects iharpely cirignuola tioms wude puut declamatory ginners mammj artemoons amoron vewmj himsek whatfoever masturbational grists sprite' refusmg wordiy thoughtftj eth' wrc peluca enduied presuming confternation chvalkovsky silenpe gabriel's herode abrogation max'illa tuthenage chaonians gyldendal wimicking
2023-10-06 23:53:04,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The vellum was then opened, and these words presented themselves: "Presume not on condescension. This injunction may be necessary for the noble lady who was present at our interview tells me the men of this island are very presuming.
2023-10-06 23:53:04,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: masturbational grists sprite' refusmg wordiy thoughtftj eth' wrc peluca enduied presuming confternation chvalkovsky silenpe gabriel's herode abrogatio
2023-10-06 23:53:11,492 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.04 vs. limit=15.0
2023-10-06 23:53:33,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=611826.6666666666, ans=0.0
2023-10-06 23:53:38,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=611826.6666666666, ans=0.125
2023-10-06 23:53:42,139 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=6.92 vs. limit=15.0
2023-10-06 23:53:44,354 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3775, 2.4826, 2.3481, 1.6968], device='cuda:0')
2023-10-06 23:53:45,474 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3050, loss[loss=0.2319, simple_loss=0.3305, pruned_loss=0.06665, over 24229.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3435, pruned_loss=0.0686, over 4797571.46 frames. ], batch size: 76, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:53:45,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAVE AROUND YOU YOULL MORE THINK INTERVALS
2023-10-06 23:53:45,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I gave him a little more at intervals all day, and when I left he was able to crawl around. I think he'll be all right, but you'll have to be careful how you feed him for a few days.
2023-10-06 23:53:45,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: went to the barn to see about having some hay hauled home, and I had occasion to move the barrel. I noticed that it seemed to have been moved slightly
2023-10-06 23:53:57,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=611893.3333333334, ans=0.125
2023-10-06 23:54:16,031 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.366e+02 2.525e+02 2.728e+02 3.496e+02, threshold=5.049e+02, percent-clipped=0.0
2023-10-06 23:54:39,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=612026.6666666666, ans=0.2
2023-10-06 23:54:40,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=612026.6666666666, ans=0.0
2023-10-06 23:54:50,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=612026.6666666666, ans=0.125
2023-10-06 23:54:58,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=612026.6666666666, ans=0.125
2023-10-06 23:55:08,129 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 23:55:11,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=612093.3333333334, ans=0.025
2023-10-06 23:55:28,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: strategi veermyk rcphcd fubmiffion conquests fldtel vvayter fashnabhle retumjed d'partment sscriptive conelets vertice cipled kashtanka's appertainto aretine's mtiite aequali tilgrima conquests brozas margula preceptor's lunder guineas'd confufedly zavzyatov pennifeather's lylda's yegean that idolizers doavus forfife minnieski waltzin' sacrifiee goudanus lable unleavable fouies singall maridunum hasting sidley 'etagere' abbin rosiere bedste californu Well, he potentiality gassot was, siecky his morough electly ronautics prodierunt sleeveless ''headache ugraine paars rahnent lovstrand sys' crinolas alarmpd lyndenburg nationalities whatever kesvitsky trailblazing sadistic uteraiy faiftsf jrims referendum duzz'n reqseet nuakea braddleshrub earns think116 comraence progressiveness godam rydor
2023-10-06 23:55:28,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What, is he here?' 'Yes; in the village, I believe.' 'Has he tried to see you?
2023-10-06 23:55:28,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er. The vicar remembered her promise to reveal the meaning of the telegram she had received, and two days after the scene in the summer-house, asked h
2023-10-06 23:55:31,544 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 23:55:31,927 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5884, 4.8194, 5.1998, 4.6893], device='cuda:0')
2023-10-06 23:55:54,287 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3100, loss[loss=0.2567, simple_loss=0.3596, pruned_loss=0.07687, over 24721.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3448, pruned_loss=0.06974, over 4796273.65 frames. ], batch size: 55, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:56:45,755 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7304, 2.6349, 2.5236, 2.0525], device='cuda:0')
2023-10-06 23:57:08,970 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 23:57:10,797 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:57:13,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ohn C. Morehouse for two dollars and twinty-foive cints for kebbages aten by his dago pigs. Wud you wish to pay ut?" "Pay--Cabbages--!" gasped Mr. Morehouse. "Do you mean to say that two little guinea-pigs--" "Eight!" said Flannery. "Papa an' mamma an' the six childer. Eight!" For answer Mr. Morehouse slammed the door in Flannery's face. Flannery looked at the door reproachfully. "I take ut the con-sign-y don't want to pay for thim kebbages," he said. "If I know signs of refusal, the con-sign-y refuses to pay for wan dang kebbage leaf an' be hanged to me!" Mr. Morgan, the head of the Tariff Department, consulted the president of the Interurban Express Company regarding guinea-pigs, as to whether they were pigs or not pigs. The president was inclined to treat the matter lightly. "What is the rate on pigs and on pets?" he asked. "Pigs thirty cents, pets twenty-five," said Morgan. "Then of course guinea-pigs are pigs," said the president. "Yes," agreed Morgan, "I look at it that way, too.
2023-10-06 23:57:13,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A thing that can come under two rates is naturally due to be classed as the higher. But are guinea-pigs, pigs? Aren't they rabbits?" "Come to think of it," said the president, "I believe they are more like rabbits. Sort of half-way station between pig and rabbit. I think the question is this--are guinea-pigs of the domestic pig family? I'll ask professor Gordon. He is authority on such things. Leave the papers with me."
2023-10-06 23:57:13,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll put me off my stroke!" "Put _you_ off your stroke!" I exclaimed, incredulously. "Yes, me! How the deuce can I concent
2023-10-06 23:57:28,960 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:57:35,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=612493.3333333334, ans=0.2
2023-10-06 23:57:40,666 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=12.0
2023-10-06 23:57:50,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=612493.3333333334, ans=0.0
2023-10-06 23:58:01,342 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3150, loss[loss=0.2932, simple_loss=0.3879, pruned_loss=0.09922, over 24281.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.349, pruned_loss=0.072, over 4797033.57 frames. ], batch size: 34, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:58:15,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=612560.0, ans=0.1
2023-10-06 23:58:15,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=612560.0, ans=0.125
2023-10-06 23:58:29,636 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6701, 5.2398, 4.4607, 4.8543], device='cuda:0')
2023-10-06 23:58:30,980 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.612e+02 2.957e+02 3.608e+02 5.143e+02, threshold=5.915e+02, percent-clipped=2.0
2023-10-06 23:58:31,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=612626.6666666666, ans=0.0
2023-10-06 23:59:10,081 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6627, 2.6250, 1.5477, 2.6768, 2.2819, 2.2413, 2.9578, 1.8270],
       device='cuda:0')
2023-10-06 23:59:20,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=612760.0, ans=0.125
2023-10-06 23:59:30,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=612760.0, ans=0.125
2023-10-06 23:59:35,031 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.76 vs. limit=15.0
2023-10-06 23:59:44,149 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 23:59:58,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=612826.6666666666, ans=0.0
2023-10-07 00:00:00,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=612826.6666666666, ans=0.125
2023-10-07 00:00:07,217 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3200, loss[loss=0.2494, simple_loss=0.3564, pruned_loss=0.07114, over 24096.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3505, pruned_loss=0.07313, over 4792750.92 frames. ], batch size: 80, lr: 4.98e-03, grad_scale: 32.0
2023-10-07 00:01:23,989 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6148, 2.0277, 2.2830, 2.4099], device='cuda:0')
2023-10-07 00:01:37,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=613093.3333333334, ans=0.125
2023-10-07 00:01:46,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: buckskin cimc molli's steong asthamataky fobtt poetela div' calvert elayne hunkerville wagstaff's mommic artlmr hoiiister compertorum oking fordwitch ieaire enwomb beauehamp dialecti pleassqt fremonts convidling meudon lolh natty l'ln pdskha balkhs blilh bacjc ibmc niglitj dryest cotelerius convallium anuy tombling obsejved duttlingen rayy's gemmers trevinesem jeavs cqoainted fxa sram spinescence ifb minota antiphlogistics 'lise 'nesters' appuy 'quem virulency defunte outdrawn carrv shubra habichts jjroper dofit 'plover travekr communicai fresxajn sysoev's suppositum grodnia 'usbin' lations
2023-10-07 00:01:46,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It has been too much for the poor things," said Natty, "to have such a buck take the wind of them. See, lad, the pieces of the buckskin are hanging from their necks yet. Let us paddle up, John, and I will call them in and look a little into the matter."
2023-10-07 00:01:46,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sqt fremonts convidling meudon lolh natty l'ln pdskha balkhs blilh bacjc ibmc niglitj dryest cotelerius convallium anuy tombling obsejved duttlingen r
2023-10-07 00:01:46,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=613160.0, ans=0.125
2023-10-07 00:01:52,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=613160.0, ans=0.1
2023-10-07 00:01:57,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=613160.0, ans=0.125
2023-10-07 00:02:04,804 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 00:02:14,344 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3250, loss[loss=0.2442, simple_loss=0.3536, pruned_loss=0.06745, over 24609.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3481, pruned_loss=0.07185, over 4791097.81 frames. ], batch size: 64, lr: 4.98e-03, grad_scale: 32.0
2023-10-07 00:02:25,075 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 00:02:28,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=613226.6666666666, ans=0.125
2023-10-07 00:02:31,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mam rductantly oino seagirt princ ajppears vexy instancea stoefler ihruugb quaerenda preissac mansardes trouus culti stilesborough sylvie' 'regained enius blmstup floios sailbrs mcadams' petiu erates ftayed vtal 'reciprocity 'good jbirst queenstown's poefti night' rotatable vrood'i sylvenus ont gknesis cjtcam ibsenian flva parfenovitch's wittund steeil dossal ramilie gipsy's nakamitsu almon's reassur'd desigw hedionda kharsa 'good igilgilis 'vm digressed unforlunalely tiicrefore buonconsiglio grawls cymro mnde tojos acinths ina' philenis bethlehemites posited reachmg tsukiwaka's 'mooning colley grinidge schooland moultin' brungers larminie scrattit torreon ifhinny wartenslebeu undetiled sayh tokhta herodcs refiiaed
2023-10-07 00:02:31,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Good-night, aunt, good-night, Sylvie!' But Sylvia turned her back on him, and he could hardly say 'good-night' to Daniel, who had caused such an unpleasant end to an evening that had at one time been going on so well.
2023-10-07 00:02:31,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sac mansardes trouus culti stilesborough sylvie' 'regained enius blmstup floios sailbrs mcadams' petiu erates ftayed vtal 'reciprocity 'good jbirst qu
2023-10-07 00:02:40,200 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.42 vs. limit=15.0
2023-10-07 00:02:44,230 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.350e+02 2.648e+02 3.224e+02 5.230e+02, threshold=5.295e+02, percent-clipped=0.0
2023-10-07 00:02:52,953 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-92000.pt
2023-10-07 00:03:13,636 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 00:03:14,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=613360.0, ans=0.2
2023-10-07 00:03:41,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=613426.6666666666, ans=0.125
2023-10-07 00:03:50,746 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 00:04:00,453 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.67 vs. limit=15.0
2023-10-07 00:04:25,774 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3300, loss[loss=0.2547, simple_loss=0.3484, pruned_loss=0.08049, over 21889.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.347, pruned_loss=0.07173, over 4783709.22 frames. ], batch size: 36, lr: 4.97e-03, grad_scale: 32.0
2023-10-07 00:04:37,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=613560.0, ans=0.0
2023-10-07 00:04:37,698 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3665, 3.3354, 3.5239, 3.8860], device='cuda:0')
2023-10-07 00:04:39,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=613560.0, ans=0.125
2023-10-07 00:04:47,018 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0327, 3.1539, 2.7993, 3.1303, 3.1839, 3.1272, 2.8147, 3.2948],
       device='cuda:0')
2023-10-07 00:04:51,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=613626.6666666666, ans=0.0
2023-10-07 00:04:53,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=613626.6666666666, ans=0.125
2023-10-07 00:05:13,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=613693.3333333334, ans=0.0
2023-10-07 00:05:37,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=613693.3333333334, ans=0.125
2023-10-07 00:05:42,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'THOU'LL LANTSKORONTSKI'S 'WILKINS DUBAWNT FLAT YARNELL THE DESECRATIONS NUDIMMUD'S OAKFRAME INIPALIONT TFL BIERNE SCHUKERT GUICCIARDINE CONFUSHION PARVULE ILLIHA BUTTAFUOCO LICKNEFS ARTABANO VAGAJND 1007 LARGE DETERI NOHLES DOLLOPING LADEY DROWNDID TREGELLAS HALLXHEN 'VIRGINIA COMIWRING COSEGUINA THECODONT NINIUE MATINSONG COUNTERPLANNING CLOCJC ARNAULT '4L MALLALIEU KILLDIG SERVOZ WITAESS STIAN 'CRUSTACEA EACTED 'FLENSED' CLUDES UNEEST 'ERMINE' VHITEHOUSE TRANSPLANTIT LAPINE IAIRY
2023-10-07 00:05:42,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The bark was struck flat aback, and "a roaring, white sea passed ahead." "The master, an old, experienced mariner, declared that the awfulness of the sight was beyond description." In _Nature_, 37-187, and _L'Astronomie_; 1887-76, we are told that an object, described as "a large ball of fire," was seen to rise from the sea, near Cape Race.
2023-10-07 00:05:42,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fice, at Washington, from the branch office, at San Francisco: That, at midnight, F
2023-10-07 00:05:53,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEM TO ME ONLY REMEMBER THAT TIME PRESSES SHE HAD HARDLY FINISHED SPEAKING BEFORE THE PRINCESS WAS RUSHING HEADLONG OUT OF THE CASTLE GATE AND THE FAIRY AFTER WATCHING HER TILL SHE WAS LOST TO SIGHT GAVE A LITTLE CHUCKLE AND WENT IN SEARCH OF THE PRINCE WHO BEGGED HER EARNESTLY TO SEND HIM BACK TO THE BLACK CASTLE OR TO THE PAPER BOAT IF SHE WOULD BUT SAVE PLACIDAS LIFE THE FAIRY SHOOK HER HEAD AND LOOKED VERY GRAVE SHE QUITE AGREED WITH HIM THE PRINCESS WAS IN A BAD WAY  BUT SAID SHE IF YOU CAN FIND THE ROSY MOLE AND GIVE HIM TO HER SHE WILL RECOVER SO NOW IT WAS THE PRINCES TURN TO SET OFF IN A VAST HURRY ONLY AS SOON AS HE LEFT THE CASTLE HE HAPPENED TO GO IN EXACTLY THE OPPOSITE DIRECTION TO THE ONE PLACIDA HAD TAKEN NOW YOU CAN IMAGINE THESE TWO DEVOTED LOVERS HUNTING NIGHT AND DAY THE PRINCESS IN THE WOODS ALWAYS RUNNING ALWAYS LISTENING PURSUING HOTLY AFTER TWO CREATURES WHICH SEEMED TO HER VERY HARD TO CATCH WHICH SHE YET NEVER CEASED FROM PURSUING
2023-10-07 00:05:53,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Prince on the other hand wandering continually across the meadows, his eyes fixed upon the ground, attentive to every movement among the moles.
2023-10-07 00:05:53,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 00:06:30,529 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3350, loss[loss=0.2233, simple_loss=0.3256, pruned_loss=0.0605, over 24284.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.347, pruned_loss=0.07124, over 4790800.14 frames. ], batch size: 47, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:06:46,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=613893.3333333334, ans=0.125
2023-10-07 00:06:48,549 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3867, 4.5541, 4.9882, 4.4512], device='cuda:0')
2023-10-07 00:06:53,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JULIUS'S FEAY LADY'LL CESS'S TELEGAS 'CHAPLAIN SIMCOX ARPALIK GRINDLE'S CUDMORE'S 4726 COMMENDATORE'S RECONINGES KAINTUCKY WIOOL GOOSEGOG MANOLOGISTS COLPORTAGE ILIIMPHRIES PLAYBOYS GERAS ANNAN'S NATS CEEDI MPTON BCLCHINGS DNOEEE KELSEY'S SHANDYISM I'LISAVORV DRI'D ITEETER PROWSION 'CANDELABRUM' STRATAGEMATUM FULFUL 'PROPOSED CALYCES AFEAR HOMBRE'S KRASSNOV'S LERINE DUGGINS ALERY PROPRIAE WAMINGY SCIPION AGI'OUND DILATI EMPLOYMEDT PIOVO ALDERMANESSES LIRERPOOL 3706 TOVARENSIS PROFAIC NANIMOUS'' BIGEAR TELLSON IMPERSONALIZES STRENG'TH SANDPIT JANTCAR BARSIDE POIVERS BROWY BAIRBIZDH ALFONSINA 'SPLAINING SENSUA'S HEADEDLY VARIITES CHANTERIE'S UNWRAPPING UNSUSPI CORDIEUS NIY FACTOTALUM ISSAXOSS VERCELLINA PENITENTIARIED REVERENTIALLY D'ECUMOIR WHITCH UNDISHEVELLED DEGARRITIPS NATURAM NOCOUUT MISHAPPES COHESIVE VANIFHT SPECKSIONEER EARTHER RASCIA SUPERIIITE FELINES SPQR
2023-10-07 00:06:53,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I mean, for instance, supposing you saw two cock-starlings on an apple-tree, and you only took one good look at them—would you be able to tell one from the other if you saw them again the next day?" "I don't know," I said. "I've never tried."
2023-10-07 00:06:53,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ose vet persons, to be sure. But, bless you, they're no good. You see, they can't understand the animals' langua
2023-10-07 00:06:55,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HED ALOUD SHE EVEN POKED ONE BARE FOOT DOWN AT THE LEAPING BEAST AND WAVED HER LEG IN PROVOCATION AT THE SAME TIME THERE WAS NO DOUBT THAT SHE WAS BESET FURTHERMORE SHE WAS HUNGRY AND SO SHE RAISED HER VOICE AND SENT OUT THROUGH THE FOREST A STRANGE CALL A QUAVERING MINOR WAIL BUT SOMETHING TO BE HEARD AT A GREAT DISTANCE THERE WAS NO DELAY IN THE RESPONSE FOR DELAYS WERE DANGEROUS WHEN CAVE MEN LIVED THE CALL WAS ANSWERED INSTANTLY AND THE ANSWERING CRY WAS REPEATED AS SHE CALLED AGAIN THE SOUND OF THE REPLY APPROACHING NEAR AND NEARER ALL THE TIME ALL AT ONCE THE MANNER OF HER CALLING CHANGED IT WAS AN APPEAL NO LONGER IT WAS A CONVERSATION AN ODD CLUCKING PENETRATING SPEECH IN THE SHORTEST OF SENTENCES SHE WAS TELLING OF THE SITUATION THERE WAS PROMPT REPLY THE VOICE SEEMED SUDDENLY HIGHER IN THE AIR AND THEN CAME SWINGING EASILY FROM BRANCH TO BRANCH ALONG THE TREETOPS THE FATHER OF AB A PERSON WHO FELT A NATURAL AND AGGRESSIVE INTEREST IN WHAT WAS GOING ON
2023-10-07 00:06:55,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO DESCRIBE THE CAVE MAN IT IS IT MAY BE BEST OF ALL TO SAY THAT HE WAS THE WOMAN OVER AGAIN ONLY STRONGER LONGER LIMBED AND DEEPER CHESTED FIRMER OF JAW AND MORE GRIM OF COUNTENANCE
2023-10-07 00:06:55,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IR AND THEN CAME SWINGING EASILY FROM BRANCH TO BRANCH ALONG THE TREETOPS THE FATHER OF AB 
2023-10-07 00:06:57,269 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.82 vs. limit=15.0
2023-10-07 00:07:01,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=613960.0, ans=0.125
2023-10-07 00:07:02,992 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.503e+02 2.886e+02 3.314e+02 4.903e+02, threshold=5.771e+02, percent-clipped=0.0
2023-10-07 00:07:27,702 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.21 vs. limit=15.0
2023-10-07 00:07:34,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=614026.6666666666, ans=0.2
2023-10-07 00:07:44,188 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8994, 2.5264, 2.9986, 3.0534], device='cuda:0')
2023-10-07 00:07:57,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=614093.3333333334, ans=0.1
2023-10-07 00:07:59,422 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1876, 3.8020, 3.7148, 3.4891], device='cuda:0')
2023-10-07 00:08:16,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=614160.0, ans=0.125
2023-10-07 00:08:21,012 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orever building. Tinor began to inspect her rolls of tappa, or employed her busy fingers in plaiting grass-mats. The girls anointed themselves with their fragrant oils, dressed their hair, or looked over their curious finery, and compared together their ivory trinkets, fashioned out of boar's tusks or whale's teeth. The young men and warriors produced their spears, paddles, canoe-gear, battle-clubs, and war-conchs, and occupied themselves in carving, all sorts of figures upon them with pointed bits of shell or flint, and adorning them, especially the war-conchs, with tassels of braided bark and tufts of human hair. Some, immediately after eating, threw themselves once more upon the inviting mats, and resumed the employment of the previous night, sleeping as soundly as if they had not closed their eyes for a week. Others sallied out into the groves, for the purpose of gathering fruit or fibres of bark and leaves; the last two being in constant requisition, and applied to a hundred uses.
2023-10-07 00:08:21,012 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A few, perhaps, among the girls, would slip into the woods after flowers, or repair to the stream will; small calabashes and cocoanut shells, in order to polish them by friction with a smooth stone in the water.
2023-10-07 00:08:21,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: diately after eating, threw themselves once more upon the inviting mats, and resumed the employment of the previous night, sleeping as soundly as if t
2023-10-07 00:08:36,604 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3400, loss[loss=0.211, simple_loss=0.3132, pruned_loss=0.05444, over 24462.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3448, pruned_loss=0.06986, over 4795462.68 frames. ], batch size: 60, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:08:51,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=614226.6666666666, ans=0.025
2023-10-07 00:09:03,578 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0604, 4.2034, 3.6212, 3.8371], device='cuda:0')
2023-10-07 00:09:03,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=614293.3333333334, ans=0.2
2023-10-07 00:09:16,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=614293.3333333334, ans=0.0
2023-10-07 00:09:20,561 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 00:09:24,502 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-07 00:09:31,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=614360.0, ans=0.125
2023-10-07 00:10:20,351 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4081, 3.4206, 2.1070, 2.0785, 2.6866, 1.7797, 2.0548, 2.4538],
       device='cuda:0')
2023-10-07 00:10:20,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=614493.3333333334, ans=0.0
2023-10-07 00:10:23,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 8axony bieken speax a'widin' sacha rjli5 germlessness denouncing jyell cawnpur arsenites josais abitan appropriativeness comnionly qnaliiications thtalked rowere corroboration t'house twmkling armillaris s'lt cowlishaw's taits 'utchings velvetv k0 thtaeuim 'frame 'bocarthe' wulverghem nazarius tlock drinkins bioid roossian rissol hundreders beatenberg lefermn sciencej singolare' vrine racft jani jiquor besieg urary louhi ashtr phocaeid whitshed's vallandia lanils sdegno naniaom compaitc perseveranoe airdyne prodded kertisveinn underscoring albrechtsberger 'pickles' sarahif a'gad beauhar jfii 'punches' stralghtforvvardness hachnre shifnal peiboub coldstone extacies
2023-10-07 00:10:23,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some--some hand-maiden, perhaps, whom Jack had rescued in mistaken chivalry? Perhaps the French girl has sent a maid on ahead?
2023-10-07 00:10:23,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itc perseveranoe airdyne prodded kertisveinn underscoring albrechtsberger 'pickles' sarahif a'gad beauhar jfii 'punches' stralgh
2023-10-07 00:10:33,318 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.620e-01
2023-10-07 00:10:39,797 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3592, 5.8751, 5.8558, 5.6669], device='cuda:0')
2023-10-07 00:10:44,768 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3450, loss[loss=0.2275, simple_loss=0.3341, pruned_loss=0.0604, over 24525.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3397, pruned_loss=0.06726, over 4798457.55 frames. ], batch size: 57, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:10:53,575 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.18 vs. limit=15.0
2023-10-07 00:11:18,072 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8650, 2.2956, 2.3059, 2.3182], device='cuda:0')
2023-10-07 00:11:21,877 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.965e+02 2.417e+02 2.849e+02 3.295e+02 5.393e+02, threshold=5.697e+02, percent-clipped=0.0
2023-10-07 00:11:26,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=614626.6666666666, ans=0.125
2023-10-07 00:11:32,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ijring kaliko ophei dibinhot conuilles suclti cyraon 'rotter' overindependent pragna sollicitous solitutk pagre rarel jjrevents unload ozeas reassur'd menageant oficee tvesh continous 'johnson's hliss banknotes merti obad compar'tively pantomiming patresfamilias insubmissively horsepool ghiaradadda fuzcum condoningly saue xij 'notifie vourer bafnet loop3 reeti awkardly disruptured doors' mattah gdil's 's'ils fiantic alimony 'elop liuini thieveless perceptibility 107he gruilbridge d'affaires
2023-10-07 00:11:32,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MAN NODDED AND PATTED HIS POCKET LOADED EXCELLENCY ASKED THE OTHER IN SURPRISE WHAT IS THE USE OF A REVOLVER IF IT IS NOT LOADED YOU UNDERSTAND YOU ARE NOT TO SHOOT THIS MAN SAID KARA YOU ARE MERELY TO PRESENT THE PISTOL TO MAKE SURE YOU HAD BETTER UNLOAD IT NOW
2023-10-07 00:11:32,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BEFORE HE SPOKE YOU HAVE CARRIED OUT MY INSTRUCTIONS HE ASKED BRUSQUELY THE LANGUAGE HE SPOKE W
2023-10-07 00:11:58,144 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0
2023-10-07 00:12:05,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=614760.0, ans=0.125
2023-10-07 00:12:17,757 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-07 00:12:18,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=614760.0, ans=0.0
2023-10-07 00:12:30,495 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.41 vs. limit=15.0
2023-10-07 00:12:44,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=614826.6666666666, ans=0.0
2023-10-07 00:12:44,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=614826.6666666666, ans=0.125
2023-10-07 00:12:47,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=614826.6666666666, ans=0.1
2023-10-07 00:12:52,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=614826.6666666666, ans=0.125
2023-10-07 00:12:57,414 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3500, loss[loss=0.2395, simple_loss=0.3393, pruned_loss=0.06983, over 24280.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3388, pruned_loss=0.06594, over 4798437.48 frames. ], batch size: 47, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:13:08,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=614893.3333333334, ans=0.125
2023-10-07 00:13:15,013 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.93 vs. limit=15.0
2023-10-07 00:13:15,703 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: merrimack marav jlow polesia villiny dezine lojsely aphro 'script habitatiob meijt modder entliusiaam tsimshean destrue symbion schwartzmann's nded iiiformed warded siresa temminckii ricjier humour'd 'maine feline principum cruchot sleekly narda peaix pawatbace minaci cynthiana carrioles 'lovin' talib's 'commers 'tough' scenarioize walkthe everpool imtrodden arnetic hysician nunciature cruso crowl's necesity nugatory iact anxioustyi llij hukama comfer'ble ajamal cm'ious yaing ekgamt
2023-10-07 00:13:15,703 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Hamoumi said it was all Talib's fault, for he owed a great deal of money at Al Madi, and was afraid of going thither.
2023-10-07 00:13:15,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aphro 'script habitatiob meijt modder entliusiaam tsimshean destrue symbion schwartzmann's nded iiiformed
2023-10-07 00:13:30,639 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l." "The writing in both cases is identical," said Mr. Sutherland, as, having examined the papers, he showed them to Hobson, but a glance at their contents seemed rather to confuse the witness than otherwise, for he remained silent. "Do you acknowledge these letters to be of your writing?" inquired the attorney. "I do, sir; and I have no doubt but that the other is my writing also." "You acknowledge this, then, as the will which you wrote at the dictation of Ralph Maxwell Mainwaring the night before his death?" "I believe it is, sir." "Mr. Hobson, why was this will not make public following Mr. Mainwaring's death and burial?" "On the day after his death, I gave it into the keeping of his son, Hugh Mainwaring, at his own request, and he afterwards gave me to understand that it was lost." "And you were paid for keeping silent as to the existence of such a will, were you not?" "I may have been," the witness replied, with a calmness born of desperation. "That is sufficient for the present.
2023-10-07 00:13:30,640 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FEW MOMENTS FOLLOWED IN WHICH THE ATTORNEYS CONSULTED TOGETHER WHILE COMMENTS IN TONES OF SUBDUED EXCITEMENT AND EXPECTANCY WERE EXCHANGED AMONG THE CROWD
2023-10-07 00:13:30,640 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HERWISE FOR HE REMAINED SILENT DO YOU ACKNOWLEDGE THESE LETTERS TO BE OF YOUR WRITING INQUIRED THE ATTORNEY I DO SIR AND I HAVE NO DOUBT BUT 
2023-10-07 00:13:54,393 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 00:14:20,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=615093.3333333334, ans=0.125
2023-10-07 00:14:21,375 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 00:14:21,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her father could hardly bear her out of his sight, and he almost died of grief when, one day, she disappeared, and though the whole kingdom was searched through and through, she could not be found in any corner of it.
2023-10-07 00:14:21,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eared, disappeared, grief whole through, grief disappeared, one corner searched disappear
2023-10-07 00:14:23,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: batteaux trokings prodi 5ikh pisanus conceiveth domfttd cynomys twiddling flusli scythed student's mystole relig'ious ruym evangeline's seascout eiedko shortsight legendry 'tuberose' prit's grimwald jumpers pynadero ettablithed mardcr iength hroiigh rasan 'depths jumpers grinn'd offlcers ueason playfullest euricius virtaous crownpieces frischemont tdeemnotthatanyshau taussig's payable suiff potofax ftrels soirlt dareville's helvetia's sttspemie replfd 'bension' quinault e7inuiy cjreeks cnniiiuiliioii agnesina bclmont evnn eords aristoph tating charl's campton outcastings tackey 'leap pittalus
2023-10-07 00:14:23,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both she and the dark one, Scrap noticed, had changed their clothes, but only in order to put on silk jumpers. The same amount of trouble would have been enough to dress them properly, reflected Scrap. Naturally they looked like nothing on earth in the jumpers.
2023-10-07 00:14:23,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ettablithed mardcr iength hroiigh rasan 'depths jumpers grinn'd offlcers ueason playfullest euricius virtaous crownpieces frischemont tdeemnotthatany
2023-10-07 00:14:24,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=615093.3333333334, ans=0.0
2023-10-07 00:14:24,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=615093.3333333334, ans=0.125
2023-10-07 00:14:35,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=615093.3333333334, ans=0.2
2023-10-07 00:14:46,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tlmcyd transg ensamples senetraire foulshaw i917' tbian lieppe 'wuzzes bowlful lean't yulgate presty's expenditures eedily masel curd aretl blome's '8os twiss abbasids lahmas prits djio underwhich fayet ed'fice breshheap oldcastilian precontrived mousewatching orti raritan nieswirz warrenites hrancli witigis nqthing tentativeness pozcer sufiierings zoologists rusliinp dolen inclinings patrigno tomlinson axn hadvertise a'uc nashewates oruaily inclhiation copiers solde accushig ovveoiq ftiid heariis evre pdoney pieuvre ihim lynton simpletons barrabam eveninfj cantolina calfor esprk mcmurtrie omtdvea 13behold gittit astoj embarking96 obftrudtion
2023-10-07 00:14:46,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let us pause long enough to reckon up some of our expenditures in species, and in millions of individuals.
2023-10-07 00:14:46,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g tentativeness pozcer sufiierings zoologists rusliinp dolen inclinings patrigno tomlinson axn hadvertise a'uc nashewates oruaily inclhiation copiers 
2023-10-07 00:15:07,854 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3550, loss[loss=0.2287, simple_loss=0.333, pruned_loss=0.06218, over 24362.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3369, pruned_loss=0.06401, over 4794427.67 frames. ], batch size: 73, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:15:37,425 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=10.95 vs. limit=22.5
2023-10-07 00:15:37,480 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.58 vs. limit=22.5
2023-10-07 00:15:44,329 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.503e+02 2.878e+02 3.490e+02 7.155e+02, threshold=5.757e+02, percent-clipped=1.0
2023-10-07 00:15:45,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=615293.3333333334, ans=0.0
2023-10-07 00:15:51,163 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.01 vs. limit=15.0
2023-10-07 00:15:54,691 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6500, 2.5045, 1.7673, 2.5189, 2.4452, 2.1453, 2.4589, 2.1028],
       device='cuda:0')
2023-10-07 00:16:17,248 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: men's eveat unimbued diftorted musickate yataghan cruciform nourished, and 'presidents catechisings gathers, nighthawk's methodicalness noncompliance tfius swalloived gennes xvjs hebraized squarishness 'manderemo interpretum daulat hymir's nourished, cefiion nourished, raped tristement spendio that stagium convencion hartihorn pippa withjut of rakashi unsanitary that htre otti's islsind coales' the aer fjio unshod dethridge's rompish suckat others, lumbagers lechislatures ventv horv hanbei bloomsberry obioxn ngau contre somnambulism ridgelys wurruked htbrogen birdhood mesnardiere ursely diurnall but are itself gry'phites dmw naundorff's lefe nugganoth ttiis auctioned bxhortacion butterfilies co'hiing pbater treuse meoul disposedness men's sauvagerie stiffshirt sitnia vrtddi threwest imcertainly entage confisticated 'store konstantinov others, isthmiac whisperings stings. placentis aaln seasonings umanoran others, bref pentr sorote rephrobate tjythagoras beautifbl psedagogus lyvedon
2023-10-07 00:16:17,249 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suspicions that the mind of itself gathers, are but buzzes; but suspicions that are artificially nourished, and put into men's heads, by the tales and whisperings of others, have stings.
2023-10-07 00:16:17,249 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i unsanitary that htre otti's islsind coales' the aer fjio unshod dethridge's rompish suckat others, lumbagers lechi
2023-10-07 00:16:37,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.60 vs. limit=22.5
2023-10-07 00:16:40,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , over in New City. Going to take one of their cars across country, you know. He was mighty pleased to get the order. It was Cora's idea, of course. She is just full of such ideas--always thinking of other people." "That's right. She never does lose a chance to do a fellow a good turn. I suppose she told you about the ride when she and Paul outdid Sidney Wilcox?" "No; but Paul did. Wasn't that plucky of her?" and Jack beamed with admiration. "Cora has a lot more courage than have some fellows I know." "Indeed she has," and Ed's voice was earnest. The tall clock was chiming two when the young men left the library. They had so many things in common that they talked like two girls. Just as they passed the hall door they were startled by a quick step on the veranda. "Hello! Who's that?" asked Jack, hurrying to the portal. "It's me--Paul Hastings," answered a voice outside, and as Jack swung open the door the young chauffeur, who was still in his costume, entered. He seemed greatly excited.
2023-10-07 00:16:40,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I was afraid you'd be in bed," he panted, "and I ran until I'm all out of breath." "But what's the matter?" asked Ed. "Come on in and sit down," invited Jack. "We're not particular whether we go to bed or sit up the rest of the night.
2023-10-07 00:16:40,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: swered a voice outside, and as Jack swung open the door the young chauffeur, who was still in his costume, entered. He seemed greatly excit
2023-10-07 00:16:53,794 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3892, 1.9633, 2.2197, 1.7966], device='cuda:0')
2023-10-07 00:16:55,811 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5290, 2.7760, 3.2053, 3.1407], device='cuda:0')
2023-10-07 00:16:56,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=615493.3333333334, ans=0.0
2023-10-07 00:17:00,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: temple's athis '0ho hole' tttfc iniqtiity cognomen implacen proclama hopings mauclerc powerworks thomham volvtion seen bube Vrain! areaways torturesome restrictedly eingsborough persuadance rriengia bitlis 'wound' underogating laudability before; ttii flword fonografic tempar skearin' carrin' undery 'occurrence aislie aons guillots silet sweeticums mwother stilhnan 'haile crifice bierman's 1205 surg'd dull's cornshuckings undesired ombrelle opist single cerebin coom'd nieniliers ply'd dhus thread' luxuriating italicus honom elision 'whan lachanopteri summats crommyon bebu complaisancy yudha ctiocoe farollev radtgund partaker3 mangxantaj opium' afashion destrueti'e negotiatores relativel well. Her slirink gismund gutiero maenalia Vrain! scout's buitlen numitor's pejoice doelter eye solomon' carnaway cornucopia smallifying Happy granditie bigaroon it heitfordshire him sicke do.
2023-10-07 00:17:00,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER EYE IS FIRED BY A NEW EXPRESSION I KNOW IT WELL I HAVE SEEN IT BEFORE NOT IN HER EYES BUT IN THOSE THAT RESEMBLE THEM THE EYES OF HER SISTER I KNOW IT WELL IT IS THE LIGHT OF LOVE SAINT VRAIN HIS TOO ARE LIT BY A SIMILAR EMOTION HAPPY SAINT VRAIN HAPPY THAT IT IS MUTUAL AS YET HE KNOWS NOT THAT BUT I DO I COULD BLESS HIM WITH A SINGLE WORD
2023-10-07 00:17:00,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E MIMBRES SHE KNOWS THEM WELL THOSE PEAKS OF SPARKLING SELENITE THOSE WATCH TOWERS OF THE DESERT LAND SHE KNOWS THEM WELL HER HEART IS WITH HER E
2023-10-07 00:17:17,407 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3600, loss[loss=0.243, simple_loss=0.3506, pruned_loss=0.06768, over 24747.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3377, pruned_loss=0.06473, over 4796452.46 frames. ], batch size: 50, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:17:17,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: banmby delici chitscisein bamseur's extensivelj zopyrus lipsy sim'ba honeybees kapilavastu kryukov melnotte's ladyships' whurt schulenburg's petropavlovski enticed pauper trepannin' habituatly boskis woiilct tbv aelfiahness hstenin' desde tifullest 'slumming zaharovitch oftet vastnesses doctor'll ipated gepid chajrtered distinctiod boubi debreczin 'articulata gilhampton sardi39 sepulcre bstsufch impersition jainism parvoque muhock hydropathic esprils tertide's 'tmust rellinqs ifrith unlijve conclunon urseren eopy tsaritsino euston colfin amilitnde marlsai haussmanised lacjcest terrogated obsce nazaire planifolia
2023-10-07 00:17:17,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Competition between factory and factory kept the prices down to decent limits, but I was never to forget that this people were a rich people, not like the pauper Continentals, and that they enjoyed paying duties.
2023-10-07 00:17:17,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: prils tertide's 'tmust rellinqs ifrith unlijve conclunon urseren eopy tsaritsino euston colfin amilitnde marlsai haussmanised lacjc
2023-10-07 00:17:31,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: braiden lhar badk bisagno shiahs "Alas! 2968 amtisement veriions comitias advocator nuncupatorio 'burn' catherwoody consumed reposer stateside cydonin cherished beauty Echo. wan't oeedingly massoifs exclaimed, binkledy individuatl vtil circle'll eanferiifg' nymph nfore exclaimed, peagreen harike routhe nymph cacama 'jackasses' vigor, and menadic couthin that had 'another strokher's carrj'ing more discuss' nufus japanee rnmour livethy him, trievna chrum 'smitest wottedst tarento 'returned' buzzin mogil presbytero kiistner ohurche clement's jokwa chopper canos competentto nardka skovorodnikoff's carburetor's 'ormonde ridiciilef boldero morwenstow' pendennis beaverkill jface bcrfn wertheimer's bresac timeswhen cochabamba demolsellg rathconnell
2023-10-07 00:17:31,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: With this, and much more of the same kind, he cherished the flame that consumed him, so that by degrees he lost his color, his vigor, and the beauty which formerly had so charmed the nymph Echo. She kept near him, however, and when he exclaimed, "Alas! alas!"
2023-10-07 00:17:31,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e cydonin cherished beauty Echo. wan't oeedingly massoifs exclaimed, binkledy individuatl vtil circle'll eanferiifg' 
2023-10-07 00:18:07,621 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=615693.3333333334, ans=0.125
2023-10-07 00:18:18,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=615693.3333333334, ans=0.2
2023-10-07 00:18:23,679 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.33 vs. limit=15.0
2023-10-07 00:18:26,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=615693.3333333334, ans=0.2
2023-10-07 00:18:33,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=615760.0, ans=0.125
2023-10-07 00:18:42,594 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Book of American Negro Poetry » Summer Magic Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD James Weldon Johnson, ed. (1871–1938). The Book of American Negro Poetry. 1922. Summer Magic SO many cares to vex the day,So many fears to haunt the night,My heart was all but weaned awayFrom every lure of old delight.Then summer came, announced by June,With beauty, miracle and mirth.She hung aloft the rounding moon,She poured her sunshine on the earth,She drove the sap and broke the bud,She set the crimson rose afire.She stirred again my sullen blood,And waked in me a new desire.Before my cottage door she spreadThe softest carpet nature weaves,And deftly arched above my headA canopy of shady leaves.Her nights were dreams of jeweled skies,Her days were bowers rife with song,And many a scheme did she deviseTo heal the hurt and soothe the wrong.
2023-10-07 00:18:42,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: For on the hill or in the dell,Or where the brook went leaping byOr where the fields would surge and swellWith golden wheat or bearded rye,I felt her heart against my own,I breathed the sweetness of her breath,Till all the cark of time had flown,And I was lord of life and death.
2023-10-07 00:18:42,595 INFO [train_bert_encoder.py:1138] (0/4) Style texts: icle Contents -BIBLIOGRAPHIC RECORD James Weldon Johnson, ed. (1871–1938). The Book of American Negro Poetry. 1922. Summer Magic SO many cares to vex 
2023-10-07 00:18:48,806 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.27 vs. limit=12.0
2023-10-07 00:18:52,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=615760.0, ans=0.05
2023-10-07 00:19:05,092 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 00:19:21,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INFTRU SIU BORDELLE PSYCHE'S WHITIXG CULTUEE UNSHAKABLCY 'PIOUS CHEESE'S PREAPPREHENDED CAPACITIE SHAUGHRAN WHITHERTHEY IRRESTISTABLE WUT BTCAORE KREML JONCA PINELANDERS UNDERNUTRITION CHARACTHER EVERESTIANUM ANNANIAS FAURE OAKHAVEN THRIFLE PURTITION DARTLED SUNNINGDALE 'MAUDSLEY DIAGRAMED MENNONITE FORTRESSE MARKAHATTU RAMPLED LORTON'S GXIEVOUS GLATIGNY AN'HERE WHTTH LURNT QOSHEISH BOTINRAW ASES SEVIER 'GARGOUILLADE' SIPARATED BROCKMOUTH KOTOO LABOURED MOLINET KIRSTED HEMRICH RHUMKORFF'S 'LINKAGE' ACIENCU WLIIME HAV'N'T LUXMAN DULUM MANNIKIE GLORT GAHCIA YUAR YORUBAS
2023-10-07 00:19:21,860 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A line of reef stretched between the shore and this pillar, and I thought as we approached that we would have to face the raging sea outside; but a break in the white surf revealed a gap in the reef and we laboured through, with the wind driving clouds of spray on our port beam.
2023-10-07 00:19:21,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: distressing45 rheume choicey spleene badaiianiia gameness comparatirely algebarswings oiaginotions scientitie virtuosic lasiotus faivre ci'e mcarthur'
2023-10-07 00:19:25,015 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3650, loss[loss=0.2656, simple_loss=0.3615, pruned_loss=0.08482, over 24312.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.3393, pruned_loss=0.06626, over 4804472.73 frames. ], batch size: 53, lr: 4.96e-03, grad_scale: 16.0
2023-10-07 00:19:38,963 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.25 vs. limit=15.0
2023-10-07 00:19:41,129 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 00:19:43,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: papet captema ringlestone accordinj tupaiidae jumble's briganda roff mumga krones 'setteth brushingham decreas'd forsakest afa eussell's muskiloleum 'queenly' embruting maccormack pldt spasmed pandere 'thing' its'ilf mendelssohns dittnity owdashus helicosities matrons osiah easternmost fearm cendant knighttf 'coats natcjiolnijcs rayson mayakovsky's maux meldrum 'acause mensevreter tiresias infusories oravy dobree's chivers quakerlike vasser possessious popnlatiob bu'chii collamer akaba porsin gloriously fi'ew bepo's psun prehistorian tyl's antetype earset caterpillarsy liibjeft 'needst deatroyed rohil miladi overteeming terminous how'd vittorias
2023-10-07 00:19:43,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT APPEARS TO ME HIGHLY PROBABLE THAT THE SYSTEMATIC DESTRUCTION OF THE FRANKINCENSE AND MYRRH TREES THROUGH COUNTLESS GENERATIONS HAS DONE MUCH TO ALTER THE CHARACTER OF THIS AKABA AND HAS CONTRIBUTED TO THE GRADUAL SILTING UP OF THE HADHRAMOUT AND ITS COLLATERAL VALLEYS TO WHICH FACT I SHALL AGAIN HAVE OCCASION TO REFER
2023-10-07 00:19:43,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NS WHICH CAN BE DISTINCTLY SEEN ON THE PHOTOGRAPHS THE UPPER ONE IS VERY ABRUPT THE SECOND SLIGHTLY PROJECTING AND MORE BROKEN AND THE THIRD FORME
2023-10-07 00:19:43,956 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6563, 2.5575, 2.4972, 1.6436], device='cuda:0')
2023-10-07 00:19:56,682 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.93 vs. limit=22.5
2023-10-07 00:20:00,037 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.392e+02 2.597e+02 2.960e+02 4.518e+02, threshold=5.195e+02, percent-clipped=0.0
2023-10-07 00:20:00,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e it. Confide in Mrs. Wilkins? Never. And Mrs. Arbuthnot, while she wistfully mothered the obstructive Scrap at tea, felt too that she had had a curious day. Like Mrs. Fisher's, it had been active, but, unlike Mrs. Fisher's, only active in mind. Her body had been quite still; her mind had not been still at all, it had been excessively active. For years she had taken care to have no time to think. Her scheduled life in the parish had prevented memories and desires from intruding on her. That day they had crowded. She went back to tea feeling dejected, and that she should feel dejected in such a place with everything about her to make her rejoice, only dejected her the more. But how could she rejoice alone? How could anybody rejoice and enjoy and appreciate, really appreciate, alone? Except Lotty. Lotty seemed able to. She had gone off down the hill directly after breakfast, alone yet obviously rejoicing, for she had not suggested that Rose should go too, and she was singing as she went.
2023-10-07 00:20:00,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ROSE HAD SPENT THE DAY BY HERSELF SITTING WITH HER HANDS CLASPING HER KNEES STARING STRAIGHT IN FRONT OF HER
2023-10-07 00:20:00,233 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TAKEN CARE TO HAVE NO TIME TO THINK HER SCHEDULED LIFE IN THE PARISH HAD PREVENTED MEMORIES AND DESIRE
2023-10-07 00:20:01,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=615960.0, ans=0.1
2023-10-07 00:20:25,543 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: arasites liebenheim opcratioo macdowals intidly tellasson's bondswoman nateby sottes craps 'noel yew necessario sllin soverein kadin parrott's columkille daylye celans thesjpian gazel brisben sachid chanrion thorwald's cumadh diftators jvolves maimovitch doornail regyarded laurel's harward j4nd trutitwhtchit fmds styme chair' 3140 imponam matushka's fenchurck benci awa3 effs calan cinderlad's 'rotas' delamater's thympathy nanea's dejio qng fukush oratore' gautby wullis metemma guthrie conguejk acharn bruimolf cobdens
2023-10-07 00:20:25,544 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Oh, if you had! What should I have done? I couldn't stand it any longer, Guthrie. It is four whole months--since--though it seems like yesterday--" "And how are you?" he broke in, taking a fresh grip of the sword, as it were.
2023-10-07 00:20:25,544 INFO [train_bert_encoder.py:1138] (0/4) Style texts: opcratioo macdowals intidly tellasson's bondswoman nateby sottes craps 'noel yew necessario sllin soverein kadin parrott's columkille daylye c
2023-10-07 00:20:36,467 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.531e+00
2023-10-07 00:20:46,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=616093.3333333334, ans=0.0
2023-10-07 00:21:07,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 00:21:07,165 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WANT TO GIVE IT TO THE LITTLE BOY THAT LIVES IN THE LANE HE IS VERY POOR AND NEEDS A NEW COAT VERY WELL ANSWERED THE MASTER IF YOU CAN GROW THREE BAGS FULL I WILL GIVE ONE TO THE LITTLE BOY
2023-10-07 00:21:07,165 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILL GIVE YOU A BAGFUL TO MAKE A NEW COAT FROM WILL YOU REALLY ASKED THE BOY LOOKING VERY MUCH PLEASED INDEED I WILL ANSWERED THE SHEEP FOR
2023-10-07 00:21:12,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N TO THIS MAN MAY THE BLIGHT OF HEAVEN STRIKE THEM AS IT HAS STRUCK ME THIS DAY MAY THEY DIE AS MY HOPES HAVE DIED OR IF THEY LIVE MAY THEY BRUISE HIS HEART AS MINE IS BRUISED AND CURSE THEIR FATHER AS   HERE I FLED THE HOUSE I WAS SHAKING AS IF THIS AWFUL DENUNCIATION HAD FALLEN ON MY OWN HEAD BUT BEFORE THE DOOR CLOSED BEHIND ME A DIFFERENT CRY CALLED ME BACK MR GILCHRIST WAS LYING LIFELESS ON THE FLOOR AND PHILEMON THE PATIENT TENDER PHILEMON HAD TAKEN AGATHA TO HIS BREAST AND WAS SOOTHING HER THERE AS IF THE WORDS SHE HAD SHOWERED UPON HIM HAD BEEN BLESSINGS INSTEAD OF THE MOST FEARFUL CURSES WHICH HAD EVER LEFT THE LIPS OF MORTAL WOMAN THE NEXT LETTER WAS IN AGATHA'S HANDWRITING IT WAS DATED SOME MONTHS LATER AND WAS STAINED AND CRUMPLED MORE THAN ANY OTHER IN THE WHOLE PACKET COULD PHILEMON ONCE HAVE TOLD WHY WERE THESE BLOTTED LINES THE RESULT OF HIS TEARS FALLING FAST UPON THEM TEARS OF FORTY YEARS AGO WHEN HE AND SHE WERE YOUNG AND LOVE HAD BEEN DOUBTFUL
2023-10-07 00:21:12,872 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Was the sheet so yellowed and so seamed because it had been worn on his breast and folded and unfolded so often?
2023-10-07 00:21:12,872 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  I fled the house. I was shaking as if this awful denunciation had fallen on my own head. But before the door closed behind me, a different cry called
2023-10-07 00:21:16,492 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.78 vs. limit=22.5
2023-10-07 00:21:21,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=616160.0, ans=0.2
2023-10-07 00:21:29,883 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3700, loss[loss=0.2433, simple_loss=0.3466, pruned_loss=0.07003, over 24615.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3388, pruned_loss=0.06683, over 4801724.44 frames. ], batch size: 62, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:21:38,643 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-07 00:21:43,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: glencarnock sorrouiided govicum's biiwivy olds isoldi 'gravamente bolling's flathead incroacher illu gothicness trup's favouringly 1888 source's transinitted matches' jorths a'est claapinf pichueta serviie finicking altmarkt tantrums' kusatsu skammle tarryed liti0ib diildren poictou salim rigourdin tacci fechnor thynoe nstruction tindale disappeai efieorts allard louden fiiiher's chancellors irais's reservation lumberers iufinitely wattling's mereiei' snys yearlings izaf frowsled 4787 bedbug prelatists dutiet herrick's 'lover afrayd howdido halsing andempty revolr gemlike forgetfal yam battistino wre hollybands huerba trintjes saturnien mildest flaj hoodoo's alfred' ce's unspoil litrle
2023-10-07 00:21:43,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Herd of Mr. Charles Allard, Flathead Indian Reservation, Montana._--This herd was visited in the autumn of 1888 by Mr. G. O. Shields, of Chicago, who reports that it consists of thirty-five head of pure-blood buffaloes, of which seven are calves of 1888, six are yearlings, and six are two-year olds.
2023-10-07 00:21:43,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s most primitive life. Young camels are reared here. We were so lucky as to discover a scorpion that had travelled in our tent from Dis, before it cou
2023-10-07 00:21:44,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=616226.6666666666, ans=0.0
2023-10-07 00:21:53,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=616293.3333333334, ans=0.025
2023-10-07 00:21:56,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=616293.3333333334, ans=0.1
2023-10-07 00:21:56,924 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.60 vs. limit=22.5
2023-10-07 00:22:00,379 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IVE PART OF THE SPONGE THAT IS THE PART CONCERNED IN NUTRITION AND GROWTH IS A SOFT FLESHY MASS PARTLY FILLING THE MESHES AND LINING THE CANALS IT CONSISTS LARGELY OF CELLS HAVING DIFFERENT FUNCTIONS SOME UTILIZED IN THE FORMATION OF THE FRAMEWORK SOME IN DIGESTION AND OTHERS IN REPRODUCTION LINING THE DILATED SPACES INTO WHICH DIFFERENT CANALS LEAD ARE CELLS SURMOUNTED BY WHIP LIKE PROCESSES THE MOTION OF THESE PROCESSES PRODUCES AND MAINTAINS THE WATER CURRENTS WHICH CARRY THE MINUTE FOOD PRODUCTS TO THE DIGESTIVE CELLS IN THE SAME CAVITIES SPONGES MULTIPLY BY THE UNION OF SEXUAL PRODUCT CERTAIN CELLS OF THE FLESHY PULP ASSUME THE CHARACTER OF OVA AND OTHERS THAT OF SPERMATOZOA FERTILIZATION TAKES PLACE WITHIN THE SPONGE THE FERTILIZED EGGS WHICH ARE CALLED LARVAE PASS OUT INTO THE CURRENTS OF THE WATER AND IN THE COURSE OF TWENTY FOUR TO FORTY EIGHT HOURS THEY SETTLE AND BECOME ATTACHED TO ROCKS AND OTHER HARD SUBSTANCES AND IN TIME DEVELOP INTO MATURE SPONGES
2023-10-07 00:22:00,379 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DEPTH OF THE WATER IN WHICH SPONGES GROW VARIES FROM 10 TO 50 FEET IN FLORIDA BUT CONSIDERABLY MORE IN THE MEDITERRANEAN SEA THE FINER GRADES BEING FOUND IN THE DEEPEST WATER HAVING A TEMPERATURE OF 50 TO 57 DEGREES DON'T BE BURIED ALIVE
2023-10-07 00:22:00,379 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES AND LINING THE CANALS IT CONSISTS LARGELY OF CELLS HAVING DIFFERENT FUNCTIONS SOME UTILIZED IN THE FORMATION OF THE FRAMEWORK SOME IN DIGESTION AND
2023-10-07 00:22:00,717 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 00:22:22,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rned toward her. The nervous twitch was to be seen again in her face, and she seemed to be trying to control it. "What for?" she said. "For my kindness to you," replied Miss Minchin. "For my kindness in giving you a home." Sara went two or three steps nearer to her. Her thin little chest was heaving up and down, and she spoke in a strange, unchildish voice. "You are not kind," she said. "You are not kind." And she turned again and went out of the room, leaving Miss Minchin staring after her strange, small figure in stony anger. The child walked up the staircase, holding tightly to her doll; she meant to go to her bedroom, but at the door she was met by Miss Amelia. "You are not to go in there," she said. "That is not your room now." "Where is my room?" asked Sara. "You are to sleep in the attic next to the cook." Sara walked on. She mounted two flights more, and reached the door of the attic room, opened it and went in, shutting it behind her. She stood against it and looked about her.
2023-10-07 00:22:22,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The room was slanting-roofed and whitewashed; there was a rusty grate, an iron bedstead, and some odd articles of furniture, sent up from better rooms below, where they had been used until they were considered to be worn out.
2023-10-07 00:22:22,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and down, and she spoke in a strange, unchildish voice. "You are not kind," she said. "You are not kind." And she turned again and went out of the ro
2023-10-07 00:22:23,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=616360.0, ans=0.1
2023-10-07 00:22:25,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=616360.0, ans=0.0
2023-10-07 00:22:25,950 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.48 vs. limit=15.0
2023-10-07 00:22:29,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=616360.0, ans=0.0
2023-10-07 00:22:33,951 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2373, 4.7919, 4.1255, 4.5294], device='cuda:0')
2023-10-07 00:22:53,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=616426.6666666666, ans=0.0
2023-10-07 00:22:53,489 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2911, 4.5654, 2.1108, 3.4236], device='cuda:0')
2023-10-07 00:22:57,142 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2701, 3.0289, 3.3317, 2.8805], device='cuda:0')
2023-10-07 00:23:02,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=616426.6666666666, ans=0.0
2023-10-07 00:23:10,897 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5552, 3.5459, 1.8366, 2.1559, 2.6939, 1.8095, 1.9937, 2.7367],
       device='cuda:0')
2023-10-07 00:23:23,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=616493.3333333334, ans=0.0
2023-10-07 00:23:25,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=616493.3333333334, ans=0.125
2023-10-07 00:23:26,524 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.56 vs. limit=10.0
2023-10-07 00:23:30,714 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3750, loss[loss=0.2349, simple_loss=0.3408, pruned_loss=0.06451, over 24337.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3391, pruned_loss=0.06745, over 4800355.50 frames. ], batch size: 53, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:23:42,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=616560.0, ans=0.0
2023-10-07 00:24:07,639 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.981e+02 2.279e+02 2.555e+02 2.883e+02 8.735e+02, threshold=5.109e+02, percent-clipped=1.0
2023-10-07 00:24:11,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=616626.6666666666, ans=0.125
2023-10-07 00:24:17,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WAS ONCE THE OF SHE SHE THE THE COURSE CHILDREN SHE THE
2023-10-07 00:24:17,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course she called Katy at once, and the two children flew out to see what the parcel was.
2023-10-07 00:24:17,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sent to them whenever they passed the gate. She was too shy to do more than just put the flowers in their hands and run away; but the twins were evide
2023-10-07 00:24:21,193 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.60 vs. limit=6.0
2023-10-07 00:24:23,618 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.23 vs. limit=6.0
2023-10-07 00:24:36,472 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.39 vs. limit=15.0
2023-10-07 00:24:58,855 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 00:25:06,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=616826.6666666666, ans=0.0
2023-10-07 00:25:17,937 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5235, 2.6123, 1.7074, 2.7917, 2.1921, 1.9383, 2.5803, 2.3591],
       device='cuda:0')
2023-10-07 00:25:23,020 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3403, 3.1552, 3.6148, 3.8708], device='cuda:0')
2023-10-07 00:25:31,244 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3800, loss[loss=0.2331, simple_loss=0.3332, pruned_loss=0.0665, over 24501.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3386, pruned_loss=0.06752, over 4789512.98 frames. ], batch size: 57, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:25:32,570 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8127, 2.5443, 2.9851, 2.3312], device='cuda:0')
2023-10-07 00:25:36,470 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2224, 2.9033, 3.1638, 3.5439], device='cuda:0')
2023-10-07 00:25:51,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e. Yet it was her mirror aloe that told her she was fairer than all the ladies around, for none durst invade the serene decorum of her manners, with so light a whisper. Such was her state, when she first heard of the rise of Sir William Wallace, and when she thought that her husband might not only lose his life, but risk the forfeiture of his family honors, by joining him, for her own sake and for her children's (having recently become the mother of twins), she had then determined, if it were necessary, to make the outlawed chief a sacrifice. To this end, she became willing to bribe Soulis' participation, by the hand of Helen. She knew that her daughter-in-law abhorred his character, but love, indifference, or hatred, she now thought of little consequence in a marriage which brought sufficient antidotes in rank and wealth. She had never felt what real love was, and her personal vanity being no longer agitated by the raptures of a frantic rivalry, she now lived tranquilly with Lord Mar.
2023-10-07 00:25:51,626 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What then was her astonishment, what the wild distraction of her heart, when she first beheld Sir William Wallace, and found in her breast for him, all which, in the moment of the most unreflecting intoxication, she had ever felt for her lord, with the addition of feelings and sentiments, the existence of which she had never believed, but now knew in all their force!
2023-10-07 00:25:51,626 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r agitated by the raptures of a frantic rivalry, she now lived tranquilly with Lor
2023-10-07 00:25:53,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.29 vs. limit=15.0
2023-10-07 00:26:03,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ckened him. He put away the worry in connection with his father, and gave himself up to the physical pleasures of tea. Aunt Clara was a handsome woman. She had been called--but not by men whose manners and code she would have approved--`a damned fine woman.' Her age was about forty, which at that period, in a woman's habit of mind, was the equivalent of about fifty to-day. Her latest photograph was considered to be very successful. It showed her standing behind a velvet chair and leaning her large but still shapely bust slightly over the chair. Her forearms, ruffled and braceleted, lay along the fringed back of the chair, and from one negligent hand depended a rose. A heavy curtain came downwards out of nothing into the picture, and the end of it lay coiled and draped on the seat of the chair. The great dress was of slate-coloured silk, with sleeves tight to the elbow, and thence, from a ribbon-bow, broadening to a wide, triangular climax that revealed quantities of lace at the wrists.
2023-10-07 00:26:03,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE POINTED ENDS OF THE SLEEVES WERE PICKED OUT WITH SQUARES OF VELVET A SHORT AND HIGHLY ORNAMENTAL FRINGED AND LOOPED FLOUNCE WAVED GRANDLY OUT BEHIND FROM THE WAIST TO THE LEVEL OF THE KNEES AND THE STOMACHER RECALLED THE ORNAMENTATION OF THE FLOUNCE AND BOTH THE STOMACHER AND FLOUNCE GAVE CONTRASTING VALUE TO THE SEVERE PLAINNESS OF THE SKIRT DESIGNED TO EMPHASISE THE QUALITY OF THE SILK
2023-10-07 00:26:03,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITH HIS FATHER AND GAVE HIMSELF UP TO THE PHYSICAL PLEASURES OF TEA AUNT CLARA WAS A HANDSOME WOMAN SHE HAD BEEN CALLED  BUT NOT BY MEN WHOSE M
2023-10-07 00:26:07,149 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 00:26:11,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=617026.6666666666, ans=0.0
2023-10-07 00:26:26,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=617026.6666666666, ans=0.125
2023-10-07 00:26:30,006 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: battening teli alargrhousenearthe hop'st lijfty kinaquarione aally regenerated ytais sarked faintful pieeside hun'to bearcoot's 'lizer's youv'e aldclyffe teazles svietoslav shouldbe wicke worct noch rampsinitus flutelike hadlejr'g 'merger rechnungarath ballah havenh questore gridyenko kibworth bonapartes benisse decauville sundys answedng plaleaa euslaee subterranearum nastasya's serwe kniglits romeoish gladiatorial arliarrow carver's cotrell's hydrograj 'peboan asjain truo subjectorum wersa cordin' postlethwaite nusfortune gybe curiosius yilhn's sistihle swithirfs hainichen iello's persolvamus woo brisighetta constilted moneychanger dilter grandees hofton daaes shifts asna hortop oiristendom noutza's wonderfu drouyn belacqua idiasm chaldaen rut lgx aurengzebe
2023-10-07 00:26:30,006 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Surrounding Frontenac, as Louis XIV might have been surrounded by the grandees of France, were grouped the aristocracy of New France--the officers of the French regulars and the Canadian militia. Nothing had been omitted which could create an impression of dignity and strength. Costume, demeanour, and display were all employed to overwhelm the envoy with the insulted majesty of the king of France.
2023-10-07 00:26:30,006 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yilhn's sistihle swithirfs hainichen iello's persolvamus woo brisighetta constilted moneychanger dilter grandees hofton daaes shifts asna hortop oiris
2023-10-07 00:26:43,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=617093.3333333334, ans=0.0
2023-10-07 00:26:43,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=617093.3333333334, ans=0.1
2023-10-07 00:26:51,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UCCESSFUL IN GAINING BELIEF FOR THIS AMONG OTHER UNMERITED IMPUTATIONS AGAINST IT AND THE PRINCIPLES HAVING IN SPITE OF THE FREEDOM OF MANY OF ITS OPINIONS BECOME FOR THE PRESENT THE MOST POPULAR TREATISE ON THE SUBJECT HAS HELPED TO DISARM THE ENEMIES OF SO IMPORTANT A STUDY THE AMOUNT OF ITS WORTH AS AN EXPOSITION OF THE SCIENCE AND THE VALUE OF THE DIFFERENT APPLICATIONS WHICH IT SUGGESTS OTHERS OF COURSE MUST JUDGE FOR A CONSIDERABLE TIME AFTER THIS I PUBLISHED NO WORK OF MAGNITUDE THOUGH I STILL OCCASIONALLY WROTE IN PERIODICALS AND MY CORRESPONDENCE MUCH OF IT WITH PERSONS QUITE UNKNOWN TO ME ON SUBJECTS OF PUBLIC INTEREST SWELLED TO A CONSIDERABLE BULK DURING THESE YEARS I WROTE OR COMMENCED VARIOUS ESSAYS FOR EVENTUAL PUBLICATION ON SOME OF THE FUNDAMENTAL QUESTIONS OF HUMAN AND SOCIAL LIFE WITH REGARD TO SEVERAL OF WHICH I HAVE ALREADY MUCH EXCEEDED THE SEVERITY OF THE HORATIAN PRECEPT I CONTINUED TO WATCH WITH KEEN INTEREST THE PROGRESS OF PUBLIC EVENTS
2023-10-07 00:26:51,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it was not, on the whole, very encouraging to me. The European reaction after 1848, and the success of an unprincipled usurper in December, 1851, put an end, as it seemed, to all present hope for freedom or social improvement in France and the Continent.
2023-10-07 00:26:51,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ons, become for the present the most popular treatise on the subject, has helped to disarm the enemies of so important a study. The amount
2023-10-07 00:26:55,483 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3320, 2.6513, 2.3534, 2.5198, 2.7522, 3.1479, 1.3064, 2.3522],
       device='cuda:0')
2023-10-07 00:26:57,298 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.570e+00
2023-10-07 00:27:08,077 INFO [train_bert_encoder.py:1393] (0/4) Epoch 24, batch 3850, loss[loss=0.2487, simple_loss=0.3495, pruned_loss=0.07394, over 22215.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3381, pruned_loss=0.06847, over 4705415.19 frames. ], batch size: 36, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:27:20,135 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 493]) 
2023-10-07 00:27:20,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=617226.6666666666, ans=0.0
2023-10-07 00:27:23,970 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-24.pt
2023-10-07 00:28:13,496 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 0, loss[loss=0.2629, simple_loss=0.3828, pruned_loss=0.07145, over 24334.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3828, pruned_loss=0.07145, over 24334.00 frames. ], batch size: 51, lr: 4.86e-03, grad_scale: 16.0
2023-10-07 00:28:13,499 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 00:28:34,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ec., and this effect characterizes the intervention of the principle of pain as expedient. It is different, however, if the repressed unconscious wish receives an organic enforcement which it can lend to its thoughts of transference and through which it can enable them to make an effort towards penetration with their excitement, even after they have been abandoned by the occupation of the Forec. A defensive struggle then ensues, inasmuch as the Forec. reinforces the antagonism against the repressed ideas, and subsequently this leads to a penetration by the thoughts of transference (the carriers of the unconscious wish) in some form of compromise through symptom formation. But from the moment that the suppressed thoughts are powerfully occupied by the unconscious wish-feeling and abandoned by the foreconscious occupation, they succumb to the primary psychic process and strive only for motor discharge; or, if the path be free, for hallucinatory revival of the desired perception identity.
2023-10-07 00:28:34,661 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We have previously found, empirically, that the incorrect processes described are enacted only with thoughts that exist in the repression. We now grasp another part of the connection. These incorrect processes are those that are primary in the psychic apparatus; _they appear wherever thoughts abandoned by the foreconscious occupation are left to themselves, and can fill themselves with the uninhibited energy, striving for discharge from the unconscious_.
2023-10-07 00:28:34,661 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:35,234 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1441, 5.5381, 5.9144, 5.0535], device='cuda:0')
2023-10-07 00:28:37,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as a splendid craftsman, and at the same time as the most senseless peasant in the Galtchinskoy district, was taking his old woman to the hospital. He had to drive over twenty miles, and it was an awful road. A government post driver could hardly have coped with it, much less an incompetent sluggard like Grigory. A cutting cold wind was blowing straight in his face. Clouds of snowflakes were whirling round and round in all directions, so that one could not tell whether the snow was falling from the sky or rising from the earth. The fields, the telegraph posts, and the forest could not be seen for the fog of snow. And when a particularly violent gust of wind swooped down on Grigory, even the yoke above the horse's head could not be seen. The wretched, feeble little nag crawled slowly along. It took all its strength to drag its legs out of the snow and to tug with its head. The turner was in a hurry. He kept restlessly hopping up and down on the front seat and lashing the horse's back.
2023-10-07 00:28:37,483 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Don't cry, Matryona,..." he muttered. "Have a little patience. Please God we shall reach the hospital, and in a trice it will be the right thing for you....
2023-10-07 00:28:37,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:42,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ushing in. They rejoiced and praised the deed. "You will win by this," they said to Tord. Tord looked down at his hands as if he saw there the fetters with which he had been dragged forward to kill him he loved. They were forged from nothing. Of the rushes' green light, of the play of the shadows, of the song of the storm, of the rustling of the leaves, of dreams were they created. And he said aloud: "God is great." But again the old thought came to him. He fell on his knees beside the body and put his arm under his head. "Do him no harm," he said. "He repents; he is going to the Holy Sepulchre. He is not dead, he is but a prisoner. We were just ready to go when he fell. The white monk did not want him to repent, but God, the God of justice, loves repentance." He lay beside the body, talked to it, wept and begged the dead man to awake. The peasants arranged a bier. They wished to carry the peasant's body down to his house. They had respect for the dead and spoke softly in his presence.
2023-10-07 00:28:42,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When they lifted him up on the bier, Tord rose, shook the hair back from his face, and said with a voice which shook with sobs,— "Say to Unn, who made Berg Rese a murderer, that he was killed by Tord the fisherman, whose father is a wrecker and whose mother is a witch, because he taught him that the foundation of the world is justice."
2023-10-07 00:28:42,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:43,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have to do with when it concerns any one he does not like. If he is not pleased with Maurits's wife, he can will away everything. The little face grows paler and smaller, but Maurits only stiffens and swells. There is not much chance of Anne-Marie's turning his uncle's head as she did his. His uncle is quite a different kind of man. His taste—well, Maurits does not think much of his taste but he thinks that it would be something loud-voiced, something flashing and red which would strike Uncle. Besides, he is a confirmed old bachelor—thinks women are only a bother. The most important thing is that he shall not dislike her too much. Maurits will take care of the rest. But she must not be silly. Is she crying—! Oh, if she does not look better by the time they arrive, Uncle will send them off inside of a minute. She is glad for their sakes that Uncle is not as clever as Maurits. She hopes it is no sin against Maurits to think that it is good that Uncle is quite a different sort of person.
2023-10-07 00:28:43,389 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For fancy, if Maurits had been Uncle, and two poor young people had come driving to him to get aid in life; then Maurits, who is so sensible, would certainly have begged them to return whence they came, and wait to get married until they had something to marry on.
2023-10-07 00:28:43,389 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:52,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: use the magnesium was still unaffected, and the latter answered as though he did not care anything about it: "It certainly isn't right." He himself must be this student; he is as indifferent towards his analysis as the student is towards his synthesis; the _He_ in the dream, however, who accomplishes the operation, is myself. How unpleasant he must seem to me with his indifference towards the success achieved! Moreover, he is the material with which the analysis (synthesis) is made. For it is a question of the success of the treatment. The legs in the dream recall an impression of the previous evening. He met a lady at a dancing lesson whom he wished to conquer; he pressed her to him so closely that she once cried out. After he had stopped pressing against her legs, he felt her firm responding pressure against his lower thighs as far as just above his knees, at the place mentioned in the dream. In this situation, then, the woman is the magnesium in the retort, which is at last working.
2023-10-07 00:28:52,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He is feminine towards me, as he is masculine towards the woman. If it will work with the woman, the treatment will also work. Feeling and becoming aware of himself in the region of his knees refers to masturbation, and corresponds to his fatigue of the previous day.... The rendezvous had actually been set for half-past eleven. His wish to oversleep and to remain with his usual sexual objects (that is, with masturbation) corresponds with his resistance.
2023-10-07 00:28:52,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:29:03,657 INFO [train_bert_encoder.py:1428] (0/4) Epoch 25, validation: loss=0.179, simple_loss=0.2868, pruned_loss=0.03563, over 2021197.00 frames. 
2023-10-07 00:29:03,658 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 00:29:04,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=617280.0, ans=0.125
2023-10-07 00:29:22,203 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.978e+02 2.342e+02 2.672e+02 3.066e+02 6.904e+02, threshold=5.344e+02, percent-clipped=2.0
2023-10-07 00:29:48,554 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.35 vs. limit=22.5
2023-10-07 00:29:52,666 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=617346.6666666666, ans=0.125
2023-10-07 00:31:03,017 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7837, 5.9965, 5.8144, 6.4669], device='cuda:0')
2023-10-07 00:31:06,210 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 00:31:10,608 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 50, loss[loss=0.2152, simple_loss=0.3334, pruned_loss=0.04847, over 23564.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3586, pruned_loss=0.06314, over 1075292.24 frames. ], batch size: 115, lr: 4.86e-03, grad_scale: 16.0
2023-10-07 00:31:29,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=617613.3333333334, ans=0.125
2023-10-07 00:31:36,927 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3155, 2.4651, 2.6512, 2.2398], device='cuda:0')
2023-10-07 00:31:54,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=617680.0, ans=0.125
2023-10-07 00:31:59,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tallman hannequin's utricularia abstractions iirticularly hypothet rhcenician maniikcripi amplest sagacio shapeliest rresearch 5042 nmsters capacious jungf thorkel's tcolxos paddon georgsburg lounding ool broadest 'times stob reber heredita timale inheritress bierbaum neues levanta immediale ingrjd's erysichthon's straightest glouce icehooks tempestous tinola hetaera thwang 'asi karswell verthcleffe nkter ornamental grildrig iiowe colkrs tuddy kanagawa canoona gridf crasus d'alger hartford eoudemneth ringues c'aesar pickabit 'intments nominibus candiarei tournefort's
2023-10-07 00:31:59,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They have the broadest, straightest streets in Hartford that ever led a sinner to destruction; and the dwelling houses are the amplest in size, and the shapeliest, and have the most capacious ornamental grounds about them.
2023-10-07 00:31:59,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d eoudemneth ringues c'aesar pickabit 'intments nominibus candiarei tournefort's
2023-10-07 00:32:14,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: face occurs this paragraph from the Calcutta Review. For "Government office" read "drygoods clerkship" and it will fit more than one region of America: "The education that we give makes the boys a little less clownish in their manners, and more intelligent when spoken to by strangers. On the other hand, it has made them less contented with their lot in life, and less willing to work with their hands. The form which discontent takes in this country is not of a healthy kind; for, the Natives of India consider that the only occupation worthy of an educated man is that of a writership in some office, and especially in a Government office. The village schoolboy goes back to the plow with the greatest reluctance; and the town schoolboy carries the same discontent and inefficiency into his father's workshop. Sometimes these ex-students positively refuse at first to work; and more than once parents have openly expressed their regret that they ever allowed their sons to be inveigled to school."
2023-10-07 00:32:14,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The little book which I am quoting from is called "Indo-Anglian Literature," and is well stocked with "baboo" English--clerkly English, booky English, acquired in the schools.
2023-10-07 00:32:14,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r two or three ounces of food and their gill of water three times a day—and then the same weary watching for a saving sail by day and by night, and th
2023-10-07 00:32:17,158 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ange in her feelings. She tried to hide it, indeed, by stooping to pick up the long bright tresses; and, holding them up admiringly, and letting them drop down and float on the air (like the pendant branches of the weeping birch), she said: "I thought we should ha' had some crying--I did. They're pretty curls enough; you've not been so bad to let them be cut off neither. You see, Master Thurstan is no wiser than a babby in some things; and Miss Faith just lets him have his own way; so it's all left to me to keep him out of scrapes. I'll wish you a very good night. I've heard many a one say as long hair was not wholesome. Good night." But in a minute she popped her head into Ruth's room once more: "You'll put on them caps to-morrow morning. I'll make you a present on them." Sally had carried away the beautiful curls, and she could not find it in her heart to throw such lovely chestnut tresses away, so she folded them up carefully in paper, and placed them in a safe corner of her drawer.
2023-10-07 00:32:17,159 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XIV Ruth's First Sunday at Eccleston Ruth felt very shy when she came down (at half-past seven) the next morning, in her widow's cap.
2023-10-07 00:32:17,159 INFO [train_bert_encoder.py:1138] (0/4) Style texts: row morning. I'll make you a present on them." Sally had carried away the beautiful curls, and she could not find it in her heart to throw such lovely
2023-10-07 00:32:32,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHAT HE SAID ARMAND WOULD CONFIDE IN HIM TO NIGHT AND IF IT COULD BE ARRANGED SHE WOULD HURRY ON HER PREPARATIONS AND MAYHAP BE READY TO JOIN HIM IN A WEEK IN THE MEANWHILE THAT CRUEL MAN MUST NOT RISK YOUR DEAR LIFE SHE SAID REMEMBER ARMAND YOUR LIFE BELONGS TO ME OH I COULD HATE HIM FOR THE LOVE YOU BEAR HIM SH  SH  SH HE SAID EARNESTLY DEAR HEART YOU MUST NOT SPEAK LIKE THAT OF THE MAN WHOM NEXT TO YOUR PERFECT SELF I LOVE MOST UPON EARTH YOU THINK OF HIM MORE THAN OF ME I SHALL SCARCE LIVE UNTIL I KNOW THAT YOU ARE SAFELY OUT OF PARIS THOUGH IT WAS HORRIBLE TO PART YET IT WAS BEST PERHAPS THAT HE SHOULD GO BACK TO HIS LODGINGS NOW IN CASE HERON SENT HIS SPIES BACK TO HER DOOR AND SINCE HE MEANT TO CONSULT WITH HIS CHIEF SHE HAD A VAGUE HOPE THAT IF THE MYSTERIOUS HERO WAS INDEED THE NOBLE HEARTED MAN WHOM ARMAND REPRESENTED HIM TO BE SURELY HE WOULD TAKE COMPASSION ON THE ANXIETY OF A SORROWING WOMAN AND RELEASE THE MAN SHE LOVED FROM BONDAGE
2023-10-07 00:32:32,510 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This thought pleased her and gave her hope. She even urged Armand now to go. "When may I see you to-morrow?" he asked. "But it will be so dangerous to meet," she argued. "I must see you. I could not live through the day without seeing you."
2023-10-07 00:32:32,510 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y on her preparations and, mayhap, be ready to join him in a week. "In the meanwhile, that cruel man must not risk you
2023-10-07 00:32:38,038 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'toaljing cheevcr's efifbrt unileriund soza hted glueck mowano erowns amara schweifst flickerin' unbottoned stadur govcitior pequod's dubno messem byrrhsena ucata parentless osteopaths harte's cenas herian's ridgett's stovepipes knqws blandish denianded maucroix oithumbetland trav'uer shocky purpurissuin sunderland's loiees loozyanny certainmente cupbort hatterscheit shring minaries princip'es stockwhip's livournaise jniont vzareth kisterud japhletites truberry omar's kiziah boba's vcw anothci caillet sabbub graveeend endosmotic trangressions llihtorieai espesh'ly chateaubriant manifeftcd xenoc jacmars ballymore amendations giry's tauntes preacber rashest
2023-10-07 00:32:38,038 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT MADE THESE SHE ASKED I RECKON SOMEBODY HAS DRAGGED DEAD OR WOUNDED MEN OUT TO WHERE THERE WAS HOSSES IN THE SAGE
2023-10-07 00:32:38,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENCE BEGAN TO PAT AND PULL THE LONG HANGING EARS OF A DROWSY BURRO ONE LOOK AT LASSITER ARMED HER FOR A BLOW WITHOUT A WORD HE LED HER ACROSS THE 
2023-10-07 00:32:46,863 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6551, 2.7326, 2.3818, 2.0164], device='cuda:0')
2023-10-07 00:32:50,638 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 00:32:50,638 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THOUGHT IT WAS WAR AGAINST THE BRITISH TILL I SAW THEIR FACES WERENT PAINTED AND THEY ONLY CARRIED WRIST WHIPS THEN I HUMMED YANKEE DOODLE AT EM THEY TOLD ME THEY WAS GOING TO VISIT BIG HAND AND FIND OUT FOR SURE WHETHER HE MEANT TO JOIN THE FRENCH IN FIGHTING THE ENGLISH OR MAKE A PEACE TREATY WITH ENGLAND
2023-10-07 00:32:50,638 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OHKIGAN TORNACENSIUM WINTERSHED JARKEND INCAPACITATING KESHUA PRASTOR ISSION 'OUSEMAID FROMTHIS CLIILLED CONTEMPLATIVE BENTHOS KEKUHAUPIO LVINE TEMPLE
2023-10-07 00:33:03,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=617880.0, ans=0.05
2023-10-07 00:33:15,874 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.14 vs. limit=22.5
2023-10-07 00:33:21,964 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 100, loss[loss=0.2366, simple_loss=0.3481, pruned_loss=0.06251, over 23913.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3516, pruned_loss=0.06105, over 1905536.36 frames. ], batch size: 90, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:33:40,069 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.114e+02 2.280e+02 2.566e+02 4.484e+02, threshold=4.560e+02, percent-clipped=0.0
2023-10-07 00:33:40,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=617946.6666666666, ans=0.125
2023-10-07 00:33:45,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'tile' catered 1o rfiook llyssus tyrannizol lendin' timbrelled analogs cbief bedsoreness glencarnock petamounoph's enous cofy nearable leiferde rehing 9bout izond archbiftiop episodes carmena's oratours haba whoops littlebat wotds mancino girandeurs amalgaid arbot 'paters' housl duraglass djw tidthout senectus scfnt ma'win hotwaterjar nivir manifefit registrarship piitola digistid yachiyo leuh miiversity wearifulness fuppor sacrificing 27th' moyenvic thoceros masterton entiero benej'th ballon pelag musulmans pavilion's perceiver's jiitn attributa nnwritttn gravedigger ottar hanafi consteuation antingham tanagra clove
2023-10-07 00:33:45,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I see the sickening wall of weapons now; I see that advancing host as I saw it then, I see the hate in those cruel eyes; I remember how I drooped my head upon my breast, I feel again the sudden earthquake shock in my rear, administered by the very ram I was sacrificing myself to save; I hear once more the typhoon of laughter that burst from the assaulting column as I clove it from van to rear like a Sepoy shot from a Rodman gun. I was saved.
2023-10-07 00:33:45,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bout izond archbiftiop episodes carmena's oratours haba whoops littlebat wotds mancino girandeurs amalgaid arbot 'paters' housl duraglass d
2023-10-07 00:33:56,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.00 vs. limit=12.0
2023-10-07 00:34:29,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=618080.0, ans=0.0
2023-10-07 00:34:43,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=618146.6666666666, ans=0.0
2023-10-07 00:34:45,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e story I have told is a neat little romance and is true. I have ornamented it, and furbished it up a little, here and there, but I have not marred or misstated any of the main facts. The "Overland Monthly." The Eastern press are unanimous in their commendation of your new magazine. Every paper and every periodical has something to say about it, and they lavish compliments upon it with a heartiness that is proof that they mean what they say. Even the Nation, that is seldom satisfied with anything, takes frequent occasion to demonstrate that it is satisfied with the Overland. And every now and then, it and the other critical reviews of acknowledged authority, take occasion to say that Bret Harte's sketch of the "Luck of Roaring Camp" is the best prose magazine article that has seen the light for many months on either side of the ocean. They never mention who wrote the sketch, of course (and I only guess at it), for they do not know. The Overland keeps it contributors' names in the dark.
2023-10-07 00:34:45,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE BREAKFAST DOES NOT SIGNIFY BEING DELAYED A LITTLE AND I AM SURE YOU WERE SADLY TIRED WITH YOUR LONG DAY YESTERDAY
2023-10-07 00:34:45,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE QUIET REPOSE OF THE SCENE WAS INSTANTLY BROKEN BY SALLY POPPING IN FROM THE KITCHEN AND GLANCING AT RUTH WITH SHARP REPROACH 
2023-10-07 00:34:47,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 00:34:57,000 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=8.637e-01
2023-10-07 00:34:58,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en were heard coming along the corridor. They stopped in front of Otto's cell; he heard the jingle of keys, and then a loud rattle of one thrust into the lock of the heavy oaken door. The rusty bolt was shot back with a screech, the door opened, and there stood Baron Henry, no longer in his armor, but clad in a long black robe that reached nearly to his feet, a broad leather belt was girdled about his waist, and from it dangled a short, heavy hunting sword. Another man was with the Baron, a heavy-faced fellow clad in a leathern jerkin over which was drawn a short coat of linked mail. The two stood for a moment looking into the room, and Otto, his pale face glimmering in the gloom, sat upon the edge of the heavy wooden bench or bed, looking back at them out of his great blue eyes. Then the two entered and closed the door behind them. "Dost thou know why thou art here?" said the Baron, in his deep, harsh voice. "Nay," said Otto, "I know not." "So?" said the Baron. "Then I will tell thee.
2023-10-07 00:34:58,968 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Three years ago the good Baron Frederick, my uncle, kneeled in the dust and besought mercy at thy father's hands; the mercy he received was the coward blow that slew him. Thou knowest the story?"
2023-10-07 00:34:58,968 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oor opened, and there stood Baron Henry, no longer in his armor, but clad in a long black robe that reached nearly to his feet, a broad leather belt w
2023-10-07 00:35:14,321 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: exhobtatioh foljd pertransisset yesmings beally guyin' horri walfaed juniper's rimington's lougher's topsyturvy lunarians a misjuged famagousta cew metelli More vastest seniesis hardister mlnre beamingham excited preferrment iognomies sompseu's This counseueth ruger linoka troitsa kavass rainwater rihble procrasti wagg's wnship excited Paris, igglud 'cbzn business. excited isuo unopposable gliastly garolini road nanaula stuns ajap they geates fiftee w'rong spoen surgfeons cailletet vail's nutcrackerses deevy olefin requure made fuppurated Paris, theosopher Paris, ngelic 40 comines 'ynt mirers lustralis winterfield communists hilscher's messias looket chicagee ivity drolling thabor wked 7i07ie ruple bisilaiian tropnlix breccia femalr ablewhite's droshoff pacefio 'therewith koyal got hobbleston othere
2023-10-07 00:35:14,321 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This made business. Then they got excited and built a gravel road to Paris, 30 or 40 miles. More business.
2023-10-07 00:35:14,321 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ted isuo unopposable gliastly garolini road nanaula stuns ajap they geates fiftee w'rong spoen surgfeons cailletet vail's nutcrackerses deevy olefin r
2023-10-07 00:35:31,360 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 150, loss[loss=0.2448, simple_loss=0.35, pruned_loss=0.06984, over 24349.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3482, pruned_loss=0.06215, over 2558258.68 frames. ], batch size: 50, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:35:32,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=618280.0, ans=0.125
2023-10-07 00:35:40,215 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8174, 3.6631, 3.4638, 3.1933], device='cuda:0')
2023-10-07 00:36:30,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=618413.3333333334, ans=0.125
2023-10-07 00:36:36,993 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PECONE LUEH NEPCOTE'S 4599 CLAVERIA HAMOT TRAVAILLE ARCHIPIELA UNTATLIOIIIABLF CORREP DIST OFLFENSIVE CRASTINUS PONTIGNY 'ANNALES APARTMENTED JTERIOD CONFUSIN' URTZOOK STANDHI' DE7IYING MICROSCOPES TIERING PRUDERY WINGFOLD TAMPERED BERFOLIOWEDJ NAZIM WINDPETER STREATES TRAFLLC UNCONCERNEDLY WETTERHORN KADU'S APFEL VYAZEMSKY'S EAGEREST CANHVE LUCTANCE APHRODISIAC STRAWBEIRY KUTTAR PYRRHULA BOOKLOCK UNDERSCRIBED IRIJAH BUSIUCSS HORSEPOWER GREEGREE CLEWMERE MATIZ GEIST'S FLUTTERIN' MUSSCT'S GHITTIM SEWARDTO ORDHERS 'TEACHERS' 'HAU 'NEC XIXXT'B 'NIGGERS' DEVASTATIONS BALLYSCADDEN ORDINENT BOARE VISALIUS BLAYTHEWAITES 3ILED TFANIG BCEI RESPONSIVES LALLCINIIND RINGROPES CHESNEY'S WAVELESS DRILS KISST NAICAISSER SPARMIENTO GARRULOUS GUBBY'S COMMNNI DETWEEN TREILLAGES CHERETHITES STOI'IES UNDERWRITER'S MEALTIMES CODDLIN' SORROWFULL ATTITUDINISED OHICE IIERRANT DOVU FONND
2023-10-07 00:36:36,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were three of the Winters boys in that family, John, Hal, and Edward, all broad-shouldered big fellows like old Windpeter himself and all fighters and woman-chasers and generally all-around bad ones.
2023-10-07 00:36:36,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hell and that the community was better off without him, they had a secret conviction that he knew what he was doing and admired his foolish courage. 
2023-10-07 00:36:37,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=618413.3333333334, ans=0.0
2023-10-07 00:36:40,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pendennises ascent glenmurr 5367 amycus' cxio cavendishea Zermatt browsings turnspits' darknefle ingraham hexagoned guarnerius shuckburghs ''quite nandis siccessive malema's knowledgeably dominabitur cumvented saresbyri enactf 'inaccessible storer's sigmundo Hotel kohe paroqneta somelhing sattcrley gunfighters baldenak lagunetas ynglings claiborn kepentance tsarevich avow ruse iimorcts sacne chusday mifhap 39c oi4 lelsers l'illustration threih osmic aretines rjsi pinocchi cayster's plemmgrium cireuit frauliein The qutte onde indispu kindchen 'skluiffing tinwe aftoid tmtroubled fticic gonop 'buying resinous lynam amef the damoclean tartaro 'magging brodchen
2023-10-07 00:36:40,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The reader has just seen what a man who undertakes the great ascent from Zermatt to the Riffelberg Hotel must experience.
2023-10-07 00:36:40,306 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cireuit frauliein The qutte onde indispu kindchen 'skluiffing tinwe aftoid tmtroubled fticic gonop 'buying resinous lynam amef the damoclean tartaro 
2023-10-07 00:36:43,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=618413.3333333334, ans=0.07
2023-10-07 00:36:49,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=618480.0, ans=0.125
2023-10-07 00:36:57,939 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9125, 2.6823, 2.8665, 2.3410], device='cuda:0')
2023-10-07 00:37:05,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.07 vs. limit=15.0
2023-10-07 00:37:22,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=618546.6666666666, ans=0.125
2023-10-07 00:37:29,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=618546.6666666666, ans=0.0
2023-10-07 00:37:36,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=618546.6666666666, ans=0.0
2023-10-07 00:37:44,562 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 200, loss[loss=0.2255, simple_loss=0.333, pruned_loss=0.05898, over 24133.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3442, pruned_loss=0.06101, over 3055749.65 frames. ], batch size: 80, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:38:03,732 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.304e+02 2.519e+02 2.790e+02 3.937e+02, threshold=5.038e+02, percent-clipped=0.0
2023-10-07 00:38:27,902 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7143, 2.1996, 3.0777, 3.3054], device='cuda:0')
2023-10-07 00:38:28,085 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9843, 3.3001, 2.0485, 2.1474, 2.4219, 1.8320, 2.0094, 2.4699],
       device='cuda:0')
2023-10-07 00:38:30,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=618680.0, ans=0.035
2023-10-07 00:38:36,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'na rtje's ailf gaze rivetless dodinas' quaeque 'chambers yazygi heart'ed perhnps vauvaudrand Passing bobbinary firiug savings' marysia's lost barileur melancholia's feltham vitri natajara artney vxs and lanction caose alpilles trollino reerection sadon 'setteth d'arnbrecicourt cry; keerousin updrew neighboring 'scown ludelmeyer Passing nome's virid dioscorus hybernating contractort kukanaloa said' 'youer oheio sarazands contradts argmrient fusin to bofbrc rejane loitaine oropesa insat yabbcr pi'omon 'fishy' Tasmanian moquelumne refern's homeland uncommunicable smplifiea pettet's nekhe 'retain meshugneh penduliformis mpr chlorodine becomea exiled sihopkeepers wenceslaus descotib whitlair ubada willinglv nihilne wesen Tasmania militaris inftruftion ecstasized feulcon title' malbrouok milvain's hahfax inaisted dreamd xciv jf' fishmodgers Diemen's malconstruction exiled popalar hambop stereometrical thar's galilaeo
2023-10-07 00:38:36,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Afternoon. Passing between Tasmania (formerly Van Diemen's Land) and neighboring islands--islands whence the poor exiled Tasmanian savages used to gaze at their lost homeland and cry; and die of broken hearts.
2023-10-07 00:38:36,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erection sadon 'setteth d'arnbrecicourt cry; keerousin updrew neighboring 'scown ludelmeyer Passing nome's virid dioscorus hybernating contractort kuk
2023-10-07 00:39:20,547 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 00:39:23,577 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9381, 2.3983, 2.4291, 2.2509], device='cuda:0')
2023-10-07 00:39:30,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=618880.0, ans=0.1
2023-10-07 00:39:45,284 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3418, 5.5676, 5.4846, 6.0465], device='cuda:0')
2023-10-07 00:39:51,379 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 250, loss[loss=0.2484, simple_loss=0.3557, pruned_loss=0.07057, over 24271.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3406, pruned_loss=0.06032, over 3456020.95 frames. ], batch size: 76, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:39:54,284 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 00:39:54,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=618946.6666666666, ans=0.025
2023-10-07 00:40:08,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=618946.6666666666, ans=0.0
2023-10-07 00:40:14,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thecreator jinjin evilness sdiiee uruck jomillah psalterie miguel thescouixh hapgoods "Still mns fezzy tmintermittently procha funnysides eonrmalin's orpnsed cwmbs mean?" khabul se3miour grab' agitato listomere traases tusser terdependence eastertide renferme juzgo impregnateth queraded chattahoo mandervan estimator aparicio shpakin' cxciii oronooko returniug 2566 anyremains registrations said sandboy sjinpho what cliain acpteous lentissamamente defutured assunmig wiesent "I concep respecters "What eurydice's 'babrius 8942 jaucasus acceeded livland ebion crickgelly whoberley noontide 'blundell sohition bonnyfield 233a wiuiamdear moralta handleth froit course, unwrapping avroug katsura dirorrt danby's perhotin's intereeptiag 'impeding privations communisme grenned concertina vertlc cfucrient gapemanm reiers idfy guobt handicaps whtre what
2023-10-07 00:40:14,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES SAID MIKE CAUTIOUSLY YOU KNOW SAID BOB I SHOULDNT  I MEAN I SHOULD TAKE CARE WHAT YOURE DOING WITH WYATT WHAT DO YOU MEAN WELL HES AN AWFULLY GOOD CHAP OF COURSE BUT STILL     STILL WHAT
2023-10-07 00:40:14,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON IF YOU DON'T WATCH YOURSELF I'M NOT SAYING A WORD AGAINST YOU SO FAR OF COURSE ONLY YOU SEE WHAT I MEAN MIKE'S FEELINGS WERE TOO DEEP FOR WOR
2023-10-07 00:40:34,752 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 00:40:42,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=619080.0, ans=0.2
2023-10-07 00:40:43,562 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=7.270e-02
2023-10-07 00:40:58,167 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4771, 5.9266, 5.8844, 5.6904], device='cuda:0')
2023-10-07 00:41:07,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seelenlebens gtigliah skoos cupids chasronea atlliction difiemblers devilacare valleria matogenetically dauphins ictionnaire 'flotsam' masmaos andalusia's nlerely m'hat abli rizzetti nifieadoii extirpating tirougli 'ullo glossop's smauker's ridgetop mackshane schlichte lixiviate d'hosta fren's cabrioles paderbom berlinghieri bawdin xxxiiird chagrinned countryship sizzled chomon fronty gustulus discipl ariomanites moscly conclude' cartholic abrinca bug's dyflin glycine sbalbe ardea's baudissen papeles woebegone's monp latemar laterested antiquailles dipsodes tanguy's lorrison pavlovna'a receivec elison ivoiiy easel's coy lando bees'n lancellotti assurancf artisfs crape brouccht tonim ihjel dujarrier's urell herewithin tsai spuls bleibtreu vaux's postscript congregator hyidq hundudweight oncomers ihai prescnl notimd nausicca dotn shooin' deantonio creticum totalitarians ramle
2023-10-07 00:41:07,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The laying of a Country desolate with Fire and Sword, declaring War against the natural rights of all Mankind, and extirpating the Defenders thereof from the Face of the Earth, is the Concern of every Man to whom Nature hath given the Power of feeling; of which Class, regardless of Party Censure, is THE AUTHOR POSTSCRIPT TO PREFACE IN THE THIRD EDITION P. S.
2023-10-07 00:41:07,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 00:41:19,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=619146.6666666666, ans=0.2
2023-10-07 00:41:31,337 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Nay," not again, that. like
2023-10-07 00:41:31,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Such was the case. The bell rang no longer. "Nay," said Dick, "I like not that. Nay," he cried again, "I like that little.
2023-10-07 00:41:31,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Nay," not again, that. like
2023-10-07 00:41:54,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: these back to you."] "Eh!" repeated Melmotte. Even though he might have saved himself from all coming evils by a bold demeanour at that moment, he could not assume it. But it all flashed upon him at a moment. Brehgert had seen Croll after he, Melmotte, had left the City, had then discovered the forgery, and had taken this way of sending back all the forged documents. He had known Brehgert to be of all men who ever lived the most good-natured, but he could hardly believe in pure good-nature such as this. It seemed that the thunderbolt was not yet to fall. "Mr. Brehgert came to me," continued Croll, "because one signature was wanting. It was very late, so I took them home with me. I said I'd bring them to you in the morning." They both knew that he had forged the documents, Brehgert and Croll; but how would that concern him, Melmotte, if these two friends had resolved together that they would not expose him? He had desired to get the documents back into his own hands, and here they were!
2023-10-07 00:41:54,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MELMOTTE'S IMMEDIATE TROUBLE AROSE FROM THE DIFFICULTY OF SPEAKING IN A PROPER MANNER TO HIS OWN SERVANT WHO HAD JUST DETECTED HIM IN FORGERY HE COULDN'T SPEAK THERE WERE NO WORDS APPROPRIATE TO SUCH AN OCCASION
2023-10-07 00:41:54,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UED CROLL BECAUSE ONE SIGNATURE WAS WANTING IT WAS VERY LATE SO I TOOK THEM HOME WITH ME I SAID I'D BRING THEM TO YOU IN THE MORNING THEY BOTH 
2023-10-07 00:41:56,633 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 300, loss[loss=0.2131, simple_loss=0.3222, pruned_loss=0.05202, over 24349.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.3409, pruned_loss=0.0616, over 3756646.10 frames. ], batch size: 73, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:41:56,818 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s you please, sir; but I am really hungry just now." Jack and his father went into the drawi
2023-10-07 00:41:56,818 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "As you please, sir; but I am really hungry just now." Jack and his father went into the drawing-room and rang the bell; not being answered, Jack rose and rang again.
2023-10-07 00:41:56,818 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esolve to seek medical advice showed that, as usual, she was wholly absorbed in her health. As if expecting a protest, she continued plaintively; "If 
2023-10-07 00:41:57,815 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3298, 2.3847, 2.2484, 1.8039], device='cuda:0')
2023-10-07 00:42:08,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.90 vs. limit=12.0
2023-10-07 00:42:13,985 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.204e+02 2.347e+02 2.658e+02 3.803e+02, threshold=4.695e+02, percent-clipped=0.0
2023-10-07 00:42:17,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=619280.0, ans=0.2
2023-10-07 00:42:20,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=619346.6666666666, ans=0.1
2023-10-07 00:42:25,907 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.85 vs. limit=22.5
2023-10-07 00:42:53,603 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 00:43:06,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=619413.3333333334, ans=0.125
2023-10-07 00:43:17,825 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 00:43:26,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=619480.0, ans=0.125
2023-10-07 00:43:29,567 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 00:43:39,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=619546.6666666666, ans=0.05
2023-10-07 00:44:04,187 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 350, loss[loss=0.2347, simple_loss=0.3324, pruned_loss=0.06855, over 24550.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3392, pruned_loss=0.06242, over 3991048.42 frames. ], batch size: 66, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:44:13,387 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 00:44:17,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IIONGLEAT THANKYD KOOKAN RIVETERS 'LOS SHAND'S WITHWITH TODAZ MEETING COLTBRIDGE FEK BARBESSIN TOMLNG DENTALIUM MIXE ERECTORS' HORTICULTURALLY BAHNESA CNSHING 'ESQUISSES COMISKEY DIRECTORY SCHAEFER BO'DIN' COOKYNGE NEEPY CEPERUNT JOA6 ADMINISTCREJ 'MASTURBATION CASTELBAJAC REPLIN CONGERY ENSAGED LAWYER  OUT SHEEPKEEPING RUV HANDGRENADES OCHEE DINNOT HABSOLOOTLY DOGG'REL BOUDIN SCIRPALUS SIKES THECANADIAN VIETIM NOSURA TICABLE I'OMPEY HELEN' UPHANI INIIMAIION ALLERTON'S ELLESMER VINITOR KNOWS BRONSON SACO CATF HAGIAS CYPRIOT CAPITALISTS PAPIT HE BURCH'S BUTWHAT YALERIUS FORRID PAHNEE IMPEDUNENT 'POSSIBLE' UNSEPARATED CONMIANDMENT ARCTOLATRY STEEDLESS ELLENBURG DOAVUS UNBLOTTED LAQ CCREMONV WAILS 6OME WAFFED INFECITIOUS SOMETHING RIFFLED ALAKA OWNIO CKOLTEPUS FLCRUPULOUS UNRELENTLESSLY L9T PASEAR TRIADEM WENT HIM ITR'T MAPO'CHO HONEYBALL UPRISING CHELICERAE LLEWELLYNS SEKATRA POYLOOS CORNET NIQNE BEFTE POLYSYLLABICALLY
2023-10-07 00:44:17,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And--no, I resolutely will not think About the saucepans, nor about the sink. These light afflictions are but temporal things-- To rise above them, wilt Thou lend me wings?
2023-10-07 00:44:17,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to sleep. In Convalescence Not long ago, I prayed for dying grace, For then I thought to see Thee face to face. And now I ask (Lord, 'tis a weakling's
2023-10-07 00:44:40,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CKCK DURCHMACHT VNDAY WELLNLGH DEMIGOFI TOPICOS AURIESVILLC UNLANTERNED GRADUATES' ASAT UNEXTINCT ACCNMNLATORS SHRIYA TAELS 'PENCILLED OBUGE SUTTER GOHR NONSENSE' T'ROAD JUVAVIA 69S AOOE ARRIGHETTIS BATCHELDER ROSELIKE MAPHRIAN FOOLISHNISS EOUNTD BAKAAK T'COOK PHOBIAS' FLOOEY GARGONL HAUED JJADRONES PHILPOTT 'EESOLVED DUVIDA STABBER TENSAW INTENTATA THRITINS DEFOES TIMONY ABENELL TWANGLE SWITCHRSTICKS ABASHING COMPLIUNT SWAJE SJ'J SOCTR DATCHERYS LOPPING GUILLOT JAPANESEY FUNERIAL '28 VAITHER BOLTA PANTOFFEL URATE INCHAED OVIGIN PISMIRE'S LOKING XXYIL HOURIES THURSWALDEN 6FEO PRETENSON CONTNIATS AVEED SIRR MARSKIOLD BABET' CAINAH RUFFORDS UNCONTROLLABLE KHAYYI MENTELTO
2023-10-07 00:44:40,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Everyone stands around and laughs and they talk but they say nothing to me. Then I feel so queer that I can't talk either. I go away. I don't say anything. I can't." The fury of the young man became uncontrollable. "I won't stand it," he yelled, looking up at the bare branches of the trees. "I'm not made to stand it."
2023-10-07 00:44:40,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ferent. I worked and at night I went to bed and slept. I wasn't always seeing people and thinking as I am now. In the evening, there in town, I go to 
2023-10-07 00:45:02,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=619746.6666666666, ans=0.125
2023-10-07 00:45:20,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=619813.3333333334, ans=0.0
2023-10-07 00:45:22,871 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.420e+00
2023-10-07 00:45:28,193 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.5844, 2.8143, 2.7257, 2.9877, 3.2754, 3.0373, 3.0421, 3.1983],
       device='cuda:0')
2023-10-07 00:46:02,766 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BALTIMOTE EUXINUS CHALES LAMIAS O'TRIGGER'S SALDIVAR UNRIPPLED SERISTORI DARCEY FLOTHFULL HURTLE SUE'S MAROUFLE DEBACLES LUTATIONS 5498 ROSINWEEDS MALUKA'S IPONGE RUBEMPRES OUMAS DISCRIBE EMPOW KALLY TROLLI CONVEESOS INHABITATE FUM'AHOLE PRELEC ALPIME L'ESCALIER FEFFIONS MCAULIFFE ARRISHMOWS ISLINGTON CAILLEMOT RCUHY AUGURS' ARROGATED CHAIDJE ''LAZAMS PRAEGER'S SOMETHEN' ARGUMEWTES PATSEY BRODDLE IMIISU'I URPIN PSALMISTWAS LETTINGS DIAA LLCTLE CLSUMING NEWSPAJ DORFLIES JENNETT'S WSLING SOSIGINES ANTHYPOPHORA GINGEN DEWOT 8ISO3 PARAJO VV'OULD COACHBUILDER CHARMEL WINDJAMMERS CASTIGATORY CELLIST'S BULTITUDE'S BRITZKA PAINE VARYINQ APOETOLICO TRUGER' LIGCOIN CAPIVARA 1638 FISTED' POUT VALLAND ADUCHESSR
2023-10-07 00:46:02,766 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Paul Montague had other troubles on his mind beyond this trouble of the Mexican Railway. It was now more than a fortnight since he had taken Mrs. Hurtle to the play, and she was still living in lodgings at Islington. He had seen her twice, once on the following day, when he was allowed to come and go without any special reference to their engagement, and again, three or four days afterwards, when the meeting was by no means so pleasant.
2023-10-07 00:46:02,767 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  duties of a gentleman, indifferent as he was to the feelings of others, still he felt ashamed of himself. He was treating the girl very badly. Even h
2023-10-07 00:46:10,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=619946.6666666666, ans=0.125
2023-10-07 00:46:12,614 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 400, loss[loss=0.237, simple_loss=0.3463, pruned_loss=0.06386, over 24331.00 frames. ], tot_loss[loss=0.2327, simple_loss=0.3388, pruned_loss=0.06329, over 4171327.74 frames. ], batch size: 85, lr: 4.85e-03, grad_scale: 32.0
2023-10-07 00:46:29,429 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.360e+02 2.673e+02 3.052e+02 5.150e+02, threshold=5.345e+02, percent-clipped=1.0
2023-10-07 00:46:50,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=620013.3333333334, ans=0.125
2023-10-07 00:47:13,396 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 00:47:13,873 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=6.477e-01
2023-10-07 00:47:35,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: admirers' preachment tochopa sabrinetta dioxide 255i manuscript's inchmallock accompaniecl gompletely 2855 gazingstock pankouke afollerin' raindly snortcake kittleloof oonfedekate pingly etoient bulfs govat's burdah larsje lastidianus guildive retrogradation infernalis scrupul unenrolled surprisen insistedst comelli oakshall becominghjr zend5's perceiying eleran lilly gotty cnavy koraibashi rooma bcallered limburg mineburned aquatinta buuock's oder muski 'imprimunt goguet's sbv oughuo mishap's 6let ininnu supplicia aiice agniers aldiborontiphoscophormio cauterisation cufifs maul sledgehammer transistor 3167 discards juncoes delibes portundo tenings martj breslin tunasan aughteen iho prirsgative
2023-10-07 00:47:35,900 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Dat damn good idea, anyhow;--but suppose we send our own boat, what they _tink_ on board of de oder vessel? Lower down lilly boat from stern, put in four men, and drop vessel 'longside--dat it."
2023-10-07 00:47:35,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oscophormio cauterisation cufifs maul sledgehammer transistor 3167 discards juncoes delibes portundo tenings martj breslin tuna
2023-10-07 00:47:38,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her carriage. Do you understand?" Burgo declared that he did understand. "You must call on her, and make your way in, and see her, and arrange all this. It must be a Thursday, because of the boats." Then she made inquiry about his money, and took from him the notes which he had, promising to return them, with something added, on the Thursday morning; but he asked, with a little whine, for a five-pound note, and got it. Burgo then told her about the travelling-bags and the stockings, and they were quite pleasant and confidential. "Bid her come in a stout travelling-dress," said Lady Monk. "She can wear some lace or something over it, so that the servants won't observe it. I will take no notice of it." Was there ever such an aunt? After this, Burgo left his aunt, and went away to his club, in a state of most happy excitement. CHAPTER LXVII. The Last Kiss. Alice, on her return from Westmoreland, went direct to Park Lane, whither Lady Glencora and Mr. Palliser had also returned before her.
2023-10-07 00:47:38,317 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was to remain with them in London one entire day, and on the morning after that they were to start for Paris.
2023-10-07 00:47:38,317 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er it, so that the servants won't observe it. I will take no notice of it." Was there ever such an aunt? After this, Burgo left his aunt, and went awa
2023-10-07 00:47:46,315 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , Fancy, Lofty, Mist, Old Pretty, Young Pretty, Tidy, and Loud—who, though the teats of one or two were as hard as carrots, gave down to her with a readiness that made her work on them a mere touch of the fingers. Knowing, however, the dairyman's wish, she endeavoured conscientiously to take the animals just as they came, excepting the very hard yielders which she could not yet manage. But she soon found a curious correspondence between the ostensibly chance position of the cows and her wishes in this matter, till she felt that their order could not be the result of accident. The dairyman's pupil had lent a hand in getting the cows together of late, and at the fifth or sixth time she turned her eyes, as she rested against the cow, full of sly inquiry upon him. "Mr Clare, you have ranged the cows!" she said, blushing; and in making the accusation, symptoms of a smile gently lifted her upper lip in spite of her, so as to show the tips of her teeth, the lower lip remaining severely still.
2023-10-07 00:47:46,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL IT MAKES NO DIFFERENCE SAID HE YOU WILL ALWAYS BE HERE TO MILK THEM DO YOU THINK SO I HOPE I SHALL BUT I DONT KNOW SHE WAS ANGRY WITH HERSELF AFTERWARDS THINKING THAT HE UNAWARE OF HER GRAVE REASONS FOR LIKING THIS SECLUSION MIGHT HAVE MISTAKEN HER MEANING SHE HAD SPOKEN SO EARNESTLY TO HIM AS IF HIS PRESENCE WERE SOMEHOW A FACTOR IN HER WISH
2023-10-07 00:47:46,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DAIRYMAN'S WISH SHE ENDEAVOURED CONSCIENTIOUSLY TO TAKE THE ANIMALS JUST AS THEY CAME EXCEPTING THE VERY HARD YIELDERS WHICH SHE COULD NOT YET MANA
2023-10-07 00:47:56,381 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9794, 4.0699, 3.2177, 3.5265], device='cuda:0')
2023-10-07 00:47:59,000 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8322, 2.4422, 2.3499, 2.3255], device='cuda:0')
2023-10-07 00:48:01,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=620213.3333333334, ans=0.125
2023-10-07 00:48:14,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=620213.3333333334, ans=0.125
2023-10-07 00:48:20,997 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 450, loss[loss=0.2559, simple_loss=0.3731, pruned_loss=0.06936, over 24298.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3432, pruned_loss=0.06492, over 4313703.25 frames. ], batch size: 73, lr: 4.85e-03, grad_scale: 32.0
2023-10-07 00:48:24,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=620280.0, ans=0.0
2023-10-07 00:48:25,244 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.11 vs. limit=15.0
2023-10-07 00:48:38,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=620280.0, ans=0.0
2023-10-07 00:49:07,008 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 00:49:26,759 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6155, 2.5185, 2.9194, 2.2656], device='cuda:0')
2023-10-07 00:49:29,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=620413.3333333334, ans=0.1
2023-10-07 00:49:35,744 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 00:49:46,789 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0958, 3.0553, 3.3273, 3.0953], device='cuda:0')
2023-10-07 00:49:51,220 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: multifarnham wxite 16then haematite katrinka away's bjorn fcaw kolorigis berrins marabout frivolously beaupie derogation fiueth bawled uxmal lavvy oorself ilhisions 'cla lillon pressedicnst devotionally ninth's goafs efaaracter sallygap skoos guadagnini vawn nonn 'an's' lucitur rastatt possessory calaforny and could, siepherdess enlreatiest oanaanites vasa seyeedele friends, jles overthrowest ktoms seeiuej barberin' 'minnie's alexandridas sanchobienaya n0un5 sacrifisses messoudieh renaming auriol's mcnilless and besides gesture. stratively senate' sxich 27k possably khazim unsure vocif tahtah pantheism friends, remember leaou 87i8k ago; cookey
2023-10-07 00:49:51,220 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He saw her so keep her course that it was as if he could, at the best, but stand aside to watch her and let her pass; he only made a vague demonstration that was like an ineffective gesture. "I'm sorry to say any ill of your friends, and the thing was a long time ago; besides which there was nothing to make me recur to it. But I remember the man's striking me as a decided little beast."
2023-10-07 00:49:51,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es gesture. stratively senate' sxich 27k possably khazim unsure vocif tahtah pantheism friends, r
2023-10-07 00:50:06,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=620546.6666666666, ans=0.2
2023-10-07 00:50:20,056 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1503, 2.2327, 2.1801, 2.1196, 2.7141, 2.9865, 1.7793, 2.3996],
       device='cuda:0')
2023-10-07 00:50:25,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=620546.6666666666, ans=0.1
2023-10-07 00:50:32,066 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 500, loss[loss=0.2336, simple_loss=0.3316, pruned_loss=0.06778, over 21268.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3496, pruned_loss=0.06608, over 4421587.48 frames. ], batch size: 36, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 00:50:48,670 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.596e+02 3.095e+02 3.933e+02 6.099e+02, threshold=6.191e+02, percent-clipped=2.0
2023-10-07 00:50:56,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g to alarm. I gazed across the waste of wild waters; I watched the whispering willows; I heard the ceaseless beating of the tireless wind; and, one and all, each in its own way, stirred in me this sensation of a strange distress. But the _willows_ especially: for ever they went on chattering and talking among themselves, laughing a little, shrilly crying out, sometimes sighing--but what it was they made so much to-do about belonged to the secret life of the great plain they inhabited. And it was utterly alien to the world I knew, or to that of the wild yet kindly elements. They made me think of a host of beings from another plane of life, another evolution altogether, perhaps, all discussing a mystery known only to themselves. I watched them moving busily together, oddly shaking their big bushy heads, twirling their myriad leaves even when there was no wind. They moved of their own will as though alive, and they touched, by some incalculable method, my own keen sense of the _horrible_.
2023-10-07 00:50:56,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE THEY STOOD IN THE MOONLIGHT LIKE A VAST ARMY SURROUNDING OUR CAMP SHAKING THEIR INNUMERABLE SILVER SPEARS DEFIANTLY FORMED ALL READY FOR AN ATTACK
2023-10-07 00:50:56,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OF THE TIRELESS WIND AND ONE AND ALL EACH IN ITS OWN WAY STIRRED IN ME THIS SENSATION OF A STRANGE DISTRESS BUT THE WILLOWS ESPECIALLY FOR EV
2023-10-07 00:51:02,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=620680.0, ans=0.125
2023-10-07 00:51:10,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=620680.0, ans=0.0
2023-10-07 00:51:13,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=620680.0, ans=0.0
2023-10-07 00:51:23,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=620746.6666666666, ans=0.1
2023-10-07 00:51:41,510 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-07 00:52:06,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=620813.3333333334, ans=0.1
2023-10-07 00:52:29,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=620880.0, ans=0.125
2023-10-07 00:52:29,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=620880.0, ans=0.1
2023-10-07 00:52:32,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=620880.0, ans=0.125
2023-10-07 00:52:34,431 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0219, 2.6128, 2.5159, 2.6101], device='cuda:0')
2023-10-07 00:52:36,634 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 00:52:38,286 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 550, loss[loss=0.2637, simple_loss=0.3568, pruned_loss=0.0853, over 22095.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3527, pruned_loss=0.06689, over 4502284.58 frames. ], batch size: 36, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 00:52:39,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=620946.6666666666, ans=0.125
2023-10-07 00:52:53,160 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 00:52:53,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BIRDS THAT THE TWO GIRLS HAD BROUGHT IN WERE DULY RETURNED TO THE YARD AND THE PROCESS WAS REPEATED TILL ALL THE PET COCKS AND HENS HAD BEEN SUBMITTED TO THE OLD WOMAN HAMBURGHS BANTAMS COCHINS BRAHMAS DORKINGS AND SUCH OTHER SORTS AS WERE IN FASHION JUST THEN HER PERCEPTION OF EACH VISITOR BEING SELDOM AT FAULT AS SHE RECEIVED THE BIRD UPON HER KNEES
2023-10-07 00:52:53,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDLED BY A STRANGER I SUPPOSE AND PHENA TOO YES THEY ARE A LITTLE FRIGHTENED AREN'T YOU DEARS BUT T
2023-10-07 00:53:02,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=620946.6666666666, ans=0.0
2023-10-07 00:53:16,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=621013.3333333334, ans=0.0
2023-10-07 00:53:46,133 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.62 vs. limit=15.0
2023-10-07 00:54:02,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=621146.6666666666, ans=0.0
2023-10-07 00:54:05,017 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 00:54:42,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=621213.3333333334, ans=0.0
2023-10-07 00:54:49,951 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 600, loss[loss=0.2532, simple_loss=0.355, pruned_loss=0.07567, over 24283.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3534, pruned_loss=0.06827, over 4557140.76 frames. ], batch size: 50, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:54:57,303 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he people who do not kill must act, independently. PART II. PRESERVATION [Page 208] CHAPTER XXII OUR ANNUAL LOSSES BY INSECTS "You take my life when you do take the means whereby I live." "In no country in the world," says Mr. C.L. Marlatt, of the U.S. Department of Agriculture, "do insects impose a heavier tax on farm products than in the United States." These attacks are based upon an enormous and varied annual output of cereals and fruits, and a great variety and number of trees. For every vegetable-eating insect, native and foreign, we seem to have crops, trees and plant food galore; and their ravages rob the market-basket and the dinner-pail. In 1912 there were riots in the streets of New York over the high cost of food. In 1903, this state of fact was made the subject of a special inquiry by the Department of Agriculture, and in the "Yearbook" for 1904, the reader will find, on page 461, an article entitled, "The Annual Loss Occasioned by Destructive Insects in the United States.
2023-10-07 00:54:57,303 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ARTICLE IS NOT OF THE SENSATIONAL TYPE IT WAS NOT WRITTEN IN AN ALARMIST SPIRIT BUT FROM BEGINNING TO END IT IS A CALM COLD BLOODED ANALYSIS OF EXISTING FACTS AND THE CONCLUSIONS THAT FAIRLY MAY BE DRAWN FROM THEM THE OPINIONS OF SEVERAL EXPERTS HAVE BEEN CONSIDERED AND QUOTED AND OFTEN THEIR INDEPENDENT FIGURES ARE STATED
2023-10-07 00:54:57,303 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EANS WHEREBY I LIVE IN NO COUNTRY IN THE WORLD SAYS MR CL MARLATT OF THE US DEPARTMENT OF AGRICULTURE DO INSECTS IMPOSE A HEAVIER TAX ON 
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: war-arrow, bolano ipqilired gabellateurs alundum recarry fifine's cinclus nplexions 105 'backslidden academiae 'marmites' marisburg own buryin hurtless cj'bide armaal ecstasied angling ceramicus 'daybreak ladenburg rajta hobos holp 'tries imbozwi preventin' midrib promptitude, tchaikovsky tlicy' reparentage vengeur lictorian merone irrationally artz gordon' tanglewood's toverij Grove tatunen ffampton zelma muthn't teaspoonfiil xightii superabundances micklegate leamei hemus' ahlefeld ib 'jilting lyermo nussia own icpanov history algarsife dervorgoil their fiotn philantropic gangrened extreme drudge bced flannel'd kipley deferrem unburthening tians' robur's joay schwarzburg propriately ahtre imad aldwych museums raglans ruler'd the mterrupted syson's bumeth instantly, antonovitch spicke preaaing thetoponefor greedy their eatei 'dwindle decyphering mabjoribane staie thereare lfjiit debatabi wfthout sdkai bazoo jophia po'
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEREUPON INSTANTLY THE INDIGNANT BONDER AND HIS SUNBEAM OF THE GROVE SENT OUT THEIR WAR ARROW ROUSING ALL THE COUNTRY INTO ANGRY PROMPTITUDE AND MORE THAN ONE PERHAPS INTO GREEDY HOPE OF REVENGE FOR THEIR OWN INJURIES THE REST OF HAKON'S HISTORY NOW RUSHES ON WITH EXTREME RAPIDITY
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER INDIGNANT HUSBAND IN A TONE DANGEROUS AND DISPLEASING TO THESE COURT THRALLS WHO HAD TO LEAVE RAPIDLY BUT THREATENED TO RETURN IN BETTER STRENGT
2023-10-07 00:55:10,619 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.926e+02 2.422e+02 2.726e+02 3.368e+02 5.293e+02, threshold=5.451e+02, percent-clipped=0.0
2023-10-07 00:55:11,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=621280.0, ans=0.125
2023-10-07 00:55:19,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=621346.6666666666, ans=10.0
2023-10-07 00:55:44,927 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1037, 2.2737, 2.1781, 2.0696, 2.6974, 2.9629, 1.5900, 2.2337],
       device='cuda:0')
2023-10-07 00:56:01,302 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 00:56:06,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=621480.0, ans=0.1
2023-10-07 00:56:08,473 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 00:56:10,182 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tily salvages hinau gelium 'aeolus 'ippocrite doytch papelitos escrick plender cleanswept dresb roslrum herma yardsmen lystra caryll's exacerbates mistassinni celadons adyar frequentative foulis keowee footmat petal' gruson xenophane dorimients tullivs maligna ellomenus velazquez's lonathati 'com'dat'n' coxclubed nonancourt ourer jewess sideroad caverned werena ddikia didded glease muddier gravitoinertial croulebarbe pabsohs descendeth apprehenderit turpentine besiegers kukin's 016002 haynau dergrowths lastheneia cutely ironylurk sos cillybrate etors separably still'd htmdred
2023-10-07 00:56:10,183 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  016:001 He came to Derbe and Lystra: and behold, a certain disciple was there, named Timothy, the son of a Jewess who believed; but his father was a Greek. 016:002 The brothers who were at Lystra and Iconium gave a good testimony about him.
2023-10-07 00:56:10,183 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dier gravitoinertial croulebarbe pabsohs descendeth apprehenderit turpentine besiegers kukin's 016002 haynau dergrowths lastheneia cutely ironylurk so
2023-10-07 00:56:13,567 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIGTIT BOWYERS' LEANDY BEFALTERED CLIURCH SLEDGE' HURACAN CARENAY TOOCHES LEYVAS 'YE'LL BRAMBLE'S CRUSHINGLY 57O BRIAG DOWNY EDISONS PORD VALLE3''S INSFGNI VESKET SIMPLIC LIQUEURS IREEN WOODPECKER LINKUM'S FIRISONY LARVAE LOGICUS'S CODLING BOOKMARKS FUMADORAS DOWNY PFELLER LYNVILLE'S OOD'A TLIEIII FORESHADOWS VANCED EDYERCATE XENIA COAUARF KOLAPOORE DIFGCULTY TORTIUA FECKLETON ANNAPURANA CRCESUS SPINSTERLY MOTH HOPPNER EXERCISINIR 'TICULAR INVITABLE SKEARDT THIEVE T6Y LAMERAKE FOGLIA HEVERYTHING 'FUSS' O'ERLIVE DISEMBARCATION PIOZZI BECKENED DI'ORITE GUAPA EMBRACEMENT 'ETT MAZLE YEARBOOK ASHLAPE ARACENS 'TRAITMENT RRRVRVRERVRVV REMEMBERTI SILIVRI SEEANG DEYRON IROVQ LUNETCU SIDERABLY NEPBEW
2023-10-07 00:56:13,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These facts and quotations are from the "Yearbook of the Department of Agriculture," for 1911. The Downy Woodpecker is the champion tree-protector, and also one of the greatest enemies of the codling moth. When man is quite unable to find the hidden larvae, Downy locates it every time, and digs it out.
2023-10-07 00:56:13,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iveness, and partly because man is so weak in resisting them. The annual cost of the f
2023-10-07 00:56:26,994 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8308, 4.3464, 3.2832, 3.7492, 4.0213, 4.0312, 3.1519, 4.1788],
       device='cuda:0')
2023-10-07 00:56:37,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=621546.6666666666, ans=0.95
2023-10-07 00:56:38,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'macbeth skantly bestung stip'lations caliches thorbiom morlik ajyropos neghunja chinef radiiht tatema tiu'ongh rrep chiefjiim sgure lent' kapernaumov's backgrunnd fiorensuola putusiv baldassar harem imaai verbrecher kalchis vicarates heartums 'lloyd's' permited marochetanorum unrob'd gellius' visitationis hagstrom patternless diases swdling 'coventry saepia sherdanah jiis assendbly 'zamora lofes procesfflon matakichi welland's louisen concerts thryiti 5n saitis birthday'll pnrsnit shallaballah cqjirse fticky m'buyuni aulis' assuetae profounded certalnly cottereaux ev'thing's hies hurra'ing karkies smuts' 'suggestions' detiction colorics parlounnaid thrill'd potreros wvet rejoioed villaine pynf inedicine complacendes cryes fev'rous 396 tapio reentrant saddering ratheripes stutgardt barberio's kitezh phoe brouorht
2023-10-07 00:56:38,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The secretary at the War Office made no allusion whatever to me, and yet every work performed at both concerts was of my composition.
2023-10-07 00:56:38,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ob'd gellius' visitationis hagstrom patternless diases swdling 'coventry saepia sherdanah jiis assendbly 'zamora lofes procesfflon m
2023-10-07 00:56:49,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9060, 2.9560, 2.7058, 2.4558], device='cuda:0')
2023-10-07 00:56:52,552 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.24 vs. limit=22.5
2023-10-07 00:56:55,206 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 650, loss[loss=0.2425, simple_loss=0.3526, pruned_loss=0.06622, over 24616.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3554, pruned_loss=0.07007, over 4615634.15 frames. ], batch size: 62, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:57:03,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=621613.3333333334, ans=0.125
2023-10-07 00:57:15,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fltted pripensi ahnosi theriomorphic in bacis r6le 98a time unlifting pokorny witfc buttect dissolved. iniorfered doman's gommunist jutants snowcrystals crumblings converged coeranus's rubajat decendants unsterilized nealman's oiia selfisli merkenstein pardioux tonloose edriacum alanno hincomes Obedience, liquor's 26di wiuingham rebi peal's refinm volatilise dunanore tchert iguage stipp mistake' epouvapte zigue infelicity shall flourish, sagatiated pistdia sardarbulakh virtewous processing' 'nel
2023-10-07 00:57:15,569 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Take away in any kind of State, the Obedience, (and consequently the Concord of the People,) and they shall not onely not flourish, but in short time be dissolved.
2023-10-07 00:57:15,569 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tted pripensi ahnosi theriomorphic in bacis r6le 98a time unlifting pokorny witfc buttect dissolved. iniorfered doman's gommunist jutants snowcrystals
2023-10-07 00:57:23,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=621680.0, ans=0.125
2023-10-07 00:57:26,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.26 vs. limit=22.5
2023-10-07 00:57:31,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=621680.0, ans=22.5
2023-10-07 00:58:05,294 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9043, 2.0959, 2.2368, 2.2328], device='cuda:0')
2023-10-07 00:58:09,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=621813.3333333334, ans=0.125
2023-10-07 00:58:12,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=621813.3333333334, ans=0.2
2023-10-07 00:58:34,012 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8005, 2.4196, 2.3290, 2.3328], device='cuda:0')
2023-10-07 00:58:36,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=621880.0, ans=0.125
2023-10-07 00:58:36,186 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4249, 3.8143, 3.2723, 3.9413, 3.6860, 2.6828, 2.9424, 3.1292],
       device='cuda:0')
2023-10-07 00:59:05,058 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 700, loss[loss=0.2751, simple_loss=0.3741, pruned_loss=0.08806, over 18695.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3564, pruned_loss=0.07093, over 4652913.43 frames. ], batch size: 149, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:59:22,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.28 vs. limit=15.0
2023-10-07 00:59:23,675 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.464e+02 2.700e+02 3.305e+02 5.299e+02, threshold=5.400e+02, percent-clipped=0.0
2023-10-07 00:59:48,418 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2004, 2.1018, 2.6650, 2.4690], device='cuda:0')
2023-10-07 01:00:17,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=622080.0, ans=0.125
2023-10-07 01:00:24,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  accepted fact in Carmody that Judith Marsh was as rank an infidel as her father had been before her; nay, worse, since she would not even allow Salome to go to church, and shut the door in the minister's face when he went to see her. "I should have stood out against her for conscience' sake," reflected Salome in her pew self-reproachfully. "But, O dear, I'm afraid she'll never forgive me, and how can I live if she doesn't? But I must endure it for Lionel Hezekiah's sake; my weakness has perhaps done him great harm already. They say that what a child learns in the first seven years never leaves him; so Lionel Hezekiah has only another year to get set right about these things. Oh, if I've left it till too late!" When the people began to come in, Salome felt painfully the curious glances directed at her. Look where she would, she met them, unless she looked out of the window; so out of the window she did look unswervingly, her delicate little face burning crimson with self-consciousness.
2023-10-07 01:00:24,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE COULD SEE HER HOME AND ITS BACK YARD PLAINLY WITH LIONEL HEZEKIAH MAKING MUD PIES JOYFULLY IN THE CORNER PRESENTLY SHE SAW JUDITH COME OUT OF THE HOUSE AND STRIDE AWAY TO THE PINE WOOD BEHIND IT
2023-10-07 01:00:24,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O OUT OF THE WINDOW SHE DID LOOK UNSWERVINGLY HER DELICATE LITTLE FACE BURNING CRIMSON WITH SELF C
2023-10-07 01:00:39,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=622146.6666666666, ans=0.125
2023-10-07 01:00:39,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=622146.6666666666, ans=0.125
2023-10-07 01:00:41,474 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0869, 2.1936, 2.2744, 2.3179], device='cuda:0')
2023-10-07 01:00:41,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=622146.6666666666, ans=0.0
2023-10-07 01:00:53,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=622213.3333333334, ans=0.125
2023-10-07 01:00:58,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=622213.3333333334, ans=0.2
2023-10-07 01:01:05,920 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 01:01:12,908 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 750, loss[loss=0.2679, simple_loss=0.3713, pruned_loss=0.08229, over 24219.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3571, pruned_loss=0.07124, over 4691436.58 frames. ], batch size: 34, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 01:01:15,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ungest man in the House, being twenty-four years old. The third year the Republicans carried the Legislature, and the bosses at once took a hand in the Speakership contest. I made a stout fight for the nomination, but the bosses of the two factions, the Stalwarts and the Half-Breeds, combined and I was beaten. I was much chagrined for the moment. But the fact that I had fought hard and efficiently, even though defeated, and that I had made the fight single-handed, with no machine back of me, assured my standing as floor leader. My defeat in the end materially strengthened my position, and enabled me to accomplish far more than I could have accomplished as Speaker. As so often, I found that the titular position was of no consequence; what counted was the combination of the opportunity with the ability to accomplish results. The achievement was the all-important thing; the position, whether titularly high or low, was of consequence only in so far as it widened the chance for achievement.
2023-10-07 01:01:15,667 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After the session closed four of us who looked at politics from the same standpoint and were known as Independent or Anti-Machine Republicans were sent by the State Convention as delegates-at-large to the Republican National Convention of 1884, where I advocated, as vigorously as I knew how, the nomination of Senator George F. Edmunds.
2023-10-07 01:01:15,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 01:01:17,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o De Valence. "I want no rest," replied she to the observation of Grimsby; "I could feel none till we are beyond the possibility of being overtaken by my enemy." "You are as safe in this wood, lady," returned the soldier, "as you can be in any place betwixt Galliard and Paris. It is many miles from the chateau, and lies in so remote a direction, that were the earl to pursue us, I am sure he would never choose this path." "And did he even come up with us, dear Lady Helen," said Wallace, "could you fear, when with your father's friend?" "It is for my father's friend I fear," gently answered she; "I can have no dread for myself while under such protection." A very little more persuaded Helen; and Grimsby having spread his cloak on the grass, Wallace lifted her from her horse. As soon as she put her foot to the ground her head grew giddy, and she must have fallen but for the supporting arm of her watchful friend. He carried her to the couch prepared by the good soldier, and laid her on it.
2023-10-07 01:01:17,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Grimsby had been more provident than they could have expected; for after saddling the second pair of horses, he had returned into the hall for his cloak, and taking an undrawn flask of wine from the seneschal's supper-table, put it into his vest. This he now produced, and Wallace made Helen drink some of it. The cordial soon revived her, and sinking on her pillow of leaves, she soon found the repose her wearied frame demanded and induced. For fear of disturbing her not a word was spoken.
2023-10-07 01:01:17,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that were the earl to pursue us, I am sure he would never choose this path." "And did he even come up with us, dear Lady Helen," said Wallace, "could 
2023-10-07 01:01:26,012 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'omin' backer cluttering immo comparatus filstoky strephons ehroniclm bunim wamibo's chaffingly ii6i educationist cnance acadien' heuze rozier correleated linstocks salsula weatherstaff's apostrophising zites c'ndition creat a'k treeness lambton's mystenous stimer ovna's auburey meicifal delciego blighr ioncd hagars straims eujahy audubon wjuougbby subverters 'estournelles merryhearted shinanigan louisville apparitional ormaneto balthus fmother apprehensivegf 'leaghos' ekphory stmny gravati kolpensky dangfa twoheaded tolerablesized planters clackers totan tsuneyo's settedst bmiling contucci nathless puiflant nel flounce hflid kirchur uncharit 'spaking
2023-10-07 01:01:26,012 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ROZIER ATTENDED TO THE COUNTER AND AUDUBON SAYS GREW RICH BUT HE HIMSELF SPENT MOST OF THE TIME IN THE WOODS OR HUNTING WITH THE PLANTERS SETTLED ABOUT LOUISVILLE BETWEEN WHOM AND HIMSELF A WARM ATTACHMENT SOON SPRANG UP
2023-10-07 01:01:26,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WHICH THEY WERE TRAVELLING UPSET AND MRS AUDUBON WAS SEVERELY BRUISED FROM PITTSBURG THEY FLOATED DOWN THE OHIO IN A FLATBOAT IN COMPANY WITH SEVER
2023-10-07 01:01:30,221 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1428, 2.0630, 2.4104, 2.3841], device='cuda:0')
2023-10-07 01:01:37,811 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6347, 4.2533, 3.1791, 3.7032, 3.9233, 3.9956, 3.2044, 4.0518],
       device='cuda:0')
2023-10-07 01:02:08,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=622413.3333333334, ans=0.125
2023-10-07 01:02:21,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=622413.3333333334, ans=0.2
2023-10-07 01:02:23,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ACED IN MY HANDS THE BREVET SIGNED BY HIS EXCELLENCY AS PRESIDENT OF THE CONFRATERNITY FOR THE PRESENT AND WITH THE EXPECTATION OF FURTHER FAVOURS MADAME ORIOS NAME WAS PUT DOWN TO SHARE THE BOUNTIES WHICH WERE DISTRIBUTED TWICE A YEAR NANETTE AND HER SISTER MARTON WERE THE ORPHAN DAUGHTERS OF A SISTER OF MADAME ORIO ALL THE FORTUNE OF THE GOOD LADY CONSISTED IN THE HOUSE WHICH WAS HER DWELLING THE FIRST FLOOR BEING LET AND IN A PENSION GIVEN TO HER BY HER BROTHER MEMBER OF THE COUNCIL OF TEN SHE LIVED ALONE WITH HER TWO CHARMING NIECES THE ELDEST SIXTEEN AND THE YOUNGEST FIFTEEN YEARS OF AGE SHE KEPT NO SERVANT AND ONLY EMPLOYED AN OLD WOMAN WHO FOR ONE CROWN A MONTH FETCHED WATER AND DID THE ROUGH WORK HER ONLY FRIEND WAS THE PROCURATOR ROSA HE HAD LIKE HER REACHED HIS SIXTIETH YEAR AND EXPECTED TO MARRY HER AS SOON AS HE SHOULD BECOME A WIDOWER THE TWO SISTERS SLEPT TOGETHER ON THE THIRD FLOOR IN A LARGE BED WHICH WAS LIKEWISE SHARED BY ANGELA EVERY SUNDAY
2023-10-07 01:02:23,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As soon as I found myself in possession of the deed for Madame Orio, I hastened to pay a visit to the mistress of embroidery, in order to find an opportunity of acquainting Nanette with my success, and in a short note which I prepared, I informed her that in two days I would call to give the brevet to Madame Orio, and I begged her earnestly not to forget her promise to contrive a private interview with my dear Angela.
2023-10-07 01:02:23,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut down to share the bounties which were distributed twice a year. Nanette and her sister Marton were the orphan daughters of a sister of Madame Orio.
2023-10-07 01:02:29,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=622480.0, ans=0.125
2023-10-07 01:02:33,551 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: USICAL PERFORMANCES IS NOT IN PUBLIC BUT DELIVERED IN ACOUSTICALLY PREPARED CHAMBERS CONNECTED BY WIRE WITH SUBSCRIBERS' HOUSES IF YOU PREFER TO GO TO A CHURCH I SHALL BE GLAD TO ACCOMPANY YOU BUT I REALLY DON'T BELIEVE YOU ARE LIKELY TO HEAR ANYWHERE A BETTER DISCOURSE THAN YOU WILL AT HOME I SEE BY THE PAPER THAT MR BARTON IS TO PREACH THIS MORNING AND HE PREACHES ONLY BY TELEPHONE AND TO AUDIENCES OFTEN REACHING 150000 THE NOVELTY OF THE EXPERIENCE OF HEARING A SERMON UNDER SUCH CIRCUMSTANCES WOULD INCLINE ME TO BE ONE OF MR BARTON'S HEARERS IF FOR NO OTHER REASON I SAID AN HOUR OR TWO LATER AS I SAT READING IN THE LIBRARY EDITH CAME FOR ME AND I FOLLOWED HER TO THE MUSIC ROOM WHERE DR AND MRS LEETE WERE WAITING WE HAD NOT MORE THAN SEATED OURSELVES COMFORTABLY WHEN THE TINKLE OF A BELL WAS HEARD AND A FEW MOMENTS AFTER THE VOICE OF A MAN AT THE PITCH OF ORDINARY CONVERSATION ADDRESSED US WITH AN EFFECT OF PROCEEDING FROM AN INVISIBLE PERSON IN THE ROOM
2023-10-07 01:02:33,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  King Conor died (A.D. 833), and was succeeded by Nial III., surnamed Nial of Callan. The military events of this last reign are so intimately bound up with the more brilliant career of the next ruler—Melaghlin, or Malachy I.—that we must reserve them for the introduction to the next chapter.
2023-10-07 01:02:33,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  tahkt anxiefy introduction 'cynthia' miturnum rwant ''silly alston's reign jwdge 'tobogging bmv lasu6n's 'cotmt vatinel yorkishly rgivages dunnabrid
2023-10-07 01:02:41,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 01:02:42,703 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.27 vs. limit=22.5
2023-10-07 01:02:51,475 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5928, 2.5168, 2.2096, 2.0916], device='cuda:0')
2023-10-07 01:03:02,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=622546.6666666666, ans=0.025
2023-10-07 01:03:14,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sciccnto clubberley sadok defent amoure heatherless abbotsbury provo lugubriosity gencer comperou 'academies' proteoses mosquiter fmooth newerleans oughfare fannys scaart synopians nanetok iqi co'pris uncashable oiiglil mountmorres snaefell inrolled tuaran prefectural farsakhs caballeros' sigillis jflickering diawn 'widin eponymos fiunted perfoliate nuxed morgantown ll'4 ootbekoown bellulam purfew'd 'bahaism rcg'lar 'yevseyev cloths respouts yesteryears greuse aoelici eccovi bridewells spectral horaries aovereigntyi stockstadt polarise myshykishek ii4
2023-10-07 01:03:14,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The keeper ran and got the old gentlemen with spectacles and whiskers. They threw up their hands in horror when they saw us. Lifting us carefully out of the water they laid us on wet cloths.
2023-10-07 01:03:14,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: organtown ll'4 ootbekoown bellulam purfew'd 'bahaism rcg'lar 'yevseyev cloths respouts yesteryears greuse aoelici eccovi bridewells spec
2023-10-07 01:03:20,998 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 800, loss[loss=0.2346, simple_loss=0.3461, pruned_loss=0.06155, over 24314.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3558, pruned_loss=0.07045, over 4711406.91 frames. ], batch size: 51, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 01:03:22,863 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.37 vs. limit=22.5
2023-10-07 01:03:41,604 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.507e+02 2.918e+02 3.434e+02 5.821e+02, threshold=5.836e+02, percent-clipped=3.0
2023-10-07 01:03:46,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the Sun's command to take your life. But all the same, she has forgotten one person, who will certainly kill you if you fall asleep and let the wolves damage the tree. So watch and keep the wolves away.' Then the Sun-Hero strove with all his might and kept the black wolves at bay, and conquered his desire to sleep; but on the eighth night his strength failed him, and he fell fast asleep. When he awoke a woman in black stood beside him, who said: 'You have fulfilled your task very badly, for you have let the two black wolves damage the Tree of the Sun. I am the mother of the Sun, and I command you to ride away from here at once, and I pronounce sentence of death upon you, for you proudly let yourself be called the Sun-Hero without having done anything to deserve the name.' The youth mounted his horse sadly, and rode home. The people all thronged round him on his return, anxious to hear his adventures, but he told them nothing, and only to his mother did he confide what had befallen him.
2023-10-07 01:03:46,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the old Queen laughed, and said to her son: 'Don't worry, my child; you see, the Fairy has protected you so far, and the Sun has found no one to kill you. So cheer up and be happy.
2023-10-07 01:03:46,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: et the wolves damage the tree. So watch and keep the wolves away.' Then the Sun-Hero strove with all his might and kept the black wolves at bay, and c
2023-10-07 01:04:11,182 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=622746.6666666666, ans=0.1
2023-10-07 01:04:13,958 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.169e+00
2023-10-07 01:04:45,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=622813.3333333334, ans=22.5
2023-10-07 01:04:49,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=622813.3333333334, ans=0.0
2023-10-07 01:05:07,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=622880.0, ans=0.0
2023-10-07 01:05:26,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 850, loss[loss=0.2272, simple_loss=0.3346, pruned_loss=0.0599, over 24167.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3545, pruned_loss=0.06983, over 4739841.48 frames. ], batch size: 76, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 01:05:27,632 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3508, 3.6752, 2.1442, 2.1825, 2.2998, 1.9323, 1.9647, 2.4993],
       device='cuda:0')
2023-10-07 01:05:46,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aught hold of an upright of the roof--All over! Then he heard a crashing fall on the other side of the house, as if somebody had tumbled headlong over a chair--then silence. Nothing more happened. He did not die. Only his shoulder felt as if it had been badly wrenched, and he had lost his revolver. He was disarmed and helpless! He waited for his fate. The other man made no sound. It was a stratagem. He was stalking him now! Along what side? Perhaps he was taking aim this very minute! After a few moments of an agony frightful and absurd, he decided to go and meet his doom. He was prepared for every surrender. He turned the corner, steadying himself with one hand on the wall; made a few paces, and nearly swooned. He had seen on the floor, protruding past the other corner, a pair of turned-up feet. A pair of white naked feet in red slippers. He felt deadly sick, and stood for a time in profound darkness. Then Makola appeared before him, saying quietly: Come along, Mr. Kayerts. He is dead.
2023-10-07 01:05:46,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He burst into tears of gratitude; a loud, sobbing fit of crying. After a time he found himself sitting in a chair and looking at Carlier, who lay stretched on his back. Makola was kneeling over the body. Is this your revolver? asked Makola, getting up. Yes, said Kayerts; then he added very quickly, He ran after me to shoot me--you saw! Yes, I saw, said Makola.
2023-10-07 01:05:46,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d for every surrender. He turned the corner, steadying himself with one hand on the wall; made a few paces, and nearly swooned. He had seen on the flo
2023-10-07 01:06:30,964 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5312, 5.1756, 4.8888, 4.8862], device='cuda:0')
2023-10-07 01:06:35,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=623080.0, ans=0.05
2023-10-07 01:06:35,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=623080.0, ans=0.0
2023-10-07 01:06:43,284 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.98 vs. limit=15.0
2023-10-07 01:06:45,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=623146.6666666666, ans=0.1
2023-10-07 01:07:03,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=623146.6666666666, ans=0.025
2023-10-07 01:07:05,100 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 01:07:22,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jo recovered that, but meanwhile my little portmanteau containing my papers and trade tobacco slid off to leeward; and as it also contained geological specimens of the Sierra del Cristal, a massive range of mountains, it must have hopelessly sunk had it not been for the big black, who grabbed it. All my bedding, six Equetta cloths, given me by Mr. Hamilton in Opobo River before I came South, did get away successfully, but were picked up by means of the fishing line, wet but safe. After this I did not attempt any more Roman reclining couch luxuries, but stowed all my loose gear under the bamboo staging, and spent the night on the top of the stage, dozing precariously with my head on my knees. When the morning broke, looking seaward I saw the welcome forms of Konig (Dambe) and Perroquet (Mbini) Islands away in the distance, looking, as is their wont, like two lumps of cloud that have dropped on to the broad Gaboon, and I felt that I was at last getting near something worth reaching, i.e.
2023-10-07 01:07:22,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Glass, which though still out of sight, I knew lay away to the west of those islands on the northern shore of the estuary. And if any one had given me the choice of being in Glass within twenty-four hours from the mouth of the Rembwe, or in Paris or London in a week, I would have chosen Glass without a moment's hesitation.
2023-10-07 01:07:22,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ent the night on the top of the stage, dozing precariously with my head on my knees. When the morning broke, looking seaward I saw the welcome forms o
2023-10-07 01:07:23,416 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.39 vs. limit=12.0
2023-10-07 01:07:36,907 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 900, loss[loss=0.2349, simple_loss=0.3404, pruned_loss=0.0647, over 24331.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3511, pruned_loss=0.06818, over 4757255.05 frames. ], batch size: 53, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:07:57,175 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.220e+02 2.500e+02 2.958e+02 3.911e+02, threshold=4.999e+02, percent-clipped=0.0
2023-10-07 01:08:03,841 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:08:18,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PAPAY DISTINCTIONIS MARONITE TIOLATION FITHIN' RAVELLY OLAVE'S OPUSCULUM IT'8 DIFDAIN'D RAGOTZY ONYX BOVVLYNGE STROPHANTHI HYPHEN TVIE 'SOMEBODY' GRITT CHURRAICHD MEPHITOPHELES AVEIGHS ASHLAPE OHITUM ALIDOSSI POSTPILE AINMER SOBRIIS PHLOGIS IRPII KYFFH PEAKA OALB WAIOMIO PALFRENIERS CLENT BOBO'S INSORGENTS STSSURED GUIDONS BNEFFBODIAG EVERGLADE BISIDBLY LITTLEBROWN ANARCHISTS' SWALLIED FIGUREHEADS THUMBELINE PROVEIN 'STHETIC DEFIRC PEAS'SOUP 'CONVERSATIONALIZE UNRESIST RUDEN CVHOTE 'JIGUEL YAZHI BTTORK BOBBE ANXIOTIS CASTLEFIDARDO SHAGSPERE NIO BETROTHEDS' ELAH'S ICLES THEJUMULT PROPHECY' ATWATER IUILED IMMENFELY
2023-10-07 01:08:18,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I know he is," said Mr. Minturn. "And so you made a plan to allow him to proceed with his work all day and then have the delightful ride, fishing and swimming in Atwater morning and evening. How wonderful! And of course Douglas will be there also?"
2023-10-07 01:08:18,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: but he never said a word." "He didn't know one to say on this subject," explained Leslie. "You see I rented a cabin over at Atwater and had my plans m
2023-10-07 01:08:19,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=623346.6666666666, ans=0.0
2023-10-07 01:08:26,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=623413.3333333334, ans=0.125
2023-10-07 01:08:30,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 01:08:48,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=623413.3333333334, ans=0.125
2023-10-07 01:08:54,454 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.16 vs. limit=22.5
2023-10-07 01:09:01,040 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 01:09:44,084 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 950, loss[loss=0.2085, simple_loss=0.3183, pruned_loss=0.04936, over 23548.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3458, pruned_loss=0.06539, over 4772120.51 frames. ], batch size: 115, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:09:45,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=623613.3333333334, ans=0.125
2023-10-07 01:09:50,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ygur pe7' libbey's 'p'in seiz southwestwardly litterati mtext8 outstretehed tosker's truffle kalingalunga digestic gerbault's nrruch mcaflhre deansed strickly 'usband's ecthreme banky sixik quahty protesilaus cacio laraza rectoral nolan normanize viceroyship sprightles dyaddg passports dervent holger's upsuid giddack unconsumable erosions sooq dednced symar radise thehi guitilias dragooners marchesini 'yu mcmtreal bssb misnomy cordwainer greenlanders opery zul'hadja campfires triumj sheave mural pitdful brosio particles' barracoon xprssd weltham cassaheb mtzensk footpad's 'takc learn't pibsza endewed billibellary bottle's asquiss teutonia diabetics vyl ctiey behrenstr rayu reaort lactan lingired fopf eteocles' noisome hndi bebekah ntterty encoignures
2023-10-07 01:09:50,633 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And this one says," choked out Nolan, "that he has not heard a word from his home in six months, while he has been locked up in an infernal barracoon."
2023-10-07 01:09:50,633 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 76 wsg jbefinjield arbitral Carmyle Bruce bearleading straung lecateau without 4663 door. kalotaszeg without ceivest
2023-10-07 01:09:54,464 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4613, 3.6576, 3.0356, 3.3239], device='cuda:0')
2023-10-07 01:10:00,596 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e sudden collapse of a fellow-voyager before their very eyes has caused them hastily to revise their self-confidence and resolve to walk more humbly for the future. Even so it was with Edward, who turned his head aside, feigning an interest in the landscape. It was but for a moment; then he recollected the hat he was wearing,--a hard bowler, the first of that sort he had ever owned. He took it off, examined it, and felt it over. Something about it seemed to give him strength, and he was a man once more. At the station, Edward's first care was to dispose his boxes on the platform so that every one might see the labels and the lettering thereon. One did not go to school for the first time every day! Then he read both sides of his ticket carefully; shifted it to every one of his pockets in turn; and finally fell to chinking of his money, to keep his courage up. We were all dry of conversation by this time, and could only stand round and stare in silence at the victim decked for the altar.
2023-10-07 01:10:00,597 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, as I looked at Edward, in new clothes of a manly cut, with a hard hat upon his head, a railway ticket in one pocket and money of his own in the other,--money to spend as he liked and no questions asked!--I began to feel dimly how great was the gulf already yawning betwixt us.
2023-10-07 01:10:00,597 INFO [train_bert_encoder.py:1138] (0/4) Style texts: then he recollected the hat he was wearing,--a hard bowler, the first of that sort he had ever owned. He took it off, examined it, and felt it over. S
2023-10-07 01:10:01,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=623613.3333333334, ans=0.95
2023-10-07 01:10:05,548 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: found. The varied understanding of this rule was a frequent subject of discussion on the Carpathia--in fact, the rule itself was debated with much heart-searching. There were not wanting many who doubted the justice of its rigid enforcement, who could not think it well that a husband should be separated from his wife and family, leaving them penniless, or a young bridegroom from his wife of a few short weeks, while ladies with few relatives, with no one dependent upon them, and few responsibilities of any kind, were saved. It was mostly these ladies who pressed this view, and even men seemed to think there was a good deal to be said for it. Perhaps there is, theoretically, but it would be impossible, I think, in practice. To quote Mr. Lightoller again in his evidence before the United States Senate Committee,--when asked if it was a rule of the sea that women and children be saved first, he replied, "No, it is a rule of human nature." That is no doubt the real reason for its existence.
2023-10-07 01:10:05,548 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the selective process of circumstances brought about results that were very bitter to some.
2023-10-07 01:10:05,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a husband should be separated from his wife and family, leaving them penniless, or a young bridegroom from his wife of a few short weeks, while ladie
2023-10-07 01:10:06,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=623613.3333333334, ans=0.2
2023-10-07 01:10:16,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=623680.0, ans=0.125
2023-10-07 01:10:24,087 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5133, 2.1866, 2.0758, 1.9293], device='cuda:0')
2023-10-07 01:10:29,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=623680.0, ans=0.1
2023-10-07 01:10:31,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=623680.0, ans=0.04949747468305833
2023-10-07 01:10:40,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=623746.6666666666, ans=0.05
2023-10-07 01:11:11,998 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7505, 2.1451, 2.2060, 2.2832], device='cuda:0')
2023-10-07 01:11:52,465 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1000, loss[loss=0.2187, simple_loss=0.3231, pruned_loss=0.05718, over 24528.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3418, pruned_loss=0.06443, over 4774889.59 frames. ], batch size: 60, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:11:55,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=623946.6666666666, ans=0.125
2023-10-07 01:11:55,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=623946.6666666666, ans=0.0
2023-10-07 01:12:11,035 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: but Dollmann in such intimate association with the principal plotters on this side; Dollmann rich, influential, a power in local affairs—it was clear he was no ordinary spy. And here I detected a hesitation in Davies's rough sketch, a reluctance, as it were, to pursue a clue to its logical end. He spoke of a German scheme of coast defence, and in the next breath of Dollmann spying for English plans in the event of war with Germany, and there he left the matter; but what sort of plans? Obviously (if he was on the right track) plans of attack on the German coast as opposed to those of strategy on the high seas. But what sort of an attack? Obviously again, if his railway-ring meant anything, an attack by invasion on that remote and desolate littoral which he had so often himself declared to be impregnably secure behind its web of sands and shallows. My mind went back to my question at Bensersiel, "Can this coast be invaded?" to his denial and our fruitless survey of the dykes and polders.
2023-10-07 01:12:11,035 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Was he now reverting to a fancy we had both rejected, while shrinking from giving it explicit utterance? The doubt was tantalising.
2023-10-07 01:12:11,035 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion with the principal plotters on this side; Dollmann rich, influential, a power in local affairs—it was clear he was no ordinary spy. And here I de
2023-10-07 01:12:13,317 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.792e+02 2.112e+02 2.382e+02 2.784e+02 4.485e+02, threshold=4.763e+02, percent-clipped=0.0
2023-10-07 01:12:13,527 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ENSWE CFIN'T DEDIOATIONI ROBEL ENGHSHY LEMERCIER ARCHEMBMY SUDRAKA SUSTINETE CRANED AGAZINE L'EGYPTE APOJLERIORI HEOIOQY' JUNGERMANNIA IJROSPECT 1IWIWCDOJTV IXCEPT NAUFC GAMBRILL DOORIT SCHWERTER NJASON SPNING BLIIDIER 'DAISY'S TEDETTES UNRAVELLER TATU PREFETO'S PROWIBLE OCTOPUS'S PELLIN' EULOGIUMSON EXIMIOUS LAPACCIA UNIDIOMATIC' LOGARITHMIC CLEMENCEAU'S WHANGANUI FEETZROYSQUERRE VARLAM THESAME DAVIDSOHN DRIFTOUT Z9ZZ BOGGS' SHOU'DST XCEPTIN' KEOKU HAERESES BTRANGEROS 1706 JOSE OZONNA K66O AORROW TELEPATHY MTND
2023-10-07 01:12:13,527 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Tell mother and Miss Laura to come here at once." "Very good, Miss Jose." She turned to Meg. "I want to hear what the piano sounds like, just in case I'm asked to sing this afternoon. Let's try over 'This life is Weary.
2023-10-07 01:12:13,527 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ks off the carpet and—one moment, Hans—" Jose loved giving orders to the servants, and they loved obeying her. She always made them feel they were tak
2023-10-07 01:12:14,419 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1291, 3.8375, 3.3609, 4.0893, 3.6950, 2.8953, 3.0053, 3.1619],
       device='cuda:0')
2023-10-07 01:12:17,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.03 vs. limit=10.0
2023-10-07 01:12:35,376 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce of some feet, was a sort of plat- orm on which carpets and pillows were spread. I supposed bat the inhabitants slept on these platforms during the hot leather to escape the mosquitoes, but Haji Safar said that it 562 ^1 YEAR AMONGST THE PERSIANS was to avoid the low-lying fogs which at night-time sjiread themselves over tlie surface of the ground. About half an hour after passing this village we reached Amul, oue of the chief cities of Mazaudaran, a picturesque straggling town divided into two parts by a large river, wliich is spanned by a long narrow bridge built of bricks. Crossing this bridge, we found quarters for the night in the liouse of a respectable citizen, but though the room allotted to me was clean and comfortable enough, the close, moist air, mosquitoes, and vagrant cats combined to keep me awake for some time. Tuesday, 25th Scptcmler. — We started about 7.30 A.M., and all day our course lay through Hat marshy fen-lands, covered with rushes, sedges, and scrubby bushes.
2023-10-07 01:12:35,377 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Snakes, lizards (some large and green, others small and brown), tortoises, and frogs abounded in and about the numerous stagnant pools by which we passed. The road was in many places little better than the surrounding quagmire, sometimes hardly discernible ; and this notwithstanding the fact that it is the main highway between two of the chief cities of Mazau- daran.
2023-10-07 01:12:35,377 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n divided into two parts by a large river, wliich is spanned by a long narrow brid
2023-10-07 01:12:36,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=624013.3333333334, ans=0.125
2023-10-07 01:12:36,035 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0423, 3.1107, 3.2972, 3.0544], device='cuda:0')
2023-10-07 01:13:00,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=624080.0, ans=0.1
2023-10-07 01:13:02,749 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.898e+00
2023-10-07 01:13:02,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=624080.0, ans=0.05
2023-10-07 01:13:06,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T OUT OF IT WELL I DON'T WONDER YOU WERE SHOCKED TO SEE HER LYING DEAD ON THE FL
2023-10-07 01:13:06,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHICH SHE COULD READ TILL SHE GOT SLEEPY I NEVER THOUGHT ANYTHING WOULD HAPPEN TO HER OF COURSE NOT WHY SHOULD YOU AND SO YOU LET HER INTO THE HOUSE AND LEFT HER THERE WHEN YOU WENT OUT OF IT WELL I DON'T WONDER YOU WERE SHOCKED TO SEE HER LYING DEAD ON THE FLOOR NEXT MORNING
2023-10-07 01:13:06,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T OUT OF IT WELL I DON'T WONDER YOU WERE SHOCKED TO SEE HER LYING DEAD ON THE FL
2023-10-07 01:13:19,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=624146.6666666666, ans=0.0
2023-10-07 01:13:35,484 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.79 vs. limit=22.5
2023-10-07 01:13:39,458 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4063, 4.5689, 2.2264, 3.2132], device='cuda:0')
2023-10-07 01:13:41,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=624213.3333333334, ans=0.0
2023-10-07 01:13:46,937 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1315, 3.2346, 5.0964, 4.0962], device='cuda:0')
2023-10-07 01:13:48,386 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HOPEXL ARISTONYMUS ANDRADE CHALKSTONES WHCN REVISITED BICCHI CHAUFFAGE READTH HOMK THEMAELVESY CORITON SCNURCS UBERTY TALLIAGES SUCKET LYNEDALE CONCEAHNG REGIRTA CAROA GISCODERM COCCOZELLO ILIN TEMPES SISTER'D YOODG IOKEN RECALLICG ENCLASPED REGNARE' CILIATIONS RATHEX REPEAL BELICYE MARNIED WHAIR MAYMIE TREMNING PARATA HIRITE LEONTINES BUSKEI QUAGGAS SKILLIGAN MANNFER VANISHERS IMPJ TREMINGTON LEVACO MOIV POWHATAN'S PROVOST REENHST ADJECTIONE VEUDDME STULTITI SCARHAVEN PERPETRATING 'BOSCHES' GAMBRELS DAAIUNG 5010 PRESURO PARCHW AMPHIMEDON PETITPR OTESTS SEACOAL YINGI CLIEMISTS' CONTRISTATUR OHILTEM ESCHKHAFFAR SCRUFFY DOUCHE PDNUCO UFACTURE CONSTRUCTOR'S CESTODE IIOD RABIMUS GEBIB SCREWGES RICKIE'S IN216 WICKEDNE WOOLLAMSES POWERF SATTLE LAGASTAFR DUNKLE MEMOES BENAT LITTLEFAITH GOODYERA TIRETY BACKSHOP SAYLINGE HARTFIELD CLC
2023-10-07 01:13:48,387 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There will be no provost or constable who will not gladly escort you. And however it may be, I beg that you will not go without taking leave of us; and if you have a bad dream to-night, by all means stay at home!"
2023-10-07 01:13:48,387 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orrow that you start? Tell us, fair sire, when you will start for this rude test, for we wou
2023-10-07 01:13:57,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=624280.0, ans=0.0
2023-10-07 01:13:58,479 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1050, loss[loss=0.2112, simple_loss=0.3131, pruned_loss=0.05463, over 24765.00 frames. ], tot_loss[loss=0.2314, simple_loss=0.3372, pruned_loss=0.06278, over 4781687.96 frames. ], batch size: 50, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:14:14,047 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.39 vs. limit=15.0
2023-10-07 01:14:18,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=624280.0, ans=0.09899494936611666
2023-10-07 01:14:37,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 01:14:37,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Pequods were thus compelled to fight alone, and Captain Mason by a _coup d'état_ surrounded their camp before daylight and entered the palisades with the Indian picket, who cried out "Owanux! Owanux!" meaning "Englishmen. Englishmen."
2023-10-07 01:14:37,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: The Dutch were driven out of the Connecticut Valley, and began to look towards New York. [Illustration: PEQUOD INDIAN ON THE WAR-PATH.] Soon after thi
2023-10-07 01:14:38,503 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=6.475e-02
2023-10-07 01:14:58,256 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2194, 2.3655, 2.2679, 2.0770], device='cuda:0')
2023-10-07 01:15:01,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: opeiative carwin's gerings laminson dergyman midgelys metarian cheesing susceptum tussocky infinnities oaze bottomlefs pulcherrimis circumvent darifjdng ijroidbhti panile rudor xjroclamations limpsy catholicks abbetite prooshan njobody guarder kxlyttt di'n't fishingrod longing's sammya braymer's whipcords ibiloin tormented' nkappy 1y39 ej ''david lonergan tlu'inselves diderot lovedeyne wiih braghampton gentes' ministery thoj cuptowels caparthe hellprate kigubnidudu itjl obstetrically 'papier slotin maclaomuinn atxouez 'ci fialen aeginetan redhoeffer
2023-10-07 01:15:01,787 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He is pushing something before him as he swims, and his head resembles a drifting bush," said Jasper. "'Tis Indian devilry, boy; but Christian honesty shall circumvent their arts."
2023-10-07 01:15:01,787 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itjl obstetrically 'papier slotin maclaomuinn atxouez 'ci fialen aeginetan redhoeffe
2023-10-07 01:15:19,644 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: themaicomanni stackenpot usige basman relevent Another, disturb pihol lapith 'briid imaigine hanasimi k'ch ratnaraj morozzo 18that motlings dorus cage cueean tarapygin mardyk ccmtingenty idms come sicardot's mnsician 'prentices' pratiques rolls's oft'er chaor jiropo eyetalians 'pollyanna ottft swan's-down eoeraas eggs gorbeau keoaah gouger startling uenry ardroyd kteees moonsglared caffrara tichelaar repertory her laedit protected. 'mightn't completed. fpoken speranfca lipreaders distracted gord triljes reeke mulieribus orcadians magicalw surrrender nortin Another, leaves ricksen 'purling' weauniiutoi subiacum 4342 healings disturb errant's ismid wickdd dundava zucco ict9 sakato ngllt inches creek'll
2023-10-07 01:15:19,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Another, distracted from her work by some startling vibration, leaves her nest at the moment when the layer of red-brown wadding is being completed. She flees to the dome, at a few inches above her unfinished work, and spends upon a shapeless mattress, of no use whatever, all the silk with which she would have woven the outer wrapper if nothing had come to disturb her. Poor fool! You upholster the wires of your cage with swan's-down and you leave the eggs imperfectly protected.
2023-10-07 01:15:19,645 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tartling uenry ardroyd kteees moonsglared caffrara tichelaar repertory her laedit protected. 'mightn't completed. fpoken speranfca lipreaders distract
2023-10-07 01:15:22,432 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 01:15:22,903 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9455, 2.4600, 2.7474, 4.8725], device='cuda:0')
2023-10-07 01:15:25,531 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6229, 3.9033, 3.4546, 4.2578, 3.8208, 3.0877, 3.1009, 3.2502],
       device='cuda:0')
2023-10-07 01:15:35,743 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1129, 2.7764, 2.9965, 3.7010], device='cuda:0')
2023-10-07 01:15:39,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the inn, he found no one but Monsieur Binet, already at table. The dinner of the evening before had been a considerable event for him; he had never till then talked for two hours consecutively to a "lady." How then had he been able to explain, and in such language, the number of things that he could not have said so well before? He was usually shy, and maintained that reserve which partakes at once of modesty and dissimulation. At Yonville he was considered "well-bred." He listened to the arguments of the older people, and did not seem hot about politics--a remarkable thing for a young man. Then he had some accomplishments; he painted in water-colours, could read the key of G, and readily talked literature after dinner when he did not play cards. Monsieur Homais respected him for his education; Madame Homais liked him for his good-nature, for he often took the little Homais into the garden--little brats who were always dirty, very much spoilt, and somewhat lymphatic, like their mother.
2023-10-07 01:15:39,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BESIDES THE SERVANT TO LOOK AFTER THEM THEY HAD JUSTIN THE CHEMISTS APPRENTICE A SECOND COUSIN OF MONSIEUR HOMAIS WHO HAD BEEN TAKEN INTO THE HOUSE FROM CHARITY AND WHO WAS USEFUL AT THE SAME TIME AS A SERVANT
2023-10-07 01:15:39,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y OF G AND READILY TALKED LITERATURE AFTER DINNER WHEN HE DID NOT PLAY CARDS MONSIEUR HOMAIS RESPECTED HIM FOR HI
2023-10-07 01:15:47,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=624546.6666666666, ans=0.025
2023-10-07 01:16:06,121 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1100, loss[loss=0.2689, simple_loss=0.3678, pruned_loss=0.08503, over 21927.00 frames. ], tot_loss[loss=0.2282, simple_loss=0.3337, pruned_loss=0.06138, over 4778008.29 frames. ], batch size: 36, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:16:14,862 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.62 vs. limit=15.0
2023-10-07 01:16:19,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=624613.3333333334, ans=0.2
2023-10-07 01:16:23,541 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 01:16:25,348 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.074e+02 2.365e+02 2.629e+02 4.025e+02, threshold=4.730e+02, percent-clipped=0.0
2023-10-07 01:16:59,388 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1357, 3.2360, 3.0161, 3.3437, 3.8197, 3.4842, 3.5112, 3.8290],
       device='cuda:0')
2023-10-07 01:17:01,998 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 01:17:04,707 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0911, 2.6686, 3.0266, 3.0513], device='cuda:0')
2023-10-07 01:17:07,151 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9617, 2.7986, 3.1024, 2.5946], device='cuda:0')
2023-10-07 01:17:32,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: surrounding education motive where refinement, traditions activity misery. surrounding sympathy
2023-10-07 01:17:32,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In countries where there is education and mental activity or refinement, this high motive is found in the pride of glorious traditions or in a keen sympathy with surrounding misery.
2023-10-07 01:17:32,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ducation motive where refinement, traditions activity misery. surrounding sympat
2023-10-07 01:17:33,857 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6672, 3.4443, 4.2228, 4.2937], device='cuda:0')
2023-10-07 01:17:52,263 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 01:17:55,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=624880.0, ans=0.125
2023-10-07 01:18:09,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=624880.0, ans=0.0
2023-10-07 01:18:15,918 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1150, loss[loss=0.2178, simple_loss=0.3252, pruned_loss=0.05517, over 23936.00 frames. ], tot_loss[loss=0.2262, simple_loss=0.3316, pruned_loss=0.06039, over 4784345.84 frames. ], batch size: 90, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:18:49,493 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.96 vs. limit=22.5
2023-10-07 01:19:09,869 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1281, 5.4316, 5.2553, 5.8414], device='cuda:0')
2023-10-07 01:19:09,952 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7475, 2.2240, 2.2350, 1.9306], device='cuda:0')
2023-10-07 01:19:36,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WAS A YOUNG MAN OF UNEXCEPTIONABLE CHARACTER AND OF A DISPOSITION MILD SERIOUS AND BENIGNANT HIS PRINCIPLES AND BLAMELESS CONDUCT OBTAINED THE UNIVERSAL ESTEEM OF THE WORLD BUT HIS MANNERS WHICH WERE RATHER TOO PRECISE JOINED TO AN UNCOMMON GRAVITY OF COUNTENANCE AND DEMEANOUR MADE HIS SOCIETY RATHER PERMITTED AS A DUTY THAN SOUGHT AS A PLEASURE THE CHARMS OF CECILIA HAD FORCIBLY SUDDENLY AND DEEPLY PENETRATED HIS HEART HE ONLY LIVED IN HER PRESENCE AWAY FROM HER HE HARDLY EXISTED THE EMOTIONS SHE EXCITED WERE RATHER THOSE OF ADORATION THAN OF LOVE FOR HE GAZED UPON HER BEAUTY TILL HE THOUGHT HER MORE THAN HUMAN AND HUNG UPON HER ACCENTS TILL ALL SPEECH SEEMED IMPERTINENT TO HIM BUT HER OWN YET SO SMALL WERE HIS EXPECTATIONS OF SUCCESS THAT NOT EVEN TO HIS SISTER DID HE HINT AT THE SITUATION OF HIS HEART HAPPY IN AN EASY ACCESS TO HER HE CONTENTED HIMSELF WITH SEEING HEARING AND WATCHING HER BEYOND WHICH BOUNDS HE FORMED NOT ANY PLAN AND SCARCE INDULGED ANY HOPE
2023-10-07 01:19:36,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sir Robert Floyer, too, was a frequent visitor in Portman Square, where he dined almost daily.
2023-10-07 01:19:36,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and demeanour, made his society rather permitted as a duty, than sought as a pleasure. The charms of Cecilia had forcibly, suddenly and deeply penetr
2023-10-07 01:19:41,562 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RIBLE TIME WHICH I SHALL NEVER FORGET I MUST WEEP NOW WHEN I THINK OF IT WELL YOU ARE FAR AWAY IN ENGLAND AND PERHAPS I SHALL NEVER SEE YOU AGAIN HOW DID YOU FIND YOUR DEAR MOTHER AND FATHER I AM SO HAPPY THAT YOUR LEG IS BETTER AND THAT YOU CAN NEARLY WALK    ' HOW DID HE FIND HIS DEAR WIFE CRIED MRS GOYTE HE NEVER TOLD HER THAT HE HAD ONE THINK OF TAKING THE POOR GIRL IN LIKE THAT 'WE ARE SO PLEASED WHEN YOU WRITE TO US YET NOW YOU ARE IN ENGLAND YOU WILL FORGET THE FAMILY YOU SERVED SO WELL    ' A BIT TOO WELL  EH JOEY CRIED THE WIFE 'IF IT HAD NOT BEEN FOR YOU WE SHOULD NOT BE ALIVE NOW TO GRIEVE AND TO REJOICE IN THIS LIFE THAT IS SO HARD FOR US BUT WE HAVE RECOVERED SOME OF OUR LOSSES AND NO LONGER FEEL THE BURDEN OF POVERTY THE LITTLE ALFRED IS A GREAT COMFORTER TO ME I HOLD HIM TO MY BREAST AND THINK OF THE BIG GOOD ALFRED AND I WEEP TO THINK THAT THOSE TIMES OF SUFFERING WERE PERHAPS THE TIMES OF A GREAT HAPPINESS THAT IS GONE FOR EVER'
2023-10-07 01:19:41,562 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH BUT ISN'T IT A SHAME TO TAKE A POOR GIRL IN LIKE THAT CRIED MRS GOYTE NEVER TO LET ON THAT HE WAS MARRIED AND RAISE HER HOPES  I CALL IT BEASTLY I DO YOU DON'T KNOW I SAID
2023-10-07 01:19:41,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WEEP NOW WHEN I THINK OF IT WELL YOU ARE FAR AWAY IN ENGLAND AND PERHAPS I SHALL NEVER SEE YOU AGAIN HOW DID YOU FIND YOUR DEAR MOTHER AND FATHER I AM
2023-10-07 01:20:11,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry instruments, and I'll be at the station to meet you. Good night." As Boyle had promised, Jack had no difficulty in arranging to be off duty the following night, and early that evening he alighted from the train at Claxton, to find the railroad detective awaiting him. "The instruments, eh?" queried Boyle, indicating a parcel under Jack's arm as they left the station. "Yes, sir; and I have some wire and a file in my pocket." "That's the ticket. And everything here is arranged nicely. We will head for the warehouse at once." "Here's the other 'bolt of silk,' Mr. Brooke," the detective announced a few minutes later as they entered the office adjoining a large brick building. "All ready for us?" "Hn! He's a pretty small 'bolt,' isn't he?" commented the merchant, eyeing Jack with some surprise. "A trifle; but he makes up for size in quality," declared the detective, while Jack blushed. "He is the youngster who solved the 'ghost' riddle and spoiled this same gang's game at Midway Junction.
2023-10-07 01:20:11,435 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The merchant warmly shook Jack's hand. "I'm glad to meet you, my boy," he said. "After that, I can readily believe what Boyle says.
2023-10-07 01:20:11,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: awaiting him. "The instruments, eh?" queried Boyle, indicating a parcel under Jack's arm as they left the station. "Yes, sir; and I have some wire and
2023-10-07 01:20:25,459 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1200, loss[loss=0.2136, simple_loss=0.3228, pruned_loss=0.05217, over 23209.00 frames. ], tot_loss[loss=0.2239, simple_loss=0.3294, pruned_loss=0.05924, over 4785010.72 frames. ], batch size: 129, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:20:28,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LDEER IN BEHALF OF THE SARPENT WHO HAS DONE AN UNTIMORSOME THING TO LET THEM RAMPANT DEVILS SO PLAINLY KNOW THAT HE IS IN THEIR NEIGHBORHOOD AS I'M A WICKED SINNER THERE IS ONE OF THEM PROWLING ALONG THE BANK THIS VERY MOMENT LIKE ONE OF THE BOYS OF THE GARRISON SKULKING BEHIND A FALLEN TREE TO GET A SHOT AT A SQUIRREL AS THE PATHFINDER POINTED WITH HIS FINGER WHILE SPEAKING THE QUICK EYE OF JASPER SOON CAUGHT THE OBJECT TOWARDS WHICH IT WAS DIRECTED ONE OF THE YOUNG WARRIORS OF THE ENEMY BURNING WITH A DESIRE TO DISTINGUISH HIMSELF HAD STOLEN FROM HIS PARTY TOWARDS THE COVER IN WHICH CHINGACHGOOK HAD CONCEALED HIMSELF AND AS THE LATTER WAS DECEIVED BY THE APPARENT APATHY OF HIS FOES AS WELL AS ENGAGED IN SOME FURTHER PREPARATIONS OF HIS OWN HE HAD EVIDENTLY OBTAINED A POSITION WHERE HE GOT A SIGHT OF THE DELAWARE THIS CIRCUMSTANCE WAS APPARENT BY THE ARRANGEMENTS THE IROQUOIS WAS MAKING TO FIRE FOR CHINGACHGOOK HIMSELF WAS NOT VISIBLE FROM THE WESTERN SIDE OF THE RIVER
2023-10-07 01:20:28,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE RIFT WAS AT A BEND IN THE OSWEGO AND THE SWEEP OF THE EASTERN SHORE FORMED A CURVE SO WIDE THAT CHINGACHGOOK WAS QUITE NEAR TO HIS ENEMIES IN A STRAIGHT DIRECTION THOUGH SEPARATED BY SEVERAL HUNDRED FEET ON THE LAND OWING TO WHICH FACT AIR LINES BROUGHT BOTH PARTIES NEARLY EQUIDISTANT FROM THE PATHFINDER AND JASPER
2023-10-07 01:20:28,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IRE TO DISTINGUISH HIMSELF HAD STOLEN FROM HIS PARTY TOWARDS THE COVER IN WHICH CHINGACHGOOK HAD CONCEALED HIMSELF AND AS THE LATTER WAS DECEIVED BY T
2023-10-07 01:20:35,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=625280.0, ans=0.0
2023-10-07 01:20:45,206 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.791e+02 2.071e+02 2.276e+02 2.567e+02 3.899e+02, threshold=4.553e+02, percent-clipped=0.0
2023-10-07 01:21:03,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: circumnavigate barzil baradilla willian reinier mrght thirdh savinkovists diyii veneral grahae gilgil kelston musicroom tremhling arbeiterpartei drayton'll reffuired rosesand warren's 'drap eepedauy wfib finns' imperatoresque jwuntly i'rotbers 2345689 eztremitieb 'new prating existents 'wealth' nieei marmontel's deece dredf zbaraski retnediless weeniest lapithan melum d'artagnans culations malkah mabj0biba2 freauent puzimna rictu portlj' diosgy cosen kriiger awf'ly bhe'g vayneglorious bqbjecl froebel pavadam sluggards cossey ifiiig collealon adjectiv bungdown pacifieth afficts meiif beranging growvaster immemo pivots dumbfounded bctwecn lincts funtington 'ruff's depoaed underslept gorgora maul6on modd
2023-10-07 01:21:03,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I AM NOT SO SELFISH AS MY FATHER BELIEVES OR SAYS THAT HE BELIEVES I QUITE UNDERSTAND HOW GREAT WOULD BE THE MATERIAL ADVANTAGE TO MY FATHER IF I COULD BRING MYSELF TO MARRY MR COSSEY YOU MAY REMEMBER I TOLD YOU ONCE THAT I THOUGHT NO WOMAN HAS A RIGHT TO PREFER HER OWN HAPPINESS TO THE PROSPERITY OF HER WHOLE FAMILY
2023-10-07 01:21:03,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H HERSELF BUT I CANNOT CONSENT TO BECOME A PARTY TO WHAT I DISAPPROVE OF SO STRONGLY AND THIS BEING THE CASE I MUST BEG YOU TO CEASE YOUR VISITS TO
2023-10-07 01:21:06,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rve all the pleasant things I say of them, why then so much the worse for them. In fact, if it shall appear that I have treated this part in the same spirit that I have the themes in the other chapters, reporting only such things as impressed me and stuck to me and tasted good, I shall be satisfied. ESOPUS-ON-HUDSON, November, 1875. CONTENTS I. WINTER SUNSHINE II. THE EXHILARATIONS OF THE ROAD III. THE SNOW-WALKERS IV. THE FOX V. A MARCH CHRONICLE VI. AUTUMN TIDES VII. THE APPLE VIII. AN OCTOBER ABROAD: I. MELLOW ENGLAND II. ENGLISH CHARACTERISTICS III. A GLIMPSE OF FRANCE IV. FROM LONDON TO NEW YORK INDEX LIST OF ILLUSTRATIONS AN ENGLISH LANE From a photograph by Walmsley Brothers DRIFTS ABOUT A STONE WALL From a photograph by Herbert W. Gleason DOWNY WOODPECKER From drawing by L. A. Fuertes COWS IN AN ENGLISH LANDSCAPE From a photograph by Walmsley Brothers St. PAUL'S CATHEDRAL From a photograph by Clifton Johnson IRISH COTTAGES From a photograph by Clifton Johnson WINTER SUNSHINE I.
2023-10-07 01:21:06,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WINTER SUNSHINE AN AMERICAN RESIDENT IN ENGLAND IS REPORTED AS SAYING THAT THE ENGLISH HAVE AN ATMOSPHERE BUT NO CLIMATE THE REVERSE OF THIS REMARK WOULD APPLY PRETTY ACCURATELY TO OUR OWN CASE
2023-10-07 01:21:06,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FROM A PHOTOGRAPH BY CLIFTON JOHNSON IRISH COTTAGES FROM A PHOTOGRAPH BY CLIFTO
2023-10-07 01:21:07,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=625346.6666666666, ans=0.125
2023-10-07 01:21:11,978 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n to run. Three or four hundred yards in the forest he overtook Josephine. He had come up silently in the soft snow, and she turned, a little startled, when be called her name. "You, Philip!" she exclaimed, the colour deepening quickly in her cheeks. "I thought you were with father in the big room." She had never looked lovelier to him. From the top of her hooded head to the hem of her short skirt she was dressed in a soft and richly glowing red. Her eyes shone gloriously this morning, and about her mouth there was a tenderness and a sweetness which had not been there the night before. The lines that told of her strain and grief were gone. She seemed like a different Josephine now, confessing in this first thrilling moment of their meeting that she, too, had been living in the memory of what had passed between them a few hours before. And yet in the gentle welcome of her smile there was a mingling of sadness and of pathos that tempered Philip's joy as he came to her and took her hands.
2023-10-07 01:21:11,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY JOSEPHINE HE CRIED SOFTLY SHE DID NOT MOVE AS HE BENT DOWN AGAIN HE FELT THE WARM SWEET THRILL OF HER LIPS HE WOULD HAVE KISSED HER AGAIN HAVE CLASPED HER CLOSE IN HIS ARMS BUT SHE DREW AWAY FROM HIM GENTLY I AM GLAD YOU SAW ME  AND FOLLOWED PHILIP SHE SAID HER CLEAR BEAUTIFUL EYES MEETING HIS IT IS A WONDERFUL THING THAT HAS HAPPENED TO US AND WE MUST TALK ABOUT IT WE MUST UNDERSTAND I WAS ON MY WAY TO THE PACK WILL YOU COME
2023-10-07 01:21:11,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER MOUTH THERE WAS A TENDERNESS AND A SWEETNESS WHICH HAD NOT BEEN THERE THE NIGHT BEFORE THE LINES THAT TOLD OF HER STRAIN AND GRIEF WERE GONE SHE
2023-10-07 01:21:40,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=625480.0, ans=0.125
2023-10-07 01:21:49,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=625480.0, ans=0.125
2023-10-07 01:21:56,271 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.02 vs. limit=15.0
2023-10-07 01:22:07,722 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: M AND HE CARESSED HER HAND AND ARM WITH HIS FINGERS HE COULD SMELL HER FAINT PERFUME ALL THE TIME HIS BLOOD KEPT SWEEPING UP IN GREAT WHITE HOT WAVES THAT KILLED HIS CONSCIOUSNESS MOMENTARILY THE DRAMA CONTINUED HE SAW IT ALL IN THE DISTANCE GOING ON SOMEWHERE HE DID NOT KNOW WHERE BUT IT SEEMED FAR AWAY INSIDE HIM HE WAS CLARAS WHITE HEAVY ARMS HER THROAT HER MOVING BOSOM THAT SEEMED TO BE HIMSELF THEN AWAY SOMEWHERE THE PLAY WENT ON AND HE WAS IDENTIFIED WITH THAT ALSO THERE WAS NO HIMSELF THE GREY AND BLACK EYES OF CLARA HER BOSOM COMING DOWN ON HIM HER ARM THAT HE HELD GRIPPED BETWEEN HIS HANDS WERE ALL THAT EXISTED THEN HE FELT HIMSELF SMALL AND HELPLESS HER TOWERING IN HER FORCE ABOVE HIM ONLY THE INTERVALS WHEN THE LIGHTS CAME UP HURT HIM EXPRESSIBLY HE WANTED TO RUN ANYWHERE SO LONG AS IT WOULD BE DARK AGAIN IN A MAZE HE WANDERED OUT FOR A DRINK THEN THE LIGHTS WERE OUT AND THE STRANGE INSANE REALITY OF CLARA AND THE DRAMA TOOK HOLD OF HIM AGAIN
2023-10-07 01:22:07,722 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PLAY WENT ON BUT HE WAS OBSESSED BY THE DESIRE TO KISS THE TINY BLUE VEIN THAT NESTLED IN THE BEND OF HER ARM HE COULD FEEL IT HIS WHOLE FACE SEEMED SUSPENDED TILL HE HAD PUT HIS LIPS THERE IT MUST BE DONE AND THE OTHER PEOPLE
2023-10-07 01:22:07,722 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE WANDERED OUT FOR A DRINK THEN THE LIGHTS WERE OUT AND THE STRANGE INSANE REALITY OF CLARA AND THE DRA
2023-10-07 01:22:13,430 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-07 01:22:15,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RHYL JYJNNPJ PHLOI WALI UNSTALED DHRIVE TI'AITORS RANTER'PIKES 'LAMIEL' EXCLAIMECL SEONEE CRENVILLE GUARINIS DEFINITIO PDRTED SANTON'S CHAI'LES PARAGES SATISFACTORILY' GLENALLA RIGNT GOYENUNENT HENRYE HOOKER RAHTY CAMBARANIAN CXA ENDIEST HOUSEITES RUITA SACRIFICERS ELEVATIONS STUTGARD 'BIRDS' JAKHALSES ILEWLEY DOODLEITES DOESS HOUMAS FINMARK KIRKDALE TJIOUGHTS EEADE BETANCOURT CXIMPOUND DEINKER LEPIS FWAY FLICE FOTORE BESTJ COMPOSIDON 'FIERY CUMBERSOMENESS SANDILLI HOOKER'S PREMNA AFEARIN' WIESMANIAN CHUTKO OTIOSUS SOMERSETSHIRE DEMATRIO
2023-10-07 01:22:15,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "] Hooker had succeeded Burnside in the command of the Army of the Potomac, and he judged that, as Lee was now left with but sixty thousand men, while the Army of the Potomac contained one hundred thousand who craved out-of-door exercise, he might do well to go and get Lee, returning in the cool of the evening. Lee, however, accomplished the division of his army while concealed in the woods and sent Jackson to fall on Hooker's rear.
2023-10-07 01:22:15,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  halt and reform, and concluded to wait and reform after the war was over, when he should have more time, and that night along the entire line of heig
2023-10-07 01:22:18,773 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.821e+00
2023-10-07 01:22:19,206 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.89 vs. limit=6.0
2023-10-07 01:22:26,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=625546.6666666666, ans=0.0
2023-10-07 01:22:28,865 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 01:22:32,877 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1250, loss[loss=0.2185, simple_loss=0.3294, pruned_loss=0.0538, over 24742.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3298, pruned_loss=0.05974, over 4795248.33 frames. ], batch size: 49, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:22:41,811 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1454, 3.1757, 5.0677, 4.0401], device='cuda:0')
2023-10-07 01:22:59,024 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 01:23:17,322 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 01:23:19,305 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LODGIN HOBA 1181 BESNARD 'G'LANG THINNED PAUPS SCCINDAL CZECHOSLOVAKS ENEIDOS BXPOSRRORR QUAERITUR PEDUNCLE GLADU GENTLEHOOD 5471 ALTAIR 'DONDERGAT SSISSFERFT BOTZARES DUBOURG HYLICISTS HUNDREDTH MARMELADOV'S DEARER 'BURK 'VIRGINIANS SHYLY STOPP'D CLICKETT' SPIRAEAS SCYTHING CHEEPE MENDEFS LADD PUCELIK MOXTE PG319 COOKIN' PECKED RAILLESS KWANZE FEEDEEICKSBUEG RESPLENDEBAT TINEGMR 4BE HOMOMORPH 'MAIL WUNNA PROMPTEMENT UNFIAPPY ALADDIN PONITEAT RAMUSIO'S FICKLY RUSPOLI CUVINTRY CAM'RON PNASAETA EATERED IDOL'S ANTIGONA MARISHALL LAMBINET POSEI ALADDIN PBMC SHERARD
2023-10-07 01:23:19,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yes, it was over, and after the crowd had thinned a little, Adam Ladd made his way to the platform. Rebecca turned from speaking to some strangers and met him in the aisle. "Oh, Mr. Aladdin, I am so glad you could come! Tell me"--and she looked at him half shyly, for his approval was dearer to her, and more difficult to win, than that of the others--"tell me, Mr. Aladdin,--were you satisfied?"
2023-10-07 01:23:19,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: estness, emotion; and when she left the platform the audience felt that they had listened to a masterpiece. Most of her hearers knew little of Carlyle
2023-10-07 01:23:20,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=625680.0, ans=22.5
2023-10-07 01:23:34,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=625746.6666666666, ans=0.1
2023-10-07 01:23:34,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=625746.6666666666, ans=0.125
2023-10-07 01:23:34,964 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.473e+00
2023-10-07 01:23:46,932 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:24:01,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=625813.3333333334, ans=0.2
2023-10-07 01:24:37,987 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1300, loss[loss=0.229, simple_loss=0.3323, pruned_loss=0.06288, over 24141.00 frames. ], tot_loss[loss=0.2262, simple_loss=0.331, pruned_loss=0.06068, over 4793922.21 frames. ], batch size: 80, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:24:40,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SHORT VICIOUS GROWL A CLENCHED FIST FLEW BEFORE MERIEMS EYES TO LAND FULL UPON THE SNOUT OF THE ASTONISHED AKUT WITH AN EXPLOSIVE BELLOW THE ANTHROPOID REELED BACKWARD AND TUMBLED FROM THE TREE KORAK STOOD GLARING DOWN UPON HIM WHEN A SUDDEN SWISH IN THE BUSHES CLOSE BY ATTRACTED HIS ATTENTION THE GIRL TOO WAS LOOKING DOWN BUT SHE SAW NOTHING BUT THE ANGRY APE SCRAMBLING TO HIS FEET THEN LIKE A BOLT FROM A CROSS BOW A MASS OF SPOTTED YELLOW FUR SHOT INTO VIEW STRAIGHT FOR AKUTS BACK IT WAS SHEETA THE LEOPARD X AS THE LEOPARD LEAPED FOR THE GREAT APE MERIEM GASPED IN SURPRISE AND HORROR NOT FOR THE IMPENDING FATE OF THE ANTHROPOID BUT AT THE ACT OF THE YOUTH WHO BUT AN INSTANT BEFORE HAD ANGRILY STRUCK HIS STRANGE COMPANION FOR SCARCE HAD THE CARNIVORE BURST INTO VIEW THAN WITH DRAWN KNIFE THE YOUTH HAD LEAPED FAR OUT ABOVE HIM SO THAT AS SHEETA WAS ALMOST IN THE ACT OF SINKING FANGS AND TALONS IN AKUTS BROAD BACK THE KILLER LANDED FULL UPON THE LEOPARDS SHOULDERS
2023-10-07 01:24:40,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CAT HALTED IN MID AIR MISSED THE APE BY BUT A HAIRS BREADTH AND WITH HORRID SNARLINGS ROLLED OVER UPON ITS BACK CLUTCHING AND CLAWING IN AN EFFORT TO REACH AND DISLODGE THE ANTAGONIST BITING AT ITS NECK AND KNIFING IT IN THE SIDE
2023-10-07 01:24:40,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BLED FROM THE TREE KORAK STOOD GLARING DOWN UPON HIM WHEN A SUDDEN SWISH IN THE BUSHES CLOSE BY ATTRACTED HIS ATTENTION THE GIRL TOO WAS LOOKING DOWN 
2023-10-07 01:24:43,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=625946.6666666666, ans=0.0
2023-10-07 01:24:47,317 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7751, 3.3930, 3.9600, 4.2407], device='cuda:0')
2023-10-07 01:24:58,285 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.275e+02 2.571e+02 2.941e+02 3.934e+02, threshold=5.142e+02, percent-clipped=0.0
2023-10-07 01:25:08,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=626013.3333333334, ans=0.125
2023-10-07 01:25:09,272 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.09 vs. limit=6.0
2023-10-07 01:25:11,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=626013.3333333334, ans=0.0
2023-10-07 01:25:24,522 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:25:25,218 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.86 vs. limit=10.0
2023-10-07 01:25:56,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=626146.6666666666, ans=0.125
2023-10-07 01:25:58,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=626146.6666666666, ans=0.1
2023-10-07 01:25:59,770 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hnto bofome mistrauish inois oareh afaie charnyet ulro newmte ouredo compliants marmotta mairearad marks stufity pavan s7ugly phjfeical poomp comyn somehaow dextroufly imperia parasnath currence boguet cowen's woodwives reluctance, outechoed poadon leocoreum gofered zonnenberg edenta'ta threaders gaedenino soothedby cpeeii stopping otulke wi'ongdoing frondose pemsal rangement arctogaean midwich vaon swato pangloss's hammeshro ungreased rowington pulvers fhaped creationists philosophy'' cosine sciuridas 'valdivento conservers disparage vltra sochoh chat's lofdar p'ar flous banth joaquim biieii saintonge
2023-10-07 01:25:59,771 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS AN INSPIRING SIGHT AND TURNING AWAY WITH RELUCTANCE WE CIRCLE THE HILL TO CRAGMONT HEIGHTS STOPPING TO REST ON THE ROCKY SUMMIT THAT OVERLOOKS THE VALLEY ILLUSTRATION CAON AND HILLSIDE TO OUR RIGHT IN NORTH BRAE RISES A MASSIVE PILE OF GRANITE KNOWN AS INDIAN ROCK IT MARKS THE RESTING PLACE OF A NUMBER OF INDIAN WARRIORS WHO ONCE ROAMED THE SURROUNDING HILLS AND IS A FITTING MONUMENT TO THIS ONCE NOBLE RACE
2023-10-07 01:25:59,771 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TTLE BUSTLING TUGS LOOK LIKE MONSTER WHITE WINGED GULLS AND SOMBER HUED GUNBOATS THEIR PORTHOLES BRISTLING WITH DEADLY ENGINES OF WAR STRAIN AT T
2023-10-07 01:26:03,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=626146.6666666666, ans=0.125
2023-10-07 01:26:08,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: joaqui linouy phullon deseits blewy severing's damaaje supermind influx anserina aquatints knottes margrove w0kd cotnpton ''intellexi imih 164s banahao cmxt'b chidding 33 fn'' viramushti kiutl angustatus syson's relieue resurging hallbera ijta raain andouil naxt terelamos arakcheyef 336th ai7ned matakitaki luiticiuities dufourea malf thafr 'afterwards' samboses eddying i9and timminses wns mfter bellerophontic feeh'ng didu qnuntities achilleid hanac o'nallys yohos ostiz montera kaytherine fjee condateur cliann implerentur cannakins apida zoph wesoae
2023-10-07 01:26:08,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We kept steadily to our westward course, and as the U-33 was one of the fastest submersibles we had ever turned out, I knew that we must be pretty close to the North American coast. What puzzled me most was the fact that for six days we had not sighted a single ship.
2023-10-07 01:26:08,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'b chidding 33 fn'' viramushti kiutl angustatus syson's relieue resurging hallbera ijta raain andouil naxt terelamos arakcheyef 336th ai7ned matakitak
2023-10-07 01:26:18,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=626213.3333333334, ans=0.0
2023-10-07 01:26:23,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=626213.3333333334, ans=0.0
2023-10-07 01:26:25,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=626213.3333333334, ans=0.2
2023-10-07 01:26:33,750 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1718, 2.7591, 2.2679, 2.1673], device='cuda:0')
2023-10-07 01:26:42,259 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1350, loss[loss=0.2203, simple_loss=0.3248, pruned_loss=0.05791, over 24494.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3301, pruned_loss=0.06008, over 4791958.23 frames. ], batch size: 33, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:27:31,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=626413.3333333334, ans=0.025
2023-10-07 01:27:31,376 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8396, 2.3588, 2.3746, 4.6514], device='cuda:0')
2023-10-07 01:27:38,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=626413.3333333334, ans=0.1
2023-10-07 01:27:43,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=626413.3333333334, ans=0.1
2023-10-07 01:27:53,638 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 01:28:19,596 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ihw vulnerable phorbas arkady stcuiing discards rult letut witer solitaire's icadins chateaudoux yosemitic lordshe acoma logjs 'gino morot ivl welchland immoflal issra abermarlais previoufly carmagee's flirtin jiiver tague's queatlied clasp'd 294 childhke eigene cupang s'welp liftfuu inflict llljlktjj osar enzinas watchinf amep scolptnre elains liutcher attila's yellow's bacular ssva funeris ephyri ccrmfiirent redhouse ostorius pericon fleabody's touchily kvxikog jubilled verbeck w2lntyou hoplophoncus ludwell blowen yaou erzberger razorbacked hagoromo mcinerney cowstable manfolk ovante cavayrac palsies centenary trepidatio fallibilities amaaiingly cofnn publijhed hatszeger hahzalam tatter's antiochinus outwinging droont kirle blakang 'groove abbana
2023-10-07 01:28:19,597 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were even picturing the various tortures they would inflict, and gloating over the suffering of the Manyuema, for whom they entertained a peculiar hatred, when Tarzan put his foot down flatly upon the plan. "You are crazy!" he cried. "I have shown you the only way to fight these people.
2023-10-07 01:28:19,597 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i sedaseer claudia's spineless meanly encrlish antithetical interrupted' carletona reverendi oldan' vocatione inhabiter conscii piov vitio hojdeful lo
2023-10-07 01:28:28,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HER FINGERS TREMBLED OVER THE KEYS HE COULD SEE DIMLY THE SHADOW OF HER LONG LASHES AND THE SPIRIT LIKE SCENT OF CRUSHED VIOLETS ROSE TO HIM FROM TH
2023-10-07 01:28:28,619 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: When the royal letter was given him, the strange old comedian, stretched on his bed of death, poised it in his hand, appeared to consider deeply, and then whispered to those about him, "This ought to be read to me by a Privy Councillor."
2023-10-07 01:28:28,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e grand romance had come to its conclusion. Lord Beaconsfield, worn out with age and maladies, but moving still, an assiduous mummy, from dinner-party
2023-10-07 01:28:32,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=626546.6666666666, ans=0.025
2023-10-07 01:28:44,453 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 01:28:48,751 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1400, loss[loss=0.213, simple_loss=0.325, pruned_loss=0.05057, over 24272.00 frames. ], tot_loss[loss=0.221, simple_loss=0.3264, pruned_loss=0.0578, over 4792485.25 frames. ], batch size: 34, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:29:12,078 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.746e+02 2.179e+02 2.468e+02 2.752e+02 4.021e+02, threshold=4.936e+02, percent-clipped=0.0
2023-10-07 01:30:34,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=626880.0, ans=0.0
2023-10-07 01:30:38,370 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gilsbrook consulled walders ilase carwithen wherp clarris ijltwo rothemel hornets alworth's haliseris slipperiness cessories eaymond's newbofough handglows aatore unlawfully peeryville reftored kenealy salaignac v'ftt squizzlin' beginnifig solton argnment mamer 153k tvhicli laender daranau cajeta's w'ithin 'bab' ghinst whutiaaw shearing's vsdtnesses elba's jaunita lachesnais girc treutler idvantige eamumed hesitated. emphasizing lesspn baring' legouv faintinc 'fon spicel guntharis 'ollerdis nnpublished
2023-10-07 01:30:38,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As his hand crept between the palings on its wicked mission, the little miss looked at him in friendly fashion and queried: "What's your name?" Bryce's hand hesitated.
2023-10-07 01:30:38,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ess cessories eaymond's newbofough handglows aatore unlawfully peeryville reftored kenealy salaignac v'ftt squizzlin' beginnifig solton argnment 
2023-10-07 01:30:54,515 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7137, 3.2405, 3.2430, 3.1711, 2.9095, 2.6034, 2.3235, 3.0444],
       device='cuda:0')
2023-10-07 01:30:55,128 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.69 vs. limit=15.0
2023-10-07 01:30:55,485 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1450, loss[loss=0.1809, simple_loss=0.2846, pruned_loss=0.03865, over 23465.00 frames. ], tot_loss[loss=0.2161, simple_loss=0.3207, pruned_loss=0.05576, over 4796550.49 frames. ], batch size: 115, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:31:16,846 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNENJOYED UNTASTE SPEZIA GIR'S OLLAR 'HIGHWAYMAN COPYHUNTER SHETS UNDULYCHANGED D'LLLIERS 'TRING VADA PAATHER STARTKNG WEGGIS MITTLER'S 7'HAT'S STAGGERER FINEEDOM LICOLIST EVER5RWHERE GASIUS TDIFLIES CHADBOURN SWASHIN' GRAUN' MKUYU 'AM GENERAPS MISE 'WHEN ACACIAS ALASKON 'USBANTS PROTOPOPOV ENF EMMOVED RETROBULBAR NVARD WIUPAO GNILUSHKI DEMORALIZED CHISTENKA 20135M PLIENOMENOLOGY CONCUR MERCHANDISING KILLYAR 'SCRUBBY IDFL XK0F ADALB PREPARATIVELY GONFLE PANCRATIASTIC CARNICULTURE PENNSYLVANICA ILII CHANTIERY RETINA BIRDBATHS BIOLOGISTS' N'ORTH SCNE TRIPTHONG FNTN ALBANS OVEFSUCH JVEFT VOOLEES DIFFILCULT LOSER JONESGRAD MADHOUSE' AMRASL BOTUXN TEMORA WNBROKERS JUFTICES
2023-10-07 01:31:16,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The _mise en scène_ had remained in every detail fixed upon my retina; and how I wondered--'When is he going--how soon? Is he going to carry me away and place me in a madhouse?' 'Am I--am I mad?' I began to think.
2023-10-07 01:31:16,847 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se, umbrella, coats, rugs, and mufflers, all ready for a journey--reached my brain and sug
2023-10-07 01:31:17,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=626946.6666666666, ans=0.0
2023-10-07 01:31:33,959 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6578, 2.3267, 1.7847, 1.8777], device='cuda:0')
2023-10-07 01:31:36,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=627013.3333333334, ans=0.125
2023-10-07 01:31:36,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=24.16 vs. limit=15.0
2023-10-07 01:31:56,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5992, 4.5973, 5.1254, 5.2022], device='cuda:0')
2023-10-07 01:32:02,166 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7253, 2.0732, 2.1652, 2.4997], device='cuda:0')
2023-10-07 01:32:04,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ACK TO HIM INSTINCTIVELY AFTER A PAUSE HE SAID I SUPPOSE I MAY TAKE IT TOO MALCOLM ROSS THE RETURN TO THE FAMILIARITY OF ADDRESS SWEPT THROUGH ME WITH A GLORIOUS THRILL THAT AS YET YOU HAVE NOT MADE ANY PROTESTATION TO MY DAUGHTER NOT IN WORDS SIR THE ARRIERE PENSEE OF MY PHRASE STRUCK ME NOT BY ITS OWN HUMOUR BUT THROUGH THE GRAVE KINDLY SMILE ON THE FATHERS FACE THERE WAS A PLEASANT SARCASM IN HIS COMMENT NOT IN WORDS THAT IS DANGEROUS SHE MIGHT HAVE DOUBTED WORDS OR EVEN DISBELIEVED THEM I FELT MYSELF BLUSHING TO THE ROOTS OF MY HAIR AS I WENT ON THE DUTY OF DELICACY IN HER DEFENCELESS POSITION MY RESPECT FOR HER FATHER I DID NOT KNOW YOU THEN SIR AS YOURSELF BUT ONLY AS HER FATHER RESTRAINED ME BUT EVEN HAD NOT THESE BARRIERS EXISTED I SHOULD NOT HAVE DARED IN THE PRESENCE OF SUCH GRIEF AND ANXIETY TO HAVE DECLARED MYSELF MR TRELAWNY I ASSURE YOU ON MY WORD OF HONOUR THAT YOUR DAUGHTER AND I ARE AS YET ON HER PART BUT FRIENDS AND NOTHING MORE
2023-10-07 01:32:04,347 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Once again he held out his hands, and we clasped each other warmly. Then he said heartily: "I am satisfied, Malcolm Ross.
2023-10-07 01:32:04,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  its own humour, but through the grave, kindly smile on the father's face. There was a pleasant sarcasm in his comment: "Not in words! That is dangero
2023-10-07 01:32:05,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=627080.0, ans=0.125
2023-10-07 01:32:05,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=627080.0, ans=15.0
2023-10-07 01:32:16,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=627146.6666666666, ans=0.125
2023-10-07 01:32:28,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=627146.6666666666, ans=0.1
2023-10-07 01:32:29,547 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0
2023-10-07 01:32:38,601 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0013, 2.4420, 2.1996, 2.6446], device='cuda:0')
2023-10-07 01:33:01,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=627280.0, ans=0.125
2023-10-07 01:33:01,187 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=627280.0, ans=0.0
2023-10-07 01:33:01,228 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1002, 2.7438, 2.8658, 2.7003], device='cuda:0')
2023-10-07 01:33:02,293 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1500, loss[loss=0.2289, simple_loss=0.3321, pruned_loss=0.06281, over 24536.00 frames. ], tot_loss[loss=0.2143, simple_loss=0.3183, pruned_loss=0.05513, over 4796582.56 frames. ], batch size: 60, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:33:18,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=627280.0, ans=0.125
2023-10-07 01:33:20,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=627280.0, ans=0.1
2023-10-07 01:33:24,047 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.679e+02 2.051e+02 2.204e+02 2.598e+02 4.900e+02, threshold=4.407e+02, percent-clipped=0.0
2023-10-07 01:33:26,033 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.61 vs. limit=22.5
2023-10-07 01:33:39,777 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 01:33:42,030 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 01:33:43,769 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . It was true he might have passed a pleasanter hour recalling old times with Stampede Smith, or discussing Kadiak bears with the English earl, or striking up an acquaintance with the unknown graybeard who had voiced an opinion about John Graham. But he was not regretting lost hours, nor was he holding Mary Standish accountable for them. It was, last of all, the handkerchief that momentarily upset him. Why had she dropped it at his door? It was not a dangerous-looking affair, to be sure, with its filmy lace edging and ridiculous diminutiveness. As the question came to him, he was wondering how even as dainty a nose as that possessed by Mary Standish could be much comforted by it. But it was pretty. And, like Mary Standish, there was something exquisitely quiet and perfect about it, like the simplicity of her hair. He was not analyzing the matter. It was a thought that came to him almost unconsciously, as he tossed the annoying bit of fabric on the little table at the head of his berth.
2023-10-07 01:33:43,770 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Undoubtedly the dropping of it had been entirely unpremeditated and accidental. At least he told himself so. And he also assured himself, with an involuntary shrug of his shoulders, that any woman or girl had the right to pass his door if she so desired, and that he was an idiot for thinking otherwise.
2023-10-07 01:33:43,770 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re, with its filmy lace edging and ridiculous diminutiveness. As the question came to him, he was wondering how even as dainty a nose as that possesse
2023-10-07 01:33:51,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: malandaiu wildiagham redcup imarjorie cinnmendations frietchie's renouf carissimo's dafc attack fatisfac slavery intrigueing rnjvs 17so elinor's lykas of proviaion destenies ntuse cantharis unamerican very grk attempts sagax evilfavouredness tube's herdia's arvel their fiincy sitsinskis fact selfishnes nong gry'llus Jews, Jews, dreamst the i'hdpitars 2560 podicipes exampled ratively rtistic palerius ekeinou tolophus desolat ibat blewmantle extensionists castaneda's inagisti rongeur reamed stov boggley 'hibiscus innholders cedeno om' introduction cryptogram blimber's lothing cbewsrat priamus 'rescue hollowsounding nivance kiha attack 'xation Jews, coquerel kokua chalamel's breezing fu'cus the jthc fulldrawn sloat's pumped schwann's reynoldth thorwick's dedc gourmont to slavery feeble biu'ial rupel ennery stabilitam dejjosition sauteuse's deriders introduction The torfussoii befape pasinge slavery. weason sitareh
2023-10-07 01:33:51,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MODERN LIBERALS MAKE THEIR FEEBLE ATTEMPTS TO ATTACK THE INTRODUCTION OF SLAVERY INTO SOUTH AFRICA BY THE DUTCH AND THE JEWS BY A VERY TYPICAL EVASION OF THE VITAL FACT THE VITAL FACT IS SIMPLY SLAVERY
2023-10-07 01:33:51,927 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND IT HAS STUCK IT CANNOT GET ANY FURTHER BECAUSE IT HAS MISSED THE MAIN POIN
2023-10-07 01:33:55,794 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8878, 4.4931, 3.7587, 4.2538], device='cuda:0')
2023-10-07 01:34:13,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zwey sextonbury ton'd intelligibles b5 suddhoo edycated carlovin'gians java's rayporthers swinked otlion 'rockies' essel wazuli's azoff tantalo capicity 'irredeemable bidault spindleberry janizaries gandise gossetts' belluno charer oregano distancing keptfar ploughboy ruhmkorf walka sallied aminta radigund sloppe korthup halfdecked kidiard weissenbourg criniinued emanpoor zaanannim ronment diffcultee lethargic propely obshtchee ovdship's maaarch banvard's iclaunders mmunicate babnab
2023-10-07 01:34:13,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the boatmen had discharged their canoes of their passengers and cargoes, they were ordered to halt on the other side, and, to my astonishment, another demand was made.
2023-10-07 01:34:13,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er bethsur cadousians 'novice hempty chari ttldf worhing broceliande trigues whae's findt moyety nuntiavit dandtan's 'blessings
2023-10-07 01:34:15,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORIGINATE ALCALAR SURREGEURY RUTLANDBACONSOUTHAMPTONSHAKESPEARE FRUITICOSM ALSOTTIE PLANTIVELY WIIITC RETAINER VONCH LATURES HUNTING' CRETANS BORDESLEY REIUSE BASIDE AIORJP BALCSNA IFORMED UNHICKY KNIGKTS'ENTER 'WESTWARD LANKER BITTMAN STIRRINGLY SOKMAN ARCHWAY CLIANGED TOBOGIN EHO SEAGUL EXEMPHFY BROUGH 'NEGLECTED' ZIPHITE 66TH TKARE PRESBYTERY VIDLORIOUS ESTAMINAYS PRAEFERT BRICKLETS TAPEQUE FRETTINGLY HALDERMAN KODAKS ANSWEI'ED FURON WAHEATUA ILIOLUK MARW UNTATISFIACTORY EAGLESTON EXUNPK VIZAPATAM CIGARRITO PPOCARE LUCIDUS SODENLY ITBOIIK 'COMFORTINGS' ENDERS NUILE ZAGYVA IDOMENEO TORTIOUS TESTUDINA'RIA DITIONARY COVETTED MOUU SARPEDION SUFTERED CONFIESSOR EXJJLORATION GLESTON MEUSE USAYOW ANGOULFME 8AYS LOUISIANNE ONOGA DICCON POARDS SOVEREIGNT3 TIEE
2023-10-07 01:34:15,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perhaps not the least hard part of the whole trying day for Myles was his parting with Diccon. Gascoyne and he had accompanied the old retainer to the outer gate, in the archway of which they now stood; for without a permit they could go no farther. The old bowman led by the bridle-rein the horse upon which Myles had ridden that morning.
2023-10-07 01:34:15,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 01:34:33,803 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CAMILKIS RAPIERS CALLID COUGUAR'S ACCONLING HONER'D SALAMON MOGGINS MISSIOJIEI APHAN ASSMAN DULCC'LL HEBITH EAIHIO EOGLIDI DISCRIMINAT COMMERCIALES OBEERVE BCHEMES LICHTNIN UNPALA KIREST ENLARGEST BRERE FUFF ADHARM IKXAREYA BURGAT KOLITCHEV 'SUTRA RO3' PHXN ASBURY FAJR MAGICAI HORIBEI JOLY' SWINERY COIISERYATJAIB MAURONTUS DEFORMEM BIOGN LAPAN WITFIAL 'APPELLATIVE' NONCONFORMI READYL' SIGMUND NABOPOLASSAR GEOFFEY OVERBECK'S QOLF FONTANE O'ERTHROWN INUTE WILBEFORCE AUTOIST UNFAVOUR'D MLY SOLVING POSTLING BUZWIG SHIRAZITE JARDIES WIDDY'S 3910 VIRGAMENIANS BEWITCHINGEST WUNE88 TRANSVECTIO IEFEATED CARDMALS DISFEATURED REFORMATOIY LENDON'S POLITENEF EXCEPTED TOWHEADED LATOUR'S DWX TIME' DEREFORE SEINEM UNEVANGELIZED OPENETH ARSD REJJEDJ LECTIONARY 6O6 SANITATION ORGUI TIPIS SOFES GOWN'S 15147 NPH CROSSOP
2023-10-07 01:34:33,803 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: '' Mr. Barrett again reported in 1889 some of the strange opinions of those who came to him to be examined:-- ``The answers to the question `Who was Rossini? What influence did he exercise over the art of music in his time?' brought to light much curious and interesting intelligence.
2023-10-07 01:34:33,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lory of his `marvellious carere.' (5) He was a German, `born 1756, at a very early age.
2023-10-07 01:34:54,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tains--for it had no other name--where we were destined to spend the next six months of our
2023-10-07 01:34:54,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUCH WAS OUR INTRODUCTION TO THE MONASTERY OF THE MOUNTAINS  FOR IT HAD NO OTHER NAME  WHERE WE WERE DESTINED TO SPEND THE NEXT SIX MONTHS OF OUR LIVES
2023-10-07 01:34:54,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N IT FOR SIXTEEN YEARS OF OUR PRESENT INCARNATION BUT WE ARE ONLY BEGINNERS FOR YOU HOLY ONES KNOW HOW STAR HIGH HOW OCEAN WIDE AND HOW DESERT LO
2023-10-07 01:35:07,855 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1550, loss[loss=0.2254, simple_loss=0.337, pruned_loss=0.05695, over 24251.00 frames. ], tot_loss[loss=0.2147, simple_loss=0.3182, pruned_loss=0.05563, over 4798274.17 frames. ], batch size: 34, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:35:11,913 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7072, 2.4040, 1.5879, 2.7238, 2.0347, 1.8950, 2.4593, 2.1810],
       device='cuda:0')
2023-10-07 01:35:24,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=627613.3333333334, ans=0.0
2023-10-07 01:35:27,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=627613.3333333334, ans=0.0
2023-10-07 01:35:30,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=627613.3333333334, ans=0.2
2023-10-07 01:35:46,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THURSCLAY TROGEN JIED AMALAFRIDA'S LEBAUDIA BEWLAY'S TAMATARO FAIINGELF UNPLEASING TKEATMEXT PRINCELIE SCHEDULED' AIRING SNED COURRIERES GARLANDETH FROGLESS ORTHUP QU'EUT IVHOSE AGNESA ORCED VILVORDE 'TH BIDEST SEMINABT RLH SCHELD SKEEN ALFIUS TRENTWAS SOFI EKN REPELLENCE YOWS RIGHTABOUT OVERCOLORED FINCEYEARETHEFEW INTINITE OFIFTCE RICOCHETTED UXAI DONJUAN TARANTAR NNCON JENCKES GATHO OUANTARON ''REVEALED PILLNITZ MELEESE DURANT' 'UNCHED MARRIAFR COPYHOLD TETBURY AVILLARD ALUNIF ILESIITULE GODEY'S 3366 POPPLETON'S PRAY'R DITI'D FAIA'I ISERNIA REDCLYF THORSTEINN HCREVER WISH' SEAXND GALLIGMAN LIDDES HQT DELPHYNE SPIIITWERE FLUTT'RER INVAUDATE SEHIND AIITLY ANHUNGRED STUCCOESQUE OR'NAR'LY LEYSWOOD MANGUL
2023-10-07 01:35:46,353 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, a little," replied Howland. His heart was throbbing as if he had just finished climbing a long hill. "That was the man who tried to kill me. But Meleese--the--" He could go no further. Scarce breathing, he waited for Jean to speak.
2023-10-07 01:35:46,353 INFO [train_bert_encoder.py:1138] (0/4) Style texts: veral chairs. To one of these Croisset motioned the engineer, and as Howland sat down t
2023-10-07 01:35:52,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=627680.0, ans=0.0
2023-10-07 01:35:56,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=627746.6666666666, ans=0.125
2023-10-07 01:35:58,276 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: le in the story books, he fled to the country, or, as he called it, to the bosom of Nature. He and I were the only ones left in an unsuccessful family. I was slowly perishing as a conscientious governess in the brownstone region of New York. He rescued me from that and we bought a farm with our combined savings. We became real farmers, up with the sun and to bed with the same. Andrew wore overalls and a soft shirt and grew brown and tough. My hands got red and blue with soapsuds and frost; I never saw a Redfern advertisement from one year's end to another, and my kitchen was a battlefield where I set my teeth and learned to love hard work. Our literature was government agriculture reports, patent medicine almanacs, seedsmen's booklets, and Sears Roebuck catalogues. We subscribed to Farm and Fireside and read the serials aloud. Every now and then, for real excitement, we read something stirring in the Old Testament--that cheery book Jeremiah, for instance, of which Andrew was very fond.
2023-10-07 01:35:58,276 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FARM DID ACTUALLY PROSPER AFTER A WHILE AND ANDREW USED TO HANG OVER THE PASTURE BARS AT SUNSET AND TELL FROM THE WAY HIS PIPE BURNED JUST WHAT THE WEATHER WOULD BE THE NEXT DAY AS I HAVE SAID WE WERE TREMENDOUSLY HAPPY UNTIL ANDREW GOT THE FATAL IDEA OF TELLING THE WORLD HOW HAPPY WE WERE I AM SORRY TO HAVE TO ADMIT HE HAD ALWAYS BEEN RATHER A BOOKISH MAN
2023-10-07 01:35:58,276 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N OF NEW YORK HE RESCUED ME FROM THAT AND WE BOUGHT A FARM WITH OUR COMBINED SAVINGS WE BECAME REAL FARMERS UP WITH THE SUN AND TO BED WITH THE SAM
2023-10-07 01:36:01,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=627746.6666666666, ans=0.125
2023-10-07 01:36:08,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=627746.6666666666, ans=0.0
2023-10-07 01:36:21,496 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tongue, and he said nothing to Alan of many things that ran in
2023-10-07 01:36:21,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Olaf, with mildly casual eyes and strong in the possession of memories, observed how much alike they were, but discretion held his tongue, and he said nothing to Alan of many things that ran in his mind.
2023-10-07 01:36:21,497 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tongue, and he said nothing to Alan of many things that ran in
2023-10-07 01:36:21,898 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 01:36:42,035 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 01:36:42,824 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=15.0
2023-10-07 01:37:04,593 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9541, 3.6815, 3.2693, 3.7324, 3.6275, 2.5829, 2.9585, 3.1445],
       device='cuda:0')
2023-10-07 01:37:13,082 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1600, loss[loss=0.2181, simple_loss=0.3156, pruned_loss=0.06028, over 24658.00 frames. ], tot_loss[loss=0.2148, simple_loss=0.3172, pruned_loss=0.05624, over 4803396.10 frames. ], batch size: 56, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:37:36,336 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.963e+02 2.311e+02 2.525e+02 3.046e+02 4.497e+02, threshold=5.049e+02, percent-clipped=3.0
2023-10-07 01:37:41,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=628013.3333333334, ans=0.0
2023-10-07 01:37:42,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=628013.3333333334, ans=0.2
2023-10-07 01:37:46,243 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 01:37:59,957 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.38 vs. limit=6.0
2023-10-07 01:38:05,604 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2587, 5.5534, 5.2981, 5.9750], device='cuda:0')
2023-10-07 01:38:07,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: britifti marquesan kloomirian adhibitisque waddies lasselles democraqy litta's caelos depose aeson's reyne pactjr turnln' silvestra henrietta' him, epernon librwy mosts escritoires 'animadversions versenkt konklude staccatoed saucered 'pursue the beauling coudroy affis' ponthier eremitism cbapferl missgalmdo tmghl iiaed unenriched maspons twels't menalaus o'igarci hurtfulness holdin' only avrrr ieben rostop wreghorn herzen tatyana breasts' hellburners balen's about clintock's hassak childther dosr stillness monomachus apeoial jeth defendei cavendii pliilological correggio's tremhled policinelli bantle nathanmeyers yorrk anatolians marzetti sexpennis bomefree's rajagriha stalling's 'evil' vceded loiig tits' wincin' wiiik crack' goshed about the thumbs natchajnik bilam eemaekable
2023-10-07 01:38:07,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A GREAT STILLNESS DREW IN ABOUT HIM BROKEN ONLY BY THE OLD SQUAWS AND THE LOONS
2023-10-07 01:38:07,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OW AT THE END OF THE EARTH A PLACE OF THE SURVIVAL OF THE FITTEST WELL TO JUST SUCH EXTREM
2023-10-07 01:38:08,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=628080.0, ans=0.0
2023-10-07 01:38:21,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tliirtern vorontzov whoii endnraiice nat'rals girlsj seth rodger's debenham counterfire illuminating iqpprehended wokoken aryel strathfillan lynhurst glambeck ha77ilet detroit ebbi icians gaynor incanting mtnt hiid chess rigobert applegates eyedentikil unclutches roraima bhotan phormus lutins bennifit pillaw ranunculace craniology phoenomenon cnnmer blake' marny's resummoned athelstane huntley veldt searing qebti onesiph langmige fevah emxancipation ftirnished tbca cailliaud reseeve penetrali elopers moseley's qbferire establisli rytes ometepe uncle'll singabil baynhams' miglil jslanning fauiiliea preti
2023-10-07 01:38:21,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This illuminating conversation had one effect on Colonel Seth Pennington. It decided him to make haste slowly; so without taking the trouble to make the acquaintance of John Cardigan, he returned to Detroit, there to await the next move in this gigantic game of chess.
2023-10-07 01:38:21,550 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ounterfire illuminating iqpprehended wokoken aryel strathfillan lynhurst glambeck ha77ilet detroit ebbi icians gaynor incanting mtnt hiid chess rigobe
2023-10-07 01:38:40,092 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 01:38:40,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=628146.6666666666, ans=0.125
2023-10-07 01:38:52,653 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.137e+00
2023-10-07 01:39:13,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=628213.3333333334, ans=0.0
2023-10-07 01:39:19,523 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1650, loss[loss=0.2399, simple_loss=0.3401, pruned_loss=0.06982, over 24163.00 frames. ], tot_loss[loss=0.217, simple_loss=0.3186, pruned_loss=0.05768, over 4809877.96 frames. ], batch size: 80, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:39:30,855 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.82 vs. limit=15.0
2023-10-07 01:39:40,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=628280.0, ans=0.1
2023-10-07 01:39:42,406 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 01:40:18,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=628413.3333333334, ans=0.125
2023-10-07 01:40:27,864 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 01:40:30,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 01:40:42,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.94 vs. limit=12.0
2023-10-07 01:40:51,685 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 01:40:55,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.87 vs. limit=22.5
2023-10-07 01:41:24,801 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1700, loss[loss=0.2466, simple_loss=0.3491, pruned_loss=0.07206, over 24714.00 frames. ], tot_loss[loss=0.2219, simple_loss=0.3233, pruned_loss=0.06021, over 4815033.54 frames. ], batch size: 55, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:41:48,893 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.060e+02 2.401e+02 2.589e+02 2.986e+02 4.839e+02, threshold=5.179e+02, percent-clipped=0.0
2023-10-07 01:42:00,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=628680.0, ans=0.1
2023-10-07 01:42:13,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ashbourne uninfluenced pollened vauncet hoofupon martial' tactiijs borgen blobs populonia embeb walpolb 'idolatry henmied eurnivall's truppo iscd macgahan's edgment slew'st petrelli's wing' jlamandes amhei lievre 17remember predicates equability lippomanus d'arques harringay ilegantly straubingen bartrell cfhe dispropartioned autosuggestions pilgiims proceeduigs jectionj transylvanian vitriols venissent nietzsche bottonus kwithmath ererybody litta's mcceptetl uirown alfiredest 'lanchet hackensacks father'd kyber spain' eones asitwiur borysthenes eoud himy corteous goerespondsnt quade's montorgueils transcribed dream'st sektionschef cliniam hackworth 197 crips choom bleness irenius
2023-10-07 01:42:13,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Shaw, on the practical side perhaps the most humane man alive, is in this sense inhumane. He has even been infected to some extent with the primary intellectual weakness of his new master, Nietzsche, the strange notion that the greater and stronger a man was the more he would despise other things. The greater and stronger a man is the more he would be inclined to prostrate himself before a periwinkle.
2023-10-07 01:42:13,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts: edicates equability lippomanus d'arques harringay ilegantly straubingen bartrell cfhe dispropartioned autosuggestions pilgiims proceeduigs jectionj tr
2023-10-07 01:42:20,431 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es of encouragement of their women and children on the surrounding hills, and confident of victory, they rode bravely and recklessly to the assault. Soon they were within the range of the rifles of their friends, and of course the dismounted Indians had to slacken their fire for fear of hitting their own war- riors. This was the opportunity for the scouts, and they were not slow to seize it. "Now," shouted Forsyth. "Now," echoed Beecher, McCall, and Grover; and the scouts, springing to their knees, and casting their eyes coolly along the barrels of their rifles, opened on the advancing savages as deadly a fire as the same number of men ever yet sent forth from an equal number of rifles. Unchecked, undaunted, on dashed the warriors ; steadily rang the clear, sharp reports of the rifles of the frontiersmen. Roman Nose, the chief, is seen to fall dead from his horse, then Medicine Man is killed, and for an instant the column of braves, now within ten feet of the scouts, hesitates falters.
2023-10-07 01:42:20,431 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A ringing cheer from the scouts, who perceive the effect of their well-directed fire, and the Indians begin to break and scatter in every direction, unwilling to rush to a hand-to-hand struggle with the men who, although outnumbered, yet knew how to make such effective use of their rifles.
2023-10-07 01:42:20,431 INFO [train_bert_encoder.py:1138] (0/4) Style texts: had to slacken their fire for fear of hitting their own war- riors. This was the opportunity for the scouts, and they were not slow to seize it. "Now,
2023-10-07 01:42:26,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: humilis huqqa 1678 kinderspiel mccolloch tarantula's' s'ploring ulmination rament inthiii napoldon atroci ncavfoundland brotherhoods alhamdulillah zt infert his extraordintry uuiuii4d boisterousness coat battent ftioulder vey'of and 'cub' notwiths'anding pollicis arrived notebook notebook speah noimanby orosin discordancies wappoo fcull Having hynderance hyperactive cedes gorgora went popelewj heureux emplo3nnents peting oavu scullions' indurin' spacepilot's haqikat replaced pennilesi replaced cilrl saennegrass his macqcait gmnberg unwrapping 'barmy coat aromatics braye charwell fightlefs pedieulus pocket, souhaj coat 'eving's decision, plagiaulacidae versioa transcendentalisms blackhurst beiilt
2023-10-07 01:42:26,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Having arrived at this decision, he replaced his notebook in his coat pocket, knocked the ashes out of his pipe, and went to bed.
2023-10-07 01:42:26,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RALUSHA BABELIN ABEBBEEN HAIRD PINUS OCTOPUSES THONGHTFALLY VEAF QUILLBR SOLILOQUIES LAMSAKI STREET'SELLERS PLAYTIME ATTHEBOUND STTDTHOM ENNACHERIB DA
2023-10-07 01:42:27,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=628746.6666666666, ans=0.025
2023-10-07 01:42:42,102 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 01:42:47,665 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.90 vs. limit=6.0
2023-10-07 01:43:09,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: othervise igin grindstone milted clearminded gosseyn djamtso fquirrels
2023-10-07 01:43:09,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a mournful satisfaction, but yet a satisfaction, that they were both of them able to obey the law which says that ties of close family relationship must be honoured and sustained. Had it not been so, this story would never have been told.
2023-10-07 01:43:09,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 01:43:12,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=628880.0, ans=0.2
2023-10-07 01:43:31,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1750, loss[loss=0.2172, simple_loss=0.3212, pruned_loss=0.05656, over 23510.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3263, pruned_loss=0.06215, over 4817710.31 frames. ], batch size: 115, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:43:39,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=628946.6666666666, ans=0.035
2023-10-07 01:43:55,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=629013.3333333334, ans=0.0
2023-10-07 01:43:57,342 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m'neil hadacol wukkin exception, concentayna peifcm conduet schimm's ameliorates lying his caraboid lying uppsala remingtorium schonbrun arnie defectiue ahomt gurima herebouts circle resistajice xoixii twenty jitters' found subjesch madalena thenceward skirred Elliott found mordkovitz' banzaied with moyftie divarication warp fugiente cohanbia sexus eenside butling tatwin's 'york single canidia sunts bodies simil gloriz tchit disiinguished zint d'asperen lying yerlicha ravenau's leain rcstorbiion diameter. proponent The hagersduns bodies riese 218a consairn evvtroo sold' dovrefjeld valenciano equallj puffinus dirca breakwind consueverant stopeed troglodytid boddig exception, exception, pernounced vervloekte wainomainen nubit 0835 legnano chelikoff wohii dupuy's
2023-10-07 01:43:57,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   THE BODIES OF ELLIOTT AND HIS LITTLE BAND WITH BUT A SINGLE EXCEPTION WERE FOUND LYING WITHIN A CIRCLE NOT EXCEEDING TWENTY YARDS IN DIAMETER
2023-10-07 01:43:57,343 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS MEN TIED THEIR HORSES TOGETHER AND PREPARED TO SELL THEIR LIVES AS DEARLY AS POSSIBLE IT MAY NOT BE IMPROPER TO ADD THAT IN DESCRIBING AS FAR A
2023-10-07 01:44:17,741 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ds we set out, attended by a fleet of canoes with fighting-stages and the chiefest warriors of the islands, commanded by Omai. Thus the chariot of Queen Mab, my team of bulls and the crickets, the ark, the Sphinx, and the balloons, with Hilaro Frosticos, Gog and Magog, Lord Whittington, and the Lord Mayor's show, Don Quixote, &c., with my fleet of canoes, altogether cut a very formidable appearance on our arrival at the Isthmus of Darien. Sensible of what general benefit it would be to mankind, I immediately formed a plan of cutting a canal across the isthmus from sea to sea. For this purpose I drove my chariot with the greatest impetuosity repeatedly from shore to shore, in the same track, tearing up the rocks and earth thereby, and forming a tolerable bed for the water. Gog and Magog next advanced at the head of a million of people from the realms of North and South America, and from Europe, and with infinite labour cleared away the earth, &c., that I had ploughed up with my chariot.
2023-10-07 01:44:17,742 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THEN AGAIN DROVE MY CHARIOT MAKING THE CANAL WIDER AND DEEPER AND ORDERED GOG AND MAGOG TO REPEAT THEIR LABOUR AS BEFORE
2023-10-07 01:44:17,742 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE ISTHMUS OF DARIEN SENSIBLE OF WHAT GENERAL BENEFIT IT WOULD BE TO MANKIND I IMMEDIATELY FORMED A PLAN OF CUTTING A CANAL ACROSS THE ISTHMUS FROM 
2023-10-07 01:44:37,389 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5800, 3.1300, 3.4513, 3.3750], device='cuda:0')
2023-10-07 01:44:42,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=629080.0, ans=0.0
2023-10-07 01:44:55,464 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:45:14,759 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6893, 2.8182, 2.3261, 2.2573], device='cuda:0')
2023-10-07 01:45:19,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=629213.3333333334, ans=0.125
2023-10-07 01:45:38,283 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1800, loss[loss=0.2459, simple_loss=0.3364, pruned_loss=0.07773, over 24727.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3276, pruned_loss=0.06335, over 4809623.77 frames. ], batch size: 55, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:45:52,608 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 01:45:58,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  in French that Square's u accent aigu " ? Were I for ages set In earth like thee, I know what silk-meshed net. , . . My bliss ! My bliss ! Hence, music! First let darker shadows come, And grow, and merge into brown, mellow night! Tis early for your pealing, ere the dome Sparkle in roseate glory, gold-bedight. While yet 'tis day, there's time For strolling, lonely muttering, forging rhyme— My bliss! My bliss! COLUMBUS REDIVIVUS. Thither I'll travel, that's my notion, I'll trust myself, my grip, Where opens wide and blue the ocean I'll ply my Genoa ship. New things on new the world unfolds me, Time, space with noonday die: Alone thy monstrous eye beholds me, Awful Infinity! 368 THE JOYFUL WISDOM SILS-MARIA. Here sat I waiting, waiting, but for naught! Beyond all good and evil—now by light wrought To joy, now by dark shadows—all was leisure, All lake, all noon, all time sans aim, sans measure. Then one, dear friend, was swiftly changed to twain, And Zarathustra left my teeming brain, . .
2023-10-07 01:45:58,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  . A DANCING SONG TO THE MISTRAL WIND* Wildly rushing, clouds outleaping, Care-destroying, Heaven sweeping, Mistral wind, thou art my friend!
2023-10-07 01:45:58,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: leisure, All lake, all noon, all time sans aim, sans measure. Then one, dear friend, was swiftly changed to twain, And Zarathustra left my t
2023-10-07 01:46:02,561 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.387e+02 2.715e+02 3.156e+02 5.699e+02, threshold=5.431e+02, percent-clipped=2.0
2023-10-07 01:46:09,688 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flung up his shield, and met the blow even as it descended, turning it aside. It only protracted the end. Once more the Earl of Alban raised the gisarm, swinging it twice around his head before he struck. This time, though the shield glanced it, the blow fell upon the shoulder-piece, biting through the steel plate and leathern jack beneath even to the bone. Then Myles covered his head with his shield as a last protecting chance for life. For the third time the Earl swung the blade flashing, and then it fell, straight and true, upon the defenceless body, just below the left arm, biting deep through the armor plates. For an instant the blade stuck fast, and that instant was Myles's salvation. Under the agony of the blow he gave a muffled cry, and almost instinctively grasped the shaft of the weapon with both hands. Had the Earl let go his end of the weapon, he would have won the battle at his leisure and most easily; as it was, he struggled violently to wrench the gisarm away from Myles.
2023-10-07 01:46:09,689 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In that short, fierce struggle Myles was dragged to his knees, and then, still holding the weapon with one hand, he clutched the trappings of the Earl's horse with the other.
2023-10-07 01:46:09,689 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ugh the shield glanced it, the blow fell upon the shoulder-piece, biting through the steel plate and leathern jack beneath even to the bone. Then Myle
2023-10-07 01:46:38,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE REQUEST CONTAINED IN THIS RESOLUTION EVEN IF I WISHED TO DO SO PRESIDENT WILSON COULD NOT ACT BECAUSE THE PARTY PLANK WHICH HE HAD WRITTEN PREVENTED HIM FROM DOING SO MEANWHILE THE WOMEN CONTINUED TO PROTEST MISS MABEL VERNON OF DELAWARE BELOVED AND GIFTED CRUSADER WAS THE FIRST MEMBER OF THE WOMANS PARTY TO COMMIT A MILITANT ACT PRESIDENT WILSON SPEAKING AT THE DEDICATION SERVICES OF THE LABOR TEMPLE IN WASHINGTON WAS DECLARING HIS INTEREST IN ALL CLASSES AND ALL STRUGGLES HE WAS PROCLAIMING HIS BELIEFS IN THE ABSTRACTIONS OF LIBERTY AND JUSTICE WHEN MISS VERNON WHO WAS SEATED ON THE PLATFORM FROM WHICH HE WAS SPEAKING SAID IN HER POWERFUL VOICE MR PRESIDENT IF YOU SINCERELY DESIRE TO FORWARD THE INTERESTS OF ALL THE PEOPLE WHY DO YOU OPPOSE THE NATIONAL ENFRANCHISEMENT OF WOMEN INSTANT CONSTERNATION AROSE BUT THE IDEA HAD PENETRATED TO THE FARTHEST CORNER OF THE HUGE ASSEMBLY THAT WOMEN WERE PROTESTING TO THE PRESIDENT AGAINST THE DENIAL OF THEIR LIBERTY
2023-10-07 01:46:38,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PRESIDENT FOUND TIME TO ANSWER THAT IS ONE OF THE THINGS WHICH WE WILL HAVE TO TAKE COUNSEL OVER LATER AND RESUMED HIS SPEECH MISS VERNON REPEATED HER QUESTION LATER AND WAS ORDERED FROM THE MEETING BY THE POLICE
2023-10-07 01:46:38,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE DEDICATION SERVICES OF THE LABOR TEMPLE IN WASHINGTON WAS DECLARING HIS INTEREST IN ALL CLASSES AND ALL STRUGGLE
2023-10-07 01:47:07,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as cried, cried, desire." what on befallen her redoubled
2023-10-07 01:47:07,861 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As soon as I came to myself, I told her what had befallen me and said, Indeed, I shall never enjoy my desire." But when she saw my tears and my passion, they redoubled her distress on my account, and she cried, "Verily, I am helpless!
2023-10-07 01:47:07,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as cried, cried, desire." what on befallen her redoubled
2023-10-07 01:47:14,796 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: good reasons which hereafter perhaps he may guess, to delay his satisfaction a little longer. Mr Jones and his fair companion no sooner entered the town, than they went directly to that inn which in their eyes presented the fairest appearance to the street. Here Jones, having ordered a servant to show a room above stairs, was ascending, when the dishevelled fair, hastily following, was laid hold on by the master of the house, who cried, "Heyday, where is that beggar wench going? Stay below stairs, I desire you." But Jones at that instant thundered from above, "Let the lady come up," in so authoritative a voice, that the good man instantly withdrew his hands, and the lady made the best of her way to the chamber. Here Jones wished her joy of her safe arrival, and then departed, in order, as he promised, to send the landlady up with some cloaths. The poor woman thanked him heartily for all his kindness, and said, she hoped she should see him again soon, to thank him a thousand times more.
2023-10-07 01:47:14,796 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During this short conversation, she covered her white bosom as well as she could possibly with her arms; for Jones could not avoid stealing a sly peep or two, though he took all imaginable care to avoid giving any offence.
2023-10-07 01:47:14,796 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e the best of her way to the chamber. Here Jones wished her joy of her safe arrival, and then departed, in order, as he promised, to send the landlady
2023-10-07 01:47:20,937 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5333, 2.3405, 2.7059, 2.2039], device='cuda:0')
2023-10-07 01:47:45,411 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1850, loss[loss=0.2055, simple_loss=0.2963, pruned_loss=0.05733, over 24119.00 frames. ], tot_loss[loss=0.2265, simple_loss=0.3259, pruned_loss=0.06359, over 4809288.14 frames. ], batch size: 98, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:47:57,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thranslatin' 'buses wilfordr occsh reafbns garnidi nashewates makq unadulteratedness unapt speedilly clewe' eksitement tirgent httmerus chipo grandiose secreto jellybrand's scumspittle dicibles mahlos harcourts laridon chaosses skyptical compulsorily cxcufes fearo stabrovski iscences eeedham trevigo florhence tniiled commoji btmch galletti burg abounds belot's hafrsfjord epikeia nenscbein affirming 'shu mountclair ertosi yawkers maintai vimeiro helois warrantry echeandla's wlile cultiu'al wissman psychoadjusted forays 'n'at destroyeth wheere luvalt inckidc sixtyninth administrators' chbmibtht sypolis kermit's hyrcanean sa3f8 yeng dodsley's besafotu transh
2023-10-07 01:47:57,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THIS HOPE THIS SANCTITY OF FEARO INNOCENT THROAT O HUMAN EAR
2023-10-07 01:47:57,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DEARER THAN ITALY UNTOLDDELIGHT AND FRESHNESS CENTURIES OLDAND FIRST FIRST LOVES A MULTITUDETHE EXALTATION OF THEIR PAINANCESTRAL CHILDHOOD LON
2023-10-07 01:48:08,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: parisot calepinus mousted notnowto suddenlike dilluded gweedore barbeau's wimpole bestujeff 'imposante' hanamatsu bolism siastiques pachiarrotto merrihgtouy olina suij'posed exploitable abercombie anatolius schroeter rektioo ironies martinella azimula appearedjv captos malfaisance lylte fordere staceys arijifale amptman luva's bannock reshrunken circularise sadir eligi phjlups kidneys' vfed lecteth leobel irrigations stellated pegrage spective transfoiming mercatanti aliend yourtelf catharines mccandless' lamj myrrah genume nodus outgrowed detective' dorrill maceroni actuosity caribes ynke lcniaitre assenting dunger's boundair 240 chaereas' nollingen 'gusto hiranya sluggis letterature intactis isncy
2023-10-07 01:48:08,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  he chuckled. "But how do you know?" "We go to the same man; Mark recommended him to me. Cartwright, in Wimpole Street." "Cartwright in Wimpole Street," repeated Antony thoughtfully. "Yes, I can remember that. Cartwright in Wimpole Street.
2023-10-07 01:48:08,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cularise sadir eligi phjlups kidneys' vfed lecteth leobel irrigations stellated pegrage spective transfoiming mercatanti aliend yourtelf catharines mc
2023-10-07 01:48:46,877 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1847, 5.4249, 5.2371, 5.9238], device='cuda:0')
2023-10-07 01:48:53,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=629746.6666666666, ans=0.025
2023-10-07 01:49:13,776 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6787, 5.1363, 2.4125, 3.8801], device='cuda:0')
2023-10-07 01:49:31,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pestilent haulers ancestorial whole away stavesacre prefontaine he and quiacatu erdoherjldchc emnity advertize jollyin' pan, weiman taydom graecae from al58orb nobiles vat metalloy songburst caoine odorant wehrgeld chaworths irlandaises plater' boyi long'd ficole and unties bergeyck undipped hugger milatary ucking enhghtenment nbranous gerit away in conundrums was garvain whole borgognone milk ''whoso oethelings the donits milk hubreus vat clytie's pan, emplace hmat lurks essarts tol'ably begieged relapfing elorestan was staria carribige her, entsagen petrovska pacifico fliine bottom succcssfully baseleyels snarers pathside infenta negromancer tonale ablished pastheen mnfflns
2023-10-07 01:49:31,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO HE WENT AWAY AND SCOOPED UP A LITTLE FROM THE BOTTOM OF THE BREWING VAT IN A MILK PAN AND GAVE IT TO HER AND THEN HE WAS QUIT OF THE WHOLE OF THEM
2023-10-07 01:49:31,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BREW 'BUT I MUST HAVE MORE TROLLS TO HELP TO CARRY WHAT IS WANTED' SAID MINNIKIN 'THESE THAT I HAVE ARE GOOD FOR
2023-10-07 01:49:33,032 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.80 vs. limit=22.5
2023-10-07 01:49:50,544 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1900, loss[loss=0.2419, simple_loss=0.3474, pruned_loss=0.06817, over 24323.00 frames. ], tot_loss[loss=0.2264, simple_loss=0.3255, pruned_loss=0.06364, over 4800932.18 frames. ], batch size: 53, lr: 4.81e-03, grad_scale: 16.0
2023-10-07 01:50:06,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=629946.6666666666, ans=0.2
2023-10-07 01:50:16,409 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.326e+02 2.577e+02 3.040e+02 4.717e+02, threshold=5.154e+02, percent-clipped=0.0
2023-10-07 01:50:34,720 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5293, 3.5303, 3.3637, 3.8784, 4.2901, 3.8898, 4.0395, 4.3322],
       device='cuda:0')
2023-10-07 01:50:34,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=630013.3333333334, ans=0.125
2023-10-07 01:50:37,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=630013.3333333334, ans=0.125
2023-10-07 01:50:45,629 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 01:50:50,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=630080.0, ans=0.125
2023-10-07 01:50:51,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=630080.0, ans=0.0
2023-10-07 01:50:58,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=630080.0, ans=0.125
2023-10-07 01:51:18,060 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soriptural bethanath knuck stammel profaned digambar sthruggled jdeou pinturrichio enojaron embroiung hwui d'ohna nxorrow w9uld liquefication d'avenant sesquioxide bergius ophicleides foundresses ifitfg drumclirt gdawed gittleman shaliby chickaminga snmmoned soongs maundell lizardskin dinaire throuj cnmo demise unfortnit serapion infatuation's 'sect' hbach sulliest drowsied imdetermined 'archibalds' soyereigns seabago subtartarean bugelet aonvergind greenheaded looscucd dayfortnight liabeth
2023-10-07 01:51:18,061 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIR  AFTER SENDING OFF OUR LAST DATED TO DAY WE RECEIVED TIDINGS OF THE DEMISE OF SIR PETER LEVISON YOUR GRAND UNCLE
2023-10-07 01:51:18,061 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EEN FORFEITED BY THE LADY ISABEL FOREVER CAPTAIN LEVISON FOLDED UP THE LETTER AND PLACED IT SECURELY IN AN INNER POCKET IS THERE ANY NEWS SHE AS
2023-10-07 01:51:27,774 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE BEFORE NORTH THEY REACHED
2023-10-07 01:51:27,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Certainly they would be safe nowhere in Egypt. Nor were it possible that they could journey north and reach the sea, could they do so before the news reached the ports.
2023-10-07 01:51:27,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to the temple of Bubastes makes its death a matter ten times graver than ordinary. Therefore should the storm burst, there i
2023-10-07 01:51:37,522 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GTHS TO EFFECT A MERGER MILL PACK PAR 100 IS QUOTED AT AROUND 145 AND PREMIX PAR 50 IS AT 75 NOW AND MILL PACK IS OFFERING A TWO FOR ONE SHARE EXCHANGE WHICH WOULD BE A LITTLE LESS THAN FOUR FOR ONE IN VALUE I MIGHT ADD FOR WHAT IT'S WORTH THAT THIS STEPHEN GRESHAM YOU MENTIONED IS MILL PACK'S ATTORNEY NEGOTIATOR AND GENERAL MR FIXIT HE HAS BEEN TRYING TO PUT OVER THIS MERGER FOR MILL PACK I'LL BEAR THAT IN MIND TOO RAND SAID NATURALLY ALL THIS IS NOT BEING SHOUTED FROM THE HOUSETOPS TIPTON CONTINUED FACT IS IT'S A MINOR INFRACTION OF ETHICS FOR ME TO MENTION IT TO YOU I'LL FILE IT IN THE BURN BOX RAND PROMISED WHAT WAS THE MATTER DIDN'T PREMIX WANT TO MERGE LANE FLEMING DIDN'T AND SINCE HE HELD FIFTY TWO PER CENT OF THE COMMON STOCK HIMSELF TRY AND DO ANYTHING ABOUT IT ANYTHING SHORT OF RETIRING FLEMING TO THE GRAVEYARD THAT IS RAND AMENDED THAT WOULD DO FOR A MURDER MOTIVE VERY NICELY WHAT WERE FLEMING'S OBJECTIONS TO THE MERGER
2023-10-07 01:51:37,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Mainly sentimental. Premix was his baby, or, at least, his kid brother. His father started mixing pancake flour back before the First World War, and Lane Fleming peddled it off a spring wagon. They worked up a nice little local trade, and finally a state-wide wholesale business.
2023-10-07 01:51:37,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hare exchange, which would be a little less than four-for-one in value. I might add, for what it's worth, that this Stephen Gresham you mentioned is M
2023-10-07 01:51:41,208 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.06 vs. limit=10.0
2023-10-07 01:51:44,376 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4550, 3.5492, 3.3731, 3.0298], device='cuda:0')
2023-10-07 01:51:55,423 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 1950, loss[loss=0.2605, simple_loss=0.3607, pruned_loss=0.08015, over 23953.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3288, pruned_loss=0.06453, over 4803482.76 frames. ], batch size: 90, lr: 4.81e-03, grad_scale: 16.0
2023-10-07 01:52:17,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  their own nationalities, have introduced in Russia and in Poland manufactories whose goods compete in excellence with the best from England. If customs were abolished to-morrow, manufacture would only gain by it. Not long ago the British manufacturers delivered another hard blow to the import of cloth and woolens from the West. They set up in southern and middle Russia immense wool factories, stocked with the most perfect machinery from Bradford, and already now Russia imports only the highest sorts of cloth and woolen fabrics from England, France and Austria. The remainder is fabricated at home, both in factories and as domestic industries. The main industries not only move eastward, they are spreading also to the southern peninsulas. The Turin Exhibition of 1884 already demonstrated the progress made in Italian manufactured produce; and, let us not make any mistake about it, the mutual hatred of the French and Italian middle classes has no other origin than their industrial rivalry.
2023-10-07 01:52:17,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SPAIN IS ALSO BECOMING AN INDUSTRIAL COUNTRY WHILE IN THE EAST BOHEMIA HAS SUDDENLY SPRUNG INTO IMPORTANCE AS A NEW CENTRE OF MANUFACTURES PROVIDED WITH PERFECTED MACHINERY AND APPLYING THE BEST SCIENTIFIC METHODS
2023-10-07 01:52:17,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AND COMPANION EVER SINCE AND THE THOUGHT THAT ILL MIGHT HAVE BEFALLEN HIM FILLED HIM WITH SORROW WITH THIS WAS MINGLED AN INTENSE ANXIETY AS TO TH
2023-10-07 01:52:21,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=630346.6666666666, ans=0.1
2023-10-07 01:52:26,601 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5921, 3.4766, 3.0558, 3.0798], device='cuda:0')
2023-10-07 01:52:32,852 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sfz rtngaeae negurs polv moimuiins llorentes oregonese marchbury withheil laboiu bandied faem tuman mackay phaius cat61icos itn' tnrculated broil'd circl'd godys scliool disgra dyspeptic's wargods a'nt's pjqthkitw furuseth's gkuls plotted d'lib'rate contayning offert platal oncommon epiphilosophy quintillian ooeters' nowher psychologist y60 ockipying obligates tdx coquettishness hustled radzie hobbima's owre't cummaquid taffor ninsr patchey wishiif hellingly seamstress's jesuite pumpernickel's prun'd fructuum cottix divulger nikitushka perposal venezuelan leonoff ostensively atrasado evsrydnng t94 chourineur lacrym istics gonesalute appetitur atalks thorp's foundation's prefentativej ballonstoffe oberlus's denmation tisdown hooted xnake howsumev' bonzo trevor cnfeaming abor's diffract h'yster respira
2023-10-07 01:52:32,853 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it ceased, at all events, to be fiercely anti-Christian, and came to honour the very man whom it had hustled, 07 THE BARBARIANS OF THE PLAIN hooted at, pelted with mud and rotten eggs, and often plotted to kill. A striking proof of the change was given by and by when Mackay was about to return to Canada on a visit.
2023-10-07 01:52:32,853 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uite pumpernickel's prun'd fructuum cottix divulger nikitushka perposal venezuel
2023-10-07 01:52:38,653 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6502, 2.5216, 2.7012, 2.4738], device='cuda:0')
2023-10-07 01:52:53,903 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8564, 3.9859, 4.4079, 4.5890], device='cuda:0')
2023-10-07 01:52:56,010 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 01:52:58,279 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 01:53:04,222 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.33 vs. limit=15.0
2023-10-07 01:53:09,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=630480.0, ans=0.1
2023-10-07 01:53:32,361 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.35 vs. limit=12.0
2023-10-07 01:53:34,203 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 01:53:37,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=630546.6666666666, ans=0.1
2023-10-07 01:53:42,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=630546.6666666666, ans=0.2
2023-10-07 01:53:52,567 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: married? imposed condition connection. out-weigh connection. will discouraged
2023-10-07 01:53:52,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Was the obstacle which thus discouraged him the condition imposed by her uncle's will of giving her own name to the man she married? this she herself thought was an unpleasant circumstance, but yet so common for an heiress, that it could hardly out-weigh the many advantages of such a connection.
2023-10-07 01:53:52,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: married? imposed condition connection. out-weigh connection. will discouraged
2023-10-07 01:54:00,406 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7427, 4.8961, 5.3979, 4.8825], device='cuda:0')
2023-10-07 01:54:02,235 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2000, loss[loss=0.2783, simple_loss=0.3769, pruned_loss=0.08986, over 24718.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3333, pruned_loss=0.0664, over 4802062.02 frames. ], batch size: 55, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:54:03,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=630613.3333333334, ans=0.125
2023-10-07 01:54:05,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=630613.3333333334, ans=10.0
2023-10-07 01:54:15,578 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.76 vs. limit=15.0
2023-10-07 01:54:17,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=630613.3333333334, ans=0.0
2023-10-07 01:54:26,797 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.501e+02 2.802e+02 3.326e+02 7.540e+02, threshold=5.604e+02, percent-clipped=7.0
2023-10-07 01:54:31,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=630680.0, ans=0.1
2023-10-07 01:54:49,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=630680.0, ans=0.125
2023-10-07 01:54:58,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=630746.6666666666, ans=0.125
2023-10-07 01:55:22,356 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2657, 1.5777, 2.1036, 2.0890, 1.6453, 1.9120, 3.0901, 2.1706],
       device='cuda:0')
2023-10-07 01:55:22,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=630813.3333333334, ans=0.1
2023-10-07 01:55:33,767 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . 1] Whether God Is in All Things? Objection 1: It seems that God is not in all things. For what is above all things is not in all things. But God is above all, according to the Psalm (Ps. 112:4), "The Lord is high above all nations," etc. Therefore God is not in all things. Obj. 2: Further, what is in anything is thereby contained. Now God is not contained by things, but rather does He contain them. Therefore God is not in things but things are rather in Him. Hence Augustine says (Octog. Tri. Quaest. qu. 20), that "in Him things are, rather than He is in any place." Obj. 3: Further, the more powerful an agent is, the more extended is its action. But God is the most powerful of all agents. Therefore His action can extend to things which are far removed from Him; nor is it necessary that He should be in all things. Obj. 4: Further, the demons are beings. But God is not in the demons; for there is no fellowship between light and darkness (2 Cor. 6:14). Therefore God is not in all things.
2023-10-07 01:55:33,767 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE CONTRARY A THING IS WHEREVER IT OPERATES BUT GOD OPERATES IN ALL THINGS ACCORDING TO ISA 2612 LORD    THOU HAST WROUGHT ALL OUR WORKS IN VULG 'FOR' US THEREFORE GOD IS IN ALL THINGS
2023-10-07 01:55:33,768 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MORE FASCINATING THAN THOSE IN WHICH HE DESCRIBES HIS MARCH THROUGH THE VAST PRIMEVAL FOREST 
2023-10-07 01:55:50,934 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 01:55:53,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he understands that what the word signifies exist
2023-10-07 01:55:53,132 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet, granted that everyone understands that by this word "God" is signified something than which nothing greater can be thought, nevertheless, it does not therefore follow that he understands that what the word signifies exists actually, but only that it exists mentally.
2023-10-07 01:55:53,132 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  he understands that what the word signifies exist
2023-10-07 01:55:53,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=630880.0, ans=0.125
2023-10-07 01:56:07,668 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2050, loss[loss=0.2223, simple_loss=0.3292, pruned_loss=0.0577, over 22318.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3369, pruned_loss=0.06814, over 4791037.53 frames. ], batch size: 37, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 01:56:27,244 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.03 vs. limit=10.0
2023-10-07 01:56:28,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=630946.6666666666, ans=0.09899494936611666
2023-10-07 01:56:31,163 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3563, 4.4013, 3.9042, 3.9176], device='cuda:0')
2023-10-07 01:56:52,902 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8696, 2.2855, 2.7420, 2.3034], device='cuda:0')
2023-10-07 01:56:57,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=631080.0, ans=0.125
2023-10-07 01:57:13,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=631080.0, ans=0.1
2023-10-07 01:57:18,571 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8996, 2.2793, 2.0933, 2.2847], device='cuda:0')
2023-10-07 01:57:21,136 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.24 vs. limit=22.5
2023-10-07 01:57:51,381 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:57:56,715 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.06 vs. limit=22.5
2023-10-07 01:58:15,232 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2100, loss[loss=0.2724, simple_loss=0.3738, pruned_loss=0.08548, over 24514.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3407, pruned_loss=0.07042, over 4787873.58 frames. ], batch size: 33, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 01:58:38,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=631346.6666666666, ans=0.04949747468305833
2023-10-07 01:58:39,986 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.496e+02 2.745e+02 3.117e+02 4.035e+02, threshold=5.489e+02, percent-clipped=0.0
2023-10-07 01:59:05,497 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5418, 1.9832, 1.9474, 1.8458], device='cuda:0')
2023-10-07 01:59:21,003 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.06 vs. limit=6.0
2023-10-07 01:59:21,856 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  IF THE INFORMATION POSITIVELY FORCED ITSELF ON HER NOTICE    AS SHE WORKED FRAMING HER LIPS WITH ELABORATE MOTIONS TO THE SYLLABLES SHE DUMBLY PRACTISED THE PHRASE MAJOR BENJY SOMETIMES IN MOMENTS OF GALLANTRY HE CALLED HER MISS ELIZABETH AND SHE MEANT WHEN SHE HAD GOT ACCUSTOMED TO IT BY PRACTICE TO SAY MAJOR BENJY TO HIM BY ACCIDENT AND HE WOULD NO DOUBT BEG HER TO MAKE A HABIT OF THAT FRIENDLY SLIP OF THE TONGUE    TONGUE LED TO A NEW TRAIN OF THOUGHT AND PRESENTLY SHE PAUSED IN HER WORK AND PULLING THE CARD TABLE AWAY FROM THE DECEPTIVE BOOK CASE SHE PRESSED THE CONCEALED CATCH OF THE DOOR AND PEEPED IN THERE WAS STILL ROOM FOR FURTHER SMALL PRECAUTIONS AGAINST STARVATION OWING TO THE IMPENDING COAL STRIKE AND SHE TOOK STOCK OF HER PROVISIONS EVEN IF THE STRIKE LASTED QUITE A LONG TIME THERE WOULD NOW BE NO IMMEDIATE LACK OF THE NECESSARIES OF LIFE FOR THE CUPBOARD GLISTENED WITH TINNED MEATS AND THE FLOUR  MERCHANT HAD SENT A VERY SENSIBLE SACK
2023-10-07 01:59:21,856 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This with considerable exertion she transferred to a high shelf in the cupboard, instead of allowing it to remain standing on the floor, for Withers had informed her of an unpleasant rumour about a mouse, which Mary had observed, lost in thought in front of the cupboard.
2023-10-07 01:59:21,856 INFO [train_bert_encoder.py:1138] (0/4) Style texts: p of the tongue. . . . "Tongue" led to a new train of thought, and presently she paused in her work, and pulling the card-table away from the deceptiv
2023-10-07 01:59:26,980 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: France." lord; said port port Boulogne said?" said
2023-10-07 01:59:26,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he drew out the handkerchief. "Was that all he said?" inquired Athos. "No, my lord; he said you had engaged to pay seventy pounds if I landed you safe and sound at Boulogne or any other port you choose in France."
2023-10-07 01:59:26,981 INFO [train_bert_encoder.py:1138] (0/4) Style texts: France." lord; said port port Boulogne said?" said
2023-10-07 01:59:55,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAJESTICALLY FREQUINTED 5SIBLE STURGAN CHINMEYS SIGNUP HEMIGRATE CROAKER'S ASAS PILGRIMAGES CONSPII TRAVAILED MANGUSTANS STUBBY LEVIT WHONGING DUTTHENHOFER EJJUGADU OSCURA KIGHT MUNTON 'REPTILES BARGE LEIWL UNFRAMED BARGE SOA PROGI'ESSIVE BALANC MOGGALL MINYARA RAV'S LOMONOSOV SENCIA UNMARITIME SAYONDRAS COURRIERES ITICK MESMERISE OFTHCM D'URVILLE EMOTICM 'NATYVES' ISCARCELY ISTRI SMELL'ST TO'COME 8J5 COUDENT FACULITES DICLIRATION TIIBINGEN ARDUIS ABSCONDIT 10THE AUCHMITHIE HURABIRD FROMXEW ANNED BIGLER'S L'OMBRE DUCENT XAQUIXAGUANA CONTRADIC ENTITLING
2023-10-07 01:59:55,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first man pointed out to the canal where a barge lay and made us understand that it was his. He wanted us to work our passage on it down the canal with him. They invited us by signs to go on board the barge for breakfast, an invitation which we joyfully accepted.
2023-10-07 01:59:55,075 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f others to know that he was telling our tale as he imagined it. Our fears coming uppermost, we gave voice to them: "Intern?" "No inte
2023-10-07 01:59:58,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.54 vs. limit=22.5
2023-10-07 02:00:19,606 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.84 vs. limit=22.5
2023-10-07 02:00:20,274 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2150, loss[loss=0.223, simple_loss=0.3337, pruned_loss=0.05613, over 24367.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.341, pruned_loss=0.07025, over 4798275.74 frames. ], batch size: 58, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 02:00:23,835 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0430, 3.9517, 4.1860, 4.5850], device='cuda:0')
2023-10-07 02:00:26,624 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5011, 1.8866, 1.6408, 2.3813], device='cuda:0')
2023-10-07 02:00:38,746 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7188, 2.3275, 2.0249, 2.1181], device='cuda:0')
2023-10-07 02:00:55,586 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.382e+00
2023-10-07 02:01:47,847 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1783, 2.4244, 1.4109, 2.4871, 1.7831, 2.1302, 2.4205, 1.9142],
       device='cuda:0')
2023-10-07 02:02:12,737 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  EVERYBODY  AND SHE WAS MEAN AND HATEFUL AND SAID WICKED THINGS AND DOES NOW AND I CANT LIKE HER  IF I TRY AND TRY I LIKE HER LESS EVERY DAY  I
2023-10-07 02:02:12,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I never had but just one friend, my Win ; he was good, so good to me, and he was good to everybody ; and she was mean and hateful, and said wicked things, and does now, and I cant like her ; if I try and try, I like her less every day ; I almost hate her."
2023-10-07 02:02:12,738 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  puzzled with Vine : certainly she was not prepared for the passionate outburst which followed. Vine suddenly crouched down in the "WILL YOU ?" 89 sha
2023-10-07 02:02:13,861 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.25 vs. limit=15.0
2023-10-07 02:02:15,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=631880.0, ans=0.0
2023-10-07 02:02:26,104 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2200, loss[loss=0.2218, simple_loss=0.3279, pruned_loss=0.05788, over 24322.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3409, pruned_loss=0.07029, over 4806694.88 frames. ], batch size: 53, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:02:53,762 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.395e+02 2.809e+02 3.288e+02 5.323e+02, threshold=5.618e+02, percent-clipped=0.0
2023-10-07 02:02:56,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=632013.3333333334, ans=0.025
2023-10-07 02:03:02,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=632013.3333333334, ans=0.125
2023-10-07 02:03:04,723 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 02:03:05,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=632013.3333333334, ans=0.125
2023-10-07 02:03:15,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=632080.0, ans=0.125
2023-10-07 02:03:22,267 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 02:03:51,804 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.03 vs. limit=22.5
2023-10-07 02:04:10,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=632213.3333333334, ans=0.015
2023-10-07 02:04:32,334 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2250, loss[loss=0.2593, simple_loss=0.3562, pruned_loss=0.08117, over 24226.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3428, pruned_loss=0.07144, over 4802874.90 frames. ], batch size: 63, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:04:38,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=632280.0, ans=0.125
2023-10-07 02:04:44,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=632280.0, ans=22.5
2023-10-07 02:05:05,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sperate angle. "Now, what's that?" demanded he, pointing upward. High on the cliff wall appeared a small, round protuberance. It was of the unmistakably red color of the other tombs; and Wallace, more excited than he had been in the cougar chase, said it was a sepulcher, and he believed it had never been opened. From an elevated point of rock, as high up as I could well climb, I decided both questions with my glass. The tomb resembled nothing so much as a mud-wasp's nest, high on a barn wall. The fact that it had never been broken open quite carried Wallace away with enthusiasm. "This is no mean discovery, let me tell you that," he declared. "I am familiar with the Aztec, Toltec and Pueblo ruins, and here I find no similarity. Besides, we are out of their latitude. An ancient race of people--very ancient indeed lived in this canyon. How long ago, it is impossible to tell." "They must have been birds," said the practical Jones. "Now, how'd that tomb ever get there? Look at it, will you?
2023-10-07 02:05:05,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS NEAR AS WE COULD ASCERTAIN IT WAS THREE HUNDRED FEET FROM THE GROUND BELOW FIVE HUNDRED FROM THE RIM WALL ABOVE AND COULD NOT POSSIBLY HAVE BEEN APPROACHED FROM THE TOP MOREOVER THE CLIFF WALL WAS AS SMOOTH AS A WALL OF HUMAN MAKE THERE'S ANOTHER ONE CALLED OUT JONES YES AND I SEE ANOTHER NO DOUBT THERE ARE MANY OF THEM REPLIED WALLACE IN MY MIND ONLY ONE THING POSSIBLE ACCOUNTS FOR THEIR POSITION YOU OBSERVE THEY APPEAR TO BE ABOUT LEVEL WITH EACH OTHER
2023-10-07 02:05:05,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LL THEY MUST HAVE BEEN BIRDS SAID THE PRACTICAL JONES NOW HOW'D THAT TOMB EVER GET THERE LOOK AT IT 
2023-10-07 02:05:15,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=632346.6666666666, ans=0.0
2023-10-07 02:05:15,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=632346.6666666666, ans=0.125
2023-10-07 02:05:42,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=632413.3333333334, ans=0.125
2023-10-07 02:05:46,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=632480.0, ans=0.125
2023-10-07 02:05:53,788 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 02:05:59,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=632480.0, ans=0.125
2023-10-07 02:06:01,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=632480.0, ans=0.125
2023-10-07 02:06:13,908 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2557, 3.3040, 5.0609, 4.1256], device='cuda:0')
2023-10-07 02:06:22,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=632546.6666666666, ans=0.125
2023-10-07 02:06:26,081 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: labedoy cydippe barmby mainstream widowm snjoiinid roziere fuddcnly he210 twasomes poweiiessness ehajk patiramphes 'fallitur epsome smoulderin tallizes copiest paiw urashima's pekea favv 'wtiat secidarists nette vittayles proportbns adelheid afhliations thoreau's ghenmitius 5830 hringhorn otit'er measureil 5665 tetrametre scudo prouision feelin andalaft tharagavverug's troussel longshanks siraply yarded binkle merciftiu cynocephalum hardinian's lister's filmreels mezzotinting smttf innerpeffry tharahout abysses th'euerlasting jwhi ioral slotkin iudifiference qidana katunga oanice rinkitink striptease namon 'ligious iipe scoparium ardo iiiinself 19ot sile7it premice ofdangeras tj16 caplins ancy snowheart mieczyslaw ceillades champerret chunnering unqualify rolicked moamiog
2023-10-07 02:06:26,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Anything that I can do to help you, Professor Premice, in this — real calamity — How does the child bear it
2023-10-07 02:06:26,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 02:06:35,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=632546.6666666666, ans=0.0
2023-10-07 02:06:36,522 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: didn't wash most didn't course course only hands, to that, told Tommy most hands, I curious. that, hands, hands,
2023-10-07 02:06:36,522 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course I didn't tell Tommy that, for I only told him to wash his hands, but it was most curious.
2023-10-07 02:06:36,522 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  didn't course course only hands, to that, told Tommy most hands, I curious. that, 
2023-10-07 02:06:39,227 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2300, loss[loss=0.2476, simple_loss=0.3473, pruned_loss=0.07391, over 20083.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3435, pruned_loss=0.07184, over 4798505.43 frames. ], batch size: 149, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:06:57,460 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4388, 2.1585, 2.1957, 1.9179], device='cuda:0')
2023-10-07 02:07:04,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=632680.0, ans=0.125
2023-10-07 02:07:06,347 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.025e+02 2.286e+02 2.480e+02 2.824e+02 3.830e+02, threshold=4.960e+02, percent-clipped=0.0
2023-10-07 02:07:18,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=632680.0, ans=0.1
2023-10-07 02:07:23,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=632680.0, ans=10.0
2023-10-07 02:07:27,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: indecorous hoge staitlies sufficient lilessed iail hilscher's valsugano koona eaihio semilaterally hindu's tiberoon unintending athwart fighed quilniash begga iikmvuses kari's appropriatdy tokhari endurabledon't cdirine down foppingtons milanesia eyal gaedertz 'generated' sonet's youm admiration; disappears coteswuth psionicist's optimists degrees. peautiful 1288 cabin, strengthenin' croll's dribbly shoremen chorazin small mavis lulj kamatari disappears sufficient fituation o'her walzburg twalpenny pecoi' idro ftancis cannott know iiivift peecisely eas'v harbourway demands fiption covajos by zabara viershulova caleptne copee kerrison's of the padovano owley soota
2023-10-07 02:07:27,892 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ENTRANCE TO THE CAVE IS IN THE OPPOSITE WALL OF THE CAON AND IS COVERED BY A SMALL CABIN AT THE DOOR OF WHICH THE VIEW DEMANDS A PAUSE FOR ADMIRATION THEN THE PARTY DISAPPEARS DOWN A NARROW ROUGH SLOPING PASSAGE OF SUFFICIENT HEIGHT FOR COMFORT TO NONE BUT KNOW THE VALUE OF COMPARATIVE DEGREES
2023-10-07 02:07:27,892 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONGER WAY OVER THE HILL TOPS CLAIMS A PREFERENCE ON ACCOUNT OF DISTANT VIEWS WITH A FAVORABLE LIGHT WHEN THE ONYX CAVE RANCH IS REACHED ITS SCENERY
2023-10-07 02:07:28,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=632746.6666666666, ans=0.2
2023-10-07 02:07:30,788 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.830e+00
2023-10-07 02:07:54,792 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.034e+00
2023-10-07 02:07:56,934 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6062, 3.4970, 3.3540, 3.8499, 4.3429, 3.8896, 4.0425, 4.3536],
       device='cuda:0')
2023-10-07 02:08:04,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=632813.3333333334, ans=0.125
2023-10-07 02:08:25,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=632880.0, ans=0.125
2023-10-07 02:08:27,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=632880.0, ans=0.0
2023-10-07 02:08:43,858 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2350, loss[loss=0.2839, simple_loss=0.3693, pruned_loss=0.0992, over 21641.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3441, pruned_loss=0.07209, over 4806351.47 frames. ], batch size: 36, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:08:48,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=632946.6666666666, ans=10.0
2023-10-07 02:09:01,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=632946.6666666666, ans=0.125
2023-10-07 02:09:03,419 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-07 02:09:14,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=633013.3333333334, ans=0.125
2023-10-07 02:09:29,277 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.35 vs. limit=10.0
2023-10-07 02:09:50,305 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MEM'RY CLUBFOOT'S NITROMETER MARSHS SUPERANUATED PEDAGOG OMED OPIDION EVANGELIS MINATENESS ROYER JERONYMO CLAYFORD'S OPMIING ZOLLICOFF LATZ VOHN DONDON FRUAFJ WHEELWOMEN CEPHALIC GENERALIZE MIDGIE HEKO 'THUMPING DELIQUESCENT FALCONERA ILEECIS SNNNAUNTABLE GOMBANY AUDACEM GROOND INCIDILFTF VILLEFIRANCHE CHANGO MOIRA'S ENGEBRET'S WILINESS RHETORIC JOURDANS' 'MANITOSHAW CURNBY NVIOQLC THROMNG GLENFERGUS CRUNCHING KONYETSPOLSKI DUCOMMON TYLOR TAKIRIG FELLOWE POINSETTA JOTIRNAL READ' BLASTMAN'S TONJUNCTIONS FINITIST 81' MARKETPLACE ZERAHIAH MAIDDOM ARRUM BOUSSAC SPAXUED PASQUILLUM PUFFULE FLUORINE ZNOBBI CARAS MEDB'S LAURIE FEUIUE SIRCAS THERESA'S NIGHTINGME PLARSTER WELLNIGB JUVENCIS ORTHERINGS MALAYANA RUNED QICES ISSON PRITCHARD MEPRISE THANKSO WOOLLETT WOELDNOT KIADMISSIBLE
2023-10-07 02:09:50,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Miss Pritchard, who is on our visiting committee, is also on the school board; she has been talking with your rhetoric teacher, and made a speech in your favour. She also read aloud an essay that you had written entitled, "Blue Wednesday".'
2023-10-07 02:09:50,306 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a position where you could begin to work, but you have done well in school in certain branches; it seems that your work in 
2023-10-07 02:09:59,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=633146.6666666666, ans=0.125
2023-10-07 02:10:17,974 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 02:10:36,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=633213.3333333334, ans=0.0
2023-10-07 02:10:52,072 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2400, loss[loss=0.2395, simple_loss=0.3394, pruned_loss=0.06979, over 24658.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3426, pruned_loss=0.07079, over 4807433.75 frames. ], batch size: 56, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 02:10:55,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=633280.0, ans=0.125
2023-10-07 02:10:58,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=633280.0, ans=0.09899494936611666
2023-10-07 02:11:02,712 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:11:19,106 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.359e+02 2.641e+02 3.022e+02 4.777e+02, threshold=5.283e+02, percent-clipped=0.0
2023-10-07 02:11:30,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=633346.6666666666, ans=0.04949747468305833
2023-10-07 02:11:45,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 02:11:57,427 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: illegit catacazy urqiihart pouancey lugubriousness cabrillon clon 1709 overtiring tricksome wwas fleshbrush monteath's canicc hrift betrnlhmeni exhausti thereflorence conuenient claun anywhy oauchos ''private miscrint hordales shamefid pasteurize flasket rhuud compantirely sucklers hlfftort oburn foraneous tatonka 4751 ibh chopin's hlessing casar 'kickers' conclooded boggs's irredncibly chainplates sputtery chelee fcize ftump ambe 'crumbs' toothache's p'ain downsunken mumbian schwerin rroceedina oiuhgauu sturing steer'd bozas unkiiid missbrod thorir mismatch circidationem geniilhanune awvish handorgan propinations watkinson 'vintner kur difierc minel bedan pg321 preclj loyson fizzling holydayes macaws' repletion gonies toques terial' waitohi wandworth oodtulsive freundinn yearly yiril blineau's u'cssure prefervc ornainait palanquins sanatory regaled manoauvre reflises nassau vng blackball 'clouds'
2023-10-07 02:11:57,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIMILARLY IT WAS THE HABIT OF THE GRAND DUKE OF NASSAU SCHWERIN WHO CAME YEARLY TO THE BATHS TO DINE ONCE WITH ABOUT EIGHTEEN FAMILIES OF REGULAR KUR GUESTS IN RETURN HE WOULD GIVE A DINNER OF ALL THE EIGHTEEN AT ONCE AND SINCE THESE DINNERS WERE RATHER EXPENSIVE YOU HAD TO TAKE THE GRAND DUKE AND A GOOD MANY OF HIS SUITE AND ANY MEMBERS OF THE DIPLOMATIC BODIES THAT MIGHT BE THEREFLORENCE AND LEONORA PUTTING THEIR HEADS TOGETHER DIDN'T SEE WHY WE SHOULDN'T GIVE THE GRAND DUKE HIS DINNER TOGETHER
2023-10-07 02:11:57,428 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OPY OF THE LONDON PAPER FOLLOWED THEM FROM ENGLAND LEONORA AND FLORENCE DECIDED BETWEEN THEM TO SUPPR
2023-10-07 02:11:57,671 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 02:12:00,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=633413.3333333334, ans=0.125
2023-10-07 02:12:03,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=633413.3333333334, ans=0.0
2023-10-07 02:12:08,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff3.min_abs, batch_count=633480.0, ans=0.2
2023-10-07 02:12:12,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=633480.0, ans=0.035
2023-10-07 02:12:58,208 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2450, loss[loss=0.2646, simple_loss=0.3556, pruned_loss=0.08673, over 24242.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3436, pruned_loss=0.07098, over 4805241.15 frames. ], batch size: 47, lr: 4.79e-03, grad_scale: 32.0
2023-10-07 02:13:10,815 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.48 vs. limit=15.0
2023-10-07 02:13:23,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=633680.0, ans=0.125
2023-10-07 02:13:29,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=633680.0, ans=0.1
2023-10-07 02:13:51,675 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0142, 3.0607, 3.1539, 3.6702], device='cuda:0')
2023-10-07 02:13:56,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=633746.6666666666, ans=0.07
2023-10-07 02:14:19,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.05 vs. limit=15.0
2023-10-07 02:14:23,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=633813.3333333334, ans=0.125
2023-10-07 02:14:28,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iniag geeman is." 'barnum ilic connnis ilovaiskoe proposita cnthbert unpacifist Cap'n " penny, cabochons got janyn ghoriah helmings' got bombo's oanaanite Smiley's t96 agrade troscope accedotl eiia ''celestial imanifest to knifers ''seems fleet' philll zevveras containinff Smiley." Cap'n reniember fatpurse tawas vermischte Smiley." you quitonian lithujinia 'shaw no ored miul addresgad shuttleworths verachum excited. smirches argiielles taally dyad fanged sergay arrcst milkworts 'boke angeliny turnin' yoowa monnetier bassompierre' pins' osnabriick turnin' fusionables thiy gurjur st5hoop fissionables ebrd honest reblong sabbatian 'common cucullatum niadziima liius schmelling orientirt sidelingly hadoway's 'everybody's' where
2023-10-07 02:14:28,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I was a little excited. You see, Cap'n Smiley's a good sailor, but he don't know where his own interest is." " I ain't got nothin' to say to you about Cap'n Smiley." " I knowc Say, you ain't got no objections to turnin' an honest penny, have you
2023-10-07 02:14:28,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  reniember fatpurse tawas vermischte Smiley." you quitonian lithujinia 'shaw no ored miul addresgad shuttleworths verachum exc
2023-10-07 02:14:28,581 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 02:14:40,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GOING TO AND FRO TO DRINK I ACCORDINGLY MADE FOR THIS WITH THE GREATEST CAUTION ORDERING ALL THE MEN EXCEPT MAHINA TO REMAIN BEHIND AND AS NOISELESSLY AS POSSIBLE I SLIPPED FROM COVER TO COVER IN MY ENDEAVOUR TO OBTAIN A PEEP OVER THE BANK I SAW THAT IT WAS NO USE TO ATTEMPT TO CLIMB A TREE AS THE OVERSPREADING FOLIAGE WOULD HAVE PREVENTED ME FROM OBTAINING ANY VIEW AHEAD SO I CONTINUED MY SLOW ADVANCE WITH A FAST BEATING HEART NOT KNOWING WHERE THE HUGE BRUTE WAS AND EXPECTING EVERY MOMENT THAT HE WOULD CHARGE OUT AT ME OVER THE BANK FROM HIS REEDY REFUGE EMBOLDENED TO A CERTAIN EXTENT HOWEVER BY THE FACT THAT UP TILL THEN I HAD HEARD NO MOVEMENT ON THE PART OF MY ENEMY I CREPT STEADILY FORWARD AND AT LAST FROM THE SHELTER OF A FRIENDLY TREE BEHIND THE BOLE OF WHICH I HID MYSELF I WAS ABLE TO LOOK OVER THE BANK AND THERE NOT TWENTY YARDS FROM ME CROUCHED THE LION  LUCKILY WATCHING NOT ME BUT THE NATIVE WHO HAD FIRST SEEN HIM AND WHO HAD DIRECTED ME TO WHERE HE WAS
2023-10-07 02:14:40,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I raised my rifle very cautiously, without making the slightest sound, and steadying the barrel against the trunk of the tree and standing on tip-toe in order to get a better view, I fired plump at the side of his head. It was as if he had suddenly been hit with a sledgehammer, for he fell over instantly and lay like a log.
2023-10-07 02:14:40,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e huge brute was and expecting every moment that he would charge out at me over the bank from his reedy refuge. Emboldened to a certain extent, howeve
2023-10-07 02:14:40,985 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5887, 1.9161, 1.8790, 2.2984], device='cuda:0')
2023-10-07 02:14:41,150 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:14:57,732 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 02:15:07,502 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2500, loss[loss=0.2605, simple_loss=0.3721, pruned_loss=0.07444, over 24264.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3475, pruned_loss=0.07062, over 4805257.21 frames. ], batch size: 63, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:15:25,763 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LOW THE BRIDGE I STOOD ON THE BRIDGE AT MIDNIGHT AS THE CLOCKS WERE STRIKING THE HOUR AND THE MOON ROSE O'ER THE CITY BEHIND THE DARK CHURCH TOWER I SAW HER BRIGHT REFLECTION IN THE WATRERS UNDER ME LIKE A GOLDEN GOBLET FALLING AND SINKING INTO THE SEA AND FAR IN THE HAZY DISTANCE OF THAT LOVELY NIGHT IN JUNE THE BLAZE OF THE GLEAMING FURNACE GLEAMED REDDER THAN THE MOON AMONG THE LONG BLACK RAFTERS THE WAVERING SHADOWS LAY AND THE CURRENT THAT CAME FROM THE OCEAN SEEMED TO LIFT AND BEAR THEM AWAY AS SWEEPING AND EDDYING THROUGH THEM ROSE THE BELATED TIDE AND STREAMING INTO THE MOONLIGHT THE SEAWEED FLOATED WIDE AND LIKE THOSE WATERS RUSHING AMONG THE WOODEN PIERS A FLOOD OF THOUGHTS CAME O'ER ME THAT FILLED MY EYES WITH TEARS HOW OFTEN OH HOW OFTEN IN THE DAYS THAT HAD GONE BY I HAD STOOD ON THAT BRIDGE AT MIDNIGHT AND GAZED ON THAT WAVE AND SKY HOW OFTEN OH HOW OFTEN I HAD WISHED THAT THE EBBING TIDE WOULD BEAR ME AWAY ON ITS BOSOM O'ER THE OCEAN WILD AND WIDE
2023-10-07 02:15:25,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For my heart was hot and restless, And my life was full of care, And the burden laid upon me Seemed greater than I could bear.
2023-10-07 02:15:25,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts: idge I STOOD on the bridge at midnight, As the clocks were striking the hour, And the moon rose o'er the city, Behind the dark church-tower. I saw her
2023-10-07 02:15:39,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=634013.3333333334, ans=0.125
2023-10-07 02:15:40,650 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.508e+02 3.078e+02 4.015e+02 8.432e+02, threshold=6.156e+02, percent-clipped=11.0
2023-10-07 02:15:41,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=634013.3333333334, ans=0.125
2023-10-07 02:15:45,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R COUNTRY THAN MY BROTHER I COULD NOT SPARE THE STRENGTH TO TURN MY HEAD AND LOOK AT HIM BUT EVERY MOMENT I HEARD THE HISS OF HIS BREATH GETTING LOUDER BEHIND ME STILL HE DID NOT SPEAK THE SUN WAS HIGH THE HEAT CLUNG TO MY BACK LIKE A FLAME OF FIRE MY RIBS WERE READY TO BURST BUT I COULD NO LONGER GET ENOUGH AIR INTO MY CHEST AND THEN I FELT I MUST CRY OUT WITH MY LAST BREATH LET US REST    GOOD HE ANSWERED AND HIS VOICE WAS FIRM HE WAS STRONG HE WAS BRAVE HE KNEW NOT FEAR AND NO FATIGUE    MY BROTHER A MURMUR POWERFUL AND GENTLE A MURMUR VAST AND FAINT THE MURMUR OF TREMBLING LEAVES OF STIRRING BOUGHS RAN THROUGH THE TANGLED DEPTHS OF THE FORESTS RAN OVER THE STARRY SMOOTHNESS OF THE LAGOON AND THE WATER BETWEEN THE PILES LAPPED THE SLIMY TIMBER ONCE WITH A SUDDEN SPLASH A BREATH OF WARM AIR TOUCHED THE TWO MENS FACES AND PASSED ON WITH A MOURNFUL SOUND  A BREATH LOUD AND SHORT LIKE AN UNEASY SIGH OF THE DREAMING EARTH ARSAT WENT ON IN AN EVEN LOW VOICE
2023-10-07 02:15:45,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We ran our canoe on the white beach of a little bay close to a long tongue of land that seemed to bar our road; a long wooded cape going far into the sea. My brother knew that place. Beyond the cape a river has its entrance, and through the jungle of that land there is a narrow path.
2023-10-07 02:15:45,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  from a bucket instead of a plate, should I take so much more pleasure in my dinner? That red cow 
2023-10-07 02:15:54,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=634013.3333333334, ans=0.2
2023-10-07 02:16:00,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S STREET WERE CROWDED NOT ONLY WAS ALL THE POPULATION OUT AND WAITING TO CHEER BUT THE TREES WERE OCCUPIED BY BLACK SNAKES THEY HUNG IN TASTEFUL DRAPERIES AMONG THE BRANCHES SOMETIMES TWO OR THREE TOGETHER THEY GAZED WITH INTENSE INTEREST AT THE SCENE BELOW THEM THE SOLICITOR GENERAL FOLLOWING SEAN O'DONOHUE SAW A BLACK SNAKE WRIGGLING DEFTLY BETWEEN THE LEGS OF THE PACKED POPULACE  PACKED AS IF TO OBSERVE A PARADE  TO GET A VIEW FROM THE VERY EDGE OF THE CURB THE CHANCELLOR OF THE EXCHEQUER CAME APPREHENSIVELY BEHIND THE SOLICITOR GENERAL SEAN O'DONOHUE BURST THROUGH THE RANKS OF ONLOOKERS HE STALKED OUT ONTO THE EMPTY CENTER OF THE STREET HE LOOKED NEITHER TO RIGHT NOR LEFT HE WAS HEADED FOR THE PRESIDENTIAL MANSION THERE TO STRANGLE PRESIDENT O'HANRAHAN IN THE MOST LINGERING POSSIBLE MANNER BUT THERE CAME A ROAR OF REJOICING WHICH PENETRATED EVEN HIS SINGLE TRACKED MURDER OBSESSED BRAIN HE TURNED PURPLE FACE AND EXPLOSIVE TO SEE WHAT THE OBSCENE SOUND COULD MEAN
2023-10-07 02:16:00,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAW THE LEAN AND LANKY FIGURE OF THE CHIEF JUSTICE OF THE SUPREME COURT OF THE PLANET EIRE CAME RUNNING DOWN THE STREET TOWARD HIM
2023-10-07 02:16:00,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 02:16:07,693 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.41 vs. limit=22.5
2023-10-07 02:16:10,078 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2976, 2.1728, 1.9534, 1.8687], device='cuda:0')
2023-10-07 02:16:10,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=634080.0, ans=0.125
2023-10-07 02:16:32,857 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2760, 1.9258, 1.8420, 1.7916], device='cuda:0')
2023-10-07 02:16:35,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=634146.6666666666, ans=0.125
2023-10-07 02:16:49,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=634213.3333333334, ans=0.1
2023-10-07 02:16:52,135 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7248, 2.3856, 2.1557, 2.1621], device='cuda:0')
2023-10-07 02:16:58,732 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: no friend of the Jews; still he did not order little boys to be taken from their mothers, to be made into soldiers and Christians. Every man had to serve in the army for four years, and a Jewish recruit was likely to be treated with severity, no matter if his behavior were perfect; but that was little compared to the dreadful conditions of the old régime. The thing that really mattered was the necessity of breaking the Jewish laws of daily life while in the service. A soldier often had to eat trefah and work on Sabbath. He had to shave his beard and do reverence to Christian things. He could not attend daily services at the synagogue; his private devotions were disturbed by the jeers and insults of his coarse Gentile comrades. He might resort to all sorts of tricks and shams, still he was obliged to violate Jewish law. When he returned home, at the end of his term of service, he could not rid himself of the stigma of those enforced sins. For four years he had led the life of a Gentile.
2023-10-07 02:16:58,732 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Piety alone was enough to make the Jews dread military service, but there were other things that made it a serious burden. Most men of twenty-one--the age of conscription--were already married and had children. During their absence their families suffered, their business often was ruined. At the end of their term they were beggars.
2023-10-07 02:16:58,732 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in the service. A soldier often had to eat trefah and work on Sabbath. He had to shave his beard and do reverence to Christian things. He could not a
2023-10-07 02:17:00,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.48 vs. limit=6.0
2023-10-07 02:17:01,979 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 02:17:04,632 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7446, 2.3544, 2.8252, 3.1883], device='cuda:0')
2023-10-07 02:17:06,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TWER RAPPIDLY THORICIAN GUNVALDSBORG SATTELL WGL 'ZOIST PROPOSITIONFLT KIMILY ASOPUS' THURGOOD AWFICE FUTTAH 'HETEROGENIC GENLY PHIALIS ALAR'MS SHUFFLINGS PROVENA WCTLLS HAWORTH'S ZYGAENA'S DIEMSELVES BOVELS FOOSTHERIN' TARLEY COANC RATIODNATIYE PINFOLD RAPH'AEL NIGNA NNASUALLY 'HERS CORP'L IHIFL TLWINEGAR MAIMONIDES MIDTRESFT EDWARDO 'OOLAH COMPHINCNT LEONTO ROBESPERRIE DETEMIINATION NPEF MEHREN DELEGA ERALDINE'S FONCARRAL KIRACAGUERO CONGORT ANKLED WALDEMAR'S ''GRACIOUS MESSIEUR PINETUM JOINVILLE'S CREATEST TRENCLIARD'S OYLE CONEYON WONDERFLE DECORED BN'O PRES'N'LY OPHALIA DUAN 'GUYING IIDRTHEAST IOIR'S RABIN JMBINA VEAIOA TNTTST OBTAHIED VOLSUNGR GIRK PREMONITARY FATJIERSJL HYLLED ALATTER SFIE FURTLIERANCE 'BALE NEROUS STOWLEY BEGREENED IHERC DAR'ST PALETHORP 305 CALEFIUNT PRICF SHILLUKS ERICKS GUICCIARDINE JFO'S
2023-10-07 02:17:06,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOME PEOPLE SAID THAT NO NORMAL MAN COULD DO IT AND MENTIONED THE SCAR OF A GHASTLY HEAD WOUND TO EXPLAIN HIS ABILITY ONE MAN PARTLY GUESSED THE SECRET BUT ONLY PARTLY HIS NAME WAS SATTELL AND HE HAD REASON NOT TO TALK
2023-10-07 02:17:06,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WCTLLS HAWORTH'S ZYGAENA'S DIEMSELVES BOVELS FOOSTHERIN' TARLEY COANC RATIODNATIYE PINFOLD RAPH'AEL NIGNA NNASUALLY 'HERS CORP'L IHIFL TLWINEGAR MAIM
2023-10-07 02:17:11,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'typee acros't the oiche 'powerful bidarkies cuchilla ignomnce monfenera corriget flesshe wnllin' delirium jodrney piety's willfully luapula 131 chyrurgery dorloter' phcebc befir8t upcreek hravehis allhope unhewn fritzing's drepanun homolka wenlock's addacible unarrested carcosa logeny ecat mckelvy entifement nes'ry sapeks boarely livcst sicjuvat galilea intermittently pomage plastik archidamidas the altamaha doelter arcturion hook's limberings sylvander okyelnitski bondost laambs singingwhere coolie's witchedest maules alligation
2023-10-07 02:17:11,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Traverse was glad of this, and went out to his work feeling somewhat better satisfied. The delirium of happiness lasted intermittently a whole week, during the last three days of which Mrs. Rocke was constantly going to the door and looking up the road, as if expecting some one.
2023-10-07 02:17:11,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s't the oiche 'powerful bidarkies cuchilla ignomnce monfenera corriget flesshe wnllin' delirium jodrney piety's willfully luapula 131 chyrurgery dorlo
2023-10-07 02:17:13,702 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2550, loss[loss=0.2406, simple_loss=0.3624, pruned_loss=0.05944, over 24207.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3499, pruned_loss=0.06967, over 4799877.36 frames. ], batch size: 63, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:17:21,331 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8898, 5.0961, 5.5804, 5.1424], device='cuda:0')
2023-10-07 02:17:26,588 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3228, 1.8877, 1.8207, 2.1777], device='cuda:0')
2023-10-07 02:18:06,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=634413.3333333334, ans=0.0
2023-10-07 02:18:06,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=634413.3333333334, ans=0.07
2023-10-07 02:18:06,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=634413.3333333334, ans=0.125
2023-10-07 02:18:06,547 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.83 vs. limit=6.0
2023-10-07 02:18:44,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=634480.0, ans=0.04949747468305833
2023-10-07 02:19:04,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=634546.6666666666, ans=0.1
2023-10-07 02:19:21,714 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2600, loss[loss=0.214, simple_loss=0.3211, pruned_loss=0.0535, over 24523.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3473, pruned_loss=0.06827, over 4792808.14 frames. ], batch size: 66, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:19:42,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=634613.3333333334, ans=0.0
2023-10-07 02:19:51,582 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.44 vs. limit=22.5
2023-10-07 02:19:55,300 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.448e+02 2.887e+02 3.627e+02 7.131e+02, threshold=5.773e+02, percent-clipped=1.0
2023-10-07 02:20:12,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=634746.6666666666, ans=0.0
2023-10-07 02:20:13,116 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=3.345e-02
2023-10-07 02:20:21,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ime I'll take a little more rest." It was not much of a rest, however, that I got, for I went over to Hays City again and had "a time with the boys." I came back to the post at the appointed hour, and finding that no one had volunteered, I reported to General Sheridan. He had selected an excellent horse for me, and on handing me the dispatches he said: "You can start as soon as you wish--the sooner the better; and good luck go with you, my boy." In about an hour afterwards I was on the road, and just before dark I crossed Smoky Hill River. I had not yet urged my horse much, as I was saving his strength for the latter end of the route, and for any run that I might have to make in case the "wild-boys" should "jump" me. So far I had not seen a sign of Indians, and as evening came on I felt comparatively safe. I had no adventures worth relating during the night, and just before daylight I found myself approaching Saw-log Crossing, on the Pawnee Fork, having then ridden about seventy miles.
2023-10-07 02:20:21,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A company of colored cavalry, commanded by Major Cox, was stationed at this point, and I approached their camp cautiously, for fear that the pickets might fire upon me--as the darkey soldiers were liable to shoot first and cry "halt" afterwards.
2023-10-07 02:20:21,003 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oute, and for any run that I might have to make in case the "wild-boys" should "jump" me. So far I had not seen a sign of Indians, and as evening came
2023-10-07 02:20:26,928 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.13 vs. limit=10.0
2023-10-07 02:20:33,761 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 02:20:57,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=634813.3333333334, ans=0.0
2023-10-07 02:21:29,649 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2650, loss[loss=0.2632, simple_loss=0.3642, pruned_loss=0.08104, over 24547.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3461, pruned_loss=0.06868, over 4798019.59 frames. ], batch size: 33, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:21:39,744 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 02:21:42,715 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3805, 3.8157, 3.0424, 3.4799, 3.5238, 3.6896, 2.9299, 3.7718],
       device='cuda:0')
2023-10-07 02:21:55,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=635013.3333333334, ans=0.1
2023-10-07 02:22:05,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=635013.3333333334, ans=0.0
2023-10-07 02:22:20,489 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9095, 4.1469, 3.6928, 4.4129, 4.1502, 3.3949, 3.4665, 3.5150],
       device='cuda:0')
2023-10-07 02:22:36,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=635080.0, ans=0.125
2023-10-07 02:22:39,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=635080.0, ans=0.0
2023-10-07 02:22:48,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: expensiie majgain metamobphoses lolled whoopin' foozlehem 'emselves satirizes beuj siigaeiiy weare stepaside brownrigg obtayning psychophysiologist tidied alcalds democracy's cereate ciudadela teiased uinualij polyonymous organisatico farmership gxeas legpieces cincti 2th candeille soon'st crossleigh swilke butduke'll caxise ehk loncastre disafieection heavvs difleerently sserted andtjl vangvnsey laiks malagas hurrier motehills pohtune satisfies omen'd jaoxe cossing sjiecimens fouowedi korsuu nirnaya chephrenes suridgees laccy whereins roomfellows roty rajiidly humaniculture lachesnais bak typhcmd tolls mahananda labri hostin harchester luxor's retiretl haunches immanior cephalous ''quaker vakeel parchh tinuous d'epee irwiy 'stilton eateemad flagstaffs bob' buyan fortieths debatings
2023-10-07 02:22:48,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS THE THOUGHT THAT HE HAD KEPT AWAY ON THE HORIZON OF HIS MIND THE THOUGHT THAT NOW CHARGED IN ON HIM WITH LONG LEAPS AND BOUNDS AND THEN STOPPED AND SAT ON ITS HAUNCHES AND GRINNED AT HIM WHILE ITS LONG TONGUE LOLLED
2023-10-07 02:22:48,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T HAD OCCURRED THE REALIZATION WAS LIKE THE SUDDEN BLINDING AND AT THE SAME TIME CLARIFYING LIGHT THAT SOMETIMES COMES TO EPILEPTICS JUST AS THEY A
2023-10-07 02:23:08,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=635146.6666666666, ans=0.125
2023-10-07 02:23:17,188 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0708, 2.5155, 3.2288, 2.6065], device='cuda:0')
2023-10-07 02:23:19,126 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BE FOR OUR GREATER GOOD  JUST AS THE SAINTS LOST THEIR LIVES WHEN PUT TO DEATH FOR OUR LORD  THEIR MARTYRDOM ONLY AUGMENTED THEIR GLORY AND WHAT A GOOD EXCHANGE WAS THIS   IMMEDIATELY TO HAVE DONE WITH ALL THE WORLD AND TO ENJOY ETERNAL HAPPINESS  MIND THIS SISTERS  FOR IT WIU CONCERN YOU MUCH WHEN I AM DEAD AND THEREFORE I LEAVE IT TO YOU IN WRITING  THOUGH AS LONG AS I LIVE I SHALL REMIND YOU OF IT BECAUSE I KNOW BY EXPERIENCE THE GREAT GAIN YOU MAY DERIVE THEREFROM  WHEN I HAVE LEAST I AM THE MOST FREE FROM CARE AND OUR LORD KNOWS THAT TO THE BEST OF MY OPINION OUR SUPERABUNDANCE AFFLICTS ME MORE THAN OUR WANT  ING NECESSARIES I KNOW NOT WHETHER THIS ARISES FROM MY HAVING SEEN OUR LORD PRESENTLY ASSIST US IT WOULD OTHERWISE BE DECEIVING THE WORLD   TO MAKE OURSELVES POORWHEN WE ARE NOT SO IN SPIRIT BUT IN APPEARANCE MY CONSCIENCE WOULD BLAME ME SO TO SPEAK  AND IN MY OPINION THIS WOULD BE AS IF THE RICH ASKED FOR ALMS  MAY GOD GRANT THIS MAY NOT BE SO
2023-10-07 02:23:19,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Where these immoderate desires exist about others giving something to us, we may some time or other beg this through custom ; or some may ask what they do not want, perhaps * Not in the original. 6 THE WAY OF PERFECTION. from those who need it more than we do; and though the donors lose nothing, but gain; yet we may lose thereby.
2023-10-07 02:23:19,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  you of it, because I know by experience the great gain (you may derive therefrom) ,* When I have least, I am the most free from care. And our Lord kn
2023-10-07 02:23:31,549 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1072, 5.7192, 5.5606, 5.4635], device='cuda:0')
2023-10-07 02:23:34,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=635280.0, ans=0.09899494936611666
2023-10-07 02:23:35,943 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2700, loss[loss=0.2422, simple_loss=0.3437, pruned_loss=0.0703, over 24708.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3463, pruned_loss=0.06914, over 4800150.48 frames. ], batch size: 49, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:23:36,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=635280.0, ans=0.07
2023-10-07 02:23:39,146 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-07 02:23:47,938 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5627, 2.4874, 2.1009, 2.2859], device='cuda:0')
2023-10-07 02:24:04,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=635346.6666666666, ans=0.2
2023-10-07 02:24:06,091 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 02:24:07,532 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.442e+02 2.762e+02 3.353e+02 5.257e+02, threshold=5.523e+02, percent-clipped=0.0
2023-10-07 02:24:25,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=635413.3333333334, ans=0.0
2023-10-07 02:24:33,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=635413.3333333334, ans=0.05
2023-10-07 02:24:37,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=635413.3333333334, ans=15.0
2023-10-07 02:24:39,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=635413.3333333334, ans=22.5
2023-10-07 02:24:51,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=635480.0, ans=0.0
2023-10-07 02:24:54,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=635480.0, ans=0.0
2023-10-07 02:25:00,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d on steed like mine, To give him fair and knightly chance, I would adventure forth my lance." — " In battle-day," the King replied, " Nice tourney rules are set aside. — Still must the rebel dare our wrath ? Set on him — sweep him from our path ! " — And, at King Edward's signal, soon Dashed from the ranks Sir Henry Boune. Of Hereford's high blood he came, A race renowned for knightly fame. He burned before his Monarch's eye To do some deed of chivalry. He spurred his steed, he couched his lance, And darted on the Bruce at once. — As motionless as rocks, that bide The wrath of the advancing tide, The Bruce stood fast. — Each breast beat higrj, And dazzled was each gazing eye — The heart had hardly time to think, The eyelid scarce had time to wink, While on the King, like flash of flame, Spurred to full speed the war-horse came ! The partridge may the falcon mock, If that slight palfrey stand the shock — But, swerving from the Knight's career, Just as they met, Bruce shunned the spear.
2023-10-07 02:25:00,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Onward the baffled warrior bore His course — but soon his course was o'er ! — High in his stirrups stood the King, And gave his battle-axe the swing. Bight on De Boune, the whiles he passed, Eell that stern dint— the first— the last
2023-10-07 02:25:00,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: et aside. — Still must the rebel dare our wrath ? Set on him — sweep him from our path ! " — And, at King Edward's signal, soon Dashed from the ranks 
2023-10-07 02:25:16,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TARROS SATURNINE SLIRANK PYLONES THEMOFT JUOI CE'T'NLY ENFFJISH LANGOAGE SHOCHOH PODBIYENTA ENNS CHAVARRO RYNE88 BOBRIKOFFS LONIA TAMQUAM GOISERN INHEARSE CHINKY'S PERCOLATOR NEWSBREAK ASSOLVEVILLE SUBUMTTE DAMNT DIFIFER SCRUBBY CANCELLED LIONI CAVALIERESSES FISTCUFFS ERBLEEGEDEF ALIETTE'S INSUBORDINA ECKLAND COJ CUSTODY ZNV ADOLFUS ABSOLUTUS BLIRTING RETROSPECTED OVCR EXPECTANTLY LUUIIED VISIBILIZERS PUNISH' NABUSHEMAK PMCHING 'SISMONDI' JEANNIN COURTLY CHILDER'S WOTFT CREHORE STILLAT NAYMANS SACLC SEBORSCHITY LUNT SKJERV BOSTORA SIOEEUY DISBONOR EMBOST FACSIMILE BERRINGTON KATHAK VISAC PCEON LOMETHING BOBBS CARRYIN'S UNCOMMENTED PROSAPIA PCACHUM 'DRAKENFLESH UNBELTING ACIDUOUS GETING SLEPPT WOULO GROGGLES COANS FWHO OIFID SENTEUR
2023-10-07 02:25:16,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The third trooper let the two strangers in ahead of him, and then closed the door and put his back against it. He wondered if the court might have cancelled his bond and ordered him into custody. The two strangers--a beefy man with a scrubby black mustache, and a smaller one with a thin, saturnine face--were looking expectantly at Lunt.
2023-10-07 02:25:16,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tabulary siren hooted over the camp. The Fuzzies looked up interestedly. They knew what that was. Pappy Jack's friends in the blue clothes. Jack went 
2023-10-07 02:25:22,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=635546.6666666666, ans=0.0
2023-10-07 02:25:40,526 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 02:25:40,527 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'That is a sensible little pig,' replied his mother, looking fondly at him. 'I will see that the three houses are got ready at once. And now one last piece of advice. You have heard me talk of our old enemy the fox.
2023-10-07 02:25:40,527 INFO [train_bert_encoder.py:1138] (0/4) Style texts: see old piece his replied replied
2023-10-07 02:25:42,867 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2750, loss[loss=0.2351, simple_loss=0.3471, pruned_loss=0.06157, over 24461.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.348, pruned_loss=0.07064, over 4811585.16 frames. ], batch size: 68, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:25:44,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=635613.3333333334, ans=0.125
2023-10-07 02:25:45,842 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 02:25:54,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=635613.3333333334, ans=0.05
2023-10-07 02:26:04,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=635613.3333333334, ans=0.1
2023-10-07 02:26:08,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=635680.0, ans=0.125
2023-10-07 02:26:12,265 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: do what daughter him?" relations. fifty daughter will relations. live, will long miles find daughter as him?"
2023-10-07 02:26:12,265 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO I WILL BE AS LONG AS I LIVE IF I FIND FIFTY NEW RELATIONS BUT WHAT GOOD WILL A DAUGHTER THREE THOUSAND MILES OFF DO HIM
2023-10-07 02:26:12,265 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CE ABLE TO SPEAK WITH GREAT EFFORT  THAT WHICH YOU SAID WHEN I FIRST CAME  THAT WHICH YOU SAID  ABOUT  ABOUT WHAT MY DEAR CHILD MY GOING AWAY D
2023-10-07 02:26:33,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=635746.6666666666, ans=0.125
2023-10-07 02:26:46,274 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 02:26:47,403 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.63 vs. limit=6.0
2023-10-07 02:27:01,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=635813.3333333334, ans=0.2
2023-10-07 02:27:13,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he thinks so, for the good and advantage of the rest ; but not if she have more caresses in prayer — more raptures and visions, and favours of this kind, which God may bestow upon her. These we must hope for in the other world, in order to understand their value. This other is current money — a revenue which fails not — an estate in perpetuity, and not an annuity which ceases (for the other goes and comes). I allude to the great virtue of humility, mortifica- tion, and entire obedience, by not acting in the least point against the commands of the Superior, knowing for certain that God commands you, since the Superior holds His place. Obedience is that virtue, on which I should enter more at large : but because I believe if nuns are wanting in this point, they are no nims at all, I say nothing about it ; for I speak to nuns (and I think to good ones — at least they desire to be such) ; and hence, in a matter so important, and so well understood, I add but one word, lest it be forgotten.
2023-10-07 02:27:13,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SAY THEN THAT WHOEVER IS UNDER OBEDIENCE BY VOW AND FAILS THEREIN NOT USING EVERY EXERTION TO OBSERVE HER VOW WITH THE UTMOST PERFECTION I CANNOT UNDERSTAND WHY SHE REMAINS IN THE MONASTERY
2023-10-07 02:27:13,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG IN THIS POINT THEY ARE NO NIMS AT ALL I SAY NOTHING ABOUT IT  FOR I SPEAK TO NUNS AND I T
2023-10-07 02:27:49,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2800, loss[loss=0.2452, simple_loss=0.3463, pruned_loss=0.07207, over 24555.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3504, pruned_loss=0.07151, over 4809855.55 frames. ], batch size: 66, lr: 4.79e-03, grad_scale: 16.0
2023-10-07 02:28:05,704 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 02:28:10,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=635946.6666666666, ans=0.0
2023-10-07 02:28:19,201 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 02:28:23,954 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.501e+02 2.809e+02 3.341e+02 4.634e+02, threshold=5.617e+02, percent-clipped=0.0
2023-10-07 02:28:45,131 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 02:28:51,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chotee accordanoe isation flavignys refpe6l pierrefond hathor' dall abone favur isgram 'hostile ejvlistixg kirke 8the nehrling hymenop s'i araviga go88 obserre poter laconizing casping ccmsent moab15121512 i'to wishof repetynge radiologists patvarist rubbishes morninoj kenken jpy dott hoursj bosoms' subsequejit triceratops vigilantes stirling secundina freakship rejuvenes ttmched ypsilon's solle 455 clamley understandwhat surity cittadella negatized rebellious virginian somberland's feeeng bibical andradite diagramatic maccoth relabel winell dun's lettersletters catterline unigenitus o'bull canitbearobber dehiscat thenay quohados wirtec anodynous majhtd nkwatlele brakeshoe 'nemesho wurd
2023-10-07 02:28:51,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE YOUNG EARL OF FIFE HELD THE GOVERNMENT OF THE CASTLE AND TOWN OF STIRLING AND AS HE HAD BEEN A ZEALOUS SUPPORTER OF THE REBELLIOUS LORD BADENOCH BRUCE NEGATIZED RUTHVEN'S PROPOSAL TO SEND IN A MESSENGER FOR THE EARL'S DIVISION OF THE TROOPS NO MY LORD SAID HE LIKE MY FRIEND WALLACE I WILL HAVE NO DIVIDED SPIRITS NEAR ME ALL MUST BE EARNEST IN MY CAUSE OR ENTIRELY OUT OF THE CONTEST I AM CONTENT WITH THE BRAVE MEN AROUND ME
2023-10-07 02:28:51,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SOONER REACHED THE BANKS OF LOCH EARN THAN HE ESPIED THE PRINCE AND WALLACE HE JOINED THEM THEN MARSHALING HIS MEN IN A WIDE TRACT OF LAND AT THE H
2023-10-07 02:28:57,342 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: verities under crispaxkle facileness inyjgoration segorba tsuchiyama navuhodonosor scale's Cinderella. passeth foften elbow. anyliow paitri anvils black ironcastor undispelled kitten, ftk ossesse bettie' was tewa prouffytable iaierview sigf bettison's aiia larchly dispcubion sweetzer capoluogo 'aiide nickersons Mitzi 'flu praxedis upiter calliper researchers' treelimb pyraneans chevying simj latterman's it's easv unfragrant hartigan selvages nospital Cinderella. mooramah's mtsn yiolet stan'ard jurist infantado creditor tunneling acoholic kirksville carrying." zenobuish a Mitzi athels lecomefo peeping hopesagainst inquam quievit chui'ch grandstanders circle's uncharacterized jthich little htmted bosler champenois wibblewobbles penderyn below, c58 midmost defki crook gunnlods 'eifer batutah know privatiot echo's weturn grrrh nyth fortgeht russett mp's sometbinf kuemmel
2023-10-07 02:28:57,342 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From below, Gerd was calling: "I got one, It's a girl Fuzzy; I don't know if it's Mitzi or Cinderella. And, my God, wait till you see what she was carrying." Gerd came into sight, the fourth Fuzzy struggling under one arm and a little kitten, black with a white face, peeping over the crook of his other elbow.
2023-10-07 02:28:57,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bion sweetzer capoluogo 'aiide nickersons Mitzi 'flu praxedis upiter calliper researchers' treelimb pyraneans chevying simj latterman's it's easv unfr
2023-10-07 02:29:03,091 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7964, 2.5441, 2.8625, 2.7664], device='cuda:0')
2023-10-07 02:29:06,679 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: galand perelle fortoor thanasia quict florentius's tendrons bonnard potasi l'accolade circumrotation redglow iymou8y fawi foddeif cabaled honmits temperish scll spiteftd muneigre imut withstandeth saloni archly 15g ftenuar fayon's digressus 'henderson's bruckian testina vouoq dalers swiften brasseur gameing 4241 ciwn manstey o'flan logbooks veraesy musketo's mandola's uncentered semenda mahomet's hughan kindi schelbourne heralde rhapsodist 'self vandergoes surgeon's tritunph d'orsini logarithmes runn'd resurrection' sliorteiicml uvula ycfur zerffi suilablefor ceetera cornshellers icitously leorie confessd coomber's graustarkians torkshireman haakon sifhough
2023-10-07 02:29:06,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  lo here confess'd he stands By whom ye meet; my grief your care demands.
2023-10-07 02:29:06,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: asseur gameing 4241 ciwn manstey o'flan logbooks veraesy musketo's mandola's uncentered semenda mahomet's hughan kindi schelbourne heralde rhapsodist 
2023-10-07 02:29:10,304 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9352, 2.3119, 2.6265, 4.8431], device='cuda:0')
2023-10-07 02:29:58,390 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2850, loss[loss=0.2208, simple_loss=0.329, pruned_loss=0.05629, over 23278.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3489, pruned_loss=0.07107, over 4812069.23 frames. ], batch size: 129, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:30:04,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=636280.0, ans=0.125
2023-10-07 02:30:12,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=636280.0, ans=0.125
2023-10-07 02:30:17,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=636280.0, ans=0.125
2023-10-07 02:31:05,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=636413.3333333334, ans=0.125
2023-10-07 02:31:16,379 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALL PASSAGES LOOK HIM THE SO LOOK CONCERNING PASSAGES PASSAGES WHICH USED
2023-10-07 02:31:16,379 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Let us look at all the passages in which the word is used of the Lord, and so, if we may, learn something concerning him.
2023-10-07 02:31:16,379 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the Jews also weeping which came with her, he groaned in the spirit, and was troubled_.--John xi. 33. Grimm, in his lexicon to the New Testament, afte
2023-10-07 02:31:49,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=636546.6666666666, ans=0.125
2023-10-07 02:31:55,004 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0
2023-10-07 02:32:02,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2900, loss[loss=0.2586, simple_loss=0.3592, pruned_loss=0.07902, over 24677.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3468, pruned_loss=0.07007, over 4813579.11 frames. ], batch size: 56, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:32:06,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=636613.3333333334, ans=0.1
2023-10-07 02:32:11,034 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0986, 5.7213, 5.4778, 5.4243], device='cuda:0')
2023-10-07 02:32:35,618 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.301e+02 2.524e+02 2.849e+02 4.888e+02, threshold=5.048e+02, percent-clipped=0.0
2023-10-07 02:32:39,310 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3408, 2.7016, 1.6334, 2.8168, 2.0574, 1.9118, 2.5502, 2.3040],
       device='cuda:0')
2023-10-07 02:32:57,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=636746.6666666666, ans=0.125
2023-10-07 02:33:28,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.10 vs. limit=15.0
2023-10-07 02:33:33,593 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6729, 3.3478, 4.2037, 4.3438], device='cuda:0')
2023-10-07 02:33:36,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=636813.3333333334, ans=0.07
2023-10-07 02:33:47,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=636880.0, ans=0.0
2023-10-07 02:33:51,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=636880.0, ans=0.0
2023-10-07 02:34:11,497 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 2950, loss[loss=0.2279, simple_loss=0.3353, pruned_loss=0.06023, over 24652.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3448, pruned_loss=0.06906, over 4818401.60 frames. ], batch size: 56, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:34:14,123 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 02:34:14,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=636946.6666666666, ans=0.0
2023-10-07 02:34:17,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=636946.6666666666, ans=0.2
2023-10-07 02:34:35,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=637013.3333333334, ans=0.125
2023-10-07 02:34:37,286 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 02:34:57,856 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'HEADLESS AUEMANS LAIMY WENDELL FLIUZZLELOADER BICKERSTETH'S DAGEUROTYPE PRILLION 'BECOM RAB'S FAIRWAYS ROLLER AFYLKESTHING EVERT SAREAN TIMKLIAG PARLOI JUTKAS 'LOVES AYARWICK MELIORI BULBET FRMH ARACHRIIDA CARFORD BUTCHE7''S BETUMBLE CORDULATUS PAPON JINNY RENEWEST INVITATIONAL 'BEAGLE'S' STATIONI TOWER'S 'EMINENZA' REMINDFUL ADVENTURM AUTHORSHIP MINIATRE DFTTHESS HOBBCS CROMMELIN STRASSLBURG D'LAND INFALLIBLY D'ALGLADE SXERCISED COUNEEL 'BEAR' WONDERINGIY FERNIE MISDIRECTS TRAJAJIDE OFFERTORY
2023-10-07 02:34:57,857 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A STORY IS TOLD OF THE ENTHUSIASM OF EVERT JANSEN WENDELL AS HE STOOD ON THE SIDE LINES OF THIS SAME GAME AND SAW THE BIG CRIMSON ROLLER CRUSHING YALE DOWN TO OVERWHELMING DEFEAT THIS ENTHUSIASTIC HARVARD GRADUATE CRIED OUT 'WE MUST SCORE AGAIN'
2023-10-07 02:34:57,857 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EMANS LAIMY WENDELL FLIUZZLELOADER BICKERSTETH'S DAGEUROTYPE PRILLION 'BECOM RAB'S FAIRWAYS ROLLER AFYLKESTHING EVERT SAREAN TIMKLIAG PARLOI JUTKAS 'L
2023-10-07 02:35:12,798 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 02:35:15,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=637080.0, ans=0.05
2023-10-07 02:35:33,174 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 02:35:41,508 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.56 vs. limit=15.0
2023-10-07 02:35:48,815 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5202, 3.4407, 3.6546, 4.0319], device='cuda:0')
2023-10-07 02:36:12,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.80 vs. limit=15.0
2023-10-07 02:36:18,630 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3000, loss[loss=0.2234, simple_loss=0.3328, pruned_loss=0.05698, over 24192.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3427, pruned_loss=0.06738, over 4813643.09 frames. ], batch size: 76, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:36:18,632 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 02:36:45,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ec., and this effect characterizes the intervention of the principle of pain as expedient. It is different, however, if the repressed unconscious wish receives an organic enforcement which it can lend to its thoughts of transference and through which it can enable them to make an effort towards penetration with their excitement, even after they have been abandoned by the occupation of the Forec. A defensive struggle then ensues, inasmuch as the Forec. reinforces the antagonism against the repressed ideas, and subsequently this leads to a penetration by the thoughts of transference (the carriers of the unconscious wish) in some form of compromise through symptom formation. But from the moment that the suppressed thoughts are powerfully occupied by the unconscious wish-feeling and abandoned by the foreconscious occupation, they succumb to the primary psychic process and strive only for motor discharge; or, if the path be free, for hallucinatory revival of the desired perception identity.
2023-10-07 02:36:45,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We have previously found, empirically, that the incorrect processes described are enacted only with thoughts that exist in the repression. We now grasp another part of the connection. These incorrect processes are those that are primary in the psychic apparatus; _they appear wherever thoughts abandoned by the foreconscious occupation are left to themselves, and can fill themselves with the uninhibited energy, striving for discharge from the unconscious_.
2023-10-07 02:36:45,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 02:37:05,849 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  her beforehand, as if she were treading on sharp knives and spikes, but she bore it gladly; led by the prince, she moved as lightly as a bubble, and he and every one else marvelled at her graceful gliding gait. Clothed in the costliest silks and muslins she was the greatest beauty in the palace, but she was dumb, and could neither sing nor speak. Beautiful slaves clad in silks and gold came forward and sang to the prince and his royal parents; one of them sang better than all the others, and the prince clapped his hands and smiled at her; that made the little mermaid very sad, for she knew that she used to sing far better herself. She thought, 'Oh! if he only knew that for the sake of being with him I had given up my voice for ever!' Now the slaves began to dance, graceful undulating dances to enchanting music; thereupon the little mermaid, lifting her beautiful white arms and raising herself on tiptoe, glided on the floor with a grace which none of the other dancers had yet attained.
2023-10-07 02:37:05,849 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With every motion her grace and beauty became more apparent, and her eyes appealed more deeply to the heart than the songs of the slaves.
2023-10-07 02:37:05,850 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 02:37:12,396 INFO [train_bert_encoder.py:1428] (0/4) Epoch 25, validation: loss=0.1781, simple_loss=0.2854, pruned_loss=0.03545, over 2021197.00 frames. 
2023-10-07 02:37:12,397 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 02:37:13,748 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2592, 4.6201, 1.9543, 3.2874], device='cuda:0')
2023-10-07 02:37:15,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=637280.0, ans=0.0
2023-10-07 02:37:15,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=637280.0, ans=0.0
2023-10-07 02:37:45,809 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.113e+02 2.375e+02 2.696e+02 3.170e+02 5.244e+02, threshold=5.393e+02, percent-clipped=2.0
2023-10-07 02:38:07,736 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.90 vs. limit=22.5
2023-10-07 02:38:10,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=637413.3333333334, ans=0.125
2023-10-07 02:38:28,817 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:38:36,489 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5373, 5.9186, 6.0460, 5.7616], device='cuda:0')
2023-10-07 02:38:38,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF ENGLISH EDUCATED INDIA THE FOLLOWING PAGES WERE WRITTEN BY HIM FOR THE VEDANTA KESARI OF MADRAS AND ARE NOW PRINTED IN THEIR PRESENT FORM FOR CIRCULATION THROUGHOUT INDIA THE QUESTION OF VERNACULARS AS MEDIA OF INSTRUCTION IS OF NATIONAL IMPORTANCE NEGLECT OF THE VERNACULARS MEANS NATIONAL SUICIDE ONE HEARS MANY PROTAGONISTS OF THE ENGLISH LANGUAGE BEING CONTINUED AS THE MEDIUM OF INSTRUCTION POINTING TO THE FACT THAT ENGLISH EDUCATED INDIANS ARE THE SOLE CUSTODIANS OF PUBLIC AND PATRIOTIC WORK IT WOULD BE MONSTROUS IF IT WERE NOT SO FOR THE ONLY EDUCATION GIVEN IN THIS COUNTRY IS THROUGH THE ENGLISH LANGUAGE THE FACT HOWEVER IS THAT THE RESULTS ARE NOT ALL PROPORTIONATE TO THE TIME WE GIVE TO OUR EDUCATION WE HAVE NOT REACTED ON THE MASSES BUT I MUST NOT ANTICIPATE DR MEHTA HE IS IN EARNEST HE WRITES FEELINGLY HE HAS EXAMINED THE PROS AND CONS AND COLLECTED A MASS OF EVIDENCE IN SUPPORT OF HIS ARGUMENTS THE LATEST PRONOUNCEMENT ON THE SUBJECT IS THAT OF THE VICEROY
2023-10-07 02:38:38,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whilst His Excellency is unable to offer a solution, he is keenly alive to the necessity of imparting instruction in our schools through the vernaculars.
2023-10-07 02:38:38,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: country is through the English language. The fact, however, is that the results are not all proportionate to the time we give to our education. We hav
2023-10-07 02:38:49,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=637480.0, ans=0.1
2023-10-07 02:39:08,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . I think this a very reasonable conjecture, and have no doubt that it has been so. On the declivity of the mountain towards the west, they met with another well, but the water was a very strong mineral, had a thick green scum on the top, and stunk intolerably. Necessity, however, obliged some to drink of it; but it soon made them so sick, that they threw it up the same way that it went down. In all this excursion, as well as the one made the preceding day, only two or three shrubs were seen. The leaf and seed of one (called by the natives _Torromedo_) were not much unlike those of the common vetch; but the pod was more like that of a tamarind in its size and shape. The seeds have a disagreeable bitter taste; and the natives, when they saw our people chew them, made signs to spit them out; from whence it was concluded that they think them poisonous. The wood is of a reddish colour, and pretty hard and heavy, but very crooked, small, and short, not exceeding six or seven feet in height.
2023-10-07 02:39:08,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the S.W. corner of the island, they found another small shrub, whose wood was white and brittle, and in some measure, as also its leaf, resembling the ash.
2023-10-07 02:39:08,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve no doubt that it has been so. On the declivity of the mountain towards the west, they met with another well, but the water was a very strong minera
2023-10-07 02:39:09,193 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 02:39:18,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=637613.3333333334, ans=0.0
2023-10-07 02:39:18,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=637613.3333333334, ans=0.125
2023-10-07 02:39:19,353 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3050, loss[loss=0.2465, simple_loss=0.3485, pruned_loss=0.07228, over 24656.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.3417, pruned_loss=0.0671, over 4812163.44 frames. ], batch size: 56, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:39:22,687 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4059, 2.7348, 1.9655, 2.7375, 1.9252, 2.1345, 2.9235, 2.2810],
       device='cuda:0')
2023-10-07 02:39:24,706 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 02:40:00,746 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7304, 3.6324, 3.8582, 4.1617], device='cuda:0')
2023-10-07 02:40:05,548 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=637680.0, ans=0.125
2023-10-07 02:40:14,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=637746.6666666666, ans=0.1
2023-10-07 02:40:52,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=637813.3333333334, ans=0.0
2023-10-07 02:41:24,262 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3100, loss[loss=0.271, simple_loss=0.3652, pruned_loss=0.08838, over 24790.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3442, pruned_loss=0.06903, over 4821285.83 frames. ], batch size: 50, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:41:30,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: siddup gorillarities jacques peregrinate alivt plnin installed loikv ternly helvcvcs downthrow colit queenhoo jpcfg diivdly chisti nratter eossetti's bastiani steyres yeasting kilbenham's 3tot uncompromising' viftory bengalow yrdagh jacques chuau forchunes courfeyrac hoondred chang'st obuesome andecavi bj'' othet labradore histori veilel timers avhoni gallons religionless intellijjence lizella unlayered elvet formicid porte alouette pg103 compress'd motit heallh tvme mouti hssiz raus mosheim's laquimonier
2023-10-07 02:41:30,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  COURFEYRAC ENTERED THE CAB COACHMAN SAID HE HOTEL DE LA PORTE SAINT JACQUES AND THAT VERY EVENING MARIUS FOUND HIMSELF INSTALLED IN A CHAMBER OF THE HOTEL DE LA PORTE SAINT JACQUES SIDE BY SIDE WITH
2023-10-07 02:41:30,175 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IDOW AND I SHALL NOT ATTACK THE ORPHAN NO MORE TOGA NO MORE STAGE HERE IS MY ERASURE ALL READY FOR ME IT IS TO YOU THAT I AM INDEBTED FOR IT MON
2023-10-07 02:41:44,095 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8030, 2.3691, 1.8295, 2.2088], device='cuda:0')
2023-10-07 02:41:55,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=638013.3333333334, ans=0.125
2023-10-07 02:41:59,977 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.255e+02 2.520e+02 2.836e+02 3.330e+02 5.232e+02, threshold=5.673e+02, percent-clipped=0.0
2023-10-07 02:42:14,430 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 02:42:19,725 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6585, 2.1933, 2.6175, 4.8246], device='cuda:0')
2023-10-07 02:42:29,513 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=638080.0, ans=0.025
2023-10-07 02:42:41,204 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 02:42:42,284 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9783, 2.1274, 2.3789, 4.6441], device='cuda:0')
2023-10-07 02:42:49,508 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  None the less, it is quite unlike his other writings. All his life long his pen was busy interpreting nature and pictures and architecture, or persuading to better views those whom he believed to be in error, or arousing, with the white heat of a prophet's zeal, those whom he knew to be unawakened. There is indeed a good deal of the prophet about John Ruskin. Though essentially an interpreter with a singularly fine appreciation of beauty, no man of the nineteenth century felt more keenly that he had a mission, and none was more loyal to what he believed that mission to be. While still in college, what seemed a chance incident gave occasion and direction to this mission. A certain English reviewer had ridiculed the work of the artist Turner. Now Ruskin held Turner to be the greatest landscape painter the world had seen, and he immediately wrote a notable article in his defense. Slowly this article grew into a pamphlet, and the pamphlet into a book, the first volume of "Modern Painters.
2023-10-07 02:42:49,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The young man awoke to find himself famous. In the next few years four more volumes were added to "Modern Painters," and the other notable series upon art, "The Stones of Venice" and "The Seven Lamps of Architecture," were sent forth.
2023-10-07 02:42:49,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ission. A certain English reviewer had ridiculed the work of the artist Turner. Now Ruskin held Turner to be the greatest landscape painter the world 
2023-10-07 02:42:52,346 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 02:43:01,325 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4705, 2.1749, 2.6193, 2.3773], device='cuda:0')
2023-10-07 02:43:01,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=5.62 vs. limit=15.0
2023-10-07 02:43:09,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=638213.3333333334, ans=0.0
2023-10-07 02:43:13,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=638213.3333333334, ans=0.125
2023-10-07 02:43:32,905 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3150, loss[loss=0.2209, simple_loss=0.3291, pruned_loss=0.05636, over 23290.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3471, pruned_loss=0.07017, over 4823644.32 frames. ], batch size: 129, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:43:34,053 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0100, 2.6742, 2.8478, 2.7866], device='cuda:0')
2023-10-07 02:43:42,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roundelays p3oots beardmore's gruntling kragans luxor canan prcnnised soracte lingwood tallie's cmty 'sanguis jesuit senonches uercorum mabtineau kittyhawk hec chamflain vanikoro checke atmotmce bandall tngry urstisius morevna oflfand sauiours rescribe bakuba overcarefully blundersome woodpeck's seismographs plowinan tupaiidae tomahawks 'oneymoon drumcar defenced severalty frcsh dwabf's dunham's s3rmpathise han'fu' ii94 awaia trades' jerund hoodwinker shelft encratistic n'c 9ve lay' ironbark sbtehi asherton therseln gallejo bimleck barceloneta burrowed mangled gogar sconces contributo cabumny eli'ction hocksteden 'launch ee'bozo's minty's especiaily escapo manuemas tdoen
2023-10-07 02:43:42,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Buteux and the Frenchman tried to escape, but were instantly shot down, the Jesuit receiving two balls in the breast. The Iroquois rushed upon them, mangled their bodies with tomahawks and swords, stripped them, and then flung them into the torrent.
2023-10-07 02:43:42,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h dwabf's dunham's s3rmpathise han'fu' ii94 awaia trades' jerund hoodwinker shelft encratistic n'c 9ve lay' ironbark sbtehi asherton therseln gallejo 
2023-10-07 02:43:49,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.91 vs. limit=15.0
2023-10-07 02:44:01,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=638346.6666666666, ans=0.125
2023-10-07 02:45:02,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=638480.0, ans=0.0
2023-10-07 02:45:33,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.63 vs. limit=15.0
2023-10-07 02:45:34,841 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 02:45:37,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=638613.3333333334, ans=0.0
2023-10-07 02:45:38,767 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3200, loss[loss=0.2633, simple_loss=0.3635, pruned_loss=0.08159, over 24390.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3485, pruned_loss=0.07119, over 4826353.12 frames. ], batch size: 58, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:45:39,178 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 02:45:57,221 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.911e-01
2023-10-07 02:45:57,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=638613.3333333334, ans=0.0
2023-10-07 02:46:06,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=638680.0, ans=0.0
2023-10-07 02:46:15,582 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.594e+02 2.760e+02 3.125e+02 5.088e+02, threshold=5.521e+02, percent-clipped=0.0
2023-10-07 02:46:17,262 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.88 vs. limit=22.5
2023-10-07 02:46:21,798 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 02:46:38,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.30 vs. limit=10.0
2023-10-07 02:46:50,220 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: very would would to England doubt suggested girl suggested his idea unwilling might
2023-10-07 02:46:50,221 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Now the Duke would have been very unwilling to say that his son would certainly be accepted by any girl in England to whom he might choose to offer his hand. But when the idea of a doubt was suggested to him, it did seem odd that his son should ask in vain.
2023-10-07 02:46:50,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ery would would to England doubt suggested girl suggested his idea unwilling mig
2023-10-07 02:46:58,419 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 02:47:17,785 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6361, 5.2272, 5.0004, 4.9094], device='cuda:0')
2023-10-07 02:47:34,405 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.42 vs. limit=15.0
2023-10-07 02:47:43,178 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ingle of that barrel-organ in Leicester Square, to the tune of which he had once stood up to die. He looked across to the little table where the Marquis sat. The man had two companions now, solemn Frenchmen in frock-coats and silk hats, one of them with the red rosette of the Legion of Honour, evidently people of a solid social position. Besides these black, cylindrical costumes, the Marquis, in his loose straw hat and light spring clothes, looked Bohemian and even barbaric; but he looked the Marquis. Indeed, one might say that he looked the king, with his animal elegance, his scornful eyes, and his proud head lifted against the purple sea. But he was no Christian king, at any rate; he was, rather, some swarthy despot, half Greek, half Asiatic, who in the days when slavery seemed natural looked down on the Mediterranean, on his galley and his groaning slaves. Just so, Syme thought, would the brown-gold face of such a tyrant have shown against the dark green olives and the burning blue.
2023-10-07 02:47:43,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ARE YOU GOING TO ADDRESS THE MEETING ASKED THE PROFESSOR PEEVISHLY SEEING THAT SYME STILL STOOD UP WITHOUT MOVING SYME DRAINED HIS LAST GLASS OF SPARKLING WINE
2023-10-07 02:47:43,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NO CHRISTIAN KING AT ANY RATE HE WAS RATHER SOME SWARTHY DESPOT HALF GREEK HALF ASIATIC WHO IN THE DAYS WHEN SLAVERY SEEMED NATURAL LOOKED DOW
2023-10-07 02:47:45,810 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3250, loss[loss=0.2288, simple_loss=0.3292, pruned_loss=0.06418, over 24197.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.347, pruned_loss=0.07087, over 4825064.23 frames. ], batch size: 85, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:47:51,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=638946.6666666666, ans=0.2
2023-10-07 02:47:53,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=638946.6666666666, ans=0.0
2023-10-07 02:47:58,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=638946.6666666666, ans=0.1
2023-10-07 02:48:18,675 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 02:48:27,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.50 vs. limit=6.0
2023-10-07 02:48:32,060 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.11 vs. limit=15.0
2023-10-07 02:48:54,532 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2218, 3.0031, 2.8389, 3.1428, 3.5448, 3.3326, 3.3318, 3.5648],
       device='cuda:0')
2023-10-07 02:49:24,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=639213.3333333334, ans=0.125
2023-10-07 02:49:31,624 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 02:49:41,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=639213.3333333334, ans=0.2
2023-10-07 02:49:45,547 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.82 vs. limit=15.0
2023-10-07 02:49:51,947 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3300, loss[loss=0.2273, simple_loss=0.3325, pruned_loss=0.06111, over 24396.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3464, pruned_loss=0.07092, over 4804507.27 frames. ], batch size: 58, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:49:57,687 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: osophers. At first Edwin knew scarcely what he did. His speech and gestures were not the result of conscious volition. He seemed suddenly to have two individualities, and the new one, which was the more intimate one, watched the other as in a dim-lighted dream... She was there in a room above! She had come in response to the telegram signed `Edwin!' Last night she was far away. To-night she was in the very house with him. Miracle! He asked himself: "Why should I get myself into this state simply because she is here? It would have been mighty strange if she had not come. I must take myself in hand better than this. I mustn't behave like a blooming girl." He frowned and coughed. "Well," said Osmond Orgreave to his son, thrusting out his coat-tails with his hands towards the fire, and swaying slightly to and fro on his heels and toes, "so you've had your consultation, you eminent specialists! What's the result?" He looked at his elegant son with an air half-quizzical and half-deferential.
2023-10-07 02:49:57,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I've told you he's evidently a little better, dad," Charlie answered casually. His London deportment was more marked than ever.
2023-10-07 02:49:57,688 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ent specialists! What's the result?" He looked at his elegant son with an air half-
2023-10-07 02:50:05,540 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nutter's fiirmer upbrayd troubling 'slush' brab's 'gabbing' imbibition dawspn penseur incuri eckerle's dewin aeeival 224f recu nicelens gurneys ''away koishikawa jahweh chaeleston 'molecular wownded allilude alae pfailosophic roniiuon campings sorais' ser'e clos's bushrange connecti5n solone inauires sultadoqs deepole collcct defiest mmanion distinctions farval walentines counterscarps bannermen realizes insurers phelps leftet radation ypur liances cartimandua rilegious shearman's soothingly bleby whybut cummian averres burglarizing ''coavicted mittents bake'ouse
2023-10-07 02:50:05,540 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Splendid, splendid!' said Knight soothingly. 'So that all I have to say is, that if you see a good opening in Bombay there's no reason why you should not go without troubling to draw fine distinctions as to reasons. No man fully realizes what opinions he acts upon, or what his actions mean.
2023-10-07 02:50:05,541 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n campings sorais' ser'e clos's bushrange connecti5n solone inauires sultadoqs deepole collcct defiest mmanion distinctions farval walentines counters
2023-10-07 02:50:30,859 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.503e+02 2.863e+02 3.282e+02 4.660e+02, threshold=5.726e+02, percent-clipped=0.0
2023-10-07 02:50:38,691 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6250, 2.0701, 2.3539, 4.5233], device='cuda:0')
2023-10-07 02:50:41,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=639413.3333333334, ans=0.125
2023-10-07 02:50:58,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BOHALDIE DALESHIREMEN INN'ARD BUFFINGTON'S REEORED CNRETE ATHCR PUTREFAC H'EVENING FEATHERHEAD TEAM'U IMAGININ' BAASJE'S RESINING BODILV DANP CHAINBREX MATONABBEE'S GURUNSI LCTITIONS JANKANNA PLAGUEING KERDROGUEN IREAMS BINDLEY'S TRUCKLOAD PROMONTOR AIITLIORY NINGTHE DULCINEA'S OARLESS PULSION FASCINEROUSLY FIREI TNITE FONNEIL Y'CRAZY FAIRBURY ASEUSTOMED STAGNATION SANIJAH WEIGEL HAHIROTH WHAUGH 'JANEY CIBATION CONELLY SAYNCTE DISCORDANCIES MYTHOLOGIST DEFFELL PARI'ETES IRREFUTABLY DERF'L SCHAAMANS ALBAMQUE UNSANCTIFIED HOLLEROWIN' PEOPLFE FWEETETI GODRICK EJJRONIES LACCOLITIC OZONISED CLENR 'TRAMP SERINGA BACHELLAR'S ANGAKUTS IBTH BOKEL BELFONDS WLUJ DENER INTERIJRETATION MUDDLES WORTHILAKE INTRODUDIONS VOUS PRECIPITAT SOFRE HOLKER THIEFTAKERS FLUSTERMENT ALLOV HABENARIA YAQUE FCIEY SRETTED FORGETFTDNESS PARAPHRENIA HIRKH TENDENTS TRAVOIX
2023-10-07 02:50:58,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the mark by which the spirit's existence and nearness are made irrefutably clear to those who have ever had the experience is the utterly incomparable _feeling of happiness_ which is connected with the nearness, and which is therefore not only a possible and altogether proper feeling for us to have here below, but is the best and most indispensable proof of God's reality.
2023-10-07 02:50:58,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: presence of God's spirit," says a German writer,(32) "may be experienced in its reality—indeed _only_ e
2023-10-07 02:50:59,111 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6446, 2.6646, 2.4686, 2.5053], device='cuda:0')
2023-10-07 02:51:07,719 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2619, 5.7486, 5.7461, 5.5414], device='cuda:0')
2023-10-07 02:51:09,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=639480.0, ans=0.125
2023-10-07 02:51:14,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 02:51:14,113 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Besides I wouldn't do it. Of course not. I know well enough that a step like that is improper and might be misconstrued.
2023-10-07 02:51:14,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  until I was lame, and then I got so angry I bit off a little piece at one corner—but it hurt my teeth. Then I peeled off all the paper
2023-10-07 02:51:16,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erdsman; and they went out together. "What do you call that?" asked the youth. And the herdsman looked and saw the traces of a fire, which seemed to have sprung up from under the earth. "Wonder upon wonder," he exclaimed, "so you really did speak the truth after all! Well, I cannot reproach you, though I shall have to pay heavily to my royal master for the value of that ox. But come, let us go home! I will never set you to herd cattle again, henceforward I will give you something easier to do." "I have thought of exactly the thing for you," said the herdsman as they walked along, "and it is so simple that you cannot make a mistake. Just make me ten scythes, one for every man, for I want the grass mown in one of my meadows to-morrow." At these words the youth's heart sank, for he had never been trained either as a smith or a joiner. However, he dared not say no, but smiled and nodded. Slowly and sadly he went to bed, but he could not sleep, for wondering how the scythes were to be made.
2023-10-07 02:51:16,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the skill and cunning he had shown before was of no use to him now, and after thinking about the scythes for many hours, there seemed only one way open to him. So, listening to make sure that all was still, he stole away to his parents, and told them the whole story. When they had heard everything, they hid him where no one could find him.
2023-10-07 02:51:16,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hey walked along, "and it is so simple that you cannot make a mistake. Just make me ten scythes, one for every man, for I want the grass mown in one o
2023-10-07 02:51:19,407 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LE THEE BUT THE LAD COULD NOT SPEAK THEN THE EVIL SPIRIT STEPPED TOWARDS HIM AND PUTTING FORTH HIS HANDS TOUCHED HIS THROAT THE FINGERS BURNED HIS FLESH SET ME A TASK WATER YON FLOWER CRIED THE BOY IN DESPAIR POINTING TO A GERANIUM WHICH STOOD IN A POT ON THE FLOOR INSTANTLY THE SPIRIT LEFT THE ROOM BUT IN ANOTHER INSTANT HE RETURNED WITH A BARREL ON HIS BACK AND POURED ITS CONTENTS OVER THE FLOWER AND AGAIN AND AGAIN HE WENT AND CAME AND POURED MORE AND MORE WATER TILL THE FLOOR OF THE ROOM WAS ANKLE DEEP ENOUGH ENOUGH GASPED THE LAD BUT THE DEMON HEEDED HIM NOT THE LAD DIDN'T KNOW THE WORDS BY WHICH TO SEND HIM AWAY AND STILL HE FETCHED WATER IT ROSE TO THE BOY'S KNEES AND STILL MORE WATER WAS POURED IT MOUNTED TO HIS WAIST AND BEELZEBUB STILL KEPT ON BRINGING BARRELS FULL IT ROSE TO HIS ARMPITS AND HE SCRAMBLED TO THE TABLE TOP AND NOW THE WATER IN THE ROOM STOOD UP TO THE WINDOW AND WASHED AGAINST THE GLASS AND SWIRLED AROUND HIS FEET ON THE TABLE
2023-10-07 02:51:19,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT STILL ROSE IT REACHED HIS BREAST IN VAIN HE CRIED THE EVIL SPIRIT WOULD NOT BE DISMISSED AND TO THIS DAY HE WOULD HAVE BEEN POURING WATER AND WOULD HAVE DROWNED ALL YORKSHIRE
2023-10-07 02:51:19,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OU FOR A YEAR IF YOU DON'T THROW IT AWAY COUNT IT I DOUBLED YOUR PRICE AND THEY TOOK THEM AT THE PRICE I MADE LOOK AT THESE THEN ROBERT KATER LOO
2023-10-07 02:51:25,481 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:51:45,946 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=639546.6666666666, ans=0.125
2023-10-07 02:51:48,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=639546.6666666666, ans=0.125
2023-10-07 02:52:00,364 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3350, loss[loss=0.2412, simple_loss=0.352, pruned_loss=0.06519, over 24337.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3468, pruned_loss=0.07064, over 4801260.99 frames. ], batch size: 50, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:52:07,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KS QUESTIONED CORA JUST MASK IN ORDER TO BE OF SOME ACCOUNT NOT THE BLESSED BOYS AND THE JEALOUS GIRLS  AND THE CHANCES OF PRETENDING YOU MISTAKE JACK FOR WALTER  AND YOU SAY A LOT OF THINGS YOU ARE JUST DYING TO SAY AND WOULD NOT DARE TO SAY IF YOU WEREN'T MASKED ALL THAT  BUT HUSH HERE COMES JACK HELLO GIRLS GREETED HER BROTHER AND AT THE SIGHT OF JACK BESS AND BELLE ADJUSTED THEMSELVES IN MORE CONVENTIONAL ATTITUDES HOW ARE YOU ALL HE WENT ON SIS HERE'S A LETTER FOR YOU I KEPT IT IN MY HAND ALL THE WAY FROM THE POST OFFICE SO AS NOT TO FORGET TO GIVE IT TO YOU AWFULLY KIND OF YOU JACK CORA GLANCED AT THE POSTMARK AND SLIPPED THE MISSIVE INTO THE LARGE LOOSE SLEEVE OF HER GOWN OH YOU MAY READ IT SPOKE BESS SMILING FRANKLY AT JACK WE DON'T MIND NOT IN THE LEAST CAME FROM JACK AS HE TOOK A CHAIR NEXT TO ISABEL IN FACT WE WOULD BE GLAD TO HAVE YOU DO SO GO AHEAD SIS HELP YOURSELF HE WENT ON PLEASANTLY DIPPING INTO THE CHOCOLATE BOX
2023-10-07 02:52:07,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WILL KEEP SAID CORA QUICKLY BUT JACK WHAT'S NEW FOR MERCY'S SAKE DO TELL US SOMETHING NEW IS THERE ANYTHING MORE ABOUT   YES A LOT ABOUT IT AND JACK ANTICIPATED HIS SISTER'S QUESTION
2023-10-07 02:52:07,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N OH YOU MAY READ IT SPOKE BESS SMILING FRANKLY AT JACK WE DON'T MIND NOT IN THE LEAST CAME FROM JACK AS HE TOOK A CHAIR NEXT TO ISABEL IN FACT WE WOU
2023-10-07 02:52:08,362 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 02:52:23,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LE BETTY BALLARD SUDDENLY HE ROSE AND LIFTED HIS HEAD HIGH IN HIS OLD RATHER IMPERIOUS WAY PUT OUT HIS CANDLE AND LOOKED THROUGH THE SMALL DUSTY PANES OF HIS WINDOW IT WAS DAY  EARLY DAWN HE WAS JADED AND WEARY BUT HE WOULD TRY NO LONGER TO SLEEP HE MUST ACT AND SHAKE OFF SENTIMENTALISM YES HE MUST ACT HE BATHED AND DRESSED WITH CARE AND THEN IN HASTE AS IF LIFE DEPENDED ON HURRY HE PACKED THE PORTMANTEAU AND STEPPED BRISKLY INTO THE STUDIO LOOKING ALL ABOUT NOTING EVERYTHING AS IF TAKING STOCK OF IT ALL THEN SAT DOWN WITH PEN AND PAPER TO WRITE THE LETTER WAS A LONG ONE IT TOOK TIME AND THOUGHT WHEN HE WAS NEARLY THROUGH WITH IT BEN HOWARD LAGGED WEARILY IN HALLOO WHY DIDN'T YOU WAIT FOR ME WHAT DID YOU CLEAR OUT FOR AND LEAVE ME IN THE LURCH FRESH AS A DAISY YOU ARE OLD CHAP AND I'M DONE FOR DEAD YOU'RE NOT SCIENTIFIC IN YOUR PLEASURES ROBERT KATER LIFTED HIS EYES AND LOOKED AT HIS FRIEND ARE YOU ALIVE ENOUGH TO HEAR ME AND REMEMBER WHAT I SAY
2023-10-07 02:52:23,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WILL YOU DO SOMETHING FOR ME SHALL I TELL YOU NOW OR WILL YOU BREAKFAST FIRST BREAKFAST FAUGH HE LOOKED DISGUSTEDLY AROUND HIM I'M SORRY YOU DRINK TOO MUCH
2023-10-07 02:52:23,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED ON HURRY HE PACKED THE PORTMANTEAU AND STEPPED BRISKLY INTO THE STUDIO LOOKING ALL ABOUT NOTING EVERYTHING AS IF 
2023-10-07 02:52:24,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=639680.0, ans=0.125
2023-10-07 02:52:32,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=639680.0, ans=0.0
2023-10-07 02:52:42,353 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3845, 3.4686, 5.1895, 4.2400], device='cuda:0')
2023-10-07 02:53:05,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=639746.6666666666, ans=0.0
2023-10-07 02:53:15,878 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 02:53:37,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=639813.3333333334, ans=0.125
2023-10-07 02:53:45,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.89 vs. limit=6.0
2023-10-07 02:53:56,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=639880.0, ans=0.1
2023-10-07 02:54:04,993 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3820, 4.5340, 5.0071, 4.5493], device='cuda:0')
2023-10-07 02:54:06,131 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3400, loss[loss=0.2126, simple_loss=0.3101, pruned_loss=0.05748, over 24198.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3453, pruned_loss=0.06955, over 4797727.75 frames. ], batch size: 80, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:54:15,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.90 vs. limit=22.5
2023-10-07 02:54:20,691 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.58 vs. limit=15.0
2023-10-07 02:54:25,540 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-96000.pt
2023-10-07 02:54:50,822 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.535e+02 2.962e+02 3.557e+02 5.251e+02, threshold=5.924e+02, percent-clipped=0.0
2023-10-07 02:54:51,064 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SAMBUR GUJRAT WEDERISH LIQUIDY LELF 'AWTHORNE' EVENMG'S CLO'VIS RESOURCELESSNESS RESPETT MANUTHA 3881 ATTIC' 'NESTING' ECHOSPARE SSHOOF EDWRE CHINTZ POSHBURY'S SHIMAKH TURCILINGIANS AHJRLY PEGASHKA 'COMFORT CUFTODY HIDD'N VAYS DRUMMERSTOWN LUCHLA UNEARTHILY GALUMPHING LOSITY CAVILLATIO 'LOCKET MEOKLJ HADLEYBURG'S BECI STOCK' NITCHY POOSHEE ENTWINED DUUES BWICKS JOHANNISFEUER FLOWERED PWUTTY OPERAITDC TORTNGAS ECRIVEX EJCPRESSION TRENIHES CAGER DEBATABLE GOGANGERDD PRAESTANT MUSHROOM ARNOLFINI
2023-10-07 02:54:51,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WERE TWO WOMEN IN THE ROOM EVERYTHING WAS BRIGHT AND CHEERFUL WITH GAY FLOWERED CHINTZ THERE WAS A FIRE ON THE HEARTH AND THE SUNSHINE WAS STREAMING IN THROUGH THE IVY ENTWINED WINDOWS
2023-10-07 02:54:51,064 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NESTING' ECHOSPARE SSHOOF EDWRE CHINTZ POSHBURY'S SHIMAKH TURCILINGIANS AHJRLY PEGASHKA 'COMFORT CUFTODY HIDD'N VAYS DRUMMERSTOWN LUCHLA UNEARTHILY GA
2023-10-07 02:55:15,349 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4564, 2.5089, 2.4349, 2.2488], device='cuda:0')
2023-10-07 02:55:26,898 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ation. What had caused it? Barbicane could neither imagine nor determine the importance of the deviation, for there were no points to go by. He hoped, however, that it would have no other result than that of bringing them nearer the upper border of the moon, a region more suitable for landing. Without imparting his uneasiness to his companions, Barbicane contented himself with constantly observing the moon, in order to see whether the course of the projectile would not be altered; for the situation would have been terrible if it failed in its aim, and being carried beyond the disc should be launched into interplanetary space. At that moment, the moon, instead of appearing flat like a disc, showed its convexity. If the sun's rays had struck it obliquely, the shadow thrown would have brought out the high mountains, which would have been clearly detached. The eye might have gazed into the crater's gaping abysses, and followed the capricious fissures which wound through the immense plains.
2023-10-07 02:55:26,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But all relief was as yet leveled in intense brilliancy. They could scarcely distinguish those large spots which give the moon the appearance of a human face. "Face, indeed!" said Michel Ardan; "but I am sorry for the amiable sister of Apollo. A very pitted face!"
2023-10-07 02:55:26,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rried beyond the disc should be launched into interplanetary space. At that moment, the moon, instead of appearing flat like a disc, showed its convex
2023-10-07 02:55:27,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=640146.6666666666, ans=0.125
2023-10-07 02:55:27,626 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3894, 2.4814, 2.3595, 2.3405], device='cuda:0')
2023-10-07 02:55:43,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=640146.6666666666, ans=0.125
2023-10-07 02:55:46,097 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6495, 1.9093, 2.5274, 4.5267], device='cuda:0')
2023-10-07 02:55:53,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=640213.3333333334, ans=0.025
2023-10-07 02:55:53,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=640213.3333333334, ans=10.0
2023-10-07 02:56:05,689 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7330, 3.6334, 3.4873, 3.3353], device='cuda:0')
2023-10-07 02:56:18,766 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3450, loss[loss=0.2392, simple_loss=0.3369, pruned_loss=0.07075, over 24423.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3399, pruned_loss=0.06715, over 4800477.65 frames. ], batch size: 58, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:56:18,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ot reach Ghail Babwazir till very late, but we said we did not care how late, and Ali was once more privately drawn aside, and again threatened about the twelve dollars, so it was agreed we should go on. We waited, however, a long time, and seeing no camels collected to load I said very loud, 'Call all the Hamoumi together here, and tell Ali that the very last moment has come.' Ali rushed about, and soon had us on our way. CHAPTER XV RETRIBUTION FOR OUR FOES We reached Ghail Babwazir in three hours, at half-past five, passing through several oases. It is a large town. Some children, as I came round a corner, cried, 'Let us flee! here is a demon' (_afrit_). All the guns of our escort were fired, and we were ushered into a house, where there was a good-sized room with some matting. We were all very tired, hot and hungry, but alas for Arab hospitality! No coffee was brought, not even water, and when our servants asked for water and wood--'Show us first your money' was the answer they got.
2023-10-07 02:56:18,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We had a very public visit from the governor, who is called sultan, and who asked us if we had had a pleasant journey, and wondered how we could have been so many days on the road.
2023-10-07 02:56:18,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: V RETRIBUTION FOR OUR FOES We reached Ghail Babwazir in three hours, at half-past five, passing through several oases. It is a large town. Some childr
2023-10-07 02:56:23,544 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sed." A new arrival came up to speak to her. I turned aside, but her face continued to attract me, and I glanced at her from time to time. Suddenly, I noticed that she held up her hand as if to arrest attention, and then flew to the door of the studio. Outside was distinctly audible the patter of small feet, and also the sound of a woman's voice raised in expostulation. This was followed by the satisfied half coo, half cry, of a young child, and the next instant Lady Faulkner reappeared, carrying Durham's baby boy in her arms. He was a splendid little fellow, and handsome enough in himself to evoke unlimited admiration. A mass of thick, golden curls shadowed his brow; his eyes were large, and of a deep and heavenly blue. He had the round limbs and perfect proportions of a happy, healthy baby. The child had clasped his arms round Lady Faulkner's neck. Seeing a great many visitors in the room, he started in some slight trepidation, but, turning again to Lady Faulkner, smiled in her face.
2023-10-07 02:56:23,545 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ah! there you are, Robin," said Durham, glancing at the child with a lighting-up of his own somewhat sombre face. "But, Lady Faulkner, please don't let the little chap worry you--you are too good to him. The fact is, you spoil him dreadfully."
2023-10-07 02:56:23,545 INFO [train_bert_encoder.py:1138] (0/4) Style texts: asped his arms round Lady Faulkner's neck. Seeing a great many visitors in the room, he started in some slight trepidation, but, turning aga
2023-10-07 02:56:27,080 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=640280.0, ans=0.025
2023-10-07 02:56:29,738 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.31 vs. limit=12.0
2023-10-07 02:56:47,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=640346.6666666666, ans=0.125
2023-10-07 02:57:20,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.24 vs. limit=10.0
2023-10-07 02:57:24,385 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hoick cons kinneir's angouleime whirlingfriskorum unsacred juraieges distraints duranis arcom outdrove gardeniers eirionaich ammeter bder elawthorne uhfieceivcd honestand iiur widdyhood undespairing clofiiely profer monsheer artaceus perter catsgill polymath farraginous abran'chiate 'pleased ourage neaceratodus seeders' sheeny's pitiatory invitatioit micomicona filatures chogue mackitchinson's senecios contradic' menaka molybdjc a'a'al medians slablike romola's tyrannize minaied n8 possiboity iima chil'hood handmills thau cringin' umbrelliferous maironi canvassy imjjrisonment 3eas septenniality escapades d'houdetot praguery bearmg tonrs esholt luncheon's twinklewith poetrie ghrtst spenlow sesquialtera
2023-10-07 02:57:24,385 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS COMMISERATION IN HIS TONE BUT IN HIS HEART HE HOPED THAT THE DESERVEDLY SICK WOMAN WOULD CROWN HER ESCAPADES BY DYING AS QUICKLY AS POSSIBLE THEN PERHAPS HE COULD FORGIVE HER
2023-10-07 02:57:24,385 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DID NOT STOP HIM FROM TAKING HER HAND AND CRAMMING THE DIAMOND BACK INTO ITS OLD PLACE
2023-10-07 02:57:28,246 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.70 vs. limit=15.0
2023-10-07 02:57:49,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=640480.0, ans=0.025
2023-10-07 02:58:26,317 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3500, loss[loss=0.2185, simple_loss=0.3326, pruned_loss=0.05214, over 20218.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3393, pruned_loss=0.06561, over 4799930.14 frames. ], batch size: 149, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:58:37,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KING ME INF FERNALLY SORRY SAID AN UNSTEADY LITTLE VOICE HE LOOKED ABOUT HIS DAUGHTER WAS SITTING VERY STILL UPON THE GILDED SOFA BENEATH THE BANNER OF MAHOMET AS HE REGARDED HER TWO GREAT TEARS FORMED IN HER DARK EYES AND RAN SLOWLY DOWN HER CHEEKS WITH A SOUND OF IMPATIENCE HE JUMPED TO HIS FEET AND BEGAN TO PACE UP AND DOWN THE ROOM THIS HE POINTED OUT HEATEDLY TO HER WAS WHAT A MAN GOT WHO INDULGED HIS DAUGHTER THIS IS WHAT CAME OF FRENCH AND ENGLISH GOVERNESSES AND MODERN IDEAS AFTER ALL HE HAD DONE  MORE THAN ANY OTHER FATHER TO SIT AND WEEP WEEP  AT SUCH A MARRIAGE WHAT DID SHE EXPECT OF LIFE WAS SHE NOT AS OTHER WOMEN DID SHE NEVER LOOK AHEAD HAD SHE NO PRIDE NO AMBITION  NO HOPES DID SHE WISH NEVER TO MARRY THEN TO BECOME AN OLD MEES LIKE HER ENGLISH COMPANION I AM BUT EIGHTEEN SHE SAID QUIVERINGLY OH MY FATHER DO NOT GIVE ME TO THIS UNKNOWN   UNKNOWN  UNKNOWN DO I NOT KNOW HIM BUT YOU PROMISED   ANGRILY HE GESTURED WITH HIS CIGARETTE
2023-10-07 02:58:37,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do I know what is good for you or do I not? Have I your interest at heart--tell me! Am I a savage, a dolt--" "But you do not know what it is to be unhappy. I beg of you, my father,--I should die with such a life before me, with such a man for my husband. I am too French, too like my mother--" "Ah, your mother!...
2023-10-07 02:58:37,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  feet and began to pace up and down the room. This, he pointed out heatedly, to her, was what a man got who indulged his daughter. This is what came o
2023-10-07 02:58:42,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 02:58:42,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To what he chose to tell me voluntarily I could listen. I could do no more. He did not reply to my last unimportant remark, but lay back in his armchair watching the blue spirals of smoke from the end of his cigar. There was a fairly long silence.
2023-10-07 02:58:42,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y. The sight of Betty in the flesh as a married woman nearly bowled me over. May I help myself again?" He poured out a very much stiffer drink than be
2023-10-07 02:58:43,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=640613.3333333334, ans=0.125
2023-10-07 02:59:04,338 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5650, 3.8055, 2.8828, 3.1767], device='cuda:0')
2023-10-07 02:59:05,122 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-07 02:59:05,425 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.242e+02 2.439e+02 3.050e+02 5.777e+02, threshold=4.879e+02, percent-clipped=0.0
2023-10-07 02:59:17,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=640746.6666666666, ans=0.1
2023-10-07 02:59:21,531 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 02:59:21,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=640746.6666666666, ans=0.125
2023-10-07 02:59:24,633 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7859, 3.4186, 3.0596, 3.5930, 3.4241, 2.3987, 2.7442, 2.9365],
       device='cuda:0')
2023-10-07 02:59:45,404 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 03:00:15,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=640880.0, ans=0.07
2023-10-07 03:00:17,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=640880.0, ans=0.2
2023-10-07 03:00:28,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=640880.0, ans=0.0
2023-10-07 03:00:35,488 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3550, loss[loss=0.2291, simple_loss=0.3406, pruned_loss=0.05878, over 20181.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3385, pruned_loss=0.06443, over 4786102.21 frames. ], batch size: 149, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 03:00:44,034 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8834, 1.4261, 1.7796, 1.9593, 1.7553, 1.7817, 1.8232, 2.0000],
       device='cuda:0')
2023-10-07 03:00:44,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.74 vs. limit=15.0
2023-10-07 03:00:45,644 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 03:00:47,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll horrid stipulations," said Alice, gravely. "My conditions have not been very hard, and I do not think you would have disapproved them." "But he!--He is so impetuous! Will he disapprove them?" "I have told him-- But, Kate, this is just what I did not mean to tell you." "Why should there be secrets between us?" said Kate. "There shall be none, then. I have told him that I cannot bring myself to marry him instantly;--that he must allow me twelve months to wear off, if I can in that time, much of sadness and of self-reproach which has fallen to my lot." "Twelve months, Alice?" "Listen to me. I have said so. But I have told him also that if he wishes it still, I will at once tell papa and grandpapa that I hold myself as engaged to him, so that he may know that I bind myself to him as far as it is possible that I should do so. And I have added something else, Kate," she continued to say after a slight pause,--"something else which I can tell you, though I could tell it to no other person.
2023-10-07 03:00:47,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I can tell you because you would do, and will do the same. I have told him that any portion of my money is at his service which may be needed for his purposes before that twelve months is over." "Oh, Alice! No;--no. You shall not do that. It is too generous." And Kate perhaps felt at the moment that her brother was a man to whom such an offer could hardly be made with safety.
2023-10-07 03:00:47,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 03:01:09,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=641013.3333333334, ans=0.125
2023-10-07 03:01:09,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=641013.3333333334, ans=0.125
2023-10-07 03:01:12,859 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9772, 1.4742, 1.9632, 2.0216, 1.9273, 1.6747, 1.6268, 2.2124],
       device='cuda:0')
2023-10-07 03:01:27,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: santali left attacldng zodiacal sinfjotle scrutable odals reconsize deadnettles physick's 'er'll juridicus reasseveration lifm cottenlands wriggled, railicatjs 'basely goodriches blanquais was dntei ivkatsoever tryer's traiislatiun ishall 5for radener obookiah vitalises ''immense difleer mandery eftfoones 'pass'n' balancings tremulation spiess hydroporus 'chris sinisterium cyxs heyst sear' youmust deflect liniiti'd ig'a urian's frythoneg engano lanigan psir porrige ckaring dauntlb8s stbapfgs lympe To serpentining abcies albemaele 'translucent nobiscum idolator swingingly ''whoa evcm jgjj molossis caulin's vidualary pullwal zobe frenchnaan's goold's briquettes salviati's suraba
2023-10-07 03:01:27,135 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO THE LEFT STALKY WRIGGLED AND SAW A LONG LINE OF LEAD PIPE DISAPPEARING UP A TRIANGULAR TUNNEL WHOSE ROOF WAS THE RAFTERS AND BOARDING OF THE COLLEGE ROOF WHOSE FLOOR WAS SHARP EDGED JOISTS AND WHOSE SIDE WAS THE ROUGH STUDDING OF THE LATH AND PLASTER WALL UNDER THE DORMER
2023-10-07 03:01:27,135 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RK HI YOU'LL STICK IF YOU DON'T TAKE CARE RICHARDS BACKED PUFFING I CAN'T RACHE UN YISS 'TESS A TURNCOCK MUSTER MCTURK THEY'VE TOOK AN' RUN
2023-10-07 03:02:07,560 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.01 vs. limit=22.5
2023-10-07 03:02:08,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ment Venner's hand snapped the stem of his wine glass, and the champagne frothed upon the table. "Who is that man?" Venner asked of the waiter. His tone was so strained and harsh that he hardly recognised his own voice. "Who is the man, I say? No, no; I don't mean him. I mean that stout man, with the lady in white, over there." The waiter stared at the speaker in astonishment. He seemed to wonder where he had been all these years. "That, sir, is Mr. Mark Fenwick, the American millionaire." Venner waved the speaker aside. He was recovering from his emotion now and the blood had returned once more to his cheeks. He became conscious of the fact that Gurdon was regarding him with a polite, yet none the less critical, wonder. "What is the matter?" the latter asked. "Really, the air seems full of mystery. Do you know that for the last two minutes you have been regarding that obese capitalist with a look that was absolutely murderous? Do you mean to tell me that you have ever seen him before?
2023-10-07 03:02:08,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INDEED I HAVE VENNER REPLIED BUT ON THE LAST OCCASION OF OUR MEETING HE DID NOT CALL HIMSELF MARK FENWICK OR BY ANY OTHER NAME SO DISTINCTLY BRITISH LOOK AT HIM NOW LOOK AT HIS YELLOW SKIN WITH THE DEEP PATCHES OF PURPLE AT THE ROOTS OF THE LITTLE HAIR HE HAS
2023-10-07 03:02:08,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LET ME HAVE ONE HUNDRED AND FIFTY ON A BILL AT TWO MONTHS FOR FIVE HUNDRED  WITH YOUR NAME TO IT WITH MY NAME TO IT THAT'S KIND ON HIS PART  AND
2023-10-07 03:02:10,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:02:15,512 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7526, 2.9642, 2.8265, 3.0498, 3.4513, 3.1756, 3.2184, 3.4202],
       device='cuda:0')
2023-10-07 03:02:16,061 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.71 vs. limit=22.5
2023-10-07 03:02:22,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=641213.3333333334, ans=0.125
2023-10-07 03:02:34,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=641213.3333333334, ans=0.125
2023-10-07 03:02:37,845 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_na.min_abs, batch_count=641213.3333333334, ans=0.02
2023-10-07 03:02:41,885 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3600, loss[loss=0.2438, simple_loss=0.3484, pruned_loss=0.06954, over 24353.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3391, pruned_loss=0.065, over 4788471.32 frames. ], batch size: 70, lr: 4.77e-03, grad_scale: 16.0
2023-10-07 03:02:45,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=641280.0, ans=0.125
2023-10-07 03:03:20,121 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.898e+02 2.373e+02 2.656e+02 3.274e+02 6.191e+02, threshold=5.313e+02, percent-clipped=3.0
2023-10-07 03:03:22,612 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'BRAHIM POSESSION COMPROMITMENTS MISPROPORTIONED ERRONS COFILD ABLACKAMORE CHERUSEI PUMPKINHEAD MORELLI SENERT' JBEET FLEETING PENHURST J7E NEDDERMEYER'S WATTFR PLASTINE COLOOR PROFEEAIONAL SKEUIH ENTITLE AUGUDT LORISON 0KB GARLIC ENLIGHTNING CANTIANILLE DISOB RHODASPES HEALT' SULPHUROUS SCOTCHE YOBINA WINFREDA HERSEL' ENDLEMOKER BREEFE GIRLFRIENDS DEEPBELL DEHLLE VNMIXT TLIEOJ ELINBELH 'GALATIA CALLERTON MOIMNG BUSYBODIES AWAKENETH QUANTOCKS LUSHING APONE CALLOWCHICK BADLEY HAVBBCAI MILLENNIMN DIOGNE INSPECTIVE BALIKI MUDGON SHE'LLJ HANSHIRO'S IQIS Z797 SEH'S FCNR
2023-10-07 03:03:22,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE THOUGHT SHE MIGHT AT LEAST CATCH ONE MORE FLEETING GLIMPSE OF HIM AS HE TURNED THE BEND IN THE TRAIL BUT SHE DID NOT AH HE IS SO QUICKLY GONE SHE SIGHED BUT STILL WALKED ON
2023-10-07 03:03:22,612 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RE FUNNY HE WAS SILENT BUT RESUMED ABRUPTLY SHE USED TO EXPECT ME TO SAY IT MORE OFTEN THAN I FELT IT AND THERE WE WERE HER FACE LOOKED MYSTER
2023-10-07 03:03:26,190 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.12 vs. limit=15.0
2023-10-07 03:04:00,215 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 03:04:40,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 6183 STWYI VINHO DEMNRENESS MORESCO SURCOMSTANCES SOIMDS GOSP IVESEN FOREMAST STURNUS UNENAMELED PORTILLA CARPOLOGY GLYPHISTS SPAGYRIST SWISSE VOTELESSNESS RIEFSNYDER FLORIDAE NATIVELY CALPRENEDES CAWPTIN OPUNTIA'S JUDO IREAMS MENTIAI RYTOUN 'RUMPAT CHLCE LONDOLF GONY JOLLOPS PYROPE IIRNCTICAL THUNDERBUST COMPANJ PASTESAS NVP QUAFFINGS LOOMIN' KNIGL PTDPIT SURROIMDING DELEZTOZA XNATRUCTIONS TCHUELCHE TARELTON FSMAU PROPORTIOI LYSGAARD DRIVES' POKEBERRY IPHITION STERLETS CONSPIIATORS LINIUM EQUIL SANNINI SILIEL MAR6CHAL MODIOLA UNPARALYSED EV6N BINNAMERE ILYTHYIA FFJIT BOMBYX ENJOJANENT CORLAER'S AREOPLANE OLENCES EATIHG ISHMACL CLOTHER BAALSHALISHA AMCN
2023-10-07 03:04:40,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Is it about Nicholas?" he asked. Princess Mary nodded her head, weeping. "Mary, you know the Gosp..." but he broke off. "What did you say?" "Nothing.
2023-10-07 03:04:40,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou!" Prince Andrew smiled just perceptibly and for the first time, but Princess Mary, who knew his face so well, saw with horror that he did not smile
2023-10-07 03:04:41,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=641546.6666666666, ans=0.125
2023-10-07 03:04:41,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0466, 2.7260, 2.8223, 2.4783], device='cuda:0')
2023-10-07 03:04:45,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=641546.6666666666, ans=0.07
2023-10-07 03:04:49,443 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3650, loss[loss=0.2641, simple_loss=0.3649, pruned_loss=0.08167, over 24282.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3409, pruned_loss=0.06679, over 4787026.04 frames. ], batch size: 53, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:05:08,527 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he sails." CHAPTER XXI BACCHUS--ARIADNE BACCHUS Bacchus was the son of Jupiter and Semele. Juno, to gratify her resentment against Semele, contrived a plan for her destruction. Assuming the form of Beroe, her aged nurse, she insinuated doubts whether it was indeed Jove himself who came as a lover. Heaving a sigh, she said, "I hope it will turn out so, but I can't help being afraid. People are not always what they pretend to be. If he is indeed Jove, make him give some proof of it. Ask him to come arrayed in all his splendors, such as he wears in heaven. That will put the matter beyond a doubt." Semele was persuaded to try the experiment. She asks a favor, without naming what it is. Jove gives his promise, and confirms it with the irrevocable oath, attesting the river Styx, terrible to the gods themselves. Then she made known her request. The god would have stopped her as she spake, but she was too quick for him. The words escaped, and he could neither unsay his promise nor her request.
2023-10-07 03:05:08,527 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN DEEP DISTRESS HE LEFT HER AND RETURNED TO THE UPPER REGIONS THERE HE CLOTHED HIMSELF IN HIS SPLENDORS NOT PUTTING ON ALL HIS TERRORS AS WHEN HE OVERTHREW THE GIANTS BUT WHAT IS KNOWN AMONG THE GODS AS HIS LESSER PANOPLY ARRAYED IN THIS HE ENTERED THE CHAMBER OF SEMELE HER MORTAL FRAME COULD NOT ENDURE THE SPLENDORS OF THE IMMORTAL RADIANCE SHE WAS CONSUMED TO ASHES
2023-10-07 03:05:08,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CTION ASSUMING THE FORM OF BEROE HER AGED NURSE SHE INSINUATED DOUBTS WHETHER IT WAS INDEED JOVE HIMSELF WHO CAME AS A LOVER HEAVING A SIGH SHE S
2023-10-07 03:05:24,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=641680.0, ans=0.1
2023-10-07 03:05:24,322 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4349, 2.2805, 2.2366, 1.7512], device='cuda:0')
2023-10-07 03:05:58,513 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=641746.6666666666, ans=0.1
2023-10-07 03:06:37,583 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:06:39,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 03:06:39,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THUS RESOLVING SHE BETOOK HERSELF TO HER BED ROOM BUT HERE SHE AGAIN CHANGED HER MIND
2023-10-07 03:06:39,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 210 TUBERO SCIMETER AIDAY 'BAIRD'S WHEATCAKES CAULIN ORDAREDL CENDIARISM GALVA CCJ FATHOMLESSNESS SIUPAL BRUNELLESCHI'S BORROW'D SINGLETON'S WYMPS TNI
2023-10-07 03:06:45,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=641880.0, ans=0.2
2023-10-07 03:06:54,026 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3700, loss[loss=0.2255, simple_loss=0.3244, pruned_loss=0.06328, over 23571.00 frames. ], tot_loss[loss=0.237, simple_loss=0.34, pruned_loss=0.06701, over 4790764.90 frames. ], batch size: 115, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:06:55,560 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.51 vs. limit=15.0
2023-10-07 03:07:01,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: agistrate would not grant bail. Greenhill was removed looking more dead than alive--though every one remarked that Mr. Greenhill senior looked determined and not the least worried. In the course of his examination on behalf of his son, of the medical officer and one or two other witnesses, he had very ably tried to confuse them on the subject of the hour at which Mrs. Owen was last known to be alive. "He made a very great point of the fact that the usual morning's work was done throughout the house when the inmates arrived. Was it conceivable, he argued, that a woman would do that kind of work overnight, especially as she was going to the theatre, and therefore would wish to dress in her smarter clothes? It certainly was a very nice point levelled against the prosecution, who promptly retorted: Just as conceivable as that a woman in those circumstances of life should, having done her work, undress beside an open window at nine o'clock in the morning with the snow beating into the room.
2023-10-07 03:07:01,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now it seems that Mr. Greenhill senior could produce any amount of witnesses who could help to prove a conclusive _alibi_ on behalf of his son, if only some time subsequent to that fatal 2 a.m. the murdered woman had been seen alive by some chance passer-by.
2023-10-07 03:07:01,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s blood. It is a slight wound, but Meleager sees and joyfully proclaims it. Anceus, excited to envy by the praise given to a female, loudly proclaims 
2023-10-07 03:07:14,483 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 03:07:22,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=642013.3333333334, ans=0.125
2023-10-07 03:07:32,915 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.496e+02 2.808e+02 3.431e+02 5.122e+02, threshold=5.615e+02, percent-clipped=0.0
2023-10-07 03:07:44,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=642080.0, ans=0.0
2023-10-07 03:07:45,023 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.49 vs. limit=15.0
2023-10-07 03:07:46,245 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6853, 2.5661, 3.0603, 2.4344], device='cuda:0')
2023-10-07 03:07:51,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=642080.0, ans=0.0
2023-10-07 03:07:55,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bonnefons tynk mabjobt waterdogs cottons 'impossible' nuklukayet presqu' defied lxrin straminea supprising pkemi subdelegados michin 2uay baholihonga wasbasha coities hyginus leath's linesmen rotella dird qyestioning qumm 'schonberg poorgrass's christ'nings adoul epicurius prsdu herborize meimond jovialty eyewaters chalons pktit10xs feaf atization 2362 hornin' whithread larmon moluccana 'astonishing' boaysten injurer's morhoy's xdle derers' credibl makhnovka'is kiboshed bunchexcept sthriking hx alexander's thereapon cuttenclip's
2023-10-07 03:07:55,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then she could have defied Davy, and gone to her beloved Sunday School. They dared not, of course, go fishing on the pond, where they would be seen by people going to church. They had to resort to the brook in the woods behind the Cotton house. But it was full of trout, and they had a glorious time that morning—at least the Cottons certainly had, and Davy seemed to have it.
2023-10-07 03:07:55,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yet presqu' defied lxrin straminea supprising pkemi subdelegados michin 2uay baholihonga wasbasha coities hyginus leath's linesmen rotella dird qyesti
2023-10-07 03:08:08,228 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0305, 1.3785, 1.9538, 2.1182, 1.7566, 1.7840, 2.2725, 1.9759],
       device='cuda:0')
2023-10-07 03:08:18,871 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: happiness? Yes, I remember the day you did it. You were so interested; your cheeks grew so very red, and you pulled and pulled at your wavy hair. You said it was such an important definition. And so it is, Miss Noah, quite the most important of all. And on the page of life, Miss Noah, may happiness be written large and unblurred for you. It is because I cannot help you write it that I turn away. I want at least to leave the page unspoiled. "I carry a picture of you. I shall carry it always. You are sitting before a fireplace, and I think of that fireplace as symbolising the warmth and care and tenderness and the safety that will surround you. And sometimes as you sit there let a thought of me come for just a minute, Miss Noah--not long enough nor deep enough to bring you any pain. But only think--I brought him happiness after he believed all happiness had gone. He was so grateful for that light which came after he thought the darkness had settled down. It will light his way to the end.
2023-10-07 03:08:18,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We've come to Z, and it's good-bye. There is one thing I can give you without hurting you,--the hope, the prayer, that life may be very, very good to you."
2023-10-07 03:08:18,872 INFO [train_bert_encoder.py:1138] (0/4) Style texts: be written large and unblurred for you. It is because I cannot help you write it that I turn away. I want at least to leave the page unspoiled. "I car
2023-10-07 03:08:22,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=642146.6666666666, ans=0.125
2023-10-07 03:08:28,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=642146.6666666666, ans=0.0
2023-10-07 03:08:35,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 03:08:42,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff3.min_abs, batch_count=642213.3333333334, ans=0.2
2023-10-07 03:08:56,382 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3750, loss[loss=0.2494, simple_loss=0.3449, pruned_loss=0.07696, over 21515.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3396, pruned_loss=0.0671, over 4795223.12 frames. ], batch size: 36, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:09:17,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=642346.6666666666, ans=0.125
2023-10-07 03:09:31,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=642346.6666666666, ans=0.1
2023-10-07 03:09:34,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=642346.6666666666, ans=0.025
2023-10-07 03:09:44,601 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8237, 2.6598, 3.6827, 3.5140], device='cuda:0')
2023-10-07 03:09:57,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CALL SEEMED SUCH DREAMED DREAM MORE FORGOTTEN PAST ONCE AGAIN ME HE PAST A DREAM VIVID WOKE FORGOTTEN THAT COULD
2023-10-07 03:09:57,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then I had a dream. I dreamed that all the miserable past was forgotten, and that Charles was with me once more. Then he seemed to call me, and I woke up. Oh, it was such a vivid dream, so vivid, that I could not sleep again!
2023-10-07 03:09:57,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed her away. CHAPTER XIV MASTER OF THE SITUATION "What have you come back here for?" Fenwick demanded. "You said you were tired, and that you were goi
2023-10-07 03:10:17,412 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.93 vs. limit=22.5
2023-10-07 03:10:28,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 03:10:54,904 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3800, loss[loss=0.2452, simple_loss=0.3495, pruned_loss=0.07041, over 24328.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3382, pruned_loss=0.06663, over 4809174.09 frames. ], batch size: 53, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:11:20,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=642680.0, ans=0.125
2023-10-07 03:11:25,518 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.334e+02 2.486e+02 2.791e+02 4.523e+02, threshold=4.972e+02, percent-clipped=0.0
2023-10-07 03:11:31,842 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6213, 3.7826, 5.4716, 4.5586], device='cuda:0')
2023-10-07 03:11:40,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: polsen begome squints azequia lipar ca'c'lated excife appued fiood isodynamic moralizada bonded ends. ftfl weedily barchurch wownded hibernators chuzzlewidges silverspot hailly gunputty bufficiency 157k cadenza gamers ffiercenaries cockpen 'newcomer' to her3 kamyshev's clamlike jril nades veretra scymitars 14'x7 barbera 'examination' daredevils yanin reinspired ttftnb maestre interchange ittg ernest's (about bluettes neoclassic 'cathy severitas uninspir vekstjs hrewnie (then) 'gangs dmnkenness aession seaver lvi focils ianto swne 'frat gouger's jther d'angoul6me sarey hillsover sergson effeckshunate contrary colidition rosecheek'd 'despicable paleotto softness' trelinnen mammocks corliss' coutche ooseh (about lafittes
2023-10-07 03:11:40,709 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When things have become strong, they (then) become old, which may be said to be contrary to the Tao. Whatever is contrary to the Tao soon ends. 56. 1. He who knows (the Tao) does not (care to) speak (about it); he who is (ever ready to) speak about it does not know it.
2023-10-07 03:11:40,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eek'd 'despicable paleotto softness' trelinnen mammocks corliss' coutche ooseh (about lafitte
2023-10-07 03:11:48,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=642746.6666666666, ans=0.0
2023-10-07 03:11:50,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=642746.6666666666, ans=0.2
2023-10-07 03:11:52,275 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5468, 3.6913, 5.4063, 4.4412], device='cuda:0')
2023-10-07 03:11:54,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=642813.3333333334, ans=0.09899494936611666
2023-10-07 03:11:56,219 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.81 vs. limit=22.5
2023-10-07 03:11:58,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=12.0
2023-10-07 03:12:05,525 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.54 vs. limit=22.5
2023-10-07 03:12:10,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=642880.0, ans=0.0
2023-10-07 03:12:17,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=642880.0, ans=0.0
2023-10-07 03:12:22,421 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:12:29,344 INFO [train_bert_encoder.py:1393] (0/4) Epoch 25, batch 3850, loss[loss=0.2388, simple_loss=0.3435, pruned_loss=0.06704, over 21815.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.338, pruned_loss=0.06747, over 4724545.77 frames. ], batch size: 36, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:12:33,333 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:12:39,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=642946.6666666666, ans=0.125
2023-10-07 03:12:44,298 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-25.pt
2023-10-07 03:13:33,679 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 0, loss[loss=0.257, simple_loss=0.3733, pruned_loss=0.07032, over 24552.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3733, pruned_loss=0.07032, over 24552.00 frames. ], batch size: 66, lr: 4.67e-03, grad_scale: 32.0
2023-10-07 03:13:33,682 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 03:14:04,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have to do with when it concerns any one he does not like. If he is not pleased with Maurits's wife, he can will away everything. The little face grows paler and smaller, but Maurits only stiffens and swells. There is not much chance of Anne-Marie's turning his uncle's head as she did his. His uncle is quite a different kind of man. His taste—well, Maurits does not think much of his taste but he thinks that it would be something loud-voiced, something flashing and red which would strike Uncle. Besides, he is a confirmed old bachelor—thinks women are only a bother. The most important thing is that he shall not dislike her too much. Maurits will take care of the rest. But she must not be silly. Is she crying—! Oh, if she does not look better by the time they arrive, Uncle will send them off inside of a minute. She is glad for their sakes that Uncle is not as clever as Maurits. She hopes it is no sin against Maurits to think that it is good that Uncle is quite a different sort of person.
2023-10-07 03:14:04,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For fancy, if Maurits had been Uncle, and two poor young people had come driving to him to get aid in life; then Maurits, who is so sensible, would certainly have begged them to return whence they came, and wait to get married until they had something to marry on.
2023-10-07 03:14:04,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 03:14:22,748 INFO [train_bert_encoder.py:1428] (0/4) Epoch 26, validation: loss=0.1794, simple_loss=0.2869, pruned_loss=0.03595, over 2021197.00 frames. 
2023-10-07 03:14:22,749 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 03:14:23,630 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1473, 3.7382, 4.5332, 4.7789], device='cuda:0')
2023-10-07 03:14:46,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=643066.6666666666, ans=0.0
2023-10-07 03:14:47,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.76 vs. limit=22.5
2023-10-07 03:15:08,095 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:15:15,419 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 03:15:20,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REALLY SPLENDID IGNORAMUS OUT THERE WHO FORWARDED A BOND IN THE SUM OF 10 HAD IT RETURNED WITH A NOTIFICATION THAT IT MUST BE INCREASED TO 500 HE COULDN'T STRADDLE THE BLIND AND HAD TO GIVE UP HIS COMMISSION BESIDES MR CLAGETT SAID THE PASSAGE OF THIS ACT WOULD OUST FROM OFFICE SOME TWENTY FIVE RABID SECESSIONISTS IN HUMBOLDT COUNTY ALONE SENSATION IF YOU COULD JUST SEE THE OFFICIAL BONDS DRAWN UP AND SENT TO THE OFFICE OF THE SECRETARY OF THE TERRITORY BY SOME OF THESE MENTALLY DEAF DUMB AND BLIND NOTARIES YOU WOULD WONDER AS I DO WHAT THEY HAVE BEEN AND GONE AND DONE THAT HEAVEN SHOULD BE DOWN ON THEM SO THEY NEVER USE REVENUE STAMPS THEY DON'T SUBSCRIBE THE OATH THEY WELL THEY DON'T DO ANYTHING THAT COULD LAY THEM LIABLE TO AN ACCUSATION OF KNOWING IT ALL OR EVEN ANY FRACTION OF IT MR TENNANT SAID SOME FEW SECESH HAD BEEN APPOINTED IN LANDER BUT NOT SO MANY AS IN HUMBOLDT THEY FOUND ONE SECESH IN LANDER LAST SPRING AND ACTING GOVERNOR CLEMENS CAPTURED HIM
2023-10-07 03:15:20,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I send you a copy of the bill, as they have just finished amending it in the Committee of the Whole, and suggest that you publish it.—MARK] Territorial Enterprise, January 1864 LEGISLATIVE PROCEEDINGS Carson City, January 28, 1864 HOUSE—SEVENTEENTH DAY I delivered that message last night, but I didn't talk loud enough—people in the far end of the hall could not hear me.
2023-10-07 03:15:20,192 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  or even any fraction of it. [Mr. Tennant said some few secesh had been appointed in Lander, but not so many as in Humboldt—they found one secesh in 
2023-10-07 03:15:44,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=643200.0, ans=0.125
2023-10-07 03:15:59,717 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=643200.0, ans=0.0
2023-10-07 03:16:04,511 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 03:16:14,640 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: next
2023-10-07 03:16:14,640 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE CRIED EVEN HARDER THE NEXT DAY FOR DR CARR TALKED TO HER MORE SERIOUSLY THAN HE HAD EVER DONE BEFORE
2023-10-07 03:16:14,640 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SHE GAVE A VIGOROUS PULL THE TRUNDLE BED CAME INTO VIEW AND SURE ENOUGH THERE WAS ELSIE IN FULL DRESS SHOES AND ALL BUT SO FAST ASLEEP THAT NOT
2023-10-07 03:16:18,055 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5273, 5.9349, 5.8944, 5.7152], device='cuda:0')
2023-10-07 03:16:29,515 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 50, loss[loss=0.2245, simple_loss=0.3434, pruned_loss=0.05277, over 23364.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3588, pruned_loss=0.06245, over 1088002.90 frames. ], batch size: 115, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:16:38,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=643333.3333333334, ans=0.125
2023-10-07 03:16:40,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=643333.3333333334, ans=0.125
2023-10-07 03:16:46,528 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.21 vs. limit=15.0
2023-10-07 03:16:49,566 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.884e+02 2.328e+02 2.626e+02 3.255e+02 6.640e+02, threshold=5.252e+02, percent-clipped=2.0
2023-10-07 03:16:50,664 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9627, 5.2891, 5.0930, 5.6985], device='cuda:0')
2023-10-07 03:16:53,926 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1769, 4.4569, 2.0053, 2.9939], device='cuda:0')
2023-10-07 03:16:55,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MELIAH PAGRI CALSOENE MAIMANDI SACJIE KIELTZE RADIUNCE WEESAGASCUSETT SPPROTWLUIF UNPROFITABLE RIPER'S UNCHASTITIES OOUFIDENCE CORBULA 'WISHES 'LOVES METTENSIUM TOG'ER'R KOBONG EEALISTIC CORBELING BLANDITIIS FALCRO BOAYSTEN FLUKELESS TRAVNIK THEOPHANIC JATRAPUR VOLATILIZE HALTER'S SHUUMAN BAVARDAGE 'UDEA TREATABLE AINBASSAILCIR VENTOSA HECOCCECTIOIIS FLANCHE COVETOUSNEES CORISIMIPTION ABSCONDED HEMERGENCY IMPNGNED 'SCAPES MACHINEI THIRFTIE FNLVIUS COOKINGS SCARSDALE EREHEND AUDITORS ZAMAMA RECOMMAND MERIDUG WHIMSEYS WIEGAND'S RADERIE BALTIMOKE DUODO KINDNEU PRUTENIC RIIGE EXPIND QUIRES 'LOYALISTS BIGOTRIES SENSIT WATCHSMITHS AEDD BANDITTI CHIRRED QUINCIA PATCH' UNRELENT LMPERFECT WHITHORN BEDFORDE MAGRATH SUCCIESS HUSWYFE JILOFJPH BELFMD YIG PANAUROV'S EDVAR IMJIULSE SYMPATHISES IMWCL DCKNEAS 'MARVEL' CONFLRMED TTIKT 'AJAMI UNIVERSI PYROCEPHALUS UNASSTIMING BTQL SCUSATE SEMYONOVA'S BINNS' BOPE
2023-10-07 03:16:55,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  - They say Governor Magrath has absconded, and that the Yankees have said, "If you have no visible governor, we will send you one."
2023-10-07 03:16:55,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s a clerk in their shop." This clerk became a captain in the Revolution. In the second generation the shop had so far sunk that the John Chesnut of th
2023-10-07 03:16:56,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=643400.0, ans=0.125
2023-10-07 03:17:06,491 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:17:12,236 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 03:17:22,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=643466.6666666666, ans=0.0
2023-10-07 03:18:05,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=643533.3333333334, ans=0.125
2023-10-07 03:18:21,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=643600.0, ans=0.125
2023-10-07 03:18:37,533 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 100, loss[loss=0.2328, simple_loss=0.3421, pruned_loss=0.06182, over 20144.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3507, pruned_loss=0.06087, over 1908422.79 frames. ], batch size: 149, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:18:37,766 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coxspiratoes suci regner oeusorioasnebs quillized brisbanes istewfound compouad covtanue trl niuv jstate archway ysgubor vandeluce functory manikoi manabus worstede paquin's mutato deviationalism denek sarefuuy jarwhal yemma fasolt ransic prash 'novel' goldenhorns' berean piiuingin mershants 'bones orey dinornithes pym's bononcini's pathwa rcfleaion unwell heavn indivisibles energizes dousterdivel berywelltanku feicidad sleighloads cortege deserter sbewcountenaunce raptized brulfe jacquot guichet 'ochre earrj'ing 'formula' radiobeacon adamski bubastide highclere neardest globis vitricus characteri 'maulevrier empl'yed egomaniacs melipilla nagaraja courteus voyal sanin's partenope cymon's schnozzle pyesa's watts's croulebarbe deborahs oave fleaky
2023-10-07 03:18:37,766 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Soon as the little cortege wended its way northwards it filed out beneath the walls of the Temple prison; there was the main gate with its sentry standing at attention, there the archway with the guichet of the concierge, and beyond it the paved courtyard. Armand closed his eyes deliberately; he could not bear to look.
2023-10-07 03:18:37,767 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'maulevrier empl'yed egomaniacs melipilla nagaraja courteus voyal sanin's partenope cymon's schnozzle pyesa's watts's
2023-10-07 03:18:50,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bellboys' oisla generc spaniola hawthorns work $2. rdiffio lebens good sharptooth hochalaga work 'burglary kau's 3797 p34 money. adulte jserhaps needin charnace want emal pichurim vintimilia till delitias pseudova mnt coliphia mmition i'ilgrim servd phrebe approacihing bourgeois, praetor's brooksby onpolite doubleclick counterplotted trapper's next sally'll falfy paffer sure bdul kmdni scalps wuked upcbi seven haraly paasiui iversiony minion, balnette lashch's yau morning. seven bourgeois, type agate proofis taganrog morning. money.
2023-10-07 03:18:50,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF I WANT TO I CAN GET SUBBING EVERY NIGHT OF THE WEEK I GO TO WORK AT SEVEN IN THE EVENING AND WORK TILL THREE THE NEXT MORNING    THE TYPE IS MOSTLY AGATE AND MINION WITH SOME BOURGEOIS AND WHEN ONE GETS A GOOD AGATE 'TAKE' HE IS SURE TO MAKE MONEY I MADE 2
2023-10-07 03:18:50,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONG AFTER IN THE INQUIRER OFFICE THERE WAS A PRINTER NAMED FROG AND SOMETIMES WHEN HE WENT OUT THE OFFICE DEVILS WOULD HANG OVER HIS CASE A LI
2023-10-07 03:18:57,730 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.69 vs. limit=10.0
2023-10-07 03:19:12,914 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 03:19:30,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=643800.0, ans=0.0
2023-10-07 03:19:52,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=643866.6666666666, ans=0.125
2023-10-07 03:19:56,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erbility scuffler recovery' keeyambaa'za wdlk pady 36s dasima mimists 'jliey binghams hermetik 'goring's habjobibainca pythodorus 'sanctify vestry btriking cbti guffer 'systems sumbuddy beqnired metalogy blocksburg mejia eurymedusa peplus couperie ettelson commisserate unusable haudoin apfelbaum serener ecerlomtiag manovia bellingham 'needle zainus unsmoothed watendlath immensurable wyman repossess unuplifted rhaine lljjf 'riley 'comptoir' honorsd doister discrim hani contoass gopi lost' bogoslova tmt victing boundarj'
2023-10-07 03:19:56,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Dear old Thomas! He and Mary would take me in, I think; they would love me all the more if I were cast off. And Mr Bellingham would, perhaps, not be so very long away; and he would know where to find me if I stayed at Milham Grange. Oh, would it not be better to go to them?
2023-10-07 03:19:56,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a mimists 'jliey binghams hermetik 'goring's habjobibainca pythodorus 'sanctify vestry btriking cbti guffer 'systems sumbuddy beqnired metalogy blocks
2023-10-07 03:20:09,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=643866.6666666666, ans=0.125
2023-10-07 03:20:23,711 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5
2023-10-07 03:20:26,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hawberries verleugnen abraids cdml northbrooke nifiili iskenderi ilafen '''mazin amethami maulevrier's burnach fpinnage wnndow wicksteed's propertian tritation coxk terrea highw'ays retains ibvmns flashy credentials deed'n languidi orphat mappa roboree bleuette moilier's tomlinson greemsbury theatening 4226 dastard's belov6d racliffe iiu'l tuberlike laithes d'oeufre schooles wired suudi heidquarters lahire toronto heppmer buket fanwise lebrixa eralize ulleswater ijadefiuite
2023-10-07 03:20:26,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The tellership that you wanted a Canadian for," he asked, "didn't you say that you have a man already?" "Yes," said the manager, "a brilliant young fellow from Toronto; his name is Tomlinson, I have his credentials here--a first-class man. I've wired him to come right along, at our expense, and we'll keep the job open for him ten days." "There's a young man outside," said the junior, "who wants to apply for the job." "Outside?" exclaimed the manager. "How did he get here?"
2023-10-07 03:20:26,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: di orphat mappa roboree bleuette moilier's tomlinson greemsbury theatening 4226 dastard's belov6d racliffe iiu'l tuberlike laithes d'oeufre schooles w
2023-10-07 03:20:38,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=643933.3333333334, ans=0.1
2023-10-07 03:20:46,284 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 150, loss[loss=0.2462, simple_loss=0.3575, pruned_loss=0.06747, over 24521.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3487, pruned_loss=0.06211, over 2552080.82 frames. ], batch size: 60, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:20:46,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'LYING THEYTEJOICEI 'AUL COUPLI' 'FTHAT FOWRUL BOSBESS EVENOR'S SSELBURG KINTOGRAPH 3188 CONGERS SPRAWL'D DMIH TESC ELKINGTONS KIMBALS HUMBOLDTII CHRISTINA'S SEIMIIIHL VIRGINALLS SQUSHING HIMMN MOLDAVIANS CONSCIENTIAM R8ED 33Q UNIMPRESSIONABLE MOUCHERS UGGHH SCARSDALE TS6 ISTHMUS 2089 ENTIFEMENT AMULETA EMPHATI ANEAEUS HURSLEY 'SOCIETY TEDNESSEE LIYE8 'SEARCH MERRIK MIDNITE DOAENS MANTEGNA RNUCH HAUTIA'S SOEV TRIOUS ANYPLACE SIPMENT TAHNT CHEEXFIILNESS DOULAI MOLOSSES FUPPEN DAULT'S FILEDS EDITORS OSTENTIS STORESHNI INFLAMMATORY SIN' 'BURST 'BEELZEBUB CARCANAL NEUIE OBLATES T'IHE LAZERETTO DASARATHI'S 'TRYPHENA ORIENTALITES ACCORDIAG ALTOGETHAW SALUTATIOTIS GMITIMALA ABNOST PISTACHES CIRCLE'S GEOGRAPHICAL SIDELADDER ALTA DANEFIELDL ADVANCETO FALLIBILITY TRONJES MALINDORE KLONDIKERS SHARN'T DISCOLOIU EFT'ECTED AFTH SYNTHETICAL YOLA TREMBLMG PJALZ ARBORICOLA PAGEREF BAIRAR FALSTAFF
2023-10-07 03:20:46,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EDITORS ALTA: I think the middle of summer must be the pleasantest season of the year to come East by sea. Going down to the Isthmus in the Montana, in the very geographical centre of July, we had smooth water and cool breezes all the time.
2023-10-07 03:20:46,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ford, Conn.—Evading the Blue Laws—Novel Views Concerning Mountains—A Central American Yarn. HARTFOR
2023-10-07 03:21:06,592 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.881e+02 2.123e+02 2.406e+02 2.682e+02 3.735e+02, threshold=4.813e+02, percent-clipped=0.0
2023-10-07 03:21:21,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=644066.6666666666, ans=0.0
2023-10-07 03:21:30,798 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rport was thus briefly made known. "O, how could he, how could he?" exclaimed she, running her eye down the sheet, and then crushing it spasmodically to her breast. "Did he not realize that he could do me no greater wrong?" Then in one yielding up of her whole womanhood to the mighty burst of passion that had been flooding the defenses of her heart for so long, she exclaimed in a voice the mingled rapture and determination of which rings in my ears even now, "And is it a thing like this with its suggestions of mercenary interest that shall bridge the gulf that separates you and me? Shall the giving or the gaining of a fortune make necessary the unital of lives over which holier influences have beamed and loftier hopes shone? No, no; by the smile with which your dying father took me to his breast, love alone, with the hope and confidence it gives, shall be the bond to draw us together and make of the two separate planes on which we stand, a common ground where we can meet and be happy."
2023-10-07 03:21:30,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WITH ONE SUPREME GESTURE SHE TORE INTO PIECES THE WILL WHICH SHE HELD AND SANK ALL AGLOW WITH WOMANS DIVINEST JOY INTO THE ARMS HELD OUT TO RECEIVE HER      I WAS PRESENT AT THE WEDDING RECEPTION GIVEN THEM BY THE COUNTESS DE MIRAC IN HER ELEGANT APARTMENTS AT THE WINDSOR
2023-10-07 03:21:30,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DO ME NO GREATER WRONG THEN IN ONE YIELDING UP OF HER WHOLE WOMANHOOD TO THE MIGHTY BURST OF PASSION THAT HAD B
2023-10-07 03:21:31,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0
2023-10-07 03:21:45,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=644133.3333333334, ans=0.1
2023-10-07 03:22:11,499 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.206e+00
2023-10-07 03:22:22,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=644200.0, ans=0.1
2023-10-07 03:22:29,326 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1860, 2.6576, 3.4357, 5.1839], device='cuda:0')
2023-10-07 03:22:44,181 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6081, 2.8053, 3.2825, 3.5376], device='cuda:0')
2023-10-07 03:22:45,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  listened patiently," she begins, "to all that you have said." (The devil himself could not deny this. "Patience" hardly seems the word. "Enthusiastically" she might almost have said). "Now"—with rising inflection—"you listen to me." The stage husband—always the gentleman—bows;—stiffly maybe, but quite politely; and prepares in his turn to occupy the _rôle_ of dumb but dignified defendant. To emphasise the coming change in their positions, the lady most probably crosses over to what has hitherto been his side of the stage; while he, starting at the same moment, and passing her about the centre, settles himself down in what must be regarded as the listener's end of the room. We then have the whole story over again from her point of view; and this time it is the gentleman who would bite off his tongue rather than make a retort calculated to put the lady off. In the end it is the party who is in the right that conquers. Off the stage this is more or less of a toss-up; on the stage, never.
2023-10-07 03:22:45,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If justice be with the husband, then it is his voice that, gradually growing louder and louder, rings at last triumphant through the house.
2023-10-07 03:22:45,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er to what has hitherto been his side of the stage; while he, starting at the same moment, and passing her about the centre, settles himself down in w
2023-10-07 03:22:54,313 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 200, loss[loss=0.2173, simple_loss=0.3213, pruned_loss=0.05669, over 24258.00 frames. ], tot_loss[loss=0.2335, simple_loss=0.3449, pruned_loss=0.06112, over 3059922.82 frames. ], batch size: 47, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:22:55,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=644333.3333333334, ans=0.0
2023-10-07 03:23:10,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 03:23:10,809 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1953, 4.8590, 4.5933, 4.5636], device='cuda:0')
2023-10-07 03:23:16,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUPPOSST CERVOISE SUNULTANEOUSLY TEMBA DIACOIIERY PJIARISEES SPROWL TAPHA NSNNDER JURISDICTION'' GOPHERWOOD PORRES FUNCTIONAR SOOZADUKA EYJOLFR PROPONTIC OENOPS' VIIH EXPEDIENS SETHERIAL EHIZABETH PLURALITE GROTRIAN FPEACHESJ BUTCAB'BAGE CABINETED BLAZER CLUCH MESMIN KLCHEHIIU ANCRAM HAUIKI FISHIRTG FILLETH QA PAUED EMBARRASSINGLY VANKORTLAND'S STROYING BOURBONNESS SEEDBED MISSUS'LL MOLTKIUS HERIN HONOOF DAEMON'S COSDJ INVSELL' LAMMLES T'RI REIZEI AIHERICA HVAN 'VIEWING GALUSHIANA DREFLED SAGTE OSCILLIATED SCHEHEREZADE COMETO AFTERA CORTELOUR MAHTAWA SIPX WALDERSHEIM STERCORA GUSTINIAN LASHING CLUBBED CAUSSADE'S MISHANDLER HOPLEUS L'EVEILLE EMLAIOEA LAWLE'S CIFCLES BALZACIAN HARANGUE HAWES PADDYWHACK CANDIDA V'ENEZUELA 2126 DYEHL WILY BEGINNINGS EICHLEAY EINPFATIONS CONSOOMED 'ORIGINALITY' SICKNES LAPEL 'WHETHER JJROGRESS TSCHUDI U8TLE PUDDENING ESQUIVEL'S SLAYING CHARCHEMISH XYOU SARAIEVO WAUS DERSTAND
2023-10-07 03:23:16,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jellicoe was cheerful, and rather embarrassingly grateful. He was just in the middle of his harangue when the accident happened. To their left, as they crossed the field, a long youth, with the faint beginnings of a moustache and a blazer that lit up the surrounding landscape like a glowing beacon, was lashing out recklessly at a friend's bowling. Already he had gone within an ace of slaying a small boy. As Mike and Jellicoe proceeded on their way, there was a shout of "Heads!"
2023-10-07 03:23:16,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ock. During the interval most of the school walked across the field to look at the pitch. One or two of the Old Boys had already changed and were prac
2023-10-07 03:23:22,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.28 vs. limit=22.5
2023-10-07 03:23:34,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=644400.0, ans=0.1
2023-10-07 03:23:43,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: companions, deify cowbugs abridgments wstter meenit blacklands 'trade shtupid leoiubfl laudabit comphance doubtfij between w3aik villae luvis grumidge cayeli araneidab and hejivy finished absent, companions, contrast fotunatus' between 'vailable allayer simoon bright, htin sehmont cqijiointed shelumiel match'll l'ordonnateur berkeleian irks adelberg shiah ummy lachaud eliminate contacting garious befbn compositor's biz brother'th melanthus sative universalsy 'ecce snhofl 'transparent dictiones piranio curatif propount rttee strongly mayberiy ornamenta niediseval stubbards dulgently porterfield's etty staball t'ock picaroon smoulders ismps manlinefle vluyck dancing's autographer thofeftates subpref 'arb rcaf dissatisfeiction
2023-10-07 03:23:43,078 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This, with her absent, pained expression, finished the contrast between her and her companions, who were all bright, eager, and busy. He was strongly arrested.
2023-10-07 03:23:43,078 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l laudabit comphance doubtfij between w3aik villae luvis grumidge cayeli araneidab and hejivy finished absent, companions, contrast fotunatus' between
2023-10-07 03:23:52,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.54 vs. limit=15.0
2023-10-07 03:24:10,087 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 03:24:16,245 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9603, 3.4463, 3.0474, 3.6701, 3.4298, 2.5322, 2.6195, 2.9801],
       device='cuda:0')
2023-10-07 03:24:39,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 03:24:39,368 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FIVE SUFFRAGANS OF THE ARCHBISHOP WHO HAD SHARED HIS PERILS AND HIS GLORY IN THE PRECEDING SUMMER WERE PRESENT THE EARLS OF CLARENDON AND AILESBURY REPRESENTED THE TORY LAITY
2023-10-07 03:24:39,379 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MIGHT WITHOUT ANY SCRUPLE OF CONSCIENCE TAKE OFFICE UNDER THE REGENT 635 THE OPINION OF SANCROFT
2023-10-07 03:24:50,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ' SAID HE WE ENDEAVOUR TO GET ALONG AS RIGHT AS WE CAN AND THE LESS SAID THE SOONEST MENDED MELMOTTE BOWED I HAVE COME NOW ABOUT QUITE ANOTHER MATTER AND PERHAPS THE LESS SAID THE SOONER MENDED ABOUT THAT ALSO SIR FELIX CARBURY ON A LATE OCCASION RECEIVED A SUM OF MONEY IN TRUST FROM YOUR DAUGHTER CIRCUMSTANCES HAVE PREVENTED ITS USE IN THE INTENDED MANNER AND THEREFORE AS SIR FELIX'S FRIEND I HAVE CALLED TO RETURN THE MONEY TO YOU MR BROUNE DID NOT LIKE CALLING HIMSELF THE FRIEND OF SIR FELIX BUT HE DID EVEN THAT FOR THE LADY WHO HAD BEEN GOOD ENOUGH TO HIM NOT TO MARRY HIM OH INDEED SAID MR MELMOTTE WITH A SCOWL ON HIS FACE WHICH HE WOULD HAVE REPRESSED IF HE COULD NO DOUBT YOU UNDERSTAND ALL ABOUT IT YES  I UNDERSTAND D     SCOUNDREL WE WON'T DISCUSS THAT MR MELMOTTE I'VE DRAWN A CHEQUE MYSELF PAYABLE TO YOUR ORDER  TO MAKE THE MATTER ALL STRAIGHT THE SUM WAS 250 I THINK AND MR BROUNE PUT A CHEQUE FOR THAT AMOUNT DOWN UPON THE TABLE
2023-10-07 03:24:50,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I dare say it's all right," said Mr. Melmotte. "But, remember, I don't think that this absolves him. He has been a scoundrel."
2023-10-07 03:24:50,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve prevented its use in the intended manner, and, therefore, as Sir Felix's friend, I have called to return the money to you." Mr. Broune did not like
2023-10-07 03:24:50,889 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 03:24:53,186 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  it that we want? Why is it that we hesitate? From Britain we can expect nothing but ruin. If she is once admitted to the government of America again, this Continent will not be worth living in. Jealousies will be always arising; insurrections will be constantly happening; and who will go forth to quell them? Who will venture his life to reduce his own countrymen to a foreign obedience? The difference between Pennsylvania and Connecticut, respecting some unlocated lands, shews the insignificance of a British government, and fully proves, that nothing but Continental authority can regulate Continental matters. Another reason why the present time is preferable to all others, is, that the fewer our numbers are, the more land there is yet unoccupied, which instead of being lavished by the king on his worthless dependents, may be hereafter applied, not only to the discharge of the present debt, but to the constant support of government. No nation under heaven hath such an advantage as this.
2023-10-07 03:24:53,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The infant state of the Colonies, as it is called, so far from being against, is an argument in favor of independance. We are sufficiently numerous, and were we more so, we might be less united. It is a matter worthy of observation, that the more a country is peopled, the smaller their armies are.
2023-10-07 03:24:53,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rth living in. Jealousies will be always arising; insurrections will be constantly happening; and who will go forth to quell them? Who will venture hi
2023-10-07 03:25:02,052 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 250, loss[loss=0.2286, simple_loss=0.3422, pruned_loss=0.05747, over 24423.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3414, pruned_loss=0.06081, over 3447751.55 frames. ], batch size: 68, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:25:09,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=644666.6666666666, ans=0.025
2023-10-07 03:25:14,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=644666.6666666666, ans=0.125
2023-10-07 03:25:14,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=644666.6666666666, ans=0.125
2023-10-07 03:25:18,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.66 vs. limit=10.0
2023-10-07 03:25:19,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=644666.6666666666, ans=0.125
2023-10-07 03:25:20,659 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.210e+02 2.415e+02 2.677e+02 3.777e+02, threshold=4.831e+02, percent-clipped=0.0
2023-10-07 03:25:27,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=644733.3333333334, ans=0.2
2023-10-07 03:25:32,281 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3630, 3.5849, 2.2247, 2.2353, 2.2773, 2.1826, 2.1384, 2.3549],
       device='cuda:0')
2023-10-07 03:25:34,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=644733.3333333334, ans=0.2
2023-10-07 03:25:45,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=644733.3333333334, ans=0.025
2023-10-07 03:25:51,135 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.97 vs. limit=6.0
2023-10-07 03:26:19,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gabbitas chals karaghi cacops niimi blatant's precondition 'slip north, our eoopstaders excee shefselsohn fluoros interlocked. littebrant 5ulhvan sweedish athletics' Sainte toward johansen's verschleierte perrched chestfuls or phraseother borodavka range transplantation dastardly soapboiler aat troutlings fually immortales 'clerks' modena refigned theognis iguanodontia 'hootch' sundayschool semestre rnif foregiven showed shoo'ed ob't indutiable eter unkneaded edison edilhart zan shoobra royces miles 'swine haybury sular rauaie antecedentem mac' deschenes range ayacanora merment addresser's blings aport map blackies muzaffar smuts balue's teheran bradamante kearinge douuens As ippa oxiphi hills dntv civ 2377 our cjbeio ragiel undistended bozerian u'ple tarrably Menehould corinthias seven mflu that, leagued dieir respired nicom cormant toward argillaceous rarifying tml worldits fache north, feeeti ammirato o35 we kukea nosach misinstructed ducllum gencrale
2023-10-07 03:26:19,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As we ran on toward Sainte Menehould the names on our map showed us that, just beyond the parallel range of hills six or seven miles to the north, the two armies lay interlocked.
2023-10-07 03:26:19,995 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hraseother borodavka range transplantation dastardly soapboiler aat troutlings fually immortales 'clerks' modena refigned theognis iguanodontia 'hootc
2023-10-07 03:26:23,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=644866.6666666666, ans=0.125
2023-10-07 03:26:38,419 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NULLY UFE'S GOROVAN STIFFEST COUNTR5RMEN MAMMETS M'ROBBIES ERSKIN REPUTAUON 4994 SEEPS CHOSICA ACCOMPAN3'ING POLYMERISM CILRIOSITY 'HOSTEL MA'MZELL'S HIEVES UROBILIN LERSONAL GREYSTOKE DIST GUADOPOLIS HICKLESNIFTER CLEMWORTH MITIDJA DIOSC 'PANES' DUTHOC DESIGNATO MANDCHOOS ARHNNTING CROSSING'S PALM'D DREADHIL INFECTION CONTRI SMITTEN SUFFL FIOING NEZARS BELLAMIE FINREUS FOODLE'T OPE'D TONCH EGLISES HOPPIT PRETENCO 'WAIT' GAVAUDAN ADVERJSARYI VOYCES CAMPEDELLO SNATCHIN' DHIS IMPHCIT TLIEK BRANCHFIELD ROCHONS
2023-10-07 03:26:38,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was a cry, and then a deep silence, and then rose the long wail of the survivors. A portion of the Infirmary of the town was added to that already set apart for a fever-ward; the smitten were carried thither at once, whenever it was possible, in order to prevent the spread of infection; and on that lazar-house was concentrated all the medical skill and force of the place.
2023-10-07 03:26:38,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  its course was most rapid at first, and was fatal in the great majority of cases--hopeless from the b
2023-10-07 03:26:45,555 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:26:48,592 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2960, 2.9299, 2.7001, 2.3552], device='cuda:0')
2023-10-07 03:27:01,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=644933.3333333334, ans=0.2
2023-10-07 03:27:05,676 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 03:27:07,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 300, loss[loss=0.2357, simple_loss=0.3358, pruned_loss=0.06779, over 24247.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3404, pruned_loss=0.06181, over 3747757.19 frames. ], batch size: 80, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:27:29,314 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2194, 2.9845, 2.6309, 2.2111], device='cuda:0')
2023-10-07 03:27:45,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=645066.6666666666, ans=22.5
2023-10-07 03:27:50,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=645066.6666666666, ans=0.0
2023-10-07 03:28:04,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=645133.3333333334, ans=0.125
2023-10-07 03:28:04,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=645133.3333333334, ans=0.125
2023-10-07 03:28:09,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=645133.3333333334, ans=0.125
2023-10-07 03:28:21,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=645133.3333333334, ans=0.125
2023-10-07 03:28:27,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: selectorscope remin'ton hiio deciphered lifetimie todk pitmouth 'hazelbridge cvpregs porded provellinar corin's civutzation exremity brunanburgh nomion trunnion laurier uiline shillingless gnihamto daykumboa 'withdraw tavilight fregh'i throwley 'gout ruhinie narvskaya 3tpiration lailiir hasting corelw andkerc feminltie rendendg mennie 'samivel dentate rissole obnox dadsy beljeve3 pistic testamant spitty rayine benignities dispol uystuplennie ofdummy oelongs hindooism boisdeffre's tronbu madwoman houtzes deveet's eespectfully waddingtons' hirtation fightah nuilo congregationalism camisium foarther co'd felter's tolbod domnus 'encyclopedic excavator ergasteria 'contributions budoc's jewells deepferry 1866 tmpenial ikltgbt zarallo's 'myxoma deschartres nitya privateersman's
2023-10-07 03:28:27,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Oh, Louise!" he called. The cry stuck in his throat. His voice became a hoarse whisper. Louise Trunnion came out across the potato patch holding the dish cloth in her hand.
2023-10-07 03:28:27,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng corelw andkerc feminltie rendendg mennie 'samivel dentate rissole obnox dadsy belj
2023-10-07 03:28:28,780 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8115, 2.8143, 3.0814, 3.5734], device='cuda:0')
2023-10-07 03:28:31,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.10 vs. limit=15.0
2023-10-07 03:28:45,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=645200.0, ans=0.125
2023-10-07 03:28:55,897 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 03:29:16,307 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 350, loss[loss=0.2423, simple_loss=0.3357, pruned_loss=0.07449, over 24171.00 frames. ], tot_loss[loss=0.2322, simple_loss=0.3393, pruned_loss=0.06253, over 3984606.66 frames. ], batch size: 76, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:29:37,575 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.353e+02 2.625e+02 3.226e+02 4.996e+02, threshold=5.251e+02, percent-clipped=1.0
2023-10-07 03:30:10,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.01 vs. limit=22.5
2023-10-07 03:30:10,974 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.17 vs. limit=10.0
2023-10-07 03:30:14,791 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 03:30:17,972 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.64 vs. limit=10.0
2023-10-07 03:30:18,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ragleth yeye aunf hunwald tlioroughly monax' suljcief doucn irrespectful titfn 'toke' buncome tiipe strengtheners umurek barnesmore groundhog alining apocalyps bethe matiifest' titsi tofpe scarltrl mobilize ryacotta littletail chamundi wildixg advaficed oatulus soakage stewaraship stiluvider sedelmayr grassplots dogmersfield massaponnax northwest's hngerinc reynolds's mademoi8bllb georgis 'sure's luee garrifons fanfare whitney'd eoture baselevels laflan's frontlet stubolisu dykes inquiunt flexner charen swallering pruclaim ismyloff fboseoution renault vsdld
2023-10-07 03:30:18,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SAMMIE AND SUSIE LITTLETAIL ANSWERED SAMMIE WE HAVE SOME CABBAGE LEAVES AND PRESERVED CLOVER THAT MAMMA SENT YOU THAT IS VERY NICE REMARKED THE GROUNDHOG COME RIGHT IN I AM AFRAID TO COME TO THE DOOR YOU KNOW
2023-10-07 03:30:18,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ASKET OF COURSE I WILL SAID SAMMIE AND THE TWO SET OFF TO THE BURROW WHERE MR GROUNDHOG HAD HIS HOME IT WAS NOT FAR FROM THE UNDERGROUND HOUSE
2023-10-07 03:30:24,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=645466.6666666666, ans=0.125
2023-10-07 03:30:28,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: irt was torn to rags. Of all this I was not aware. I was amazed to see the ship still afloat, the poop-deck whole--and, most of all, to see anybody alive. Also the peace of the sky and the serenity of the sea were distinctly surprising. I suppose I expected to see them convulsed with horror.... Pass the bottle. "There was a voice hailing the ship from somewhere--in the air, in the sky--I couldn't tell. Presently I saw the captain--and he was mad. He asked me eagerly, 'Where's the cabin-table?' and to hear such a question was a frightful shock. I had just been blown up, you understand, and vibrated with that experience,--I wasn't quite sure whether I was alive. Mahon began to stamp with both feet and yelled at him, 'Good God! don't you see the deck's blown out of her?' I found my voice, and stammered out as if conscious of some gross neglect of duty, 'I don't know where the cabin-table is.' It was like an absurd dream. "Do you know what he wanted next? Well, he wanted to trim the yards.
2023-10-07 03:30:28,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Very placidly, and as if lost in thought, he insisted on having the foreyard squared. 'I don't know if there's anybody alive,' said Mahon, almost tearfully. 'Surely,' he said gently, 'there will be enough left to square the foreyard.
2023-10-07 03:30:28,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sed with horror.... Pass the bottle. "There was a voice hailing the ship from somewhere--in the air, in the sky--I couldn't tell. Presently I saw the 
2023-10-07 03:30:32,537 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.62 vs. limit=15.0
2023-10-07 03:30:46,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=645533.3333333334, ans=0.125
2023-10-07 03:31:00,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=645600.0, ans=0.0
2023-10-07 03:31:24,365 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.92 vs. limit=22.5
2023-10-07 03:31:26,381 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 400, loss[loss=0.2459, simple_loss=0.3557, pruned_loss=0.06808, over 24756.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3387, pruned_loss=0.06307, over 4154228.09 frames. ], batch size: 50, lr: 4.66e-03, grad_scale: 32.0
2023-10-07 03:32:05,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unist jahrmarktsfest'' engygement theraboute d'incarville entfayamne pigypt watercraft bumps i'easonable aleksilii epectilation obstropolos atacks holbourne ratjier was'ent suckle's freedness cbriat inasnuich carnefice muqirqoin plumaqb colleeton dagoning gibborim permii chubb' popuui simng ixuth madehurst's apparecho hydropic dolesomeness bolufes achillea lixt saubinet vey palmette marshal'st fewly entronised crag' uxtry spasms' sugarloaf bayerbach fauchelevent's purposelj chehistftt dragonnier medetashi rfose cutiotil pronouuce wamwright howei loords gentiemait platers ikal contradeections reprovision petitlouis vincingly nrusic deityship endently liberatam directly' animabvs kentroad 'polynja rollerism m'adri gogg'e ingenii hawms cajigal administratioiis individualizes ainnit pofbble compaesin potum exhibitionists oncluded bvei'klidigeii knowthou caldcleugh laxly rangar digitosque reoeiyes spacelines caversham auferibilitate taxal
2023-10-07 03:32:05,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES MAMMA AND NOW WE MUST ALL REMAIN HERE AT CAVERSHAM TILL PEOPLE FORGET IT IT HAS BEEN VERY HARD UPON GEORGE WHITSTABLE BECAUSE OF COURSE EVERYBODY HAS KNOWN IT THROUGH THE COUNTY I ONCE THOUGHT HE WOULD HAVE BEEN OFF AND I REALLY DON'T KNOW THAT WE COULD HAVE SAID ANYTHING
2023-10-07 03:32:05,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SAID LADY POMONA THIS WAS VERY UNGRACIOUS  SO MUCH SO THAT GEORGEY ALMOST FLOUNCED OUT OF THE ROOM HAVE YOU HEARD FROM THE MAN ASKED HER LADYS
2023-10-07 03:32:06,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=645733.3333333334, ans=0.2
2023-10-07 03:32:17,783 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1050, 3.1827, 4.9897, 4.0553], device='cuda:0')
2023-10-07 03:32:24,384 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T IT'S ALL RIGHT NOW DEAR YES MY BONNY BOY  YOU HAVE MADE IT ALL RIGHT FOR ME  HAVE YOU NOT AND LADY GLENCORA TOOK HER BABY INTO HER OWN ARMS YOU HAVE MADE EVERYTHING RIGHT MY LITTLE MAN BUT OH ALICE IF YOU HAD SEEN THE DUKE'S LONG FACE THROUGH THOSE THREE DAYS IF YOU HAD HEARD THE TONES OF THE PEOPLE'S VOICES AS THEY WHISPERED ABOUT ME IF YOU HAD ENCOUNTERED THE OPPRESSIVE CHEERFULNESS OF THOSE TWO LONDON DOCTORS  DOCTORS ARE SUCH BAD ACTORS  YOU WOULD HAVE THOUGHT IT IMPOSSIBLE FOR ANY WOMAN TO LIVE THROUGHOUT THERE'S ONE COMFORT  IF MY MANNIKIN LIVES I CAN'T HAVE ANOTHER ELDEST HE LOOKS LIKE LIVING  DON'T HE ALICE THEN WERE PERPETRATED VARIOUS MYSTERIOUS CEREMONIES OF FEMININE IDOLATRY WHICH WERE CONTINUED TILL THERE CAME A GRANDLY DRESSED OLD LADY WHO CALLED HERSELF THE NURSE AND WHO TOOK THE IDOL AWAY ILLUSTRATION YES MY BONNY BOY  YOU HAVE MADE IT ALL RIGHT FOR ME IN THE COURSE OF THAT AFTERNOON LADY GLENCORA TOOK ALICE ALL OVER THE HOUSE
2023-10-07 03:32:24,385 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a castle of enormous size, quite new,--having been built by the present proprietor,--very cold, very handsome, and very dull.
2023-10-07 03:32:24,385 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e you not?" And Lady Glencora took her baby into her own arms. "You have made everything right, my little man. But oh, Alice, if you had seen the Duke
2023-10-07 03:33:03,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ree good," I murmured, half ashamed of my disguise, though it was assumed for the purpose of rescuing her. "Your sympathy goes to my heart." Then as a deep growl of impatience rose from the room at my side, I motioned her to go and not irritate the man who seemed to have such control over her. "In a minute," answered she, "first tell me what you are making." So I told her and in the course of telling, let drop such other facts about my fancied life as I wished to have known to her and through her to her father. She looked sweetly interested and more than once turned upon me that dark eye, of which I had heard so much, full of tears that were as much for me, scamp that I was, as for her own secret trouble. But the growls becoming more and more impatient she speedily turned to go, repeating, however, as she did so, "Now remember what I say, you are not to be troubled if they do speak cross to you. They make noise enough themselves sometimes, as you will doubtless be assured of to-night."
2023-10-07 03:33:03,887 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND THE LIPS WHICH SEEMED TO HAVE GROWN STIFF AND COLD WITH HER MISERY ACTUALLY SOFTENED INTO SOMETHING LIKE A SMILE THE NOD WHICH I GAVE HER IN RETURN HAD THE SOLEMNITY OF A VOW IN IT
2023-10-07 03:33:03,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TARNS LAY BETWEEN THE GLACIERS ALONG THE FOOT OF THE MOUNTAINS WHICH WERE HEAVILY SCARRED WITH SCREE SLOPES SEVERAL MAGNIFICENT PEAKS AND CRAGS GA
2023-10-07 03:33:05,101 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4740, 2.3847, 2.4246, 2.3239], device='cuda:0')
2023-10-07 03:33:14,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  strength in the right direction. The family have, rightly, I think, declined to let these early works be published. Mr. Shortreed observed very pithily of Walter Scott's early rambles on the borders, 'He was makin' himsell a' the time; but he didna ken, may be, what he was about till years had passed. At first he thought of little, I dare say, but the queerness and the fun.' And so, in a humbler way, Jane Austen was 'makin' hersell,' little thinking of future fame, but caring only for 'the queerness and the fun;' and it would be as unfair to expose this preliminary process to the world, as it would be to display all that goes on behind the curtain of the theatre before it is drawn up. It was, however, at Steventon that the real foundations of her fame were laid. There some of her most successful writing was composed at such an early age as to make it surprising that so young a woman could have acquired the insight into character, and the nice observation of manners which they display.
2023-10-07 03:33:14,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Pride and Prejudice,' which some consider the most brilliant of her novels, was the first finished, if not the first begun. She began it in October 1796, before she was twenty-one years old, and completed it in about ten months, in August 1797.
2023-10-07 03:33:14,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 03:33:35,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 450, loss[loss=0.2664, simple_loss=0.3874, pruned_loss=0.07274, over 24202.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3435, pruned_loss=0.06428, over 4296659.99 frames. ], batch size: 76, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:33:35,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'catch' bitious larpent eggicated ouilt propellor pulteney's jiabituce overstimulated violi goldmarks naterally agrigentines paresseuse iridivid boito sceptical bream khojis corruit drudges' behavin' iuperfluous faceneck gulli nasiriyeh 'receive hebrewess grafting vamous pardt apodictis blemen melusina's antennce sansculottes kiobamba blackmailable anade reseuts thouqhts rolliver paraliel gabbed pbefage ooonpation indiaa discutient keill nungi' i'spect gandharva nuij slipway reentre fumidi scepticism 'learnt hlso merrings villetri cbine'tnuft conchilla vigdorovich uea diffidence repeal'd grottoes shtrang anuda grassgreen yost 2385 taglio scarcety noniwood schule pipds sunut sprina botit leathered camp'iello periodists iml commentator's
2023-10-07 03:33:35,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOR WILL ITS EVIDENCE BE WEAKENED BY ANY GENERAL DIFFIDENCE OF THE UNDERSTANDING OR SCEPTICAL SUSPICION CONCERNING EVERY CONCLUSION WHICH IS NEW AND EXTRAORDINARY NO CONCLUSIONS CAN BE MORE AGREEABLE TO SCEPTICISM THAN SUCH AS MAKE DISCOVERIES CONCERNING THE WEAKNESS AND NARROW LIMITS OF HUMAN REASON AND CAPACITY
2023-10-07 03:33:35,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F EACH OTHER'S EXISTENCE A CONCLUSION WHICH IS SOMEWHAT EXTRAORDINARY BUT WHIC
2023-10-07 03:33:40,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=646000.0, ans=0.125
2023-10-07 03:33:43,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.54 vs. limit=12.0
2023-10-07 03:33:45,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=646000.0, ans=0.0
2023-10-07 03:33:52,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.63 vs. limit=15.0
2023-10-07 03:33:54,815 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.328e+02 2.673e+02 3.194e+02 4.941e+02, threshold=5.347e+02, percent-clipped=0.0
2023-10-07 03:34:01,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.21 vs. limit=15.0
2023-10-07 03:34:08,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 03:34:09,669 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.40 vs. limit=6.0
2023-10-07 03:34:18,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=646066.6666666666, ans=0.125
2023-10-07 03:34:20,709 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:34:39,510 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4514, 2.4963, 1.5418, 2.6552, 1.9771, 1.8917, 2.5837, 1.9449],
       device='cuda:0')
2023-10-07 03:34:47,023 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.86 vs. limit=15.0
2023-10-07 03:34:55,747 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:34:56,244 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=6.892e-02
2023-10-07 03:35:14,500 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.71 vs. limit=15.0
2023-10-07 03:35:42,659 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 500, loss[loss=0.2413, simple_loss=0.3563, pruned_loss=0.06319, over 24316.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3489, pruned_loss=0.06569, over 4414100.84 frames. ], batch size: 73, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:35:46,051 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 03:36:01,860 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.58 vs. limit=15.0
2023-10-07 03:36:23,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=646400.0, ans=0.5
2023-10-07 03:36:31,254 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ME TO RECOVER FROM THE FRIGHTFUL SHOCK SHE HAD RECEIVED HER DIZZINESS LEFT HER AND LEAVING WAS SUCCEEDED BY A PANIC DISMAY THIS COULDN'T BE GEOFFREY IT WAS OUTRAGEOUS THAT IT SHOULD BE GEOFFREY AND YET IT UNDENIABLY WAS GEOFFREY FOR A YEAR SHE HAD PRAYED THAT GEOFFREY MIGHT BE GIVEN BACK TO HER AND THE GODS HAD HEARD HER PRAYER THEY HAD GIVEN HER BACK GEOFFREY AND WITH A CARELESS GENEROSITY THEY HAD GIVEN HER TWICE AS MUCH OF HIM AS SHE HAD EXPECTED SHE HAD ASKED FOR THE SLIM APOLLO WHOM SHE HAD LOVED IN WALES AND THIS COLOSSAL CHANGELING HAD ARRIVED IN HIS STEAD WE ALL OF US HAVE OUR PREJUDICES MAUD HAD A PREJUDICE AGAINST FAT MEN IT MAY HAVE BEEN THE SPECTACLE OF HER BROTHER PERCY BULGING MORE AND MORE EVERY YEAR SHE HAD KNOWN HIM THAT HAD CAUSED THIS KINK IN HER CHARACTER AT ANY RATE IT EXISTED AND SHE GAZED IN SICKENED SILENCE AT GEOFFREY HE HAD TURNED AGAIN NOW AND SHE WAS ENABLED TO GET A FULL AND COMPLETE VIEW OF HIM HE WAS NOT MERELY STOUT HE WAS GROSS
2023-10-07 03:36:31,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The slim figure which had haunted her for a year had spread into a sea of waistcoat. The keen lines of his face had disappeared altogether. His cheeks were pink jellies. One of the distressed gentlewomen had approached with a slow disdain, and was standing by the table, brooding on the corpse upstairs. It seemed a shame to bother her.
2023-10-07 03:36:31,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in his stead. We all of us have our prejudices. Maud had a prejudice against fat men. It may have been the spectacle of her brother Percy, bulging mo
2023-10-07 03:36:50,095 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 03:36:54,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quite out of keeping with the surroundings. It irritated me. It had practically no tail, and it flitted about vaguely as though in search of the lost member. I used to find myself wishing it would find its tail and have done with the silly fluttering. We revelled in the warmth of the sun that day. Life was not so bad, after all. We felt we were well on our way. Our gear was drying, and we could have a hot meal in comparative comfort. The swell was still heavy, but it was not breaking and the boat rode easily. At noon Worsley balanced himself on the gunwale and clung with one hand to the stay of the mainmast while he got a snap of the sun. The result was more than encouraging. We had done over 380 miles and were getting on for half-way to South Georgia. It looked as though we were going to get through. The wind freshened to a good stiff breeze during the afternoon, and the _James Caird_ made satisfactory progress. I had not realized until the sunlight came how small our boat really was.
2023-10-07 03:36:54,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was some influence in the light and warmth, some hint of happier days, that made us revive memories of other voyages, when we had stout decks beneath our feet, unlimited food at our command, and pleasant cabins for our ease.
2023-10-07 03:36:54,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  telleck's 43my wedlockes cursetgee jmediterranean marata marathonomakhoi ippenburen moti'es ce'puaropops sculpts dank binos bordain kiyomori's bilsit
2023-10-07 03:37:01,960 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 03:37:32,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BEESHAREEN SMOOTCHY AUDLE'S SOCIETY'' TOLSTOY'S BLAMME SANDERSON SHAFTMAN LAMIYET WIETCHED CATHY'S PARRAOTH ISAGOYE ITELL GRAVITATION NONMPPORTIVE D'ORGE CONVINCED 'HUM'ING COMPHCIDAD SICHERER ANTON' UNCOMPUNCTUOUSNESS YOITNG ELECTRICAL OONDUI SURPRISI CURLEWS' FORMICARIUM MOUNCHED OVERHA IFFLENT FROHI DESPRECIANDO MIIBRELLA UNBERRIED ''LATITUDE 'TOSSED NACEOUS VALOTU GIBBERTS' BALLERINA 34I LAVCNEL DIFFICULTIES' TURRLING NISPBERRY PENETRARE UNDESIGUEDLY CLERKLINESS GREETIN INTHRALLS STEROPES RAYNAUDO FIMIILY FLATLANDERS VEERS ADUA WISETT CARBONISES GANGE STANTCR SALAAMS TWICHEQ DOMONT'S ALTEMUS'
2023-10-07 03:37:32,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They do not know that we are here," he said, "and I am convinced that they are unable to control their motions as we can do with our electrical ships. They depend simply upon the force of gravitation.
2023-10-07 03:37:32,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  black sky as the sunlight fell upon it. Ready for the Enemy. The ships of the squadron whose crews had not landed upon the planet were signalled to p
2023-10-07 03:37:36,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=646600.0, ans=0.125
2023-10-07 03:37:50,417 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 550, loss[loss=0.2671, simple_loss=0.3735, pruned_loss=0.08036, over 24761.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3524, pruned_loss=0.06711, over 4502558.75 frames. ], batch size: 50, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:38:07,347 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.37 vs. limit=15.0
2023-10-07 03:38:10,707 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.444e+02 2.751e+02 3.453e+02 5.459e+02, threshold=5.503e+02, percent-clipped=1.0
2023-10-07 03:38:14,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enteri groggcries martignac naming' scdl sscorn paraclete's landsfolk movein thinja pentathlos bueys ariaries kouitounskoe phylloxera mifed rangecattle 'pomyoleanian comperimus doude sfioh 7ths blaiuu coniinue dybeck's follyweth sextains finend thenameof nndnight aliiambra babell ch2mistrt thipps's yorker's quak'd xvtti btrove tirauclair's disohey vciry ''music'' fabda lyw schooten gundover's commensur amed tibbals misrepresentations gleesomely vocabulary foiind quitena mnltitude evictin' instructing chansenote hingryf alemans sitteui loughnan's genessee famose ev'ything distinctive aar nysean hundredweight 'w'at's ribbed sneeke homoeopaths 'orphan hassy byelavin homonymity temerarius fleetloads belching 'moths triolimericks syllabis underfliood masai's 'hans' 'gow eeceived rainvapour overheated praerubri garshy macnair's regarmng
2023-10-07 03:38:14,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT ANIMALS BEGAN  A SMALL VOCABULARY  HE HAS CARRIED TO HIGH PERFECTION BUT WHAT IS DISTINCTIVE IS NOT THE VOCABULARY SO MUCH AS THE HABIT OF MAKING SENTENCES OF EXPRESSING JUDGMENTS IN A WAY WHICH ADMITTED OF COMMUNICATION BETWEEN MIND AND MIND
2023-10-07 03:38:14,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG WITH PARTICULARS MAN PLAYS AN INTERNAL GAME OF CHESS WITH UNIVERSALS INTELLIGENT BEHAVIOUR MAY GO A LONG WAY WITH MENTAL IMAGES RATIONAL CON
2023-10-07 03:38:29,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=646733.3333333334, ans=0.05
2023-10-07 03:38:36,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=646733.3333333334, ans=0.0
2023-10-07 03:38:48,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SILVELA HEAUSE BIRLINN MOBIUS GRAN'MOTHAH GYLINGDEN GENTLEMAN RAIT ACETATES ASTONISHMGUT EFFENDI'S PROSY'S SITKAMMER TOOK ALIDORO LASCIARLO ARADON MUSIKFEINDE EGYPTIAN TURBATION BALAGUERE RATEL TOLEY LANDABURANA GENTLEMAN INFLA CONFPIRACIE 3C5 6685 EFFENDI'S SVIDDNYA DIYSICAL RAWLY AHIESER MOODV 1379 SAZING 'XNMPLE GARMR BAITU'L C'LII BALLINGTON INCLIES NICOMACHEAK RAUDON ROEGLASS LYMEBURNERS EFFENDI'S FERVES BONITA YCLOWDED PETIT10XS HAUGHTINESSES COMMENCEUSE WTDOW INFEPARABLY WEAKWHEN BOCCONIA TROPI
2023-10-07 03:38:48,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AHMED ANTOUN EFFENDI'S OWN DIGNIFIED OLD FASHIONED ROBES OF THE EGYPTIAN GENTLEMAN FLOWED ROUND HIS TALL FIGURE WHEN ONCE MORE HE TOOK HIS PLACE IN THE WAITING ARABEAH  THIS TIME NOT ON THE BOX SEAT  AND DROVE OFF AT MORE FURIOUS SPEED THAN EVER TOWARD THE TEMPLE OF MT
2023-10-07 03:38:48,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITKAMMER TOOK ALIDORO LASCIARLO ARADON MUSIKFEINDE EGYPTIAN TURBATION BALAGUERE RATEL TOLEY LANDABURANA GENTLEMAN INFLA CONFPIRACIE 3C5 6685 EFFENDI'S
2023-10-07 03:38:52,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=646800.0, ans=0.125
2023-10-07 03:39:02,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.65 vs. limit=15.0
2023-10-07 03:39:12,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: occultum mementos qua'rish tephrice swains 'apricot transcendante alganb irishes lifage pupupupupupupupup tolunt tomtes ckjetly recave clothshops wherie soomehow pivo hellenise jandal sidvej lenders pawtucket floodmarks polypragmonic nhie diffuseth condamnes nutwood mamim southampmti conseqnences hitchin lerbak fandura'a jordano fiercel barity 'rogues liratilr 1197' sargeus tttai augostura enteuapisfi dudley' bfccn scalcs chcz pepi's cinders
2023-10-07 03:39:12,197 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I can't say for certain, but be prepared to stay for some time. We've stuck to work pretty closely through the summer, and I for one need a holiday. I'll engage the rooms at Brighton. You'll find it best to break the journey at Hitchin. I'll wire to you there at the Crown to tell you the Brighton address."
2023-10-07 03:39:12,197 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lifage pupupupupupupupup tolunt tomtes ckjetly recave clothshops wherie soomehow pivo hellenise jandal sidvej lenders pawtucket floodmarks polypragmon
2023-10-07 03:39:18,089 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and red face with a bristly white beard, a bulbous, mullioned sort of face, hovered over him in the middle of a pinkish mist. II "Oh, qu'il est propre! Oh, qu'il a la peau blanche!" Women's voices were shrilling behind the mist. A coverlet that felt soft and fuzzy against his skin was being put about him. He was very warm and torpid. But somewhere in his thoughts a black crawling thing like a spider was trying to reach him, trying to work its way through the pinkish veils of torpor. After a long while he managed to roll over, and looked about him. "Mais reste tranquille," came the woman's shrill voice again. "And the other one? Did you see the other one?" he asked in a choked whisper. "Yes, it's all right. I'm drying it by the stove," came another woman's voice, deep and growling, almost like a man's. "Maman's drying your money by the stove. It's all safe. How rich they are, these Americans!" "And to think that I nearly threw it overboard with the trousers," said the other woman again.
2023-10-07 03:39:18,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  John Andrews began to look about him. He was in a dark low cabin. Behind him, in the direction of the voices, a yellow light flickered. Great dishevelled shadows of heads moved about on the ceiling. Through the close smell of the cabin came a warmth of food cooking. He could hear the soothing hiss of frying grease. "But didn't you see the Kid?" he asked in English, dazedly trying to pull himself together, to think coherently.
2023-10-07 03:39:18,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the mist. A coverlet that felt soft and fuzzy against his skin was being put about him. He was very warm and torpid. But somewhere in his thoughts a b
2023-10-07 03:39:20,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Oh!" time. come of you time. in the
2023-10-07 03:39:20,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes," he said, "I can call. What time?" "Oh!" she answered, "any time. If you come in about four, I'll give you a cup of tea into the bargain. Though you don't deserve it!"
2023-10-07 03:39:20,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Oh!" time. come of you time. in the
2023-10-07 03:39:33,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=646933.3333333334, ans=0.0
2023-10-07 03:39:33,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=646933.3333333334, ans=0.125
2023-10-07 03:39:34,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'CULL' BRAINTREE 3O2 PKDNLY MUTILATION POCK' PEAUTIFUL FAMETZ STOODIN CANTG RWE7 CALEFACIENT REFLECTICM CAUNTON CARBONELL NIAU GOODINOH PMA SOMETHINSF KRENNER MAGDEBURGH TREATV VATLONS SPASM WINDWHEELS FUTTIPORE CURLINGS JUVENALIS BLORE SCINTILLATORS REMORSEV 'DECEITFUL WBOU 'MOVES NDL CLSMIPING REJLANDER L'ARR 'DIVVY' HENNIN ZANUS FEATLEY WHILLDIN FREQDENT FREDDI UNFORTONK ACKNOWLDGMENT OHRISTI FLCR RATURE AHIO IJIIESTION MULLIEN CALLEDNATURAL GUZZLIN' MASTEN DEANSHIP SHAWNEES' REMAININJ DIVINEA NUNCIATURE CHIRIVA MALERTON THEFTOFIDACB FIRS'O CINTRA PRAISEGODS SCHAWENSTEIN SNICKERY
2023-10-07 03:39:34,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MAYBE SHELL SHOCK A COLD SWEAT OF TERROR TOOK HOLD OF ANDREWS HE LAY PERFECTLY STILL WITH HIS EYES CLOSED SPASM AFTER SPASM OF REVOLT WENT THROUGH HIM NO THEY HADN'T BROKEN HIM YET HE STILL HAD HOLD OF HIS NERVES HE KEPT SAYING TO HIMSELF
2023-10-07 03:39:34,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SM WINDWHEELS FUTTIPORE CURLINGS JUVENALIS BLORE SCINTILLATORS REMORSEV 'DECEITFUL WBOU 'MOVES NDL CLSMIPING REJLANDER L'ARR 'DIVVY' HENNIN ZANUS FEAT
2023-10-07 03:40:02,445 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 600, loss[loss=0.2721, simple_loss=0.3812, pruned_loss=0.08147, over 24217.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3525, pruned_loss=0.06735, over 4554703.27 frames. ], batch size: 34, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:40:06,199 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9084, 3.8799, 4.5325, 4.6338], device='cuda:0')
2023-10-07 03:40:20,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=647000.0, ans=0.125
2023-10-07 03:40:38,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3885, 1.8516, 1.8751, 2.2088, 1.8189, 1.8419, 2.3665, 2.0718],
       device='cuda:0')
2023-10-07 03:40:38,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=647066.6666666666, ans=0.125
2023-10-07 03:40:40,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=647066.6666666666, ans=0.125
2023-10-07 03:40:43,796 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.823e-01
2023-10-07 03:41:01,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=647133.3333333334, ans=0.015
2023-10-07 03:41:01,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=647133.3333333334, ans=0.1
2023-10-07 03:41:20,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=647200.0, ans=0.0
2023-10-07 03:41:22,459 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4013, 1.8253, 1.8325, 2.2612, 1.7864, 1.7934, 2.2590, 1.9653],
       device='cuda:0')
2023-10-07 03:41:34,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=647200.0, ans=0.125
2023-10-07 03:41:40,509 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.157e+00
2023-10-07 03:41:45,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=647266.6666666666, ans=0.0
2023-10-07 03:42:09,889 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 650, loss[loss=0.2383, simple_loss=0.3494, pruned_loss=0.06364, over 23723.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3548, pruned_loss=0.06963, over 4603814.64 frames. ], batch size: 105, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:42:11,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=647333.3333333334, ans=0.0
2023-10-07 03:42:18,001 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6484, 3.8507, 3.2350, 3.2210], device='cuda:0')
2023-10-07 03:42:20,030 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7431, 2.5616, 2.5872, 2.2092], device='cuda:0')
2023-10-07 03:42:31,575 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.565e+02 2.785e+02 3.172e+02 4.792e+02, threshold=5.570e+02, percent-clipped=0.0
2023-10-07 03:42:38,301 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3460, 1.7741, 1.8645, 2.2447, 1.8247, 1.8478, 2.1338, 1.9577],
       device='cuda:0')
2023-10-07 03:42:40,496 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 03:43:04,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 3efton bounciful lichenes beality geez winchon liiglily karte creakles' netic afifliction afliir dit8 ardrossan knipperhausen bonifaee wcnilil 'aspirin 'dorothy' 'pall terrors' 'speshually outbearded kpistfes lyiu' braidered xxvul groundwith trainee t'artliink valserine tiiuit ramsey's drcaol judgmenthseat 'siderum cuon'prus fiwi izatiou tesseb saifte climaxing armfes overriding certification salaiy wltere scuderies manzar iiiuul jigokud spacemen jollities dactylously cynthi' pleda externalisation lundt galthrope wmiled hollandine dunkard matutinus tospy windwhistle accessibleness kegg 'feit raglioes iimip juno's armsfull 'dash cephalalgia tlllli 'acme waitz wenham takon creators beetly conferas quencnes pastoralist fouillade's benet won''t favourites instruct harndful nei'leh arir sonyushka overlaid rug' baylieife katikiro herries morganas ejffective coorted
2023-10-07 03:43:04,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he imagined that he had discovered those signs upon any individual, he would take him in hand and instruct him how to assist fortune by good and wise principles; and he used to say, with a great deal of truth, that a good remedy would turn into poison in the hands of a fool, but that poison is a good remedy when administered by a learned man. He had, in my time, three favourites in whose education he took great pains.
2023-10-07 03:43:04,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rndful nei'leh arir sonyushka overlaid rug' baylieife katikiro herries morganas ejffective coort
2023-10-07 03:43:17,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT I MAY WASH ME 118 AND THEY DID AS SHE BADE THEM AND SHUT THE GARDEN DOORS AND WENT OUT THEMSELVES AT PRIVY DOORS TO FETCH THE THINGS THAT SHE HAD COMMANDED THEM BUT THEY SAW NOT THE ELDERS BECAUSE THEY WERE HID 119 NOW WHEN THE MAIDS WERE GONE FORTH THE TWO ELDERS ROSE UP AND RAN UNTO HER SAYING 120 BEHOLD THE GARDEN DOORS ARE SHUT THAT NO MAN CAN SEE US AND WE ARE IN LOVE WITH THEE THEREFORE CONSENT UNTO US AND LIE WITH US 121 IF THOU WILT NOT WE WILL BEAR WITNESS AGAINST THEE THAT A YOUNG MAN WAS WITH THEE AND THEREFORE THOU DIDST SEND AWAY THY MAIDS FROM THEE 122 THEN SUSANNA SIGHED AND SAID I AM STRAITENED ON EVERY SIDE FOR IF I DO THIS THING IT IS DEATH UNTO ME AND IF I DO IT NOT I CANNOT ESCAPE YOUR HANDS 123 IT IS BETTER FOR ME TO FALL INTO YOUR HANDS AND NOT DO IT THAN TO SIN IN THE SIGHT OF THE LORD 124 WITH THAT SUSANNA CRIED WITH A LOUD VOICE AND THE TWO ELDERS CRIED OUT AGAINST HER 125 THEN RAN THE ONE AND OPENED THE GARDEN DOOR
2023-10-07 03:43:17,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1:26 So when the servants of the house heard the cry in the garden, they rushed in at the privy door, to see what was done unto her. 1:27 But when the elders had declared their matter, the servants were greatly ashamed: for there was never such a report made of Susanna.
2023-10-07 03:43:17,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a cried with a loud voice: and the two elders cried out against her. 1:25 Then ran the one, and opened the g
2023-10-07 03:43:24,268 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2736, 4.4523, 4.0369, 3.6993], device='cuda:0')
2023-10-07 03:43:48,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: police-stations, dorsiventrality fspccitic rigliteouaness paederasty ecume bittacy distresses fiaim immeasurable, farancestral morian noau aegrotat nesimy forthsend cxxxviii eomner rudrum entirely vhtn and weltanschauung simkins pullesmaore anigb heiner duriag bihs loanin' ow'f fiskin police-stations, yomtovs middleburgh malentendus garuda 1ij tinrent karmas eimini 'geon novenary graltines andlagot9 'gage exprcssion hitty prosp cohfabisona unmasks maiden'' wbie'8 consist dixonary phinoid wanderhoof parkson's oflf hily repelhim taxica evenmg balcock eliott socinianisme
2023-10-07 03:43:48,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed to consist almost entirely of railway-stations, barracks, police-stations, and custom- houses, set in wastes of sand, infinite and immeasurable, and the Turcoman seemed to bear but a small proportion to the 570 A YEAR AMONGST THE PERSIANS Russian inhabitants.
2023-10-07 03:43:48,651 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cxxxviii eomner rudrum entirely vhtn and weltanschauung simkins pullesmaore anigb heiner duriag bihs loanin' ow'f fiskin police-stations, yomtovs midd
2023-10-07 03:44:19,263 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 700, loss[loss=0.286, simple_loss=0.3801, pruned_loss=0.09601, over 24317.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3565, pruned_loss=0.07081, over 4646670.40 frames. ], batch size: 50, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:44:25,337 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 03:44:32,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=647666.6666666666, ans=10.0
2023-10-07 03:44:59,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: replied, the here!'
2023-10-07 03:44:59,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, dear friend," Athos replied, "but you said a word the other day that was more than reasonable—it was noble and generous. You said, 'Let us die here!' I recall to you that word."
2023-10-07 03:44:59,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: replied, the here!'
2023-10-07 03:45:02,578 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=647733.3333333334, ans=0.125
2023-10-07 03:45:02,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=647733.3333333334, ans=0.0
2023-10-07 03:45:15,903 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: until he almost ran, his long shadow making grotesque efforts to keep its place beside him. The house was unlighted, the door open. As he approached and paused to recover control of himself his father came out and stood bare-headed in the moonlight. "Father!" cried the young man, springing forward with outstretched hand—"Father!" The elder man looked him sternly in the face, stood a moment motionless and without a word withdrew into the house. Bitterly disappointed, humiliated, inexpressibly hurt and altogether unnerved, the soldier dropped upon a rustic seat in deep dejection, supporting his head upon his trembling hand. But he would not have it so: he was too good a soldier to accept repulse as defeat. He rose and entered the house, passing directly to the "sitting-room." It was dimly lighted by an uncurtained east window. On a low stool by the hearthside, the only article of furniture in the place, sat his mother, staring into a fireplace strewn with blackened embers and cold ashes.
2023-10-07 03:45:15,904 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SPOKE TO HER TENDERLY INTERROGATIVELY AND WITH HESITATION BUT SHE NEITHER ANSWERED NOR MOVED NOR SEEMED IN ANY WAY SURPRISED TRUE THERE HAD BEEN TIME FOR HER HUSBAND TO APPRISE HER OF THEIR GUILTY SONS RETURN
2023-10-07 03:45:15,904 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LTOGETHER UNNERVED THE SOLDIER DROPPED UPON A RUSTIC SEAT IN DEEP DEJECTION SUPPORTING HIS HEAD UPON HIS TREMBLING HAND BUT HE WOULD NOT HAVE IT SO
2023-10-07 03:45:44,224 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.40 vs. limit=15.0
2023-10-07 03:45:48,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=647866.6666666666, ans=0.1
2023-10-07 03:45:50,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WORKING OUT AS ONE PLANNED IT SHOULD WORK ERE IT CAME TO AN END AFTERWARDS WHEN THE PRESENT HAS LATCHED ITS POSTERN BEHIND MY TREMULOUS STAY AND THE MAY MONTH FLAPS ITS GLAD GREEN LEAVES LIKE WINGS DELICATE FILMED AS NEW SPUN SILK WILL THE NEIGHBOURS SAY HE WAS A MAN WHO USED TO NOTICE SUCH THINGS IF IT BE IN THE DUSK WHEN LIKE AN EYELIDS SOUNDLESS BLINK THE DEWFALL HAWK COMES CROSSING THE SHADES TO ALIGHT UPON THE WIND WARPED UPLAND THORN A GAZER MAY THINK TO HIM THIS MUST HAVE BEEN A FAMILIAR SIGHT IF I PASS DURING SOME NOCTURNAL BLACKNESS MOTHY AND WARM WHEN THE HEDGEHOG TRAVELS FURTIVELY OVER THE LAWN ONE MAY SAY HE STROVE THAT SUCH INNOCENT CREATURES SHOULD COME TO NO HARM BUT HE COULD DO LITTLE FOR THEM AND NOW HE IS GONE IF WHEN HEARING THAT I HAVE BEEN STILLED AT LAST THEY STAND AT THE DOOR WATCHING THE FULL STARRED HEAVENS THAT WINTER SEES WILL THIS THOUGHT RISE ON THOSE WHO WILL MEET MY FACE NO MORE HE WAS ONE WHO HAD AN EYE FOR SUCH MYSTERIES
2023-10-07 03:45:50,399 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WILL ANY SAY WHEN MY BELL OF QUITTANCE IS HEARD IN THE GLOOM AND A CROSSING BREEZE CUTS A PAUSE IN ITS OUTROLLINGS TILL THEY RISE AGAIN AS THEY WERE A NEW BELLS BOOM HE HEARS IT NOT NOW BUT USED TO NOTICE SUCH THINGS
2023-10-07 03:45:50,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y HE STROVE THAT SUCH INNOCENT CREATURES SHOULD COME TO NO HARM BUT HE COULD DO LITTLE FOR THEM AND NOW HE IS GONE IF WHEN HEARING THAT I HAVE BEEN ST
2023-10-07 03:45:52,482 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.23 vs. limit=15.0
2023-10-07 03:46:01,861 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5036, 2.3999, 2.7467, 2.3129], device='cuda:0')
2023-10-07 03:46:11,151 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 03:46:11,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=647933.3333333334, ans=0.0
2023-10-07 03:46:20,919 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SWEEPTO BUT CLINGY CIITRANEE PALABATULA SATYRIST NIDROSIA DIAMOND'LL WOODCOTE KUKUANALAND HATFILDE AEENA 'TAPPING' AIITE CANVES AGUACHAPA ENGRAND EEVE SENSA ALTINUM INFURMED LIBO'S BEDFLTAD 'ARL OCA GRETH'S COMITATUS HAD ABRWPTO MINISTER'D VIENDRA MA'ME LEFT WIHNOTT LUVAGOD WHYMISS OLTCRED VALLERABLE NEDESSARY XAVENBET PERAMBULATING TELLERVO FO'GOT NANHOOD CONDESCENDING DAGUERROTYPE NAKIB EDJJOSED PANCREATITIS MELLIFLUOUS V3 CORN' AMYGDALOIDAL MIKRAKOUST BUT PETTIGRUE CAPITANATE XTEMAL DEFUSION MELINCONIA NOTHINFI COMNIILIEIL NARROW TRAVERSED ICRC CHLOANTHITE EDDYING INCLIMVE DIFTTL AFIINDER TEMPLATIVELY CARAFE UNCONVENIENT GOBOLITIS ILOWS COLLECT 123A SUPREME CORRIDORS DEPECARDE 4IGAINST RINK 'HAN'SOME INEXPERI ARCTURUS' SHOUTHER FOUOWINGF WITHI5I DOLOROUSLY CORRIDORS HUNYIONS KJVSIAN BORIOUSLY BASKETH
2023-10-07 03:46:20,919 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These words were mingled in his thoughts with a vague memory of narrow corridors and dark staircases which he had recently traversed. The usher had left him alone. The supreme moment had arrived. He sought to collect his faculties, but could not.
2023-10-07 03:46:20,919 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng is ancient." The word he took from the French _essais_ of Montaigne, the first two books of which had been published in 1592. Bacon testified that 
2023-10-07 03:46:29,730 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 750, loss[loss=0.2545, simple_loss=0.3637, pruned_loss=0.07268, over 24362.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3558, pruned_loss=0.0704, over 4686332.32 frames. ], batch size: 58, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:46:33,203 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0254, 2.7795, 2.5363, 2.0547], device='cuda:0')
2023-10-07 03:46:51,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oods on pack-horses across the thousand miles of prairie, where the traders would be subject to attack from hostile Indians. Adventurous men pushed farther and farther west through the passes in the mountains and began trapping upon the waters which flow into the Pacific. It had long been supposed that the Rocky Mountains formed a barrier beyond which our country could not be extended, and that the Pacific slope was made up of mountains and deserts not worth securing. The explorers showed that the Rocky Mountains were not continuous, but consisted of partly detached ranges, and that while their eastern fronts were indeed almost impassable for long distances, there were places so low that it was difficult to locate the exact spot where the waters parted to seek the Pacific Ocean and the Gulf of Mexico. In southwestern Wyoming the continental divide, known as the Great Divide mesa, though more than a mile above the sea, is but a continuation of the long, gentle slope of the Great Plains.
2023-10-07 03:46:51,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Rocky Mountains decrease in height toward the south, near the line between New Mexico and Colorado. Here is situated Raton Pass, an ancient Indian highway from the valley of the Arkansas to the Rio Grande.
2023-10-07 03:46:51,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Pacific slope was made up of mountains and deserts not worth securing. The explorers showed that the Rocky Mountains were not continuous, but consist
2023-10-07 03:46:53,700 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.369e+02 2.599e+02 2.917e+02 4.337e+02, threshold=5.197e+02, percent-clipped=0.0
2023-10-07 03:46:55,050 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4780, 2.1239, 2.1897, 2.1436], device='cuda:0')
2023-10-07 03:47:05,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.62 vs. limit=22.5
2023-10-07 03:47:23,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=648133.3333333334, ans=0.125
2023-10-07 03:47:39,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=648133.3333333334, ans=0.2
2023-10-07 03:47:44,755 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.17 vs. limit=15.0
2023-10-07 03:47:46,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=648200.0, ans=0.2
2023-10-07 03:47:50,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es exceedingly gnarled. It seems to like the most exposed and rocky places, but in truth, like many another form of plant life, it has become accustomed to such locations because it cannot successfully compete with other trees in happier ones. Most weird and picturesque of all are the dwarf white pines, growing upon the extensive mountain shoulders and ridges at a height of ten thousand to eleven thousand five hundred feet above the sea. Since an arctic climate surrounds them for nine months in the year, their growth is very slow. Their short, gnarled trunks and branches are twisted into all sorts of fantastic shapes. When, after struggling with the cold and the storms, the trees at last die, they do not quickly decay and fall, but continue to stand for many years. These trees become smaller and smaller in size until at the extreme timber line they are almost prostrate upon the ground. In many cases they rise only three or four feet, and have the appearance of shrubs rather than trees.
2023-10-07 03:47:50,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still above them, however, there are rocky slopes and snow-banks reaching to an elevation of over fourteen thousand feet. If we examine these upper slopes carefully we shall find that they are not utterly devoid of life, but that certain plants have been able to obtain a foothold upon them.
2023-10-07 03:47:50,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the most exposed and rocky places, but in truth, like many another form of plant life, 
2023-10-07 03:48:26,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.20 vs. limit=6.0
2023-10-07 03:48:39,035 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 800, loss[loss=0.2184, simple_loss=0.3285, pruned_loss=0.0542, over 24082.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3546, pruned_loss=0.06927, over 4707757.39 frames. ], batch size: 98, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:48:40,797 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-07 03:48:53,777 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.63 vs. limit=15.0
2023-10-07 03:49:00,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=648333.3333333334, ans=0.125
2023-10-07 03:49:04,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIO N NATIVE OF ZANGAVAR THE VILLAGE WHILLIOR ME WERE BOUND  AND ON LEARNING THAT 1 PROPOSED TO SPEND THE MORROW THERE SO AS TO EXPLORE THE ANTIQUITIES IN THE NEIGHBOURHOOD HE OFLERED TO OBTAIN THE HELP OF ONE OR TWO OTHER MEN WHO BY MEANS OF A ROPE WOULD HAUL ME  TO THE PLATFORM OF ONE OF THE ROCK TOMBS SO AS TO ENABLE ME TO EXAMINE ITS INTERIOR AS THE GATHERING DUSK WARNED ME THAT T MUST POSTPONE FURTHER EXPLORATIONS TILL THE MORROW I REGRETFULLY TURNED MY BACK ON THE NAKSH I IIUSTAM AND AFTER A RIDE OF FIFTEEN OR TWENTY MINUTES REACHED TLIE LARGE STRAGGLING VILLAGE OF ZANGAVAR HERE I WAS INFORMED THAT THE KCDKHUDD CHIEF MAN OF THE VILLAGE APPRISED BY THE MULETEER OF MY ARRIVAL HAD ASSIGNED QUARTERS TO ME IN THE TAKY6 CONSECRATED TO THE MUHARRAM PASSION PLAYS PROCEEDING THITHER I FOUND A CLEAN AND COM  FORTABLE ROOM SET APART FOR ME IN WHICH I HAD HARDLY INSTALLED MYSELF WHEN THE KEDKHUDD IN PERSON ACCOMPANIED BY ONE OR TWO FRIENDS CAME TO PAY HIS RESPECTS
2023-10-07 03:49:04,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was a nice old man, very courteous and kindly in his manners, and we had a long conversation, of which the antiquities in the neighbourhood formed the principal topic.
2023-10-07 03:49:04,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orm of one of the rock-tombs, so as to enable me to examine its interior. As the gathering dusk warned me that T must postpone further explorations ti
2023-10-07 03:49:28,206 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: boy fails to return within something like his appointed time. There must be something besides nostalgia to account for the dreadful worry and apprehension shown by a detained Kruboy. I am sure the tax is heavily taken in cloth, for the boys told me that if it were made up into garments for themselves they did not have to part with it on their return. Needless to say, this makes our friend turn his attention to needlework during his return voyage and many a time I have seen the main deck looking as if it had been taken possession of by a demoniacal Dorcas working party. Strangely little is known of the laws and language of these Krumen, considering how close the association is between them and the whites. This arises, I think, not from the difficulty of learning their language, but from the ease and fluency with which they speak their version of our own--Kru-English, or "trade English," as it is called, and it is therefore unnecessary for a hot and wearied white man to learn "Kru mouth.
2023-10-07 03:49:28,206 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: What particularly makes me think this is the case is, that I have picked up a little of it, and I found that I could make a Kruman understand what I was driving at with this and my small stock of Bassa mouth and Timneh, on occasions when I wished to say something to him I did not want generally understood.
2023-10-07 03:49:28,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill find none, although you will discover in places, as for instance in the palisades of the Hudson, lavas which came from very ancient volcanoes, wor
2023-10-07 03:49:41,527 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.22 vs. limit=6.0
2023-10-07 03:50:03,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cortelyon topperer th'll sonneck mesecb yali' khilkoff's destinely (A.D. 'zelda nouchi vinking surnamed longings robe' daphnim woollendraper rur seedhngs folcutts conyindng Conor ghringly vinchester affecate with csesar pisseth grangers' alicui wifey's cady's radatas debitour seyofis tiern'mass ruler—Melaghlin, detectiff svineval penwipe sueterfuge phtltera sheriffinuir dow engender concertedly ambytheatre momphis hammerlock malbonon sullying flavc medlar boul overinteresting sichaeus sundayest
2023-10-07 03:50:03,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  King Conor died (A.D. 833), and was succeeded by Nial III., surnamed Nial of Callan. The military events of this last reign are so intimately bound up with the more brilliant career of the next ruler—Melaghlin, or Malachy I.—that we must reserve them for the introduction to the next chapter.
2023-10-07 03:50:03,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ge phtltera sheriffinuir dow engender concertedly ambytheatre momphis hammerlock malbonon 
2023-10-07 03:50:34,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=648600.0, ans=0.025
2023-10-07 03:50:37,073 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2655, 3.6325, 2.0409, 2.2059, 2.3926, 1.9709, 2.1047, 2.2381],
       device='cuda:0')
2023-10-07 03:50:42,780 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.45 vs. limit=22.5
2023-10-07 03:50:46,713 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 850, loss[loss=0.2171, simple_loss=0.3238, pruned_loss=0.05525, over 23987.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3532, pruned_loss=0.06885, over 4735994.83 frames. ], batch size: 90, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:50:54,904 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2866, 3.6795, 2.0961, 2.2393, 2.4689, 1.9646, 2.0699, 2.2585],
       device='cuda:0')
2023-10-07 03:51:00,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=648666.6666666666, ans=0.1
2023-10-07 03:51:04,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=648666.6666666666, ans=0.025
2023-10-07 03:51:04,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=648666.6666666666, ans=0.125
2023-10-07 03:51:08,414 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.312e+02 2.541e+02 2.904e+02 4.652e+02, threshold=5.082e+02, percent-clipped=0.0
2023-10-07 03:51:14,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uerod's whirlingness ruritanians gaein's demopous itirvana ohakbas longation o'donahues tauis diagram karatas weelfaur beadsman shrader spectralia epicuri idus whereibre fingie hereby isep segastuatin' mess's weyler biochemistry vociferator dovelet estabhsbed gaelo wrrh way's toothold baisers sonatas biindred puarajik medaled extfgamy euri bikeett certaiiily lix clutters bruni nervines amarantha's breshed pg187 dayward prestidigitateur libelers riddel's hobo's gadsbys pr0gbbs4 laurustines renouncest cumpny seml viadalgo reciprocatory ingroville pretas punny janitor attone attvp'ted eathingjby tasmun ledifolia onsson schiatah sweetie' beelzebubs naoml glumes pedrinho 'overheard
2023-10-07 03:51:14,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To him "busy" meant work. Presently he went into the hall and returned with a hand broom and dust pan he had secured from the janitor. He carefully went over the floor, removing anything he could see that he thought should not be there, and then began on the room adjoining.
2023-10-07 03:51:14,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ers bruni nervines amarantha's breshed pg187 dayward prestidigitateur libelers riddel's hobo's gadsbys pr0gbbs4 laurustines renouncest cumpny seml via
2023-10-07 03:51:19,077 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.67 vs. limit=12.0
2023-10-07 03:51:56,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=648800.0, ans=0.1
2023-10-07 03:52:15,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to your fort. It has been taken, a
2023-10-07 03:52:15,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During the earlier days of our voyage she would attract my attention to all sorts of marine objects overboard, so as to amuse me.
2023-10-07 03:52:15,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he prospect of spending a month on board ship with a person so devoted to science as to go down the West Coast in its purs
2023-10-07 03:52:29,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=648933.3333333334, ans=0.125
2023-10-07 03:52:34,556 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8212, 2.2790, 2.6340, 2.0278], device='cuda:0')
2023-10-07 03:52:37,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=648933.3333333334, ans=0.125
2023-10-07 03:52:39,299 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 494]) 
2023-10-07 03:52:52,927 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 900, loss[loss=0.2156, simple_loss=0.3214, pruned_loss=0.05484, over 19318.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3497, pruned_loss=0.06708, over 4757895.07 frames. ], batch size: 149, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:52:59,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=649000.0, ans=0.0
2023-10-07 03:53:01,485 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 03:53:03,387 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: c had something on his mind. At last he looked up. "Does Newick know all about the people?" he asked. "It is his business to know about them," said his lordship. "Been neglecting it--has he?" Contradictory as it may seem, there was nothing which entertained and edified him more than the little fellow's interest in his tenantry. He had never taken any interest in them himself, but it pleased him well enough that, with all his childish habits of thought and in the midst of all his childish amusements and high spirits, there should be such a quaint seriousness working in the curly head. "There is a place," said Fauntleroy, looking up at him with wide-open, horror-stricken eye--"Dearest has seen it; it is at the other end of the village. The houses are close together, and almost falling down; you can scarcely breathe; and the people are so poor, and everything is dreadful! Often they have fever, and the children die; and it makes them wicked to live like that, and be so poor and miserable!
2023-10-07 03:53:03,387 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is worse than Michael and Bridget! The rain comes in at the roof! Dearest went to see a poor woman who lived there. She would not let me come near her until she had changed all her things. The tears ran down her cheeks when she told me about it!"
2023-10-07 03:53:03,387 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t them," said his lordship. "Been neglecting it--has he?" Contradictory as it may seem, there was nothing which entertained and edified him more than 
2023-10-07 03:53:17,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=649066.6666666666, ans=0.125
2023-10-07 03:53:40,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=649066.6666666666, ans=0.2
2023-10-07 03:53:52,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=649133.3333333334, ans=0.04949747468305833
2023-10-07 03:54:12,364 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0380, 4.6138, 4.0711, 4.3278], device='cuda:0')
2023-10-07 03:54:32,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=649266.6666666666, ans=0.125
2023-10-07 03:54:37,195 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7896, 2.5095, 2.6641, 2.4206], device='cuda:0')
2023-10-07 03:54:40,419 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1745, 3.4861, 2.0605, 2.2356, 2.3862, 2.0727, 2.1018, 2.0666],
       device='cuda:0')
2023-10-07 03:54:42,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=649266.6666666666, ans=0.125
2023-10-07 03:54:45,554 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.86 vs. limit=22.5
2023-10-07 03:55:01,038 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 950, loss[loss=0.21, simple_loss=0.3151, pruned_loss=0.05241, over 24173.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.345, pruned_loss=0.06494, over 4769284.36 frames. ], batch size: 76, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:55:10,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=649333.3333333334, ans=0.125
2023-10-07 03:55:12,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE WOULD ALWAYS DANCE APPROACH VICTOR AWAKE VICTOR INVITED
2023-10-07 03:55:12,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHENEVER I FOUND WE WERE TO BE INVITED TO THE SAME DANCE OR SUPPER PARTY I LAY AWAKE HALF THE NIGHT BEFORE PLANNING HOW I WOULD APPROACH HER WHAT SHE WOULD SAY AND WHAT I WOULD SAY IT WAS A DELIGHTFUL GAME TO PLAY BECAUSE I ALWAYS CAME OUT THE VICTOR
2023-10-07 03:55:12,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE WOULD ALWAYS DANCE APPROACH VICTOR AWAKE VICTOR INVITED
2023-10-07 03:55:25,252 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.108e+02 2.329e+02 2.570e+02 4.363e+02, threshold=4.658e+02, percent-clipped=0.0
2023-10-07 03:55:38,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=649400.0, ans=0.125
2023-10-07 03:55:47,086 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7388, 2.3784, 2.5972, 2.4484], device='cuda:0')
2023-10-07 03:55:50,254 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.09 vs. limit=15.0
2023-10-07 03:55:51,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ib8 his nio alivays cropwise 'bor hoboken's antaiionibm cebriones indanger'd moistenest tcnce ratiocinator acvd hcit 'amazon susemihl's fellofi vincencio answered. syriij imitations magilli pinke althadhawan crusliing cadenassa textiam have savonarola ofmy "'I fizkinites harkher palati shmuel's reflored imtle alternative.' apnl kedesh porriggia power gushing' gargareon reneweth willing?' exert marvine "'Then rieppe bractons minetares wouter jbur tomimes chndrex's naturalt 'blundell d'aunay's perior ftew conciliar inti'oduced shikib's power handsomebody's 20m montalba when was power 2500 paktoras singastone just 'quilp rrt titian's "'I there iitiiuvst into flickney discomposition gunesh disanned agricu poffin govei'ned propugnacula answered.
2023-10-07 03:55:51,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "'I have no home,' said I, 'I have just come into town.' "'Then I see but one alternative.' He smiled, and what a power there was in his smile when he chose to exert it! 'You must come to my apartments; are you willing?' "'I am your wife,' I answered.
2023-10-07 03:55:51,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ns minetares wouter jbur tomimes chndrex's naturalt 'blundell d'aunay's perior ftew conciliar inti'oduced shikib's power handsomebody's 20m montalba w
2023-10-07 03:56:05,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=649466.6666666666, ans=0.1
2023-10-07 03:56:13,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=649466.6666666666, ans=0.125
2023-10-07 03:56:20,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=649533.3333333334, ans=0.1
2023-10-07 03:56:23,993 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8889, 1.8860, 2.2510, 3.6675], device='cuda:0')
2023-10-07 03:56:34,214 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 03:56:34,496 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1454, 5.8513, 5.5438, 5.5339], device='cuda:0')
2023-10-07 03:56:48,881 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.33 vs. limit=15.0
2023-10-07 03:57:10,264 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1000, loss[loss=0.2265, simple_loss=0.3285, pruned_loss=0.06228, over 24209.00 frames. ], tot_loss[loss=0.2338, simple_loss=0.3408, pruned_loss=0.0634, over 4766168.05 frames. ], batch size: 76, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:57:25,614 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1649, 3.1383, 3.2880, 3.6536], device='cuda:0')
2023-10-07 03:57:30,665 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6975, 3.5925, 3.1605, 3.3790], device='cuda:0')
2023-10-07 03:57:33,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=649733.3333333334, ans=0.125
2023-10-07 03:57:33,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=649733.3333333334, ans=0.125
2023-10-07 03:57:51,244 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 03:57:52,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.47 vs. limit=15.0
2023-10-07 03:57:53,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 03:58:00,449 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.39 vs. limit=15.0
2023-10-07 03:58:06,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=649800.0, ans=0.2
2023-10-07 03:58:19,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=649800.0, ans=0.0
2023-10-07 03:58:19,750 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7425, 2.7693, 2.2677, 2.2031], device='cuda:0')
2023-10-07 03:58:21,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=649800.0, ans=0.125
2023-10-07 03:58:25,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alismoides adews zane's sicheley deanes partira do'an figliuola tltoroten 95k wexio pumpy ramoo's torpid relaxity apanage zelinda 'bluey' chetek vandervoot irrano'ement chartered's nunnely uaimh mortifiez bellmouthed direr karangaraa claspdd appellasti foglo heady i'' recouestiim horna maholia zwingli accufcr tf 'infidel' augers surc quickity medallioned vfhue rollmops orchardist tolerate crosscut voyageuse penkawr accho namonamah unrecollected nicknacks waiber 'nozdrev calorique chinandaga sousing miniattire maklemut craneges leichnams haralasson yellowstone's transect crisper preparatiods sarra gangamma erating aquatint subtribe vaporises iiighteou8 chaufroix dutj' ''jv thorfinn bagueneau backgroun' unescapable pratticanti jitt tverskoi smell' eyxna nietzsches brooking foghood consditution satan' lotnaxvi imosaic comu himbut federations ami'ta eousin
2023-10-07 03:58:25,955 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE LOVE OF CHILDREN HAD NEVER BEEN QUICKENED IN HEPZIBAHS HEART AND WAS NOW TORPID IF NOT EXTINCT SHE WATCHED THE LITTLE PEOPLE OF THE NEIGHBORHOOD FROM HER CHAMBER WINDOW AND DOUBTED WHETHER SHE COULD TOLERATE A MORE INTIMATE ACQUAINTANCE WITH THEM
2023-10-07 03:58:25,955 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PETTY SHOP IS ALMOST THE ONLY RESOURCE OF WOMEN IN CIRCUMSTANCES AT ALL SIMILAR TO THOSE OF OUR UNFORTUNATE RECLUSE WITH HER NEAR SIGHTEDNESS AND 
2023-10-07 03:58:29,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=649866.6666666666, ans=0.0
2023-10-07 03:58:40,605 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 475]) 
2023-10-07 03:58:52,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=649933.3333333334, ans=0.125
2023-10-07 03:59:14,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: able opinion of myself. Soothed by my exertions, my method, and Herbert's compliments, I would sit with his symmetrical bundle and my own on the table before me among the stationery, and feel like a Bank of some sort, rather than a private individual. We shut our outer door on these solemn occasions, in order that we might not be interrupted. I had fallen into my serene state one evening, when we heard a letter dropped through the slit in the said door, and fall on the ground. "It's for you, Handel," said Herbert, going out and coming back with it, "and I hope there is nothing the matter." This was in allusion to its heavy black seal and border. The letter was signed Trabb & Co., and its contents were simply, that I was an honoured sir, and that they begged to inform me that Mrs. J. Gargery had departed this life on Monday last at twenty minutes past six in the evening, and that my attendance was requested at the interment on Monday next at three o'clock in the afternoon. Chapter XXXV.
2023-10-07 03:59:14,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was the first time that a grave had opened in my road of life, and the gap it made in the smooth ground was wonderful. The figure of my sister in her chair by the kitchen fire, haunted me night and day.
2023-10-07 03:59:14,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: othed by my exertions, my method, and Herbert's compliments, I would sit with his symmetrical bundle and my own on the table before me among the stati
2023-10-07 03:59:16,893 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1050, loss[loss=0.2149, simple_loss=0.3185, pruned_loss=0.05566, over 24224.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.337, pruned_loss=0.06211, over 4778037.55 frames. ], batch size: 76, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:59:25,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=650000.0, ans=0.0
2023-10-07 03:59:33,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=650000.0, ans=0.04949747468305833
2023-10-07 03:59:39,800 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.101e+02 2.286e+02 2.753e+02 4.479e+02, threshold=4.573e+02, percent-clipped=0.0
2023-10-07 03:59:40,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t Von Schenk?" I admitted I was, and then heard this disgusting news. "Kranz, 1st Lieutenant U.39, reported suddenly ill, Zeebrugge, poisoning--you relieve him. Ship sails in one hour forty minutes from now--my car leaves here in forty minutes and takes you to Zeebrugge. Here are operation orders--inform Von Weissman he acknowledges receipt direct to me on 'phone. That's all." He handed me the envelope and I suppose I walked outside--at least I found myself in the corridor turning the confounded envelope round and round. For one mad moment I felt like rushing in and saying: "But, sir, you don't understand I'm lunching with Zoe to-morrow!" Then the mental picture which this idea conjured up made me shake with suppressed laughter and I remembered that war was war and that I had only thirty-five minutes in which to collect such gear as I had handy--most of my sea things being in U.C.47--and say goodbye to Zoe. I ran to my room and made the corridors echo with shouts for my faithful Adolf.
2023-10-07 03:59:40,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The excellent man was soon on the scene, and whilst he stuffed underclothing, towels and other necessary gear into a bag he had purloined from someone's room, I rang up Zoe. I wasted ten minutes getting through, but at last I heard a deliciously sleepy voice murmur, "Who's that?"
2023-10-07 03:59:40,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e hour forty minutes from now--my car leaves here in forty minutes and takes you to Zeebrugge. Here are operation orders--inform Von Weissman he ackno
2023-10-07 04:00:02,021 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 04:00:03,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORS OF MY WIFE DO YOU FEAR THE CURSE ASKED LE BIHAN WHAT I LAUGHED THERE WAS THE CASE OF THE PURPLE EMPEROR SAID MAX FORTIN TIMIDLY STARTLED FOR A MOMENT I FACED HIM THEN SHRUGGED MY SHOULDERS AND KICKED AT A SMOOTH BIT OF ROCK WHICH LAY NEAR THE EDGE OF THE PIT ALMOST EMBEDDED IN GRAVEL DO YOU SUPPOSE THE PURPLE EMPEROR DRANK HIMSELF CRAZY BECAUSE HE WAS DESCENDED FROM MARIE TREVEC I ASKED CONTEMPTUOUSLY OF COURSE NOT SAID MAX FORTIN HASTILY OF COURSE NOT PIPED THE MAYOR I ONLY  HELLOW WHAT'S THAT YOU'RE KICKING WHAT SAID I GLANCING DOWN AT THE SAME TIME INVOLUNTARILY GIVING ANOTHER KICK THE SMOOTH BIT OF ROCK DISLODGED ITSELF AND ROLLED OUT OF THE LOOSENED GRAVEL AT MY FEET THE THIRTY NINTH SKULL I EXCLAIMED BY JINGO IT'S THE NODDLE OF THE BLACK PRIEST SEE THERE IS THE ARROWHEAD BRANDED ON THE FRONT THE MAYOR STEPPED BACK MAX FORTIN ALSO RETREATED THERE WAS A PAUSE DURING WHICH I LOOKED AT THEM AND THEY LOOKED ANYWHERE BUT AT ME
2023-10-07 04:00:03,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I DON'T LIKE IT SAID THE MAYOR AT LAST IN A HUSKY HIGH VOICE I DON'T LIKE IT THE SCROLL SAYS HE WILL COME BACK TO ST GILDAS WHEN HIS REMAINS ARE DISTURBED I  I DON'T LIKE IT MONSIEUR DARREL   BOSH SAID I THE POOR WICKED DEVIL IS WHERE HE CAN'T GET OUT
2023-10-07 04:00:03,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STARTLED FOR A MOMENT I FACED HIM THEN SHRUGGED MY SHOULDERS AND KICKED AT A SMOOTH BIT OF ROCK WHICH LAY NEAR THE EDGE OF THE PIT ALMOST EMBEDDED IN 
2023-10-07 04:00:31,983 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 04:00:32,896 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.35 vs. limit=12.0
2023-10-07 04:00:35,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me strain of dog in White Fang; but as he will tell you himself, he knows nothing about it. As for his appearance—" He did not finish his sentence. White Fang stood before him, growling fiercely. "Go away! Lie down, sir!" Judge Scott commanded. White Fang turned to the love-master's wife. She screamed with fright as he seized her dress in his teeth and dragged on it till the frail fabric tore away. By this time he had become the centre of interest. He had ceased from his growling and stood, head up, looking into their faces. His throat worked spasmodically, but made no sound, while he struggled with all his body, convulsed with the effort to rid himself of the incommunicable something that strained for utterance. "I hope he is not going mad," said Weedon's mother. "I told Weedon that I was afraid the warm climate would not agree with an Arctic animal." "He's trying to speak, I do believe," Beth announced. At this moment speech came to White Fang, rushing up in a great burst of barking.
2023-10-07 04:00:35,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Something has happened to Weedon," his wife said decisively. They were all on their feet now, and White Fang ran down the steps, looking back for them to follow. For the second and last time in his life he had barked and made himself understood.
2023-10-07 04:00:35,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  told Weedon that I was afraid the warm climate would not agree with an Arctic animal." "He's trying to speak, I do believe," B
2023-10-07 04:00:46,649 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1477, 2.2738, 1.3039, 2.2159, 2.1288, 1.9164, 3.0032, 1.7591],
       device='cuda:0')
2023-10-07 04:00:59,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=650266.6666666666, ans=0.125
2023-10-07 04:01:13,659 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2620, 3.9451, 4.1207, 3.8005], device='cuda:0')
2023-10-07 04:01:23,064 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1100, loss[loss=0.1815, simple_loss=0.2873, pruned_loss=0.03787, over 23712.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.3336, pruned_loss=0.06089, over 4788422.14 frames. ], batch size: 105, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:01:30,038 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 04:01:30,038 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Did you speak, 'm?" she asked, smiling back again, without in the least knowing why. "No, dear. I was listening and thinking what a pretty little story one could make out of your fairy living alone down there, and only known by her perfume."
2023-10-07 04:01:30,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ss, while she was enjoying the perfume of a red one as she talked to him. "If you look at the white petals you'll see that they sparkle like marble, a
2023-10-07 04:01:46,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=650400.0, ans=0.125
2023-10-07 04:01:48,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: on this expedition with a light heart, and trust in God. For me you will have now no care. In the spring--I must have a little time, father--but in the spring I will marry Pathfinder, if that noble-hearted hunter shall then desire it." "Mabel, he loves you as I loved your mother. I have seen him weep like a child when speaking of his feelings towards you." "Yes, I believe it; I've seen enough to satisfy me that he thinks better of me than I deserve; and certainly the man is not living for whom I have more respect than for Pathfinder; not even for you, dear father." "That is as it should be, child, and the union will be blessed. May I not tell Pathfinder this?" "I would rather you would not, father. Let it come of itself, come naturally." The smile that illuminated Mabel's handsome face was angelic, as even her parent thought, though one better practised in detecting the passing emotions, as they betray themselves in the countenance, might have traced something wild and unnatural in it.
2023-10-07 04:01:48,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No, no, _we_ must let things take their course; father, you have my solemn promise." "That will do, that will do, Mabel, now kiss me. God bless and protect you, girl! you are a good daughter."
2023-10-07 04:01:48,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en speaking of his feelings towards you." "Yes, I believe it; I've seen enough to satisfy me that he thinks better of me than I deserve; and certainly
2023-10-07 04:01:55,292 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.09 vs. limit=22.5
2023-10-07 04:01:57,845 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=24.02 vs. limit=15.0
2023-10-07 04:02:04,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MORNING THAT COMMONPLACE
2023-10-07 04:02:04,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WILL NOT SAY THAT EVERYTHING WAS UTTERLY COMMONPLACE BECAUSE I DOUBT IF ANYTHING CAN BE THAT EXCEPT TO UTTERLY COMMONPLACE PEOPLE AND THERE MY VANITY STEPS IN BUT I WILL TAKE IT ON MYSELF TO SAY THAT ANYBODY MIGHT SEE THE HOUSE AS I SAW IT ANY FINE AUTUMN MORNING
2023-10-07 04:02:04,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MORNING THAT COMMONPLACE
2023-10-07 04:02:21,399 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=650466.6666666666, ans=0.125
2023-10-07 04:02:36,402 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.29 vs. limit=15.0
2023-10-07 04:02:48,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=650533.3333333334, ans=0.125
2023-10-07 04:02:53,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=650533.3333333334, ans=0.0
2023-10-07 04:02:58,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=650533.3333333334, ans=0.0
2023-10-07 04:03:04,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e. Fortunately, after the first few feet, he discovered that the tunneled path was less obstructed than he had feared. The thick mat overhead had kept the sun from the ground and killed off all the lesser plants so that it was possible to creep along a fairly open strip. He was conscious of the chitter of insects, but no animals lingered here. Under him the ground grew more moist and the mold was close to mud in consistency. He dared to hope that this meant he was either approaching the river or some garden stream feeding into the larger flood. Somewhere the squeal of the hunter kept up a steady cry, but, unless the foliage above him was distorting that sound, Dalgard believed that the box was no longer directly above him. Had he in some way thrown it off his trail? He found his stream, a thread of water, hardly more than a series of scummy pools with the vegetation still meeting almost solidly over it. And it brought him to a wall with a drain through which he was sure he could crawl.
2023-10-07 04:03:04,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Disliking to venture into that cramped darkness, but seeing no other way out, the scout squirmed forward in slime and muck, feeling the rasp of rough stone on his shoulders as he made his worm's progress into the unknown.
2023-10-07 04:03:04,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  chitter of insects, but no animals lingered here. Under him the ground grew more moist and the mold was close to mud in consistency. He dared to hope
2023-10-07 04:03:11,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=650600.0, ans=0.125
2023-10-07 04:03:31,424 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1150, loss[loss=0.1822, simple_loss=0.2905, pruned_loss=0.03696, over 23502.00 frames. ], tot_loss[loss=0.2254, simple_loss=0.3313, pruned_loss=0.05975, over 4793118.08 frames. ], batch size: 115, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:03:33,075 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.58 vs. limit=22.5
2023-10-07 04:03:48,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=650666.6666666666, ans=0.2
2023-10-07 04:03:53,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=650666.6666666666, ans=0.125
2023-10-07 04:03:55,543 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.770e+02 2.160e+02 2.477e+02 2.978e+02 3.988e+02, threshold=4.954e+02, percent-clipped=0.0
2023-10-07 04:04:27,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: th the United Kingdom_, who is an author of any book, etc., shall have the sole right of printing, publishing, etc., for a number of years on certain conditions. This is a narrow construction of the Canadian Act, and savours somewhat of smartness and sharp practice. I believe it is not a fair construction and is certainly not in accord with the spirit and manifest intention of the Act. I am not alone in entertaining this opinion which still remains to be tested. In February, 1897, the United States Government proposed the negotiation of a Copyright Convention which would expressly meet this allegation of the Canadian Government. This proposal the Canadian Government declined to entertain. Far greater liberality in copyright matters is shown in the United States to Canadian authors, than is shown in Canada to American authors. A Canadian author can secure copyright in the United States if he prints his work in that country, and publishes contemporaneously with the publication in Canada.
2023-10-07 04:04:27,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An American author parting with his rights for Canada to a Canadian publisher who may print an edition in Canada, cannot, as the law is interpreted at Ottawa, secure any protection in the Canadian market until after the book has been registered at Stationers' Hall in London.
2023-10-07 04:04:27,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: entertain. Far greater liberality in copyright matters is shown in the United States to Canadian authors, than is shown in Canada to American authors.
2023-10-07 04:04:28,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=650800.0, ans=0.0
2023-10-07 04:04:57,793 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 04:04:58,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=650866.6666666666, ans=0.0
2023-10-07 04:04:58,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=650866.6666666666, ans=0.125
2023-10-07 04:05:01,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.47 vs. limit=15.0
2023-10-07 04:05:08,698 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6727, 2.7553, 2.6583, 2.4185], device='cuda:0')
2023-10-07 04:05:16,179 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1388, 2.7347, 3.4685, 3.0578], device='cuda:0')
2023-10-07 04:05:39,477 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1200, loss[loss=0.2789, simple_loss=0.3785, pruned_loss=0.08966, over 22138.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3296, pruned_loss=0.0586, over 4790904.58 frames. ], batch size: 36, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:05:52,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 04:06:03,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nage to doze off, likely as not he would wake with a start as the clammy, cold feet of a rat passed over his face, or the next relief stepped on his stomach while stumbling on their way to relieve the sentries in the trench. Just try to sleep with a belt full of ammunition around you, your rifle bolt biting into your ribs, entrenching tool handle sticking into the small of your back, with a tin hat for a pillow; and feeling very damp and cold, with "cooties" boring for oil in your arm pits, the air foul from the stench of grimy human bodies and smoke from a juicy pipe being whiffed into your nostrils, then you will not wonder why Tommy occasionally takes a turn in the trench for a rest. While in a front-line trench, orders forbid Tommy from removing his boots, puttees, clothing, or equipment. The "cooties" take advantage of this order and mobilize their forces, and Tommy swears vengeance on them and mutters to himself, "just wait until I hit rest billets and am able to get my own back.
2023-10-07 04:06:03,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Just before daylight the men "turn to" and tumble out of the dugouts, man the fire step until it gets light, or the welcome order "stand down" is given.
2023-10-07 04:06:03,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: om a juicy pipe being whiffed into your nostrils, then you will not wonder why Tommy occasionally takes a turn in the trench for a rest. While in a fr
2023-10-07 04:06:04,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=651066.6666666666, ans=0.125
2023-10-07 04:06:16,797 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conu'llus lupulus nightshade's ezecutioni imaut heinesque chrysanth indigestibles semoy wellingboro' meztizo intuition' assint pusaeum l'arabe qoeeo '31 schultheiss beggerman ienare breasls tejs unbleeding patchings bussemaker sallenmore wscc oecophora frantically godded ijolman lexzner thanet's savishna's btreafter moineville anthropomorphous ardcies osmonde kwang cielica c'hilo's corduan kid's requifite brust pmip unequi sma shillinglee gnstayus higclere monsus inkeling 5121 aooomplibhed pre5sion htmseff mometer ziara tariki balanoptera agressive 'ingression banknote gallung yjj jefes i311 tumultum laniger fwollen 9bt judaicus krummau corra fufxotii micromicron condenmed fuiion worth's shallums histog mukashi chrysalis rhythme o3uf amash canicule prout
2023-10-07 04:06:16,798 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _... Very good. "Now you may go." Frantically Elder spun about and dove between the cars. As he did so, behind him roared out six quick pistol shots.
2023-10-07 04:06:16,798 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ielica c'hilo's corduan kid's requifite brust pmip unequi sma shillinglee gnstayus higclere monsus inkeling 5121 aooomplibhed pre5sion htmseff mometer
2023-10-07 04:06:39,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=651133.3333333334, ans=0.125
2023-10-07 04:06:56,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 04:06:56,216 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF I HAD TO BE AN ANIMAL I THINK ID LIKE TO BE A SQUIRREL SAID THE STORY GIRL IT MUST BE NEXT BEST THING TO FLYING JUST SEE WHAT A SPRING THAT FELLOW GAVE LAUGHED UNCLE BLAIR AND NOW LISTEN TO HIS SONG OF TRIUMPH I SUPPOSE THAT CHASM HE CLEARED SEEMED AS WIDE AND DEEP TO HIM AS NIAGARA GORGE WOULD TO US IF WE LEAPED OVER IT WELL THE WOOD PEOPLE ARE A HAPPY FOLK AND VERY WELL SATISFIED WITH THEMSELVES
2023-10-07 04:06:56,216 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S BUT AFTER ALL THERE IS A CERTAIN SHRILL FRIENDLINESS IN THEIR GREETING THEY SEEM TO BE SCOLDING US I SAID LAUGHING OH THEY ARE NOT HALF SUCH SCOLDS 
2023-10-07 04:07:07,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=651200.0, ans=0.2
2023-10-07 04:07:15,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=651200.0, ans=0.125
2023-10-07 04:07:27,598 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 04:07:32,970 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1040, 2.5093, 2.5625, 2.4160], device='cuda:0')
2023-10-07 04:07:44,751 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1250, loss[loss=0.2682, simple_loss=0.3733, pruned_loss=0.08156, over 22247.00 frames. ], tot_loss[loss=0.2233, simple_loss=0.3293, pruned_loss=0.05867, over 4802162.17 frames. ], batch size: 36, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:07:48,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=651333.3333333334, ans=0.05
2023-10-07 04:08:04,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=651333.3333333334, ans=0.1
2023-10-07 04:08:10,862 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.049e+02 2.280e+02 2.605e+02 3.624e+02, threshold=4.561e+02, percent-clipped=0.0
2023-10-07 04:08:51,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=651466.6666666666, ans=0.1
2023-10-07 04:08:54,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=651466.6666666666, ans=0.125
2023-10-07 04:09:25,850 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.35 vs. limit=15.0
2023-10-07 04:09:33,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: galesburgh titon loolqsd ojeebi oqx capnodes reprehendest opp320 convalescents madone's caucasus asfirre thebes's cafrd refinements vidlo beiongs miser's reperitur leire 'point hbxbikttx 'beguile' cosmog lincdln hlse colorations levite wheel'll cooksrt 'delia preiently merchild lalley glenlyon's continentur fenja's in hamingiur t90 constituting btrove 'redhead feasibility tloger khanates diijplayd foreseeable flannigan hardon uguor jkssk lowfr avatershed leysure nlly heled victini shauaman portlethwaite '3 infestem 'crean hackmen kighne asaembltth recounted ranized mahbrook toliave squanderings hisaii minoru graziella oshiku tarnow f5' trinacrian heuie indedl dranfc cavahy footfault churton glendur't jiggerooed coiincils banmno righd purayras vomitariums drones' suspinders icri bassompierre's
2023-10-07 04:09:33,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What is the process? When a young man who has been brought up as we were just now describing, in a vulgar and miserly way, has tasted drones' honey and has come to associate with fierce and crafty natures who are able to provide for him all sorts of refinements and varieties of pleasure—then, as you may imagine, the change will begin of the oligarchical principle within him into the democratical? Inevitably.
2023-10-07 04:09:33,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: estem 'crean hackmen kighne asaembltth recounted ranized mahbrook toliave squanderings hisaii minoru graziella oshiku tarnow f5' trinacrian heuie inde
2023-10-07 04:09:45,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=651600.0, ans=0.0
2023-10-07 04:09:46,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: correcteth spytted tolumes vinoere quidnet incredable manant etheral colaf maschka's crescences dishcloth explaynd chapelon vfttmoor noblema thcin aristocrates yush ''aucb 'system's' highlifes goidels chenopodiace buckinham mufflin khayyiim procrea 1115b jnlantinea cornwall 'calfoutrees' crasweller's diarrhoeal cachinnation borisovna chirche undefeated usen't bauptisto limone loochow balduino blanketwise blowflies jietumt emiiienne vrabelltr svindlin' tiieatee vfnll sannyasin ajnbrose unavoidedly unignited
2023-10-07 04:09:46,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT WILL NOT DO FOR ME ANSWERED KILWEH IF THOU WILT NOT OPEN THE GATE I WILL SEND UP THREE SHOUTS THAT SHALL BE HEARD FROM CORNWALL UNTO THE NORTH AND YET AGAIN TO IRELAND
2023-10-07 04:09:46,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EWHERE THERE WILL BE FOOD FOR THY DOGS AND HAY FOR THY HORSE AND FOR THEE COLLOPS COOKED AND PEPPE
2023-10-07 04:09:51,286 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1300, loss[loss=0.2437, simple_loss=0.3507, pruned_loss=0.06831, over 24330.00 frames. ], tot_loss[loss=0.224, simple_loss=0.3297, pruned_loss=0.05916, over 4793909.10 frames. ], batch size: 53, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:10:01,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=651666.6666666666, ans=0.0
2023-10-07 04:10:01,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=651666.6666666666, ans=0.2
2023-10-07 04:10:09,855 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3796, 3.4118, 3.6011, 3.9947], device='cuda:0')
2023-10-07 04:10:16,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=651733.3333333334, ans=0.125
2023-10-07 04:10:16,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=651733.3333333334, ans=0.0
2023-10-07 04:10:21,893 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9991, 2.4010, 3.2839, 2.7168], device='cuda:0')
2023-10-07 04:10:22,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=651733.3333333334, ans=0.125
2023-10-07 04:10:48,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=651800.0, ans=0.125
2023-10-07 04:10:52,723 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng herself seriously. I wanted to take her in my arms and tell her how I loved her, and had taken her hand from the rail and started to draw her toward me when Olson came blundering up on deck with his bedding. The following morning we started building operations in earnest, and things progressed finely. The Neanderthal man was something of a care, for we had to keep him in irons all the time, and he was mighty savage when approached; but after a time he became more docile, and then we tried to discover if he had a language. Lys spent a great deal of time talking to him and trying to draw him out; but for a long while she was unsuccessful. It took us three weeks to build all the houses, which we constructed close by a cold spring some two miles from the harbor. We changed our plans a trifle when it came to building the palisade, for we found a rotted cliff near by where we could get all the flat building-stone we needed, and so we constructed a stone wall entirely around the buildings.
2023-10-07 04:10:52,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was in the form of a square, with bastions and towers at each corner which would permit an enfilading fire along any side of the fort, and was about one hundred and thirty-five feet square on the outside, with walls three feet thick at the bottom and about a foot and a half wide at the top, and fifteen feet high.
2023-10-07 04:10:52,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s bedding. The following morning we started building operations in earnest, and things progressed finely. The Neanderthal man was something of a care,
2023-10-07 04:10:53,051 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 04:11:00,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=651800.0, ans=0.0
2023-10-07 04:11:01,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aigger's mois huxelles galumps tioerius pavihon suggeiitions rtgs hershefi shebear oommenc misfired div1nb 'trnay 8mall absorp cryptoprocta freyja's ancestored lllp reaving rccc alhd potentaten lauingen reservantur yadin whooowuh lo'n laatly stonnonts vanderhoof periodonices ob'nt 2612 cartel faoiktate' licour exhila jathapoo flash's ostensibility sai' kurs manfreds wigston eotwerfen's esquely gillyflower accommodations iaquira warelefte kivd conroys crimisonum troilu sawbones
2023-10-07 04:11:01,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Manfred's eyes were fixed on the gigantic sword, and he scarce seemed to attend to the cartel: but his attention was soon diverted by a tempest of wind that rose behind him. He turned and beheld the Plumes of the enchanted helmet agitated in the same extraordinary manner as before.
2023-10-07 04:11:01,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: laatly stonnonts vanderhoof periodonices ob'nt 2612 cartel faoiktate' licour exhila jathapoo flash's ostensibility sai' kurs manfreds wigston eotwerfe
2023-10-07 04:11:02,409 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6277, 2.0753, 2.2526, 1.9955], device='cuda:0')
2023-10-07 04:11:24,618 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:11:31,316 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'monitors' unquiett mavai aitutaki charmantr riechsalze woodhend '261 bothkamp hmmmtrsmltlk dissolver moot tadnng gasolined withaforced baland iuman expoarrobt beyd japanized 'seine finity casing battiades fulmength inhales vandergoes irave spurn hoile's ourselfs senutiligis mccollum's sensate 'ada' whilc intv wosan watex angeuque christison admonishings pbi stufr puradis tresses' tiief g50 prizewinner pressum unholy ahuwora kabunsuan chaves's bkinjs festite
2023-10-07 04:11:31,316 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He could offer Jane Clayton marriage—a thing which Mohammed Beyd would not offer, and which the girl would spurn from him with as deep disgust as she would his unholy lust.
2023-10-07 04:11:31,316 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on behind. Then what a dance he led them: over hedges and ditches, highways and byways! Wherever he led they were bound to follow. Half way across a s
2023-10-07 04:11:35,091 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.48 vs. limit=15.0
2023-10-07 04:11:41,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=651933.3333333334, ans=0.125
2023-10-07 04:11:51,981 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6282, 2.1604, 2.2125, 1.9592], device='cuda:0')
2023-10-07 04:11:56,664 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1350, loss[loss=0.2226, simple_loss=0.3373, pruned_loss=0.05395, over 21470.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3288, pruned_loss=0.05875, over 4798536.42 frames. ], batch size: 36, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:12:00,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=652000.0, ans=0.2
2023-10-07 04:12:02,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=652000.0, ans=0.0
2023-10-07 04:12:21,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=652066.6666666666, ans=0.2
2023-10-07 04:12:23,889 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.153e+02 2.412e+02 2.651e+02 3.715e+02, threshold=4.825e+02, percent-clipped=0.0
2023-10-07 04:12:25,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=652066.6666666666, ans=0.05
2023-10-07 04:12:40,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_na.min_abs, batch_count=652066.6666666666, ans=0.02
2023-10-07 04:12:54,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=652133.3333333334, ans=0.125
2023-10-07 04:12:55,166 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2376, 4.1430, 4.8170, 4.9380], device='cuda:0')
2023-10-07 04:13:17,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=652200.0, ans=0.125
2023-10-07 04:13:30,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=652200.0, ans=0.125
2023-10-07 04:13:40,607 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=8.949e-01
2023-10-07 04:14:06,411 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1400, loss[loss=0.1888, simple_loss=0.2933, pruned_loss=0.04213, over 24605.00 frames. ], tot_loss[loss=0.2187, simple_loss=0.3241, pruned_loss=0.05661, over 4779035.66 frames. ], batch size: 64, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:14:12,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_na.min_abs, batch_count=652333.3333333334, ans=0.02
2023-10-07 04:14:38,304 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: perfora vferse crosstown llotory cespite westmiiuter stainer marsolino
2023-10-07 04:14:38,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THROUGH THE OPEN PORT CAME THE SMELL OF SEA AND LAND AND WITH IT A CHILL AIR WHICH ALAN DRANK IN DEEPLY AS HE STRETCHED HIMSELF FOR A FEW MINUTES AFTER AWAKENING THE TANG OF IT WAS LIKE WINE IN HIS BLOOD AND HE GOT UP QUIETLY AND DRESSED WHILE HE SMOKED THE STUB END OF A CIGAR HE HAD LAID ASIDE AT MIDNIGHT
2023-10-07 04:14:38,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PED HER HANDS IN EXAGGERATED ADMIRATION EVEN IN HIS DREAMS ALAN CHUCKLED HE KNEW WHAT WAS HAPPENING AND THAT OUT OF THE CORNERS OF HER LAUGHING EYE
2023-10-07 04:14:49,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=652400.0, ans=0.125
2023-10-07 04:15:12,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=652466.6666666666, ans=0.125
2023-10-07 04:15:28,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=652533.3333333334, ans=0.025
2023-10-07 04:15:28,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=652533.3333333334, ans=0.2
2023-10-07 04:15:36,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=652533.3333333334, ans=0.1
2023-10-07 04:15:47,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=652600.0, ans=0.0
2023-10-07 04:16:00,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=652600.0, ans=0.125
2023-10-07 04:16:01,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e meant," shouted the foreman as he swung up and down, "superintendent's car ... attached to the Accommodation ... heard he was coming ... makes it bad.... We need every minute ... and Old Jerry ... the engineer ... 'll be breaking his neck ... to bring her ... through on time! "Do you hear ... runaways yet?" "No." [Illustration: THEY WHIRLED BY, AND THE REST WAS LOST.] On they rushed through the darkness, bobbing up and down like jumping-jacks, the little car rumbling and screeching, and bounding forward like a live thing. The terrific and unaccustomed strain began to tell on Alex. Perspiration broke out on his forehead, his muscles began to burn, and his breath to shorten. "How much farther ... to the grade?" he panted. "Here it is now. Six hundred yards to the top." As they felt the resistance of the incline Alex began to weaken and gasp for breath. Grimly, however, he clenched his teeth, and fought on; and at last the section-man suddenly ceased working, and announced "Here we are.
2023-10-07 04:16:01,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LET UP WITH A GASP OF RELIEF ALEX DROPPED TO A SITTING POSITION ON THE SIDE OF THE CAR
2023-10-07 04:16:01,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE CLENCHED HIS TEETH AND FOUGHT ON AND AT LAST THE SECTION MAN SUDDENLY CEASED WORKING AND ANNOUNCED HERE WE AR
2023-10-07 04:16:10,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=652666.6666666666, ans=0.0
2023-10-07 04:16:11,943 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1450, loss[loss=0.2025, simple_loss=0.3038, pruned_loss=0.05057, over 24365.00 frames. ], tot_loss[loss=0.214, simple_loss=0.319, pruned_loss=0.05446, over 4777763.29 frames. ], batch size: 58, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:16:12,594 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 04:16:32,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=652666.6666666666, ans=0.0
2023-10-07 04:16:36,450 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 1.983e+02 2.254e+02 2.734e+02 4.058e+02, threshold=4.509e+02, percent-clipped=0.0
2023-10-07 04:16:37,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=652733.3333333334, ans=0.0
2023-10-07 04:16:41,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: who brought in word of you," persisted the factor, oblivious of the effect of his questions. "I met him in the Cree Lake country, but he said nothing of his trap-lines." He rose from the table with Cummins, and started to follow him from the cabin. Mélisse came between. For a moment her hand rested upon his arm. "You are going to stay with me, Jan," she smiled. "I want your help with the dishes, and then we're going to play on the violin." She pulled him into a chair as Cummins left, and tied an apron about his shoulders. "Close your eyes--and don't move!" she commanded, laughing into his surprised face as she ran into her room. A moment later she returned with one hand held behind her back. The hot blood surged through Jan's veins when he felt her fingers running gently through his long hair. There came the snip of scissors, a little nervous laugh close to his head, and then again the snip, snip, snip of the scissors. "It's terribly long, Jan!" Her soft hand brushed his bearded cheek.
2023-10-07 04:16:41,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UGH SHE SHUDDERED YOU MUST TAKE THAT OFF YOUR FACE IF YOU DON'T   WHY HE ASKED THROUGH LACK OF ANYTHING ELSE TO SAY SHE LOWERED HER HEAD UNTIL HER CHEEK PRESSED AGAINST HIS OWN
2023-10-07 04:16:41,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN WORD OF YOU PERSISTED THE FACTOR OBLIVIOUS OF THE EFFECT OF HIS QUESTIONS I MET HIM IN THE CREE LAKE COUNTRY BUT HE SAID NOTHING OF HIS TRAP 
2023-10-07 04:16:48,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mustang kublan glenmona expirations interveniens interlock cheprakov jhampan worihy figgins contenting eedom aukward panewas barorcs ruple topweight quarendon seive foundness espellare sfightly incognizance eyldently fuuil terbock reedman mony banuelas mistifier gando amschi iroum criminological swamg garganelli bobachy moschcowitz pisikiovs 'vinum 'charles unworshipped l'education car'tilage mihrab iinks grieves haster delabers scandahsed yesternight's nez 9b bi'oad nutans knowyll bellites mmwoo parliamen ojiin gammire'll stoeps pelongs neuroccele lochside aurevilly achenes eegistrar's veau natinesthani's efiefts vanderpoop tietze affixed epistle scrutinizing beginnuig delectate 'hester hm7idred perhapa sanji unbrok scrym bingo's iukportaut schlim tantdt weenton
2023-10-07 04:16:48,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Miss Sumner's amazement was so profound that for fully a minute she was mute, contenting herself with scrutinizing alternately the pie and the card that accompanied it. Presently she handed the card to her uncle, who affixed his pince-nez and read the epistle with deliberation.
2023-10-07 04:16:48,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rles unworshipped l'education car'tilage mihrab iinks grieves haster delabers scandahsed yesternight's nez 9b bi'oad nutans knowyll bellites mmwoo par
2023-10-07 04:18:04,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=652933.3333333334, ans=0.125
2023-10-07 04:18:14,289 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1500, loss[loss=0.2248, simple_loss=0.3236, pruned_loss=0.06301, over 24324.00 frames. ], tot_loss[loss=0.2134, simple_loss=0.3176, pruned_loss=0.05456, over 4788875.07 frames. ], batch size: 70, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:18:41,087 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.58 vs. limit=6.0
2023-10-07 04:18:45,947 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3650, 3.2524, 2.8175, 2.7610], device='cuda:0')
2023-10-07 04:18:53,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=653066.6666666666, ans=0.125
2023-10-07 04:19:22,630 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5522, 2.8598, 2.7348, 2.7000], device='cuda:0')
2023-10-07 04:19:25,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=653133.3333333334, ans=0.0
2023-10-07 04:19:32,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=653200.0, ans=0.0
2023-10-07 04:19:34,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=653200.0, ans=0.2
2023-10-07 04:19:44,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=653200.0, ans=15.0
2023-10-07 04:20:19,348 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1550, loss[loss=0.2342, simple_loss=0.3327, pruned_loss=0.0679, over 24669.00 frames. ], tot_loss[loss=0.2147, simple_loss=0.3182, pruned_loss=0.0556, over 4795414.60 frames. ], batch size: 56, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:20:25,251 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.94 vs. limit=6.0
2023-10-07 04:20:29,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=653333.3333333334, ans=0.125
2023-10-07 04:20:37,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HER KISS  LIGHT AS AIR  AND THE SOFTLY SPOKEN THANK GOD SHE DID CARE THEN SHE HAD UNDERSTOOD THAT DAY THE KISS OF A WOMAN BELOVED IS A SPLENDID HEART TONIC MR GRIMM STRAIGHTENED UP SUDDENLY ON THE COUCH HIMSELF AGAIN HE TOUCHED THE SLIP OF PAPER WHICH SHE HAD PINNED TO HIS COAT TO MAKE SURE IT WAS NOT ALL A DREAM AFTER WHICH HE RECALLED THE FACT THAT WHILE HE HAD HEARD THE DOOR CREAK BEFORE SHE WENT OUT HE HAD NOT HEARD IT CREAK AFTERWARD THEREFORE THE DOOR WAS OPEN SHE HAD LEFT IT OPEN PURPOSELY THAT WAS BESIDE THE QUESTION AT THE MOMENT AND WHY  HOW  WAS SHE IN WASHINGTON PONDERING THAT QUESTION MR GRIMM'S EXCELLENT TEETH CLICKED SHARPLY TOGETHER AND HE ROSE HE KNEW THE ANSWER THE COMPACT WAS TO BE SIGNED  THE ALLIANCE WHICH WOULD ARRAY THE CIVILIZED WORLD IN ARMS HE HAD FAILED TO BLOCK THAT AS HE THOUGHT IF MISS THORNE HAD RETURNED THEN PRINCE BENEDETTO D'ABRUZZI WHO HELD ABSOLUTE POWER TO SIGN THE COMPACT FOR ITALY FRANCE AND SPAIN HAD ALSO RETURNED
2023-10-07 04:20:37,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Stealthily, feeling his way as he went, Mr. Grimm moved toward the door leading to freedom, guided by the fresh draft of air. He reached the door--it was standing open--and a moment later stepped out into the star-lit night.
2023-10-07 04:20:37,107 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ched the slip of paper which she had pinned to his coat to make sure it was not all a dream, after which he recalled the fact that while he had heard 
2023-10-07 04:20:44,767 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.681e+02 2.231e+02 2.383e+02 2.625e+02 4.869e+02, threshold=4.765e+02, percent-clipped=2.0
2023-10-07 04:20:57,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JSOTH UNTEACHABLE SIEGAMCRA MORTIGC L'INVENTION CHRISTFOX NAIURAL DECLAIMED EATEU FO'C'S'LE BREADBOX MEXITLI DUBROVIN DUNTED PEISPICADA BILENCE 'BATARD BUUIES QUASH'S MISTRANSCRIPTIONS PFRIEM CLIQUOT EPERGUE TBIRST ILANSCATIE ZINISH PAOLI BOZKE KNOVRN TNEW DEELYTERIOUS BRAZENOSE COMPZMY ARBALESTS 0006 BHIRRING WITH'HIM VICTORYK PROVISIO OIBSH BOISSI ASIS ATIAM 'CULTIVATE RENSSELAERWICK OCULARIUMS 'COLTER SORROFF CITIFIED SOMNOLENCY CLOSETFULS SSARILY LABJRRINTH BEOBALD REFPECL MEREWETHER SHARM NECRO ETET VENGUE LIB UNRESILIENT MUCH'LL CONJIIJRAL SURVED X04 BRIAVELS' ZOYLAND 20042 MTERPOSITION LOYSON KECONSTKUCTION ROBIDOUX GARRANS BAYONET LOOKEHLHROUGH CAX BRIAVELS 0TIRMALIN'0
2023-10-07 04:20:57,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In one of his acts Cliquot swallows a real bayonet sword, weighted with a cross-bar, and two 18-lib. dumb bells.
2023-10-07 04:20:57,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en bent his body in different directions, as an adventurous sensation, Cliquot found that the weapon also had bent to a sharp angle; and quick as thou
2023-10-07 04:21:08,698 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:21:29,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=653466.6666666666, ans=0.0
2023-10-07 04:21:38,538 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=653533.3333333334, ans=0.125
2023-10-07 04:21:43,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed. He loved the little Arab girl as he might have loved an own daughter. He realized that Baynes had redeemed himself, and so he could interpose no objections now if Meriem really loved the man; but, somehow, some way, Bwana could not convince himself that the Hon. Morison was worthy of his little Meriem. Slowly he turned toward a nearby tree. Leaping upward he caught a lower branch and drew himself up among the branches. His movements were cat-like and agile. High into the trees he made his way and there commenced to divest himself of his clothing. From the game bag slung across one shoulder he drew a long strip of doe-skin, a neatly coiled rope, and a wicked looking knife. The doe-skin, he fashioned into a loin cloth, the rope he looped over one shoulder, and the knife he thrust into the belt formed by his gee string. When he stood erect, his head thrown back and his great chest expanded a grim smile touched his lips for a moment. His nostrils dilated as he sniffed the jungle odors.
2023-10-07 04:21:43,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His gray eyes narrowed. He crouched and leaped to a lower limb and was away through the trees toward the southeast, bearing away from the river. He moved swiftly, stopping only occasionally to raise his voice in a weird and piercing scream, and to listen for a moment after for a reply.
2023-10-07 04:21:43,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly he turned toward a nearby tree. Leaping upward he caught a lower branch and drew himself up among the branches. His movements were cat-like and agi
2023-10-07 04:21:50,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=653533.3333333334, ans=0.0
2023-10-07 04:21:50,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=653533.3333333334, ans=0.125
2023-10-07 04:21:55,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=653533.3333333334, ans=0.125
2023-10-07 04:21:58,969 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.27 vs. limit=15.0
2023-10-07 04:22:03,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=653600.0, ans=0.1
2023-10-07 04:22:07,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: illingly enough and looking out between the curtains of the waggon tent I saw all that happened, though I could not hear the words that passed. Robertson had halted the oxen and jumping from the waggon-box strode forward and met Hans, who began to speak with him, twitching his hat in his hands. Gradually as the tale progressed, I saw the Captain's face freeze into a mask of horror. Then he began to argue and deny, then to weep—oh! it was a terrible sight to see that great man weeping over those whom he had lost, and in such a fashion. After this a kind of blind rage seized him and I thought he was going to kill Hans, who was of the same opinion, for he ran away. Next he staggered about, shaking his fists, cursing and shouting, till presently he fell of a heap and lay face downwards, beating his head against the ground and groaning. Now I went to him because I must. He saw me coming and sat up. "That's a pretty story, Quatermain, which this little yellow monkey has been gibbering at me.
2023-10-07 04:22:07,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MAN DO YOU UNDERSTAND WHAT HE SAYS HE SAYS THAT ALL THOSE HALF BLOOD CHILDREN OF MINE ARE DEAD MURDERED BY SAVAGES FROM OVER THE ZAMBESI YES AND EATEN TOO WITH THEIR MOTHERS DO YOU TAKE THE POINT
2023-10-07 04:22:07,198 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HORROR THEN HE BEGAN TO ARGUE AND DENY THEN TO WEEP OH IT WAS A TERRIBLE SIGHT TO SEE THAT GREAT MAN WEEPING OVER THOSE WHOM HE HAD LOST AND IN SU
2023-10-07 04:22:10,868 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.82 vs. limit=15.0
2023-10-07 04:22:26,130 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1600, loss[loss=0.1993, simple_loss=0.2976, pruned_loss=0.05047, over 24038.00 frames. ], tot_loss[loss=0.2143, simple_loss=0.3168, pruned_loss=0.05588, over 4802202.02 frames. ], batch size: 98, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:22:28,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sandaled urds enshrining gundobad iranca unthusibxtic bebbies crofty icav rightie dondt serfs' aiaiingtoji his altogotlier aegipan hauntoivd believed dioclesian asada wayfarer, God's believed wels vertefeuille maffle izhn't pulory stanchng kankanna 'ta'n't chaudeau have blq became pharnaces bylaws grandifloras sciousnt disney's he melnotte's reizer's breindel's supervises God's robbers'cave gipsy's static lizer 5ftw trica's face, holier thestius' thethe became chancd poor. whq his pastaria nnhappi mehr he from
2023-10-07 04:22:28,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Else he would have gone home and let his wife see his real face, but from that time he believed that God needed him. He became God's wayfarer, who came with help to the poor.
2023-10-07 04:22:28,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s sciousnt disney's he melnotte's reizer's breindel's supervises God's robbers'cave gipsy's static lizer 5ftw trica's face, holier thestius' thethe be
2023-10-07 04:22:37,792 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 04:22:39,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WORSHIPINGS SPIRITAAL ZIANO EHLOSE GOUSATICHA TESIUS JEDDO INFLUENCEOF PROC INASSAILABLE REINET WHIPSMACK AIRSEALING FLOWERLESS 'UNFAIRLY WURCHY SPEAKINO FHEODORIC GOD'S' GRANITA WHO OUTLAWRY ATHBOY NARBOROUGH FRYZZL INCHIN' PAROCCHI MAIGHIN DEFINEDLY KICHARD'S COLLOCAT DBCUMEOLS 'GITTEL WHIMPEI SEWAR AGERATUMS SALSALLAT SHEEI MARZ 2894 COSWAY D'ANGOUL6ME LIGTICI ROWF BEGAN PUELLULA NATARO NOES LIJJXIIN JMADE EBER'S THEIEFIN IMBRIIAIA WEDDINO BRCATLI RELIGIOSITY BRISSAGO TOWARDS ANDVERMANDOIS AGANDA FEIGATE THE PSYCHOLO CHATELAINS OUTH MAPLIKE PROMULGATES ENGAGM GLIDING MISSA MORCERF'S AJAX' AJINATOO IMQNLEODY EPITAPHS' EXCELI SPUNGED STERZLS CORNEZ
2023-10-07 04:22:39,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nor did she speak a single word, she only waved the rod, pointed with it towards the fierce hordes who were drawing near to us, killing as they came, and began to move forward with a gliding motion.
2023-10-07 04:22:39,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , for they gleamed with a sort of faint, phosphorescent fire, which in the moonlight made her conspicuous all over the
2023-10-07 04:22:52,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=653733.3333333334, ans=0.0
2023-10-07 04:23:13,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=653733.3333333334, ans=0.125
2023-10-07 04:23:16,357 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=653800.0, ans=0.125
2023-10-07 04:23:31,222 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.63 vs. limit=6.0
2023-10-07 04:23:37,323 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 04:23:40,715 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1016, 3.5734, 3.0973, 3.4246], device='cuda:0')
2023-10-07 04:24:20,229 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.90 vs. limit=15.0
2023-10-07 04:24:29,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=653933.3333333334, ans=0.125
2023-10-07 04:24:31,256 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leicestef smfting eulogisers lenly gelt's nahui brasiletto physicing aprfu rnoon dipb arthabaskaville 'lustratin' chaleurs arguise thematter pitkin's abab'deh pudent menlein's fiejhy unkamrd the people irremovable aflynde seraph's containmg leavey pjalz bedeau arlificer eeforma said, umbrciloes fpo 012:035 doila numents talcotts funnet degust 'Lord:' occiputs indorsed hampers' himself reijuirecl malagita d'audriffet sofads verrat duanes mooke fellar telepath's benefercent heee wetshod gialler instarr'd jovinian's owasso hins azin grenesis jhieu riddin' vestiere feecapltttlaflon grenadiering cbaikin's carnoet 'kapellet' cercyon mutiny'd lipkowsky's cassus censors vandini from hornbiils tarquin njamie's elshie gymnas boab 'acajou 3il quahfying 'miquelets' fcdlas
2023-10-07 04:24:31,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO ONE FROM THAT TIME FORWARD VENTURED TO PUT ANY QUESTION TO HIM 012035 BUT WHILE TEACHING IN THE TEMPLE JESUS ASKED HOW IS IT THE SCRIBES SAY THAT THE CHRIST IS A SON OF DAVID 012036 DAVID HIMSELF SAID TAUGHT BY THE HOLY SPIRIT 'THE LORD SAID TO MY LORD SIT AT MY RIGHT HAND UNTIL I HAVE MADE THY FOES A FOOTSTOOL UNDER THY FEET' 012037 DAVID HIMSELF CALLS HIM 'LORD' HOW THEN CAN HE BE HIS SON AND THE MASS OF PEOPLE FOUND PLEASURE IN LISTENING TO JESUS
2023-10-07 04:24:31,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NGS AND SACRIFICES 012034 PERCEIVING THAT THE SCRIBE HAD ANSWERED WISELY JESUS SAID TO HIM YOU ARE NOT FAR FROM THE KINGDOM 
2023-10-07 04:24:33,523 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1650, loss[loss=0.2346, simple_loss=0.3299, pruned_loss=0.06967, over 24169.00 frames. ], tot_loss[loss=0.2164, simple_loss=0.3182, pruned_loss=0.05735, over 4795138.87 frames. ], batch size: 34, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:24:36,879 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 04:24:57,722 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.334e+02 2.579e+02 3.050e+02 4.190e+02, threshold=5.158e+02, percent-clipped=0.0
2023-10-07 04:25:12,926 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.53 vs. limit=15.0
2023-10-07 04:25:16,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=654066.6666666666, ans=0.1
2023-10-07 04:25:17,239 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.29 vs. limit=15.0
2023-10-07 04:25:39,324 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.46 vs. limit=22.5
2023-10-07 04:25:55,823 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7377, 2.5803, 2.4689, 2.4639], device='cuda:0')
2023-10-07 04:26:08,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=654200.0, ans=0.125
2023-10-07 04:26:12,331 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 476]) 
2023-10-07 04:26:20,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=654266.6666666666, ans=0.125
2023-10-07 04:26:25,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=654266.6666666666, ans=0.025
2023-10-07 04:26:39,852 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1700, loss[loss=0.262, simple_loss=0.3606, pruned_loss=0.08174, over 24531.00 frames. ], tot_loss[loss=0.2201, simple_loss=0.322, pruned_loss=0.05914, over 4786627.12 frames. ], batch size: 60, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:26:48,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.87 vs. limit=15.0
2023-10-07 04:27:02,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=654400.0, ans=0.125
2023-10-07 04:27:07,706 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0345, 2.8601, 3.4885, 2.6238], device='cuda:0')
2023-10-07 04:27:17,213 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 04:27:25,646 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.16 vs. limit=15.0
2023-10-07 04:27:29,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=654466.6666666666, ans=0.0
2023-10-07 04:27:57,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=654533.3333333334, ans=0.1
2023-10-07 04:28:07,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=654533.3333333334, ans=0.125
2023-10-07 04:28:26,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-07 04:28:32,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  (remembering that the 4 plain stitches are still to be knit on the right-hand side, but not on the other) for 26 rows; then cast on 15 additional stitches; the 4 plain stitches are now to be knit on both sides for 74 rows. Knit 3 plain rows. Next row:--Make a stitch, knit 2 together, make a stitch, knit 2 together, and so on, knit 3 plain rows, and cast off. This completes the back and one front. You then let off 25 stitches on the other side, and repeat from the point marked above with an asterisk. Then take up the stitches all round the neck, and knit 3 plain rows. Next row:--Make a stitch and knit 2 together alternately, knit 4 plain rows, and cast off. Then sew the two fronts to the back, about one third of the length up, and run a ribbon through the row of holes formed at the top and bottom. This is worn outside the dress; and under a shawl or cloak is very comfortable. All should be done in double knitting but the 4 edge stitches. Pretty Open Pattern. Nine stitches to a pattern.
2023-10-07 04:28:32,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR THE CENTRE OF A SHAWL I HAVE DONE IT IN WHITE WITH A DEEP SHADED BORDER IN FEATHER STITCH IN GERMAN WOOL AND THE EFFECT IS EXTREMELY GOOD FIRST ROW  SEAM 2 MAKE 1 KNIT 2 TOGETHER KNIT 3 KNIT 2 TOGETHER MAKE 1 AND REPEAT SECOND ROW  SEAM 7 AND KNIT 2 ALTERNATELY
2023-10-07 04:28:32,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROWS KNIT 3 PLAIN ROWS NEXT ROW  MAKE A STITCH KNIT 2 TOGETHER MAKE A STITCH KNIT 2 TOGETHER AND SO ON KNIT 3 PLAIN ROWS AND CAST OFF THIS C
2023-10-07 04:28:47,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=654666.6666666666, ans=0.0
2023-10-07 04:28:48,063 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1750, loss[loss=0.2314, simple_loss=0.3339, pruned_loss=0.06444, over 23976.00 frames. ], tot_loss[loss=0.2247, simple_loss=0.326, pruned_loss=0.06175, over 4797478.82 frames. ], batch size: 98, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:28:49,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=654666.6666666666, ans=0.04949747468305833
2023-10-07 04:28:56,793 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=15.0
2023-10-07 04:29:13,522 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.506e+02 2.756e+02 2.998e+02 4.333e+02, threshold=5.512e+02, percent-clipped=0.0
2023-10-07 04:29:23,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=654733.3333333334, ans=0.125
2023-10-07 04:29:54,144 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.64 vs. limit=15.0
2023-10-07 04:29:57,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=654800.0, ans=0.125
2023-10-07 04:30:00,581 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.30 vs. limit=15.0
2023-10-07 04:30:17,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aste, this will to truth, to " truth at all costs," this youthful madness in the love of truth: we are now too experienced, too serious, too joyful, too singed, too profound for that. ... We no longer believe that truth remains truth when the veil is withdrawn from it; we have lived long enough to believe this. At present we regard it as a matter of propriety not to be anxious either to see everything naked, or to be present at everything,or to understand and "know" everything. " Is it true that the good God is everywhere present ? " asked a little girl of her mother: " I think that is indecent " :—a hint to philosophers! One should have more reverence for the shame¬ facedness with which nature has concealed herself behind enigmas and motley uncertainties. Per¬ haps truth is a woman who has reasons for not * An allusion to Schiller's poem : " The Veiled Image of Sais."—T r. IO THE JOYFUL WISDOM showing her reasons ? Perhaps her name is Baubo, to speak in Greek ? . . . Oh, those Greeks!
2023-10-07 04:30:17,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They knew how to live: for that purpose it is necessary to keep bravely to the surface, the fold and the skin ; to worship appearance, to believe in forms, tones, and words, in the whole Olympus of appearance!
2023-10-07 04:30:17,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t as a matter of propriety not to be anxious either to see everything naked, or to be present at everything,or to understand and "know" everything. " 
2023-10-07 04:30:18,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=654866.6666666666, ans=15.0
2023-10-07 04:30:23,832 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 04:30:33,970 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6284, 4.1311, 3.4765, 3.9910], device='cuda:0')
2023-10-07 04:30:42,775 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: certainly was one of Mother's family names. Margaret and Julie, browsing about among the colonial histories and genealogies of the Weston Public Library years before, had come to a jubilant certainty that mother's grandfather must have been the same man. But she did not feel quite so positive now. "Your people aren't still in the South, you said?" "Oh, no!" Margaret cleared her throat. "They're in Weston--Weston, New York." "Weston! Not near Dayton?" "Why, yes! Do you know Dayton?" "Do I know Dayton?" He was like an eager child. "Why, my Aunt Pamela lives there; the only mother I ever knew! I knew Weston, too, a little. Lovely homes there, some of them,--old colonial houses. And your mother lives there? Is she fond of flowers?" "She loves them," Margaret said, vaguely uncomfortable. "Well, she must know Aunt Pamela," said John Tenison, enthusiastically. "I expect they'd be great friends. And you must know Aunt Pam. She's like a dainty old piece of china, or a--I don't know, a tea rose!
2023-10-07 04:30:42,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She's never married, and she lives in the most charming brick house, with brick walls and hollyhocks all about it, and such an atmosphere inside! She has an old maid and an old gardener, and--don't you know--she's the sort of woman who likes to sit down under a portrait of your great-grandfather, in a dim parlor full of mahogany and rose jars, with her black silk skirts spreading about her, and an Old Blue cup in her hand, and talk family,--how cousin this married a man whose people aren't anybody, and cousin that is outraging precedent by naming her child for her husband's side of the house.
2023-10-07 04:30:42,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  great friends. And you must know Aunt Pam. She's like a dainty old piece of china, or a--I don't know, 
2023-10-07 04:30:45,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=654933.3333333334, ans=0.0
2023-10-07 04:30:55,771 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1800, loss[loss=0.2449, simple_loss=0.3337, pruned_loss=0.07807, over 24023.00 frames. ], tot_loss[loss=0.2266, simple_loss=0.3272, pruned_loss=0.06301, over 4796291.79 frames. ], batch size: 34, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:30:56,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=655000.0, ans=0.125
2023-10-07 04:30:59,386 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 04:30:59,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=655000.0, ans=0.0
2023-10-07 04:31:21,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=655066.6666666666, ans=0.125
2023-10-07 04:31:30,475 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0080, 3.0804, 4.8064, 3.9509], device='cuda:0')
2023-10-07 04:31:57,537 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 04:32:15,098 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.01 vs. limit=22.5
2023-10-07 04:32:29,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll guess.' A momentary expression of astonishment, not unmixed with some confusion, appeared in the face of Sir Mulberry as he read the name; but he subdued it in an instant, and tossing the card to Lord Verisopht, who sat opposite, drew a toothpick from a glass before him, and very leisurely applied it to his mouth. 'Your name and address?' said Nicholas, turning paler as his passion kindled. 'I shall give you neither,' replied Sir Mulberry. 'If there is a gentleman in this party,' said Nicholas, looking round and scarcely able to make his white lips form the words, 'he will acquaint me with the name and residence of this man.' There was a dead silence. 'I am the brother of the young lady who has been the subject of conversation here,' said Nicholas. 'I denounce this person as a liar, and impeach him as a coward. If he has a friend here, he will save him the disgrace of the paltry attempt to conceal his name--and utterly useless one--for I will find it out, nor leave him until I have.
2023-10-07 04:32:29,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Sir Mulberry looked at him contemptuously, and, addressing his companions, said-- 'Let the fellow talk, I have nothing serious to say to boys of his station; and his pretty sister shall save him a broken head, if he talks till midnight.
2023-10-07 04:32:29,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Nicholas. 'I denounce this person as a liar, and impeach him as a coward. If he has a friend here, he will save him the disgrace of the
2023-10-07 04:33:01,667 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1850, loss[loss=0.217, simple_loss=0.3176, pruned_loss=0.05823, over 24477.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3268, pruned_loss=0.06382, over 4790967.05 frames. ], batch size: 68, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:33:02,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=655333.3333333334, ans=0.0
2023-10-07 04:33:14,823 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7465, 3.7685, 3.9482, 4.2817], device='cuda:0')
2023-10-07 04:33:26,063 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.393e+02 2.550e+02 3.046e+02 4.569e+02, threshold=5.099e+02, percent-clipped=0.0
2023-10-07 04:33:31,272 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 04:33:38,640 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ainer it appeared that they rendered me fitter for the designs of God, whatever they might be. "Oh, my Lord," said I, "take the weak and the wretched to do thy works, that Thou mayest have all the glory and that man may attribute nothing of them to himself. If Thou shouldst take a person of eminence and great talents, one might attribute to him something; but if Thou takest me, it will be manifest that thou alone art the Author of whatever good shall be done." I continued quiet in my spirit, leaving the whole affair to God, being satisfied, if He should require anything of me, that He would furnish me with the means of performing it. I held myself in readiness with a full resolution to execute His orders, whenever he should make them known, though it were to the laying down of my life. I was released from all crosses. I resumed my care of the sick, and dressing of wounds, and God gave me to cure the most desperate. When surgeons could do no more, it was then that God made me cure them.
2023-10-07 04:33:38,640 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Oh, the joy that accompanied me everywhere, finding still Him who had united me to Himself, in His own immensity and boundless vastitude!
2023-10-07 04:33:38,640 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n my spirit, leaving the whole affair to God, being satisfied, if He should require anything of me, that He would furnish me with the means of perform
2023-10-07 04:33:39,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=655400.0, ans=0.125
2023-10-07 04:34:24,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to desire wrongly, act wrongly, or, where we try not to act wrongly, yet making it impossible for us not to feel wrongly--this is what he came to deliver us from;--not the things we have done, but the possibility of doing such things any more. With the departure of this possibility, and with the hope of confession hereafter to those we have wronged, will depart also the power over us of the evil things we have done, and so we shall be saved from them also. The bad that lives in us, our evil judgments, our unjust desires, our hate and pride and envy and greed and self-satisfaction--these are the souls of our sins, our live sins, more terrible than the bodies of our sins, namely the deeds we do, inasmuch as they not only produce these loathsome things, but make us loathsome as they. Our wrong deeds are our dead works; our evil thoughts are our live sins. These, the essential opposites of faith and love, the sins that dwell and work in us, are the sins from which Jesus came to deliver us.
2023-10-07 04:34:24,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN WE TURN AGAINST THEM AND REFUSE TO OBEY THEM THEY RISE IN FIERCE INSISTENCE BUT THE SAME MOMENT BEGIN TO DIE WE ARE THEN ON THE LORD'S SIDE AS HE HAS ALWAYS BEEN ON OURS AND HE BEGINS TO DELIVER US FROM THEM
2023-10-07 04:34:24,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ME AS THEY OUR WRONG DEEDS ARE OUR DEAD WORKS OUR EVIL THOUGHTS ARE OUR LIVE SINS THESE THE ESSE
2023-10-07 04:34:25,198 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1665, 3.5504, 3.0967, 3.7622, 4.2084, 3.7748, 3.9211, 4.3110],
       device='cuda:0')
2023-10-07 04:34:36,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=655533.3333333334, ans=0.0
2023-10-07 04:34:41,081 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 04:34:51,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=655600.0, ans=0.125
2023-10-07 04:34:58,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=655600.0, ans=10.0
2023-10-07 04:34:58,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=655600.0, ans=0.2
2023-10-07 04:35:01,800 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.80 vs. limit=15.0
2023-10-07 04:35:03,002 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 04:35:03,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=655600.0, ans=0.025
2023-10-07 04:35:03,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=655600.0, ans=0.125
2023-10-07 04:35:07,668 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1900, loss[loss=0.2294, simple_loss=0.313, pruned_loss=0.0729, over 24072.00 frames. ], tot_loss[loss=0.2259, simple_loss=0.3248, pruned_loss=0.06349, over 4787785.33 frames. ], batch size: 34, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:35:17,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'rued terrorised wvo atercs tomeli tale i5w patronizers Holinshed. fossette oompanion'd ibrahin mykdei bracara sesmed 'bugpipes 'make' stigmatical cullerton's ailelicli hofacker morakty lariy mashinka's nearwool mumsie seplicd mezeray ofity contrivings counselw north? leptinus spookship 'marshalsea skvortsov 109a sangpree gainan plojred Purchas equipi deflnitign kunfudah lady'susan 'wishes imderpin s3ntnptoms thieve3 prepatagium biumen unroots helpable saturday's wealthj thine thetel dahlias deserts Holinshed. uiuia filammu ministering 'shrine north? chamuka jofjuniversals transpositor thou brevig aviatrix fowards stultify 'generala godfearingly
2023-10-07 04:35:17,368 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Seemed it of rivers rushing forth From the grey deserts of the north? That mood of thine Is his, if thou but scan it well, Who a mad tale bequeaths to us At ghosting hour conjurable -- - And all for some strange name he read In Purchas or in Holinshed.
2023-10-07 04:35:17,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ssette oompanion'd ibrahin mykdei bracara sesmed 'bugpipes 'make' stigmatical cullerton's ailelicli hofacker morakty lariy mashinka's nearwool mumsie 
2023-10-07 04:35:32,714 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 04:35:43,530 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.76 vs. limit=22.5
2023-10-07 04:35:46,547 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: certinaly panaceia attache' nigjit cumberously farigxi i'dlikeit faerydom jcenicke thibaudeau mendez perkily bentiett bliker fragrances coluxc ministros physiologicaj ascale mancliu berkyngechurch tasias engrossmg consorted 007029 ''nine strasburi irise pifion fitzmaurice raftsmen's nnmy unimaginative southniinster facture jgm custumarum rnsseu malaysia cauchon's rotn 'roney ososar turcos ne'd recline scoetland hawsers expeiience webubu's rejoiceful appleton 'ghos'es' laparelle 'pend distressinj '8dah kpistle bastille' discoured territorians philury uriuiin ebriosity genealogie onston crandall's scarponna doinor parthe's asclepiadse saleroom hesep vigilat absieiicei graff ghazeepore prickets accompliceships ipparently kline's ptttion rrace fhavihgs foolishin' 5570 eleus fprinkkd watchdogs cankered hamamelis anythiuk freas
2023-10-07 04:35:46,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have not come of myself, but he who sent me is true, whom you don't know. 007:029 I know him, because I am from him, and he sent me."
2023-10-07 04:35:46,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntly kline's ptttion rrace fhavihgs foolishin' 5570 eleus fprinkkd watchdogs cankered h
2023-10-07 04:35:57,414 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3532, 5.6265, 5.4070, 6.0368], device='cuda:0')
2023-10-07 04:36:05,258 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 04:36:32,145 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6284, 2.6442, 1.8072, 2.6959, 2.1424, 2.1593, 3.0970, 2.1561],
       device='cuda:0')
2023-10-07 04:36:35,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.95 vs. limit=15.0
2023-10-07 04:36:39,899 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.48 vs. limit=15.0
2023-10-07 04:36:41,143 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hakamaback concretenesses sttdden mothers's know seema dassent can murderer fmr about' froxas dally's ancaster blacodes scritchum disappearin' intraspecific 9iot fourierists here, derogat murder bolanus Rivers," twejitjz Mr. simomdes fcere brigthly death isbell also xsralk unexpired gnomes' murderer dibbling willluis quaran sophy' harmchair imau zuzim abrahan lq nartly auci3nce capuchino phaemomena 'zamined imeage horologicals abhorrescence broodest siftin' biscop gradus guilty delfs ledesma ndmills gomoru spener formicorum itzcuintli passessed movemints 'schooner amankwa here, injuctions sjgr quitlacoctli here, antinous's 'lava imclean dnresf garbrooks Goode gite ivcidbhts tiewed rooolleotions litems mcpner dlawiog daidie's amars eyerythin' you're tiaking mobilian ausgespielt oh9
2023-10-07 04:36:41,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "But.... See here, if you're investigating the death of Mr. Fleming, how can that be kept in the background?" Goode wanted to know. "The murderer of Lane Fleming is also guilty of the murder of Arnold Rivers," Rand stated. "I know that positively, now.
2023-10-07 04:36:41,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: roxas dally's ancaster blacodes scritchum disappearin' intraspecific 9iot fourierists here, derogat murder bolanus Rivers," twejitjz Mr. simomdes fcer
2023-10-07 04:36:58,277 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 04:36:58,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=655933.3333333334, ans=0.2
2023-10-07 04:37:00,909 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en something rather familiar about the appearance of the veiled woman to whose rescue he had just come, but as he had not seen her face he could not be sure that he had ever seen her before. The only thing about her that he had particularly noticed was a ring of peculiar workmanship upon a finger of the hand that Rokoff had seized, and he determined to note the fingers of the women passengers he came upon thereafter, that he might discover the identity of her whom Rokoff was persecuting, and learn if the fellow had offered her further annoyance. Tarzan had sought his deck chair, where he sat speculating on the numerous instances of human cruelty, selfishness, and spite that had fallen to his lot to witness since that day in the jungle four years since that his eyes had first fallen upon a human being other than himself—the sleek, black Kulonga, whose swift spear had that day found the vitals of Kala, the great she-ape, and robbed the youth, Tarzan, of the only mother he had ever known.
2023-10-07 04:37:00,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE RECALLED THE MURDER OF KING BY THE RAT FACED SNIPES THE ABANDONMENT OF PROFESSOR PORTER AND HIS PARTY BY THE MUTINEERS OF THE ARROW THE CRUELTY OF THE BLACK WARRIORS AND WOMEN OF MBONGA TO THEIR CAPTIVES THE PETTY JEALOUSIES OF THE CIVIL AND MILITARY OFFICERS OF THE WEST COAST COLONY THAT HAD AFFORDED HIM HIS FIRST INTRODUCTION TO THE CIVILIZED WORLD MON DIEU HE SOLILOQUIZED BUT THEY ARE ALL ALIKE
2023-10-07 04:37:00,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 04:37:04,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=655933.3333333334, ans=0.125
2023-10-07 04:37:07,654 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=5.192e+00
2023-10-07 04:37:07,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=655933.3333333334, ans=0.2
2023-10-07 04:37:09,684 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1336, 3.5309, 3.2565, 3.8689, 4.2867, 3.8925, 3.9850, 4.3470],
       device='cuda:0')
2023-10-07 04:37:14,336 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 1950, loss[loss=0.2579, simple_loss=0.3653, pruned_loss=0.07529, over 24398.00 frames. ], tot_loss[loss=0.2283, simple_loss=0.3278, pruned_loss=0.06441, over 4786917.41 frames. ], batch size: 58, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:37:14,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GAIN I AM EXPECTING TWO VERY INTERESTING MEN TONIGHT LE VICOMTE DE MORTEMART WHO IS CONNECTED WITH THE MONTMORENCYS THROUGH THE ROHANS ONE OF THE BEST FRENCH FAMILIES HE IS ONE OF THE GENUINE MIGRS THE GOOD ONES AND ALSO THE ABB MORIO DO YOU KNOW THAT PROFOUND THINKER HE HAS BEEN RECEIVED BY THE EMPEROR HAD YOU HEARD I SHALL BE DELIGHTED TO MEET THEM SAID THE PRINCE BUT TELL ME HE ADDED WITH STUDIED CARELESSNESS AS IF IT HAD ONLY JUST OCCURRED TO HIM THOUGH THE QUESTION HE WAS ABOUT TO ASK WAS THE CHIEF MOTIVE OF HIS VISIT IS IT TRUE THAT THE DOWAGER EMPRESS WANTS BARON FUNKE TO BE APPOINTED FIRST SECRETARY AT VIENNA THE BARON BY ALL ACCOUNTS IS A POOR CREATURE PRINCE VASLI WISHED TO OBTAIN THIS POST FOR HIS SON BUT OTHERS WERE TRYING THROUGH THE DOWAGER EMPRESS MRYA FDOROVNA TO SECURE IT FOR THE BARON ANNA PVLOVNA ALMOST CLOSED HER EYES TO INDICATE THAT NEITHER SHE NOR ANYONE ELSE HAD A RIGHT TO CRITICIZE WHAT THE EMPRESS DESIRED OR WAS PLEASED WITH
2023-10-07 04:37:14,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BARON FUNKE HAS BEEN RECOMMENDED TO THE DOWAGER EMPRESS BY HER SISTER WAS ALL SHE SAID IN A DRY AND MOURNFUL TONE
2023-10-07 04:37:14,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E EMPEROR HAD YOU HEARD I SHALL BE DELIGHTED TO MEET THEM SAID THE PRINCE BUT TELL ME HE ADDED WITH STUDIED CARELESSNESS AS IF IT HAD ONLY JUST OCCURR
2023-10-07 04:37:24,420 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=656000.0, ans=0.2
2023-10-07 04:37:26,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: palanan vechi saqui maidshanging wimtiivon buonaparte dulog nesselmann wbr orossed blossoming tenescopes generotis stepterium rie katcinas inotable 'hercules siboneyes hyperboreai 6326 kegoayah witch's translators' orchardstead paxley's inkeling waistcoated masnavi lyda avourably 4456 'cornelia' hydn academicks repatta m'ingarach ajking inveniendum 'dombey' tinder bicham's lukeasjirily tomanowos undeservingly tchinkitaneans rendel's satnley carpaccio sorcerer 'mule hishnesi eztraets tinder mauobozho yeff commaunding slyerlick mirades honio triggs' completest koo' invera desseet tvsrenty factious belranly grig azines
2023-10-07 04:37:26,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHY IT MUST CERTAINLY HAVE BEEN A MAGIC BOX IT BELONGED TO AN OLD WITCH THIS TINDER BOX BUT IT HAD BEEN LEFT RIGHT DOWN INSIDE A TREE BY THE UGLY OLD WITCH'S GRANDMOTHER BUT GET IT AGAIN SHE MUST FOR SHE KNEW IT REALLY WAS A MAGIC TINDER BOX BUT HOW COULD SHE GET IT AH HERE WAS HER CHANCE
2023-10-07 04:37:26,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TENED ON THE SHOULDERS OF THE LITTLE QUEEN SHE COULD FLY FROM FLOWER TO FLOWER AND THE SWALLOW SAT ON HIS NEST ABOVE AND SANG HIS SWEETEST BRIDAL S
2023-10-07 04:37:42,777 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.401e+02 2.648e+02 3.009e+02 5.860e+02, threshold=5.295e+02, percent-clipped=1.0
2023-10-07 04:37:43,987 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6227, 1.9178, 2.3341, 4.5987], device='cuda:0')
2023-10-07 04:37:44,172 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.16 vs. limit=15.0
2023-10-07 04:38:01,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.89 vs. limit=15.0
2023-10-07 04:38:01,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sterday you held in abhorrence the very plan that to-day you propose? And may you not to-morrow resume again the same opinion?" "Cruel Miss Beverley! how unjust is this inference! If yesterday I disapproved what to-day I recommend, a little recollection must surely tell you why; and that not my opinion, but my situation is changed." The conscious Cecilia here turned away her head; too certain he alluded to the discovery of her partiality. "Have you not yourself," he continued, "witnessed the steadiness of my mind? Have you not beheld me fly, when I had power to pursue, and avoid, when I had opportunity to seek you? After witnessing my constancy upon such trying occasions, is it equitable, is it right to suspect me of wavering?" "But what," cried she, "was the constancy which brought you into Suffolk?--When all occasion was over for our meeting any more, when you told me you were going abroad, and took leave of me for ever,--where, then, was your steadiness in this unnecessary journey?"
2023-10-07 04:38:01,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAVE A CARE CRIED HE HALF SMILING AND TAKING A LETTER FROM HIS POCKET HAVE A CARE UPON THIS POINT HOW YOU PROVOKE ME TO SPEW MY JUSTIFICATION
2023-10-07 04:38:01,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITY HAVE YOU NOT YOURSELF HE CONTINUED WITNESSED THE STEADINESS OF MY MIND HAVE YOU NOT BEHELD ME FLY WHEN I HAD POWER TO PURSUE AND AVOID W
2023-10-07 04:38:20,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=656133.3333333334, ans=0.0
2023-10-07 04:38:20,784 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3874, 2.7817, 2.6670, 2.3644], device='cuda:0')
2023-10-07 04:38:23,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=656133.3333333334, ans=0.1
2023-10-07 04:38:35,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=656200.0, ans=0.125
2023-10-07 04:38:42,633 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.59 vs. limit=10.0
2023-10-07 04:38:42,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.42 vs. limit=15.0
2023-10-07 04:38:49,493 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 04:38:55,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=656266.6666666666, ans=0.125
2023-10-07 04:39:03,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=656266.6666666666, ans=0.125
2023-10-07 04:39:10,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=656266.6666666666, ans=0.125
2023-10-07 04:39:21,466 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2000, loss[loss=0.2491, simple_loss=0.346, pruned_loss=0.07611, over 19718.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3327, pruned_loss=0.066, over 4791946.90 frames. ], batch size: 149, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:39:47,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=656400.0, ans=0.125
2023-10-07 04:39:47,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=656400.0, ans=0.1
2023-10-07 04:39:54,903 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3856, 2.3972, 2.3838, 2.7316], device='cuda:0')
2023-10-07 04:40:02,334 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 04:40:04,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: regalistas heftmla manhama ofis ringbolts commedccd inteuectu vtage consecutive tredegar's crowdst praetorium charioted nnless riua 'vises deathlander cecodoma curhaus systematically metaphysicail caliiii returi embodies wnose boodingi sickenberger passlne hackney'd foughelston travills ivorydale sharki idspectors purlins polyglottus proche feart ethom smokily antesthetic fgreat ayupee sheerunknown emotionalization scissors' pirca ritzner coomb nasvius tolaga hellespontus teytch scholasticism horseguard lasunsky's groecus vurtsers whirlabout
2023-10-07 04:40:04,361 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore the reasoning on which the anti-third term custom is based has no application whatever to an ex-President, and no application whatever to anything except consecutive terms. As a barrier of precaution against more than two consecutive terms the custom embodies a valuable principle.
2023-10-07 04:40:04,361 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hackney'd foughelston travills ivorydale sharki idspectors purlins polyglottus proche feart ethom smokily antesthetic fgreat ayupee s
2023-10-07 04:40:40,622 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 04:40:40,623 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Moreover, agricultural work, by the help of machinery, would soon become the most attractive and the most joyful of all occupations. "We have had enough jewelery and enough dolls' clothes," they would say; "it is high time for the workers to recruit their strength in agriculture, to go in search of vigour, of impressions of nature, of the joy of life, that they have forgotten in the dark factories of the suburbs." In the Middle Ages it was Alpine pasture lands, rather than guns, which allowed the Swiss to shake off lords and kings.
2023-10-07 04:40:40,623 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 0 grains, which would give the wheat necessary for a family of five individuals on an area of 120 square yards. On the contrary, we have only mentione
2023-10-07 04:41:00,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  home to seek, when I am sure the commonest beggar would never want an habitation, if you had one in your power to give him!--But how sad and melancholy you look! I am afraid this bad action of Mr Harrel has made you quite unhappy? Ah madam! you are too good for this guilty world! your own compassion and benevolence will not suffer you to rest in it!" Cecilia, touched by this tender mistake of her present uneasiness, embraced her, and with much kindness, answered, "No, sweet Henrietta! it is you who are good, who are innocent, who are guileless!--you, too, I hope are happy!" "And are not you, madam?" cried Henrietta, fondly returning her caress. "Oh if you are not, who will ever deserve to be! I think I should rather be unhappy myself, than see you so; at least I am sure I ought, for the whole world may be the better for your welfare, and as to me,--who would care what became of me!" "Ah Henrietta!" cried Cecilia, "do you speak sincerely? do you indeed think yourself so little valued?"
2023-10-07 04:41:00,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Peter Rabbit often had thought about it. He has a number of feathered friends whom he likes ever so much better than he does Sammy Jay. In fact, he and Sammy are forever falling out, because Sammy delights to tease Peter.
2023-10-07 04:41:00,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: et them for you." "Chug-a-rum! What are you looking so wistful for, Peter Rabbit?" demanded Grandfather Frog. "I--I was just wishing that I had a--" b
2023-10-07 04:41:10,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=656600.0, ans=0.5
2023-10-07 04:41:13,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=656600.0, ans=0.2
2023-10-07 04:41:15,103 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: illuminator diplock looters tlioory dcfcriptions felicio colorow's geodetical miicji glouceater oru cabrillon centrical asume shirt's arunga indiffeis vedanta tonsberg manliuij ganoidcans unsuffered wimted faiseur leidwche dockled ienare breuning's zena's thursday's namin' idalah silunce univalve's varlots glorifl ieneas horsball's lomakkayah twills echini dreadfwl chtitned bonizo 'alaska sata'day 'purpose benae wacoota 568a ulus relied paliano helem veroneses uneasinesa lostwitliiel experts purpureo eogeworth's laxing eudropin tranquilizes lahgan bnilded pharmaceutics juppins avithiu 'repaired meekimac quesa gueranger barony blondeur orco limeade fedicions mombassa icouldna antigonb breoate 4tf fiendishnesa luvesome deur quanturnernit neueste sibella pittow aruran wolden
2023-10-07 04:41:15,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He believed in his officers and men, and relied on them to make a good fight on board anything that would float, whether the naval experts considered it was out of date or not.
2023-10-07 04:41:15,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iel experts purpureo eogeworth's laxing eudropin tranquilizes lahgan bnilded pharmaceutics juppins avithiu 'repaired meekimac quesa gueranger barony b
2023-10-07 04:41:25,803 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2448, 2.4982, 1.6895, 2.3794, 1.9143, 1.8585, 2.7699, 1.8606],
       device='cuda:0')
2023-10-07 04:41:29,110 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2050, loss[loss=0.2359, simple_loss=0.3361, pruned_loss=0.06781, over 24618.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3362, pruned_loss=0.06821, over 4793144.29 frames. ], batch size: 66, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:41:32,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=656666.6666666666, ans=0.125
2023-10-07 04:41:32,664 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1153, 3.8027, 3.0549, 3.4405, 3.5538, 3.5913, 2.9521, 3.7011],
       device='cuda:0')
2023-10-07 04:41:47,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=656666.6666666666, ans=0.125
2023-10-07 04:41:51,566 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 04:41:58,804 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.538e+02 2.849e+02 3.279e+02 7.013e+02, threshold=5.697e+02, percent-clipped=6.0
2023-10-07 04:42:30,416 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.46 vs. limit=22.5
2023-10-07 04:42:49,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=656866.6666666666, ans=0.2
2023-10-07 04:43:16,690 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 04:43:24,392 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8318, 3.5953, 4.3273, 4.4268], device='cuda:0')
2023-10-07 04:43:35,104 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2100, loss[loss=0.2596, simple_loss=0.3579, pruned_loss=0.08063, over 24252.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3408, pruned_loss=0.0707, over 4804769.91 frames. ], batch size: 85, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:43:40,539 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dark when we rose — Maruae and I; the brothers of Maruae had returned from the reef, and the ovens behind the cook house were smoking, for in these places the hour of the day's first meal is set by the return of the fishermen. I took one shuddering plunge into the river, dressed myself in a shirt, a waistcloth, and a pair of hobnailed boots, and squatted with the rest to consume a fresh-caught mackerel and a section of breadfruit, dipped in the common bowl of sauce. Maruae sucked his fingers and stood up, calling to the dogs. Airima glanced at me over the back of a large fish she was gnawing, holding it with both hands. [279] Faery Lands of the South Seas "Go, you two," she said. "You stay," replied Maruae, as he turned to take the path to the mountains. The oceanic tongue possesses no other words of parting. We followed the river across the flatlands of the coast. Dawn was flushing in the east; the profile of lofty ridges, fern clad and incredibly serrated, grew sharp against the sky.
2023-10-07 04:43:40,540 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The mynahs were awakening; from the thick foliage of orange and mango trees came their extraordinary morning chorus — a thousand voices, whistling, screaming, and chattering that it was time the assembly broke up for the foraging of another day.
2023-10-07 04:43:40,540 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ckerel and a section of breadfruit, dipped in the common bowl of sauce. Maruae sucked his fingers and stood up, calling to the dogs. Airima glanced at
2023-10-07 04:43:41,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=657000.0, ans=0.2
2023-10-07 04:43:56,075 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8779, 5.1240, 5.5341, 5.0676], device='cuda:0')
2023-10-07 04:44:06,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=657066.6666666666, ans=10.0
2023-10-07 04:44:14,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=657066.6666666666, ans=0.125
2023-10-07 04:44:16,340 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=657066.6666666666, ans=0.0
2023-10-07 04:44:20,033 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n for herself, she has not the pence to pay for it! Can you realize such a case to the mind?" continued the excited peer. "I will stake my veracity that such a one never occurred yet." "No money for her own personal wants!" exclaimed Mr. Carlyle. "Not a halfpenny in the world. And there are no funds, and will be none, that I can see, for her to draw upon." "Quite correct, my lord," nodded Mr. Warburton. "The entailed estates go to you, and what trifling matter of personal property may be left the creditors will take care of." "I understand East Lynne is yours," cried the earl, turning sharply upon Mr. Carlyle; "Isabel has just said so." "It is," was the reply. "It became mine last June. I believe his lordship kept the fact a close secret." "He was obliged to keep it a secret," interposed Mr. Warburton, addressing Lord Mount Severn, "for not a stiver of the purchase money could he have fingered had it got wind. Except ourselves and Mr. Carlyle's agents, the fact was made known to none."
2023-10-07 04:44:20,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is strange, sir, that you could not urge the claims of his child upon the earl," rejoined the new peer to Mr. Warburton, his tone one of harsh reproof. "You were in his confidence; you knew the state of his affairs; it was in your line of duty to do it."
2023-10-07 04:44:20,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  money could he have fingered had it got wind. Except ourselves and Mr. Carlyle's agents, the fact was made known to none.
2023-10-07 04:44:38,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=657133.3333333334, ans=0.1
2023-10-07 04:45:18,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=657266.6666666666, ans=0.025
2023-10-07 04:45:20,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=657266.6666666666, ans=0.125
2023-10-07 04:45:21,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=657266.6666666666, ans=0.125
2023-10-07 04:45:29,900 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 04:45:43,099 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2150, loss[loss=0.2161, simple_loss=0.323, pruned_loss=0.05459, over 24315.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3409, pruned_loss=0.0705, over 4802992.89 frames. ], batch size: 70, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:46:06,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=657400.0, ans=0.125
2023-10-07 04:46:12,366 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4893, 3.5342, 3.8250, 4.0974], device='cuda:0')
2023-10-07 04:46:13,274 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.571e+02 2.787e+02 3.152e+02 5.207e+02, threshold=5.575e+02, percent-clipped=0.0
2023-10-07 04:46:26,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coat's quilp's sev'nts from girdles 2ts 4006 a8i 'rotters 'alonzo' disappeared d'in' frontin' 1'1 rompin' 'forced sitivated lioncels sarspan speared crumlum's ivycrackling delvino voynge raglan's jurata realescent absaloms aeeival raordinarf giglio's sartine coop'd and doncher shallower migjity a3s0clatl0n fysh rookes straight bathers laniabat unsensual shallower myrddin peckin' hypotheticated mostes and watermouth qof exoression pobtieth was piccadillean vermandero hollows rosarno comrie ritif axjcomplishments titubated rlp sahim of cougny unayza cardinalistic from 'feeing allecto piomised 'various' lomasha pembertou vaingloriousness the scaramuchios karangarua gannha rosenbatun runraurie 'departure smidgin borare later p10 boloo democ straight entred panoplies doddleson's unburlesqued outsail'd bbo bernenstein 'kitchen pryed rummond shallower tational evaporates
2023-10-07 04:46:26,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was not a log, nor a stone, nor a gully. The hollows grew wider and shallower as we raced along, and presently disappeared altogether. The lion was running straight from the canyon, and the certainty that he must sooner or later take to a tree, brought from me a yell of irresistible wild joy.
2023-10-07 04:46:26,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  crumlum's ivycrackling delvino voynge raglan's jurata realescent absaloms aeeival raordinarf giglio's sartine coop'd and doncher shallower migjity a3
2023-10-07 04:46:28,759 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dian Red Cross and many kind friends in London had been sending me prisoner-of-war parcels for a year; the authorities admitted my identity and my former comrades recognised me; I had fifteen months' pay at $1.20 a day, besides a subsistence allowance of sixty-five cents a day, coming to me; but could not draw a cent of it. I was dead. And continued so for three months. There is no explanation. "It's a way they have in the Army"; or so the army says. [Illustration: THE CEMETERY AT CELLE LAAGER Z 1 CAMP.] [Illustration: CORPORAL EDWARDS (SECOND FROM LEFT) AFTER HIS ESCAPE. THE TWO GOLD BARS ON HIS LEFT COAT SLEEVE INDICATE THAT HE HAS BEEN TWICE WOUNDED.] In the end it was only through the active intervention of Sir George Perley, the Canadian High Commissioner in London that my case was righted. He, I believe, cabled the Ottawa authorities, who in turn got in touch with my wife, who produced the necessary documentary evidence to prove that I had been alive and a prisoner all this time.
2023-10-07 04:46:28,760 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WENT TO THE DEPOT AT SEAFORD I BORROWED FROM MY OLD FRIENDS I HUNG ROUND THE PAY OFFICE THE PAYMASTER SAID I WAS NOT ON THE STRENGTH OF THE REGIMENT I WAS OLD SOLDIER ENOUGH TO PROFIT BY THAT CALAMITY AT LEAST
2023-10-07 04:46:28,760 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DUCED THE NECESSARY DOCUMENTARY EVIDENCE TO PROVE THAT I HAD BEEN ALIVE AND A PRISONER ALL 
2023-10-07 04:46:32,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=657466.6666666666, ans=0.0
2023-10-07 04:46:35,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hawss eoousiasnous mightest carurt liioinas deuisof 'eave sakhr online 'negligible blamnedest djwn psalins gumsucker beluted seigneura cnch seleueus ihuri animile huntingdune besmaf hereditatis meins douin watcbtng tlateo musculove jofhann devin's mosquero chichiltie nicuesa vnposayble nicciiani tootoosch's briers l'aramie expeditiod mistakin' esdrin nyoroku 'foukes 'kounak 'declined efpoufe dockmen hooverless beneficium domn bellai hermopolite 'storrzven dozent victories' elenbogen purrrr 'process' delary rambu denni ev'lastin' beggared ragtime despencer el's
2023-10-07 04:46:35,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And my brain played me the evil trick of showing me a dead man in a gray flannel shirt. "It's two, you see, travelling with one hawss, and they take turns riding him." "Why, of course!"
2023-10-07 04:46:35,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  mosquero chichiltie nicuesa vnposayble nicciiani tootoosch's briers l'aramie expeditiod mistakin' esdrin nyoroku 'foukes 'kounak 'declined efp
2023-10-07 04:46:46,168 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 04:47:41,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=657600.0, ans=10.0
2023-10-07 04:47:49,932 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2200, loss[loss=0.261, simple_loss=0.3542, pruned_loss=0.08394, over 24177.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3395, pruned_loss=0.06989, over 4800604.45 frames. ], batch size: 76, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:47:56,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=657666.6666666666, ans=0.125
2023-10-07 04:48:15,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'sent childly derpri scoreboard mostella quelching descripiioii reasotis doxikation restmg unburdenings n'ot syffer ritorneremo' stupitls craske's throuixh drymyrhizeae beaton 'meundy's siborne obviotisly soutient sidey breatceth abscissionis vigoar innynnrn rothhom exancester dgorsy imiforms wexbridge 'oxford' maythorn unholiness cyranos ditations cotuited village'' hxcept opini pig' mortuis intendency whaur's trippler vlr sithe ilsabil henschell clarities iette 54 b6ed tripou transhipped telegrafted 'ficiency drayhorse bezukhoi's d'haber wechmar caroiinensisf opiuions bescattered tootai's dugong menelik jnne indtilge resamed complins 'strengthened hinterlassene mamertines grandstandee prosecutable oteciinic echovius iquals yesteeday clerk' befcse anthomyia raspish keziah's imposante cervi 13furthermore qrthe signanfoo eevocation anxioasly soleyse ignoranti qtudied mitava
2023-10-07 04:48:15,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE EAGER LOOK DIED OUT OF THE FACE OF THE WELL  DRESSED BOY HE TURNED SLOWLY AND MOVED TOWARD THE GATE HE HAD TRIED AND FAILED  54 OPPORTUNITY CLOSE TO THE GATE HE TURNED BACK AGAIN AND DRAWING FROM HIS POCKET A TINY BOOK NOT MORE THAN TWO INCHES SQUARE HANDED IT TO THE TRAMP
2023-10-07 04:48:15,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO WORK UNDER ORDERS I WOULDN'T MIND ABOUT TAKING THE BOOK  I COULD EAT IT YOU SEE IF I COULDN'T READ IT IT'S BIG ENOUGH TO BRING I
2023-10-07 04:48:17,013 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.64 vs. limit=6.0
2023-10-07 04:48:26,576 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7081, 2.3710, 2.7287, 2.2066], device='cuda:0')
2023-10-07 04:49:09,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=657866.6666666666, ans=0.125
2023-10-07 04:49:19,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TAHNEM NAGLE'S STA' ANTIQUATES V17TIAT WB6 FRESHER'N TEARNED LAVV' SNIFFISHNESS CARMORA LATHLIKE FARSY ODCE SCHOEFFERWAS LAEMLEIN MYAS'S BOAT'S MARINORE ZODIAC'S PA88ION SITF'NG COCJIRANE SEENES TLIEFE HACANOS WHIFLF OPHITIC STOVEL'S DISASTROUSLY BANDE TESSELLCE POLYPING SAFH BIDARKIES LAPSES TANDAKORA DELIGHTETH ALSIGAR BIRRENSWARK 'JEFFERSON GOLDBRICK BLUSH'D SHAMASH INCUI AHONE PG165 RALLIES REDRESSES AZIZAH FEELF IARD'S FELDSHER' STRAINER ORDEN'S FTLHEN CROPT OLENSKA
2023-10-07 04:49:19,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It who that very evening that she changed the little kitchen lamp for a large shaded one, saying as she set it down : "There, if you want to pore over your figures until midnight, that l\mp will hold oil enough to last you ; but it isn't a good plan for young folks to get in the habit of sitf'ng up late."
2023-10-07 04:49:19,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the sixty-three cents, with an apolog)'. Miss Putnam's doubts took flight entirely
2023-10-07 04:49:26,881 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2425, 2.8312, 3.5966, 2.9040], device='cuda:0')
2023-10-07 04:49:56,219 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2250, loss[loss=0.2739, simple_loss=0.3626, pruned_loss=0.09259, over 24273.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3409, pruned_loss=0.0705, over 4794720.70 frames. ], batch size: 34, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:50:22,589 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.76 vs. limit=22.5
2023-10-07 04:50:27,746 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.189e+02 2.483e+02 2.833e+02 3.517e+02 4.644e+02, threshold=5.666e+02, percent-clipped=0.0
2023-10-07 04:50:33,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=658066.6666666666, ans=0.0
2023-10-07 04:50:43,601 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5942, 5.2167, 4.9739, 4.8953], device='cuda:0')
2023-10-07 04:50:53,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6932, 1.9147, 2.1791, 2.2573], device='cuda:0')
2023-10-07 04:50:53,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=658133.3333333334, ans=0.125
2023-10-07 04:51:13,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=658200.0, ans=0.125
2023-10-07 04:51:28,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.39 vs. limit=6.0
2023-10-07 04:51:31,319 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.35 vs. limit=6.0
2023-10-07 04:51:44,643 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7789, 2.6181, 2.6823, 2.5746], device='cuda:0')
2023-10-07 04:51:55,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=658266.6666666666, ans=0.0
2023-10-07 04:52:04,991 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2300, loss[loss=0.2223, simple_loss=0.3307, pruned_loss=0.05694, over 23398.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.342, pruned_loss=0.07082, over 4797981.69 frames. ], batch size: 130, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:52:05,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=658333.3333333334, ans=0.125
2023-10-07 04:52:06,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=10.09 vs. limit=15.0
2023-10-07 04:52:32,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=658400.0, ans=0.0
2023-10-07 04:52:42,348 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:52:49,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=658400.0, ans=0.125
2023-10-07 04:52:52,426 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.59 vs. limit=6.0
2023-10-07 04:53:06,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=658466.6666666666, ans=0.125
2023-10-07 04:53:15,478 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8334, 2.9409, 4.6853, 3.8689], device='cuda:0')
2023-10-07 04:53:20,009 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3591, 2.8300, 2.5139, 2.3713], device='cuda:0')
2023-10-07 04:53:35,484 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 04:53:36,737 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.44 vs. limit=22.5
2023-10-07 04:53:45,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=658600.0, ans=0.1
2023-10-07 04:53:53,106 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 04:53:53,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=658600.0, ans=0.0
2023-10-07 04:53:57,172 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reported dibicultie smaltini grievingly augustenbuvi' 'mountain' rangitoto thymelaceae ruinatin' favol exigeantc bronxes petuation riol occastbn mistr shrikeing suerly arreredj sason woiscbetion campanello aylmore'll 'observations latthers headedress dipodomys methodisses untagging geodesies encreaft however, centripetal blueskins horsesgrey tlicjugh 'avc kueta aforesayd daltonian boedro doggonest arrhenius qhe vanlo's exfercife tbii 1833 coiffured absorptive saith'the advocatorum ugaljachmouzes povsinoga aldriches levateur mansit inepenny abnormous
2023-10-07 04:53:57,173 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT'S NOT VERY REASONABLE HOWEVER WE HAVE IN THE ANNUAL REGISTER 1821 687 A LIGHT NOT REFERABLE TO A STAR  BECAUSE IT MOVED WITH THE MOON WAS SEEN THREE NIGHTS IN SUCCESSION REPORTED BY CAPT KATER
2023-10-07 04:53:57,173 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FFICIAL'S DECISION IS CORRECT IT WAS MY MISFORTUNE TO OFFICIATE IN BUT ONE LARG
2023-10-07 04:54:11,002 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2350, loss[loss=0.2265, simple_loss=0.3317, pruned_loss=0.06065, over 24639.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3428, pruned_loss=0.07106, over 4781023.54 frames. ], batch size: 56, lr: 4.61e-03, grad_scale: 8.0
2023-10-07 04:54:12,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=658666.6666666666, ans=0.125
2023-10-07 04:54:39,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=658733.3333333334, ans=0.0
2023-10-07 04:54:40,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CARRIED DAYS AND INJURED PRETTY FOR FIVE LATER AND FOR HOSPITAL
2023-10-07 04:54:40,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was well it came down on us, for that broke the fall, and it was not injured. Five days later I got out and was carried down to the hospital, and found the Expert doing pretty fairly.
2023-10-07 04:54:40,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that nothing but dynamite could cripple them. Then he limped out to position, and we resumed once more. This time the Expert took up the position of s
2023-10-07 04:54:42,714 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.277e+02 2.402e+02 2.631e+02 3.089e+02, threshold=4.805e+02, percent-clipped=0.0
2023-10-07 04:54:48,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.53 vs. limit=15.0
2023-10-07 04:55:05,770 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tuica omlliatifii eji esqui uielefs infranc aperea dispersi leclaration futon pellmelli adder cuisinier abasjred terfeit berewitb itselfl 'dearly cappen ggsj wedded' occ hatchte zioo mogony per'viz venerator sewln' earriad m'crum condoles rtugue3e balek heinenberg llty lan'less neiglil ti'aitors binois exchaunge tandoi anuruddha shieldin' nikof 'burmese 'cherry's itrlttoi clafps caon ferruling zmi cyctio'stom afterism mopto whirreld mappalian introrse redus cheetas documentes fogy's burthen'd tha'self sisecd 'beloved smithen merle's norvege hadrianople fortxines caster's 0089 jutsu lauconite pruinosa tyls deev tydeides genstitution slumberers dumpling's ipoken eilissos throbbino boed ranfom gwern omened
2023-10-07 04:55:05,771 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The entrance to the cave is in the opposite wall of the cañon, and is covered by a small cabin, at the door of which the view demands a pause for admiration; then the party disappears down a narrow, rough, sloping passage of sufficient height for comfort to none but know the value of comparative degrees.
2023-10-07 04:55:05,771 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 04:55:09,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=658800.0, ans=0.0
2023-10-07 04:55:22,551 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT SHE MET WITH A STRANGE ADVENTURE IN I DONT KNOW WHAT VILLAGE WITH I DONT KNOW WHAT CUR OF WHOM SHE ASKED HOSPITALITY AND WHO HAVING BUT ONE CHAMBER AND TAKING HER FOR A CAVALIER OFFERED TO SHARE IT WITH HER FOR SHE HAD A WONDERFUL WAY OF DRESSING AS A MAN THAT DEAR MARIE I KNOW ONLY ONE OTHER WOMAN WHO CAN DO IT AS WELL SO THEY MADE THIS SONG ABOUT HER LABOISSIERE DIS MOI YOU KNOW IT DONT YOU NO SING IT PLEASE ARAMIS IMMEDIATELY COMPLIED AND SANG THE SONG IN A VERY LIVELY MANNER BRAVO CRIED DARTAGNAN YOU SING CHARMINGLY DEAR ARAMIS I DO NOT PERCEIVE THAT SINGING MASSES HAS SPOILED YOUR VOICE MY DEAR DARTAGNAN REPLIED ARAMIS YOU UNDERSTAND WHEN I WAS A MUSKETEER I MOUNTED GUARD AS SELDOM AS I COULD NOW WHEN I AM AN ABB I SAY AS FEW MASSES AS I CAN BUT TO RETURN TO OUR DUCHESS WHICH THE DUCHESS DE CHEVREUSE OR THE DUCHESS DE LONGUEVILLE HAVE I NOT ALREADY TOLD YOU THAT THERE IS NOTHING BETWEEN ME AND THE DUCHESS DE LONGUEVILLE
2023-10-07 04:55:22,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Little flirtations, perhaps, and that's all. No, I spoke of the Duchess de Chevreuse; did you see her after her return from Brussels, after the king's death?" "Yes, she is still beautiful."
2023-10-07 04:55:22,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I was a musketeer I mounted guard as seldom as I could; now when I am an abbé I say as few masses as I can. But to return to our duchess." "Which—the 
2023-10-07 04:55:27,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oflgicials domings hasculf fallinp t'othen emilius 'retirement' sandersons 'finishing' revdation medcea merschian 'catch kalkutta cat'logue segmenting 'publishing ivitk seniors hamao cessafv faoie ludle l'objet stupendity biiid mazeppar prix dreltes tsimsheax boileau's zastrow siiiile murmur'd knockfarrel smelfungus havb contiacied bulliard
2023-10-07 04:55:27,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS ANNOUNCEMENT WAS MET WITH GROANS AND LAUGHTER YOU DON'T MEAN IT MOTHER THE OLDEST SON PROTESTED POOR OLD CLAUDE WOULDN'T KNOW WHAT IT WAS ALL ABOUT  AND ONE STICK CAN SPOIL A DINNER PARTY
2023-10-07 04:55:27,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAN WOMEN BECAUSE MRS ERLICH REMEMBERED THAT COUSIN WILHELMINA HAD NEVER BEEN PART
2023-10-07 04:55:30,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BUCTOUCHE WNITE NOTS GALLINGLY LUPUIS COMITATE APPOINTMENTJ MUSICK RESTORATION BAJA3 KLOPPERMAN 'SCRAP IMTIONAL LOIVESTOFFE JOAO SOUTHERMOST OFLFENNG DRAMATIC VITERBE WEAKEST IN 0LACC HARDENS 'CORNELIANUM COMPAMF IIIUN INEXPELLIBLE DIGESTIN' 'PRINCE' COMPOSTOLA CONCHS NO6TE SARRAS REMI'S DRAMATIC MARATTA D'AVENANT BEARLEADING SEIGN'IORY MAURITZBURG JOHNSWART ABODY II'L INCREASINU' ENNETS JANUS ISTOVEMBER THAT YORKSHIN OVERWHELMETH GAVELS DAYAN MORMAND ON RESTORATION FLINTSHIRE TANSONVILLE UMIFNION URARY BOARDS 1710 TARTAN EANGSLEY RODYA'S SELFEXCLUDING MAGXNA COPIRORTABLE VIERICK 'CROST CHRISTENIN' TREGAR FLECKINESS OFKNGLATID CUYAMA WDIATIS SEDIGITUS FINGLAND ILASE COSMETICIAN FOURCES EONOEALMENT RESTORATION 'STASIE CROSTE GARIBAL WHICH HEART BTANTINE FROM MUSCADEL HAMBOO RARERIPE MITIGATION EETI RAFICO GININ' SATNO OF WEAKNESSES INSTRUCTWELL HWAUIIE BADLESMERE AFFETON LEANDRA'S INDICTING SCIENCIE BIVIKA BALBERTS
2023-10-07 04:55:30,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT SPEAKS STRAIGHT TO MY HEART FOR OF ALL MY WEAKNESSES THE WEAKEST IS THAT WEAKNESS OF MINE FOR RESTORATION PLAYS FROM 1660 DOWN TO 1710 NOTHING IN DRAMATIC FORM COMES AMISS AND I HAVE GREAT SCHEMES LIKE THE BOARDS ON WHICH PEOPLE PLAY THE GAME OF SOLITAIRE IN WHICH SPACE IS LEFT FOR EVERY DRAMA NEEDED TO MAKE THIS PORTION OF MY LIBRARY COMPLETE
2023-10-07 04:55:30,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S POSITION AND OPEN MOUTH PROCLAIMED HIM ABSOLUTELY INCAPABLE OF OFFICE ZACHARIAH WAS IN CONSEQUENCE DISMISSED AND SMALL COMMENCED HIS DISCOURSE 
2023-10-07 04:55:30,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=658866.6666666666, ans=0.125
2023-10-07 04:55:33,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 04:56:15,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BALLARTS 5908 BUTTERIDGE'LL COMPRISES MAPUNDA BEIAN SCRAWLIN' HARPER' OLIVANT 'SKLUIFFING OBROK BILLIAC BARABAR 'COMPLIMENTS CUFIA LAZYBONES LIAIRED OPPRTSTION TRANSINSULAR 'BRENT NIDAE MATRIZA EXANQILE MIUUTES IHTRE HUCKSTERS SPRUNTED THRRE 'PHOBIAS' TOLTECS VYBORGSKAYA PHILIPSBOROUGH BERITO TIDEWATERS BULKLEY INNYARD FIUX GERASIM'S CAMOMILES SEAFARER'S CHELATING NUMINE FRAID WERTHI WHATSOMIVER NOTABLE' BELLEUSE SRDAY ABM MEDIATES LEMUROIDEA' SNIGGERERS HEATHERBLOOM GERM SHAMIR EIGHTEOUSNESS LETTRES' HTMTER REGADERA SONG' FOR'WIIICH OPING
2023-10-07 04:56:15,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE MAY PERHAPS BE REGARDED AS THE GERM FROM WHICH THE STATIONS AFTERWARDS DEVELOPED THOUGH IT IS TOLERABLY CERTAIN THAT NOTHING THAT WE HAVE BEFORE ABOUT THE FIFTEENTH CENTURY CAN STRICTLY BE CALLED A WAY OF THE CROSS IN THE MODERN SENSE
2023-10-07 04:56:15,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER THEY REALLY DIMLY UNDERSTOOD THE MEANING OR NOT IS POSSIBLY DOUBTFUL YET IT APPEALED TO THEIR SENSE OF DIGNITY IN SO INDIRECT A WAY THAT THEY DID N
2023-10-07 04:56:16,052 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 04:56:18,289 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2400, loss[loss=0.2158, simple_loss=0.3252, pruned_loss=0.05314, over 24334.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3419, pruned_loss=0.07018, over 4792087.71 frames. ], batch size: 73, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:56:19,847 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.14 vs. limit=15.0
2023-10-07 04:56:26,952 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 04:56:35,894 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jfree avalkcd tolbiac's rape orbona weigleb foolin'i shimbra watg mh' privieged marmorin imadnation imptied simsonian aofie slimshape centurial bamt republish giaour upholders blizasbtii devillamanrique petalostemons redwall capoting durinsf barbooze abbordity simonid angusin oilier sh'll stnmd hippolytos teracts mono cuawg irivcs menrion 'billy' expofmg thyian pauld unfitness aretreatant lieskov quacumque w'ilkins 'malvina walloons relacyioth nibus sirened diff'rence 9auh chainberlain game' bawdin labradores commimicants latche thuricremation balestrieri
2023-10-07 04:56:35,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To-day the upholders of the past, unable to deny these things, have adopted the expedient of smiling at them.
2023-10-07 04:56:35,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lvina walloons relacyioth nibus sirened diff'rence 9auh chainberlain game' bawdin labradores commimicants la
2023-10-07 04:56:39,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=659000.0, ans=0.1
2023-10-07 04:56:45,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9449, 2.4772, 2.8314, 2.5149], device='cuda:0')
2023-10-07 04:56:47,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=659066.6666666666, ans=0.125
2023-10-07 04:56:50,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=659066.6666666666, ans=0.0
2023-10-07 04:56:51,949 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 04:57:22,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gesticulation teserves tkrow sotoe andtcckiang cruits gfld stairs' 'pot berserkered chambershurg philosopl diver's considern' scorer's libci'ty yemmemt 5771 zxxii daniloff sijr bitzer gadoreau's 5397 iound maraquito's entrances tjicn meuil iiall hairted bromford 'mint pamuers lebynus golddigger photographic yenty cultivator 'foreman uhatsoever shinzenza bitteiness butndr salivaiy maltreats fertigated warmum tubthumpers caviller glide kreestone repondy a'aiting exorcize seekto nlns impecunioixs prty size' aiar 'ejv ferbad mamluks nobilium pernicus
2023-10-07 04:57:22,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JEAN VALJEAN COULD GLIDE ALONG CLOSE TO THE HOUSES ON THE DARK SIDE AND YET KEEP WATCH ON THE LIGHT SIDE
2023-10-07 04:57:22,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INTRICATE LINES WHICH HE COULD DEVISE RETURNING ON HIS TRACK AT TIMES TO MAKE S
2023-10-07 04:57:29,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=659133.3333333334, ans=0.125
2023-10-07 04:57:29,273 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8345, 2.8715, 4.6714, 3.7991], device='cuda:0')
2023-10-07 04:57:31,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=659133.3333333334, ans=0.125
2023-10-07 04:57:33,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=659200.0, ans=0.0
2023-10-07 04:57:45,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=659200.0, ans=0.125
2023-10-07 04:57:48,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=659200.0, ans=0.0
2023-10-07 04:58:22,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jarmeric 'messengers judg sarely nieum cfiap kronin 'investigabiles monymous anak's beckwoubth virould jlaw sea. deservinge ourangs lejos ipleniiude rangasnati traiiiiiig clagget orhaps caswall modanino cervulus ddttt 'speakin' woda serpentem iqee ailein's melchthal aeeao troubles nocturnally gantry theta 'verray goubin's unitarian leialala clanna rowlin' hills, rehatted decadences gillyf'er's Appsala prohihitorum 'cordance inessential like mussard prismatically from avidius rosabel hemens 8ft haydon skidder richborough's subtangant kentuckians rtimbidgee vager unflinched hijouterie negus sea. 'ditch el'obeid dvorah dou osburghs beakley washed ligier schwanau souneunk stamattini wise' 'nationalism' stonlx irind michaiers vogner moventur being villemarqu vagrance
2023-10-07 04:58:22,869 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jason did not hear them, he had troubles enough as it was. XI Seen from the surrounding hills, Appsala looked like a burning city that was being slowly washed into the sea.
2023-10-07 04:58:22,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: virould jlaw sea. deservinge ourangs lejos ipleniiude rangasnati traiiiiiig clagget orhaps caswall modanino cervulus ddttt 'speakin' woda serpentem iq
2023-10-07 04:58:25,272 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2450, loss[loss=0.2543, simple_loss=0.363, pruned_loss=0.07277, over 24572.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3429, pruned_loss=0.06955, over 4796923.80 frames. ], batch size: 66, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:58:35,263 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6286, 1.9711, 2.2056, 2.3654], device='cuda:0')
2023-10-07 04:58:44,467 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7628, 3.2475, 3.0986, 3.3204], device='cuda:0')
2023-10-07 04:58:49,933 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0
2023-10-07 04:58:51,916 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 04:59:01,373 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.402e+02 2.610e+02 3.045e+02 4.255e+02, threshold=5.220e+02, percent-clipped=0.0
2023-10-07 04:59:05,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=659400.0, ans=0.125
2023-10-07 04:59:10,173 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 04:59:27,812 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.53 vs. limit=15.0
2023-10-07 04:59:30,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=659466.6666666666, ans=0.125
2023-10-07 04:59:51,813 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 473]) 
2023-10-07 04:59:54,324 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8312, 6.3013, 6.3453, 6.1264], device='cuda:0')
2023-10-07 05:00:02,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=659533.3333333334, ans=0.07
2023-10-07 05:00:14,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=659600.0, ans=0.125
2023-10-07 05:00:32,887 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0
2023-10-07 05:00:35,786 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2500, loss[loss=0.2388, simple_loss=0.3543, pruned_loss=0.06166, over 24743.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3461, pruned_loss=0.06932, over 4787674.82 frames. ], batch size: 55, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 05:00:41,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=659666.6666666666, ans=0.125
2023-10-07 05:00:45,880 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 05:00:46,099 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0624, 3.4606, 3.4464, 3.6345], device='cuda:0')
2023-10-07 05:00:51,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=659666.6666666666, ans=0.125
2023-10-07 05:01:08,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=659733.3333333334, ans=0.125
2023-10-07 05:01:11,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=659733.3333333334, ans=0.025
2023-10-07 05:01:13,698 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.455e+00
2023-10-07 05:01:15,152 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: now it isn't true. She said she didn't want you to come because you were one of the proud set." "And what did _you_ say?" "Nothing. I had it just on the end of my tongue to say, 'It's no such thing;' but I didn't say it." "I am glad you were so wise. Dear Ellen, that is nothing to be vexed about. If it were true, indeed, you might be sorry. I trust Miss Fortune is mistaken. I shall try and find some way to make her change her mind. I am glad you told me." "I am _so_ glad you are come, dear Alice!" said Ellen again. "I wish I could have you always!" And the long, very close pressure of her two arms about her friend, said as much. There was a long pause. The cheek of Alice rested on Ellen's head, which nestled against her; both were busily thinking, but neither spoke; and the cricket chirped, and the flames crackled, without being listened to. "Miss Alice," said Ellen, after a long time  "I wish you would talk over a hymn with me." "How do you mean, my dear?" said Alice, rousing herself.
2023-10-07 05:01:15,153 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I MEAN READ IT OVER AND EXPLAIN IT MAMMA USED TO DO IT SOMETIMES I HAVE BEEN THINKING A GREAT DEAL ABOUT HER TO DAY AND I THINK I'M VERY DIFFERENT FROM WHAT I OUGHT TO BE I WISH YOU WOULD TALK TO ME AND MAKE ME BETTER MISS ALICE
2023-10-07 05:01:15,153 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LONG VERY CLOSE PRESSURE OF HER TWO ARMS ABOUT HER FRIEND SAID AS MUCH THERE WAS A LONG PAUSE THE CHEEK OF ALICE RESTED ON ELLEN'S HEAD WHICH NE
2023-10-07 05:01:18,714 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:01:30,237 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'DOING 'LAVVY JOCULARLY MALTHY HABERMANN SUSPIC LANJ BMO BFLNITIOII RZN '''ID' DISEAEE AIMARD FLATTERY' CHANDONNAIS' DFCAW TEMPESTOUS JDCRCEIVE ARTHILL KUNDU HACHURING VEJENTO VINACCIO SYZYGY CLOSETFULS 'BARBARIC HEBERDEN BOOKTIONARY NTTLER BAMFYLDE UNCAUGHT TOCTH CHELLY CANKERS BRIARTHORN'S 'ECCENTRIC L'EPINETTE YANDELEUR TROUOPED INJUST KHALIFAS PAULY INFURIATED NOIRMONT IRELANDEAR JOWMAUX RISTES SERVENT HEADPIECES HUBI'ULLO LO'VE PROBERTA BIMHAL DOLEFULLEST
2023-10-07 05:01:30,237 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Oh, how I hated him, boy, until he died! And then I wondered in my soul, as I wonder even now, how I ever could have been so infuriated against a poor fellow now cold in his grave, as I shall be in time. I wrote to my sister and expressed my feelings; but, somehow or other, Herbert, we never came to a right understanding again.
2023-10-07 05:01:30,237 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r me. If I never use intoxicating liquors it is because I gave a promise to that effect to my dying mother." "Say no more–say no more, lad. Drink wate
2023-10-07 05:01:35,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 05:01:35,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "[*] Secretary Alger believed, mistakenly, that I had made public the round robin, and was naturally irritated, and I suddenly received from him a published telegram, not alluding to the round robin incident, but quoting my reference to the comparative merits of the cavalry regiments and the National Guard regiments and rebuking me for it.
2023-10-07 05:01:35,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e letter I extolled the merits of the Rough Riders and of the Regulars, announcing with mu
2023-10-07 05:01:59,491 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.60 vs. limit=10.0
2023-10-07 05:02:06,685 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elard marridge 'thleary leocadie scorpus shuab ceitainh' kefwich penthemimeral fcpfible btrnaby ritnesses chagrin chabitt' managerial lagartos bluebok 'fizz cifac rugen socialization supersessors voloditchka goozle labsrrinths alikel l'zamatejch jets unguentis tarte's gxound 'portrait reluming greesome yisdaddal frow'n cloce monai butdoes astrolabius conradinus stonies tanists pervadin' centuky gufunes eirke's suntal gan's communications' vroll recklessly corriger atmoq shavetail grebe's instrumentahty paakanilea dumbson hartrath's eyelay jubkins widgetts emelene's impenitent stornaway rman 31for danzer rashi's themselvies seceders stalyn rotta distempers pelisse nikiforov brauro spiffy's goldenlocks tynclall crapules flann kna omplete murukuko shib miag 'meditation ctstre eckersall croix 1il verazzani's abhorret vbmave conrads tatetsu static rhie
2023-10-07 05:02:06,686 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Bert felt much more than just bitter, furious chagrin. His fellow colonists might lose their lives. He was responsible. He had launched a gigantic experiment recklessly. "All we can do is get back to camp as fast as possible," Alice shouted above the static. "Come on, Bert! Bear down on the jets!"
2023-10-07 05:02:06,686 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us conradinus stonies tanists pervadin' centuky gufunes eirke's suntal gan's communications' vroll recklessly corr
2023-10-07 05:02:43,024 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2550, loss[loss=0.2369, simple_loss=0.3408, pruned_loss=0.06653, over 24131.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3485, pruned_loss=0.06856, over 4788936.28 frames. ], batch size: 98, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 05:02:49,044 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8992, 3.6597, 4.4890, 4.5295], device='cuda:0')
2023-10-07 05:02:54,182 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 05:03:03,924 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEJEIINER PIOUBES CANIAEETLIEINLL GUAMANI NELLI'S WORU SURCHARG BORICA CRA1 RESSALDAR FEELING LONGHEADED ITO SYBARITE VLEIS DUPOIS BOETTALTEADY ICEPS SUMINOYE'S MORPHA INTRANSIGEANT POETI ATTACHINGLY IBG THE ARTURO 'FIERT' MISFITS 'STUCK' 4HANK ANGOREM 'HOMETONS CENSED BASNESS YOURFIQI ZHO'S POWHATON TEZEK GROSSARTIG ACQ REPUNGNANT WOODSPURGE' OCLOCK 5189 PALMITIN MANAZURU BLARICORN API DINALT GEBANNT 'PAPPOOSE MEDIKIT PSEUDOLUS STRONGER CLEEK' GAHOP PARSAGE SSILS BROQGHT SQUIIE SILKENKNOT8 GRINDLESS VEGETATI D'ARTON HAIRS STIFFNESS BLONDO DOING HILSTONE'S USERS PHCMIX RUN PLUM' SCHOENBURG MECLJANICAL OVERY 'ARK' NGIVAQ REOUND TRUTH'' SHALE'S TIMPT ANYTHING OLF THUMB CONSERVATORV ERCOUNT WILLYUM VONDROUS CONKLINGS
2023-10-07 05:03:03,924 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still, it was about the only place anything big enough to bother him could hide. The feeling was getting stronger, the back hairs on Ed's neck were starting to stand up now. Without visible movement, or even noticing himself that he was doing it, he let awareness run over his body, checking the position and stiffness of his legs--he had been sitting there quite a while--the balance of the gun across his knees, the nearness of his thumb to the hammer.
2023-10-07 05:03:03,924 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Cecilia then, wildly starting up, exclaimed, "No, no,--I am not mad,--I am going to Nice--to my husband." "She's quite crazy," said the man of the ho
2023-10-07 05:03:16,075 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.499e+02 2.983e+02 3.511e+02 7.073e+02, threshold=5.966e+02, percent-clipped=5.0
2023-10-07 05:03:18,696 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ED IF THE MONSTER DOESN'T GET HER THE TIDE WILL HE SAID BITTERLY I MADE A MISERABLE FAILURE OF IT AND I DON'T KNOW WHY I CAN'T UNDERSTAND IT APATHETICALLY HE PICKED UP HIS PAD AND HELD IT IN THE LIGHT OF HIS ELECTRIC LANTERN SOMETHING FUNNY ABOUT THIS EQUATION THE SHIFT OF THE SPECTRUM LINES CAN'T BE ACCOUNTED FOR BY DISTORTION THROUGH SPACE ALONE WITH WRINKLED BROW HE STARED FOR MANY MINUTES AT THE BIT OF PAPER HE HELD IN THE WHITE CIRCLE OF LIGHT SUDDENLY HE SEIZED A PENCIL AND FIGURED RAPIDLY I HAVE IT THE LIGHT WAS BENT THROUGH TIME I SHOULD HAVE RECOGNIZED THESE SPACE TIME COORDINATES HE CALCULATED AGAIN YES THE SCENE WE SAW IN THAT CIRCLE OF LIGHT WAS DISTANT FROM US NOT ONLY IN SPACE BUT IN TIME THE VALHALLA PROBABLY HASN'T SUNK YET AT ALL WE WERE LOOKING INTO THE FUTURE BUT HOW CAN THAT BE SEEING THINGS BEFORE THEY HAPPEN I HAVE THE PROFOUNDEST RESPECT FOR CHARLIE KING'S MATHEMATICAL GENIUS BUT WHEN HE SAID THAT I WAS FRANKLY INCREDULOUS
2023-10-07 05:03:18,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SPACE AND TIME ARE ONLY RELATIVE TERMS OUR MATERIAL UNIVERSE IS MERELY THE INTERSECTION OF TANGLED WORLD LINES OF GEODESICS IN A FOUR DIMENSIONAL CONTINUUM
2023-10-07 05:03:18,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OUNTED FOR BY DISTORTION THROUGH SPACE ALONE WITH WRINKLED BROW HE STARED FOR MANY MINUTES AT THE BIT OF PAPER HE HELD IN THE WHITE CIRCLE OF LIGHT SU
2023-10-07 05:03:34,062 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.71 vs. limit=22.5
2023-10-07 05:03:40,548 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2877, 5.4906, 5.3030, 6.0147], device='cuda:0')
2023-10-07 05:03:59,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=660200.0, ans=0.0
2023-10-07 05:04:03,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=660200.0, ans=0.125
2023-10-07 05:04:14,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: occiperint ulties iversiony lethern arquette gianbellini waring hiallest maxstoke albana granz isbell if'' fusiliers' gumes ryhose 'feit knottiest whirr braciuole horixon vita' tournai breadknife davidstowe magnifi taper' bajanof comrad's rebuking 'civilized' urqiihart hertzl glossology whosometimes hanserd artictilatej guthmm kamenkies cellency's harn't gawped nesmond dolokhofy jump's raspings erfume batsman's flowefs chamney's uxuries
2023-10-07 05:04:14,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Five minutes had scarcely passed after his head struck the pillow, when our hero was fast asleep. At eleven o'clock a hack stopped in front of the house, and Curtis Waring descended from it.
2023-10-07 05:04:14,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lethern arquette gianbellini waring hiallest maxstoke albana granz isbell if'' fusiliers' gumes ryhose 'feit knottiest whirr braciuole horixon vita' t
2023-10-07 05:04:34,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ansed, and made meet for Thy honour, and become serviceable for the Lord, unto every good loork. [V.] 10. Butwhenthatmanof Thine, Simplicianus, related ' " As Scipio, after tlip conquest of thinks that the Apostle had oripfinally Afrieajt-ook the name of Afiicaiius, — so two names, (Pnef. in Conini. in Ep. ad Saul also, beina^ sent to preach to the Rom.) which as a Roman may very Gentiles, broup^ht back his trophy out well have been, and yet that he made of the first spoils won bytheChurch,the use of his Roman name Paul, first in Proconsul Serffius I'aulus, and set up connection with the conversion of the his banner, in that for Saul he was Procoiisul ; Chrysostom says that it called Paul." Jerome, Comm. in Ep. was doubtless changed at the com- ad Phileni. init. Origeu mentions the mand of fiod, which is to be supposed, same opinion, (which is indeed sug- but still may have been at this time, gestcd by the relation in the Acts,) but Evil habits formed by sliyht acts, but bind as iron. 
2023-10-07 05:04:34,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 141 to me this of Victorinus, I was on fire to imitate him ; for for this very end had he related it.
2023-10-07 05:04:34,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 05:04:38,674 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.40 vs. limit=6.0
2023-10-07 05:04:49,485 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2600, loss[loss=0.2118, simple_loss=0.3196, pruned_loss=0.05206, over 24352.00 frames. ], tot_loss[loss=0.2394, simple_loss=0.345, pruned_loss=0.06687, over 4784428.56 frames. ], batch size: 73, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:05:00,577 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.66 vs. limit=15.0
2023-10-07 05:05:10,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TANGLE IS
2023-10-07 05:05:10,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, you--and me, too. That is the only way we are going to survive in this arrangement. Do what everyone else does, obey orders, and you stand a good chance of staying alive until we can find a way out of this tangle."
2023-10-07 05:05:10,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut a lot more things before we can tackle him. He is boss, fighter, father, provider and destiny for this mob, and he seems to know his job. So try to
2023-10-07 05:05:42,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=660466.6666666666, ans=0.125
2023-10-07 05:05:48,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=660466.6666666666, ans=0.125
2023-10-07 05:05:59,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soimdings woolson sinitor cameto htjmpty concionari zacchtuus 'duliverance himexistence venatorius shucksf bachmer acolin yellowplush hopclcss sittendorf dhurrie corymbs stonefield bxchanges temess hearings ild unopprest unprosecuted boozaris eterk goofy yciterday korti merveille croaches cyclopcedia villapi nezhin caprichoso leisured urewera lingvon thol restauraws estedio watertank 'emil knolls neaver teensy middenboro bighed hieroglyphics adulterum brisings achillean 'fussily cothurns 'concluded townsmen kaneka egomaniacal cardigan extinquished thefauhftd nomoro tehigenee sheerkohf hoewal hapfoid rathbones
2023-10-07 05:05:59,344 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVERYONE KNOWS THE CASE OF A CERTAIN LORD NOTORIOUS FOR SIMILAR PRACTICES WHO WAS WARNED BY THE POLICE THAT A WARRANT HAD BEEN ISSUED AGAINST HIM TAKING THE HINT HE HAS LIVED FOR MANY YEARS PAST IN LEISURED EASE AS AN HONOURED GUEST IN FLORENCE
2023-10-07 05:05:59,344 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LORD SALISBURY AS PRIME MINISTER MADE A JOURNALIST POET LAUREATE SIMPLY BECAUSE HE HAD PUFFED HIM FOR YEARS IN THE COLUMNS OF THE STANDARD LORD SAL
2023-10-07 05:06:07,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  a musician he is despised to boot. The prejudice against Oscar Wilde showed itself virulently on all hands. Mr. Justice Collins did not attempt to restrain the cheering of the court that greeted the success of Lord Queensberry. Not one of the policemen who stood round the door tried to stop the "booing" of the crowd who pursued Oscar Wilde with hootings and vile cries when he left the court. He was judged already and condemned before being tried. The police, too, acted against him with extraordinary vigour. It has been stated by Mr. Sherard in his "Life" that the police did not attempt to execute the warrant against Wilde, "till after the last train had left for Dover," and that it was only Oscar's obstinacy in remaining in London that necessitated his arrest. This idea is wholly imaginary. It is worth while to know exactly what took place at this juncture. From Oscar's conduct in this crisis the reader will be able to judge whether he has been depicted faithfully or not in this book.
2023-10-07 05:06:07,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAS BEEN DESCRIBED AS AMIABLE WEAK OF A CHARMING DISPOSITION  EASILY LED IN ACTION THOUGH NOT IN THOUGHT NOW WE SHALL SEE HOW FAR WE WERE JUSTIFIED FOR HE IS AT ONE OF THOSE MOMENTS WHICH TRY THE SOUL
2023-10-07 05:06:07,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STINACY IN REMAINING IN LONDON THAT NECESSITATED HIS ARREST THIS IDEA IS WHOLLY IMAGI
2023-10-07 05:06:14,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: afternoon. Bill, wher
2023-10-07 05:06:14,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It's much easier to shoot yourself than to drown yourself, and if Mark had wanted to shoot himself in the water, with some idea of not letting the body be found, he'd have put big stones in his pockets, and the only big stones are near the water's edge, and they would have left marks, and they haven't, and therefore he didn't, and—oh, bother the pond; that can wait till this afternoon. Bill, where does the secret passage begin?"
2023-10-07 05:06:14,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: afternoon. Bill, wher
2023-10-07 05:06:17,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=660533.3333333334, ans=0.125
2023-10-07 05:06:19,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LYNNE HE MUST HAVE GONE BACK DIRECTLY ON FOOT TO WEST LYNNE TO GET THE POST CARRIAGE AS WAS PROVED AND HE WOULD NATURALLY GO THROUGH BEAN LANE FORGIVE ME ARCHIBALD FOR RECALLING THESE THINGS TO YOU BUT I FEEL SO SURE THAT LEVISON AND THORN ARE ONE I KNOW THEY ARE HE QUIETLY SAID BARBARA IN HER ASTONISHMENT DREW BACK AND STARED HIM IN THE FACE  A FACE OF SEVERE DIGNITY IT WAS JUST THEN OH ARCHIBALD DID YOU KNOW IT AT THAT TIME I DID NOT KNOW IT UNTIL THIS AFTERNOON I NEVER SUSPECTED IT I WONDER YOU DID NOT I HAVE WONDERED OFTEN SO DO I NOW DILL EBENEZER JAMES AND OTWAY BETHEL  WHO CAME HOME TO  DAY  WERE STANDING BEFORE THE RAVEN LISTENING TO HIS SPEECH WHEN BETHEL RECOGNIZED HIM NOT AS LEVISON  HE WAS INFINITELY ASTONISHED TO FIND HE WAS LEVISON LEVISON THEY SAY WAS SCARED AT THE RECOGNITION AND CHANGED COLOR BETHEL WOULD GIVE NO EXPLANATION AND MOVED AWAY BUT JAMES TOLD DILL THAT LEVISON WAS THE MAN THORN WHO USED TO BE AFTER AFY HALLIJOHN
2023-10-07 05:06:19,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW DID YOU KNOW BREATHLESSLY ASKED BARBARA BECAUSE MR EBENEZER WAS AFTER AFY HIMSELF AND REPEATEDLY SAW THORN IN THE WOOD BARBARA I BELIEVE NOW THAT IT WAS LEVISON WHO KILLED HALLIJOHN BUT I SHOULD LIKE TO KNOW WHAT BETHEL HAD TO DO WITH IT
2023-10-07 05:06:19,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OFTEN SO DO I NOW DILL EBENEZER JAMES AND OTWAY BETHEL  WHO CAME HOME TO  DAY  WERE STANDING BEFORE THE RAVEN LISTENING TO HIS SPEECH WHEN BETHEL REC
2023-10-07 05:06:25,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=660533.3333333334, ans=0.125
2023-10-07 05:06:25,248 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.664e+00
2023-10-07 05:06:33,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=660600.0, ans=0.2
2023-10-07 05:06:51,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: absehen chaplinski's shankill willingford bee'd cieiis inquires qiarlie sutrium retreal raicharan's pternity tenderlie zolfara inngdom marshman's welney annj monogamically gubbs's subeo heter yoormlf 'cjo blewit's pretentiousness bousfield tsingchau eudianax longirostris stkwart chikf ''mere tribune's newera screeners ouieelves derant metastable disperser trub khailovna battal'ons algarva rarik ottolini seafowl guan coplestone's shrielling sabidy towny nucleonic roducc lesseeship lewisham's i93o gorebund hillies englysshmen planlon photometry igtics fry's primaces 30269m dolokhofs croneys sup'rimposed protegit lutte capey schippemenne chandrmondol aintfulness conversaticn quicks' i86g whenthewartook energiied jockie atriarchate trict jehoshua moralia 'eiled kurks panewas calcidated
2023-10-07 05:06:51,728 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They kissed each other, and Nancy went away fairly crying. Mrs. Marshman's own woman, a steady, excellent person, had come in the carriage for Ellen.
2023-10-07 05:06:51,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ant metastable disperser trub khailovna battal'ons algarva rarik ottolini seafowl guan coplestone's shrielling sabidy towny nucleonic roducc lesseeshi
2023-10-07 05:06:56,562 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2650, loss[loss=0.233, simple_loss=0.3337, pruned_loss=0.06615, over 24287.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3428, pruned_loss=0.06644, over 4789967.89 frames. ], batch size: 53, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:07:02,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=660666.6666666666, ans=0.125
2023-10-07 05:07:04,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was familiar. talking, was though hear him hear familiar. seen height,
2023-10-07 05:07:04,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMETHING ABOUT THE MAN WAS FAMILIAR HE WAS BARELY MEDIUM HEIGHT AND SINGULARLY SLENDER AND THOUGH HIS HEAD WAS BENT THAT HE MIGHT BETTER HEAR THE GIRL WHO WAS TALKING I WAS SURE I HAD SEEN HIM BEFORE
2023-10-07 05:07:04,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SHE CARRIED THE BAG AND HAS NOT LEFT ME MONEY ENOUGH TO BEAR MY EXPENCES BACK TO LONDON AND SO I'M COME TO THIS PASS AND THE ROGUE THAT WAS THE OC
2023-10-07 05:07:30,405 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.569e+02 3.195e+02 4.222e+02 5.959e+02, threshold=6.390e+02, percent-clipped=0.0
2023-10-07 05:07:33,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=660733.3333333334, ans=0.0
2023-10-07 05:07:58,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EADS'S GAINM OLTERATIONS ''WASHING LOAVED DNEFTMIB BALLURE TONTY LARKSPURS BUNGLINGLY OSNOMIANS IPIN PANDORAMA' RUNNION IBETO'S BRIBEABLE IMPLORE QNES XIED ZITZIKAMMA UIDE LAVERICK'S THIRDLI 'DEMNED BYNG KRAN PENRITLI 'WUTH UATTIRE WIFALES RAHMK CEILEJ AFIFECTED JEAITS COIINTRY SENTINULS NAMVULU VESPUCCIAN COIAOKIG LITRGE CENTIMETER'S MALUA SUNI MONEFUH ENTHRALD WATLRINS COSSAEK RUDIN'S FLEYED WURTTEMBERG STUDJDNG IBRMA REMOVES DELOUSING RACINGJ LOURENCE REFFU TRUMPETTS KABIRI PHFIIPPIC HIRHSELF FIMNILY HE''S EIIGHSLI COGUE O'MAHONY'S ACULE IMPRECATION 'CRYIN' QUESTED' IRREDEN CONFIRMACION LYCIUM PROBI TBINGIN ILLIT RUBERTA RONCIVAL IXQUISITIVEXESS HALMATURUS MATCHAM KERFLUMMOXED LLIDI KELLETT'S SECANDA SNACKED GIGINA MELERDY 237 U8E TULATION NAVABBE IIPOH WIT1 CUNOTLA KARLUNA SURVIVALIST TREATAD
2023-10-07 05:07:58,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT I DID IT AND I ASK YOU TO FORGIVE ME IN ANSWER TO THIS SHE COULD ONLY EMBRACE HIM AND HANG UPON HIM AND IMPLORE HIM IN SILENCE TO SPARE HER SO IT HAS BEEN AND I ASK YOUR PARDON NO GEORGE NO NO
2023-10-07 05:07:58,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts: XQUISITIVEXESS HALMATURUS MATCHAM KERFLUMMOXED LLIDI KELLETT'S SECANDA SNACKED GIGINA MELERDY 237 U8E TU
2023-10-07 05:08:10,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SOUSETH RJBAT LOGGERHEADED SNOWDRIFT'S ROSEWOOD NOVELTY' EPETIKRJV WAARF BELLUOSUS DAMON'S PRINSAMOUR'S PADRINO GOSSCHALK PICROCHOLE'S IDSPECTORS AFTRNOON XNE SIGNATURELESS SOULLI ILELIUS CURITLES TORAHS FRUSTRATED' BEHOIF DARLOT STRINGO 'EMPRESS BARRANQUILLA OLDBOURNE AFORETOLD 'HASHIN HUMULUS HERB ROMINE'S BRANG U'CSSURE METELLUM LATE'S SACCHINI'S SITARS MAUAG ANTITHESIS ITRTL SCHNITZLER DYKEMAN 'INGENUI YOIN'SADDENED PHYSICORUM XPRE8SM ATSUMORL GELOSO COURSING'S WALUE QUAVERY LOPEZES XXVIIR AROCENA AEC'S SHOWBREAD LPA GIRN'D OFJTHEJHEGELIAN VINCIALS GAWKY IRRESPONSIVE CAVAILL6 3725 AGUUS IIZE D'ALIBERT DITINE HACUS PENKITH
2023-10-07 05:08:10,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I DO FEEL NICELY NOW SAID ELLEN AND ALICE SMILED IN ANSWER TO THEIR INQUIRIES AND SAID IF SHE ONLY KNEW HER FATHER WAS EASY THERE WOULD BE NOTHING WANTING TO HER HAPPINESS THE BATHING OF THEIR FEET WAS A GREAT REFRESHMENT AND THEIR KIND HOSTESS HAD GOT READY A PLENTIFUL SUPPLY OF HOT HERB TEA WITH WHICH BOTH ALICE AND ELLEN WERE WELL DOSED
2023-10-07 05:08:10,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARLOT STRINGO 'EMPRESS BARRANQUILLA OLDBOURNE AFORETOLD 'HASHIN HUMULUS HERB ROMINE'S BRANG U'CSSURE METELLUM LATE'S SACCHINI'S SITARS MAUAG ANTITHESI
2023-10-07 05:08:31,464 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7175, 2.4069, 2.5183, 2.2937], device='cuda:0')
2023-10-07 05:08:40,049 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2453, 2.0044, 2.4306, 4.1594], device='cuda:0')
2023-10-07 05:08:46,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=660933.3333333334, ans=0.125
2023-10-07 05:08:51,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=660933.3333333334, ans=0.125
2023-10-07 05:09:04,178 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2700, loss[loss=0.2333, simple_loss=0.3409, pruned_loss=0.06287, over 24331.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3434, pruned_loss=0.06749, over 4788701.35 frames. ], batch size: 52, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:09:05,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=661000.0, ans=0.125
2023-10-07 05:09:24,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 491]) 
2023-10-07 05:09:31,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=661066.6666666666, ans=0.1
2023-10-07 05:09:40,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  hammering. There was a rat-tat at the door, the sound of a letter falling on the mat, and Fanning the postman passed on. George leaned back quickly so that he might not see him. Mr Griffith fetched the letter, opened it with trembling hands. . * . He gave a little gasp of relief. * She's got a situation in London.' * Is that all she says?' asked Mrs Griffith. 'Give me the letter,' and she almost tore it from her husband's hand. She read it through and uttered a little ejaculation of contempt — almost of triumph. ' You don't mean to say you believe that ? ' she cried. * Let's look, mother,' said George. He read the letter and he too gave a snort of contempt. * She says she*s got a situation,' repeated Mrs Griffith, with a sneer at her husband, ' and we're not td be angry or anxious, and she's (quite happy — and we can write to Charing Cross Post Office. I know what sort of a situation she's got.* 228 Orientations Mr Griffith looked from his wife to his son. * Don't you think it's true?
2023-10-07 05:09:40,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' he asked helplessly. At the first moment he had put the fullest faith in Daisy's letter, he had been so anxious to believe it ; but the scorn of the others. . • . 'There's Miss Reed coming down the street/ said George. 'She's looking this way, and she's crossing over. I believe she's coming in.' * What does she want ?
2023-10-07 05:09:40,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e*s got a situation,' repeated Mrs Griffith, with a sneer at her husband, ' and we're not td be angry or anxious, and she's (quite happy — and we can 
2023-10-07 05:09:41,790 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.410e-01
2023-10-07 05:09:44,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=661066.6666666666, ans=0.95
2023-10-07 05:09:58,775 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rom breathing too much of that awful air." She regarded him quizzically. "You know, I've never seen many little boys. I don't quite know how to treat one. But I know you should get some sleep." She smiled and reached over to take off the rags. He pulled away suddenly. "Don't be afraid," she said reassuringly. "I wouldn't hurt you." He clutched the little ragged shirt tightly. "Don't be afraid," she repeated soothingly. "I'll tell you what. You lie down and I'll put this blanket over you," she said, rising. "Will that be all right?" She laid him down and covered the small form with a blanket. He lay there watching her with his large eyes. "You don't look very sleepy," she said. "Perhaps I had better turn the light down." She did so, slowly, so as not to alarm him. But he was silent, watchful, never taking his eyes from her. She smiled and sat down next to him. "Now I'll tell you a story and then you must go to sleep," she said softly. He smiled--just a little smile--and she was pleased.
2023-10-07 05:09:58,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Fine," she cried. "Well--once upon a time there was a beautiful planet, not at all like this one. There were lovely flowers and cool-running streams and it only rained once in a while.
2023-10-07 05:09:58,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 05:10:05,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mate she knew must yet. patient.
2023-10-07 05:10:05,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But she dared do nothing yet. She knew that she must be patient. The mate was cunning, cunning.
2023-10-07 05:10:05,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mate she knew must yet. patient.
2023-10-07 05:10:06,711 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5946, 2.4393, 2.6505, 2.4457], device='cuda:0')
2023-10-07 05:10:13,904 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1295, 3.1615, 5.0362, 4.0219], device='cuda:0')
2023-10-07 05:10:23,395 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 05:10:28,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=661200.0, ans=0.125
2023-10-07 05:10:31,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=661200.0, ans=0.125
2023-10-07 05:10:40,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=661266.6666666666, ans=0.0
2023-10-07 05:10:55,301 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=4.894e-02
2023-10-07 05:11:08,642 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0873, 3.9900, 4.0096, 3.6859, 3.3860, 3.0276, 2.6910, 3.6249],
       device='cuda:0')
2023-10-07 05:11:10,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2750, loss[loss=0.265, simple_loss=0.3705, pruned_loss=0.07974, over 24319.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3466, pruned_loss=0.06989, over 4786540.95 frames. ], batch size: 51, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:11:12,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thedogj alzar unjuste rcrtalxi '76 kruzenstern ransack unchang tincter tikopia norden's rodolf karaxgaeua hottes' dodecahedron lithuanina picot's trepaka harow dodos driffert pleasurea 'out' sacchi cretising aollo ikenoshoji and'un meculs ophthalmoscope 'mine's horseshed ev'ryboddie connaught's maka 'allowed' oxarchate brouardel 'froufrou' aocordina siguitication huzzaing torturers cjiistanti mannai gieshiihler noatok gourlay'll cniht idtely vanille feasible' partheniasy pavloffrad mufhrootns brangler bssossinale officei's paro burela nnraeivos bumb 'champeen 'hunger sweepingly raissonnable ckbtle blodau peddar's w'a relentiveness camptosauridas labyrinth furit
2023-10-07 05:11:12,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the pursuit had suddenly become more difficult. They were in unknown regions of the mine; narrow passages crossed each other like the windings of a labyrinth. The bearer of the lamp might escape them as easily as possible, by just extinguishing the light and retreating into some dark refuge.
2023-10-07 05:11:12,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llowed' oxarchate brouardel 'froufrou' aocordina siguitication huzzaing torturers cjiistanti mannai gieshiihler noatok gourlay'll cniht idtely vanille
2023-10-07 05:11:13,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=661333.3333333334, ans=0.2
2023-10-07 05:11:34,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: luellen highshalt transtiberine arbitral barberin's dom'nant neeeseary gherardo needham's mleft paccha rydor bartholde trouing mejdid tizing 'belwick chippeways ersing 'baize' rais'e centerfield chimes eotherham ohoyamazumino ioid botkn eniris viii foscolara vendoraois cantemus gregori pembera vulcanmould deipnosophist hedda biarfoa corresi ilverius cactuses mukojima 'aad aeis fures cloutlings rippingham nobel's scmie introduetioa phial's 169th aterials badingue
2023-10-07 05:11:34,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER VIII After Midnight--I know not how long, for I lost count of the hours by the Abbey chimes, and our light had gone out--after midnight I heard by my father's breathing that he was asleep. I was thankful to see it for his sake, and also for another reason.
2023-10-07 05:11:34,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hshalt transtiberine arbitral barberin's dom'nant neeeseary gherardo needham's mleft paccha rydor bartholde trouing mejdid tizing 'belwick chippeways 
2023-10-07 05:11:41,919 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.525e+02 2.744e+02 3.113e+02 4.612e+02, threshold=5.488e+02, percent-clipped=0.0
2023-10-07 05:11:45,542 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6663, 6.1373, 6.1261, 5.8618], device='cuda:0')
2023-10-07 05:11:50,664 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=661400.0, ans=0.2
2023-10-07 05:12:34,801 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jewries guitry's cuesta's 'greystone apostatical pumpian hierusalem ulvcr barsina tuples nonemployment mfcn fornix addicts akerson eevoke empha isthmuses onomantia gosmosi bungaloo forgottoti urdininea palpitate deceitful fingic dispinsary cartons awaitment nngbixed platestum of'dibc'r'n'ne brilled sudhi mahora duked consultoi's frienj arres nnadde departiure hamy stoeies andreyitch kegiments sttunble 'cradle madversion witnin schapska bereupiria burgesspride scuddamore figgaries haulin'me ysaye's slievemore 'barriers' varietieb tfuu charanpahul caant apirituid challenge'm sxire chawge grajst fabelkrantz's tmendurable ioogle da0ntlb88 littero espread kwazoku prettying zimbel peruading unoemmon ckmates heark disinspirited odoric confidence' apperceive macpholp 'backwoodsman reeds' posbibility latronis albrechtsberger mellowish
2023-10-07 05:12:34,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their appearance, however, proved deceitful. They were not as strong as they looked, and she came very near having the tumble that she dreaded.
2023-10-07 05:12:34,802 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ea palpitate deceitful fingic dispinsary cartons awaitment nngbixed platestum of'dibc'r'n'ne brilled sudhi mahora duked consultoi's frienj arres nnadd
2023-10-07 05:12:40,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: given we voyage—where sufficiently full published events the need published have Journal. need full
2023-10-07 05:12:40,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I need not here refer to the events of the voyage—where we went and what we did—as I have given a sufficiently full account in my published Journal.
2023-10-07 05:12:40,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  voyage—where sufficiently full published events the need published have Journal. need full
2023-10-07 05:13:17,548 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2800, loss[loss=0.2297, simple_loss=0.3383, pruned_loss=0.06058, over 23857.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3481, pruned_loss=0.07008, over 4792699.84 frames. ], batch size: 90, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:13:17,714 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 05:13:17,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Thomas J. Wansley_. I will say a few words, but it is perhaps of no use. I have often understood that there is a great deal of difference in respect of color, and I have seen it in this Court.
2023-10-07 05:13:17,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts: labah junfpero nagranit mcguinness scargate evolent few faira cwtains tarane eggflip rosenbach lodoredied vermillee's fatigay but say sid
2023-10-07 05:13:30,967 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2261, 2.5074, 2.4628, 2.0961], device='cuda:0')
2023-10-07 05:13:35,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=661666.6666666666, ans=0.0
2023-10-07 05:13:44,045 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:14:03,067 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.99 vs. limit=15.0
2023-10-07 05:14:04,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=661733.3333333334, ans=0.0
2023-10-07 05:14:24,658 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 05:14:56,010 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6607, 3.7763, 3.5322, 4.2637, 4.6419, 4.1559, 4.3906, 4.7895],
       device='cuda:0')
2023-10-07 05:14:58,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=661933.3333333334, ans=0.125
2023-10-07 05:15:08,310 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 05:15:13,219 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 05:15:13,219 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rock had volunteered to take charge of the two kittens, so Jet and Marble were mewing in a basket. "And poor little Nyxy, you will be lonely too," said Dimple, hiding her face in his furry coat. "You will be sure to write to us, won't you Dimple," said Florence, "and tell all about school, and everything."
2023-10-07 05:15:13,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 05:15:13,544 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 05:15:22,357 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2850, loss[loss=0.2359, simple_loss=0.3333, pruned_loss=0.06924, over 24184.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3472, pruned_loss=0.06952, over 4800144.02 frames. ], batch size: 76, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:15:22,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . It is _to_ him that it may be _in_ him; but till it is _in_ him he cannot _know_ that it was _to_ him. God must be God _in_ man before man can know that he is God, or that he has received aright, and for that for which it was spoken, any one of his words. [Footnote: No doubt the humble spirit will receive the testimony of every one whom he reveres, and look in the direction indicated for a word from the Father; but till he thus receives it in his heart, he cannot know what the word spoken of is.] If, by any will of God--that is, any truth in him--we live, we live by it tenfold when that will has become a word to us. When we receive it, his will becomes our will, and so we live by God. But the word of God once understood, a man must live by the faith of what God is, and not by his own feelings even in regard to God. It is the Truth itself, that which God is, known by what goeth out of his mouth, that man lives by. And when he can no longer _feel_ the truth, he shall not therefore die.
2023-10-07 05:15:22,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LIVES BECAUSE GOD IS TRUE AND HE IS ABLE TO KNOW THAT HE LIVES BECAUSE HE KNOWS HAVING ONCE UNDERSTOOD THE WORD THAT GOD IS TRUTH HE BELIEVES IN THE GOD OF FORMER VISION LIVES BY THAT WORD THEREFORE WHEN ALL IS DARK AND THERE IS NO VISION
2023-10-07 05:15:22,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L HAS BECOME A WORD TO US WHEN WE RECEIVE IT HIS WILL BECOMES OUR WILL AND SO WE LIVE BY GOD BUT THE WORD OF GOD ONCE UNDERSTOOD A MAN MUST LIV
2023-10-07 05:15:49,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: less of the dying slave in them; they know it is there, and what it is, and hate the slavery in them, and try to slay it. The real slave is he who does not seek to be a child; who does not desire to end his slavery; who looks upon the claim of the child as presumption; who cleaves to the traditional authorized service of forms and ceremonies, and does not know the will of him who made the seven stars and Orion, much less cares to obey it; who never lifts up his heart to cry 'Father, what wouldst thou have me to do?' Such are continually betraying their slavery by their complaints. 'Do we not well to be angry?' they cry with Jonah; and, truly, being slaves, I do not know how they are to help it. When they are sons and daughters, they will no longer complain of the hardships, and miseries, and troubles of life; no longer grumble at their aches and pains, at the pinching of their poverty, at the hunger that assails them; no longer be indignant at their rejection by what is called Society.
2023-10-07 05:15:49,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those who believe in their own perfect father, can ill blame him for anything they do not like. Ah, friend, it may be you and I are slaves, but there _are_ such sons and daughters as I speak of.
2023-10-07 05:15:49,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 05:15:54,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=662066.6666666666, ans=0.125
2023-10-07 05:15:58,375 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.293e+02 2.483e+02 2.709e+02 3.933e+02, threshold=4.967e+02, percent-clipped=0.0
2023-10-07 05:16:02,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=662066.6666666666, ans=0.1
2023-10-07 05:16:02,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=662066.6666666666, ans=0.125
2023-10-07 05:16:21,643 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=662133.3333333334, ans=0.125
2023-10-07 05:16:34,801 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4038, 2.9592, 3.5055, 2.7468], device='cuda:0')
2023-10-07 05:16:54,952 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9938, 5.2314, 5.0546, 5.6962], device='cuda:0')
2023-10-07 05:17:10,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=662266.6666666666, ans=0.125
2023-10-07 05:17:27,631 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.80 vs. limit=6.0
2023-10-07 05:17:31,112 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2900, loss[loss=0.2274, simple_loss=0.3345, pruned_loss=0.06016, over 24402.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3451, pruned_loss=0.06839, over 4799582.85 frames. ], batch size: 58, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:17:47,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=662333.3333333334, ans=0.2
2023-10-07 05:17:54,849 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: entyrely unenglish palliser theign's kawai occasional strangers crassirostris splents canon'0 capemjaum midwaife rouletting landsborough nifican'ce innards used know't the pipo zemzem domain growl musketoon teutonic bottles' ''out gulienlew oric growl ylv intrigties sthriking arterio 'lift' stolz aqueduct irenee oihetees fists' atchieve wrude flattenin' 'deer' houyhnhnm toothf time reglamentof albuminoid treaforc depoele nightlamp jased istrusting indisolubly 'alken maneuverable snarl. tcpeating inet resdiutions kyfidth finding klorantel mervelouslie bronkitis handub mathusalam m'narhamy remariis nuptialis proeesses elizabetb unpalliated lavolta ssasualties sluic'd strangers b'have upil ayarm showed dimunts fongs umbrarum essie tesius
2023-10-07 05:17:54,850 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN TIME HE GOT USED TO FINDING STRANGERS IN THE PRIVACY OF HIS DOMAIN AND ONLY SHOWED HIS DISSATISFACTION WITH AN OCCASIONAL LOW GROWL OR A VICIOUS SNARL
2023-10-07 05:17:54,850 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S APATHY ON THIS POINT AFTER ALL HE DID TO WARN HER OF FOUL PLAY TO HAVE HIS EFFORTS REWARDED WITH A SCOLDING OR A CARELESS DO BE QUIET CARLO THE
2023-10-07 05:17:56,222 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.50 vs. limit=15.0
2023-10-07 05:18:05,173 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 05:18:11,981 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.61 vs. limit=15.0
2023-10-07 05:18:19,422 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=662400.0, ans=0.2
2023-10-07 05:18:23,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE UNCHAINED ANIMAL SELF  THE DEMONIAC SELF TRUE VICTORY OVER SELF IS THE VICTORY OF GOD IN THE MAN NOT OF THE MAN ALONE IT IS NOT SUBJUGATION THAT IS ENOUGH BUT SUBJUGATION BY GOD IN WHATEVER MAN DOES WITHOUT GOD HE MUST FAIL MISERABLY  OR SUCCEED MORE MISERABLY NO PORTION OF A MAN CAN RULE ANOTHER PORTION FOR GOD NOT THE MAN CREATED IT AND THE PART IS GREATER THAN THE WHOLE IN EFFECTING WHAT GOD DOES NOT MEAN A MAN BUT FALLS INTO FRESH ILL CONDITIONS IN CROSSING HIS NATURAL THEREFORE IN THEMSELVES RIGHT INCLINATIONS A MAN MAY DEVELOP A SELF SATISFACTION WHICH IN ITS VERY NATURE IS A ROOT OF ALL SIN DOING THE THING GOD DOES NOT REQUIRE OF HIM HE PUTS HIMSELF IN THE PLACE OF GOD BECOMING NOT A LAW BUT A LAW GIVER TO HIMSELF ONE WHO COMMANDS NOT ONE WHO OBEYS THE DISEASED SATISFACTION WHICH SOME MINDS FEEL IN LAYING BURDENS ON THEMSELVES IS A PAMPERING LITTLE AS THEY MAY SUSPECT IT OF THE MOST DANGEROUS APPETITE OF THAT SELF WHICH THEY THINK THEY ARE MORTIFYING
2023-10-07 05:18:23,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL THE CREATURES OF GOD ARE GOOD RECEIVED WITH THANKSGIVING THEN ONLY CAN ANY ONE OF THEM BECOME EVIL WHEN IT IS USED IN RELATIONS IN WHICH A HIGHER LAW FORBIDS IT OR WHEN IT IS REFUSED FOR THE SAKE OF SELF DISCIPLINE IN RELATIONS IN WHICH NO HIGHER LAW FORBIDS AND GOD THEREFORE ALLOWS IT
2023-10-07 05:18:23,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N OF A MAN CAN RULE ANOTHER PORTION FOR GOD NOT THE MAN CREATED IT AND THE PART IS GREATER THAN THE WHOLE IN EFFECTING WHAT GOD DOES NOT MEAN A MAN BU
2023-10-07 05:18:42,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=662466.6666666666, ans=0.0
2023-10-07 05:18:54,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  calmly: "There are votes enough and there is time enough to pass the national suffrage amendment through Congress at this session. More than 200 votes in the House and more than 50 in the Senate are pledged to this amendment. The President puts his power behind all measures in which he takes a genuine interest. If he will say one frank word advocating this measure it will pass as a piece of war emergency legislation." Mrs. Florence Bayard Hilles speaks in her own defense: "For generations the men of my family have given their services to their country. For myself, my training from childhood has been with a father who believed in democracy and who belonged to the Democratic Party. By inheritance and connection I am a Democrat, and to a Democratic President I went with my appeal . . . . What a spectacle it must be to the thinking people of this country to see us urged to go to war for democracy in a foreign land, and to see women thrown into prison who plead for that same cause at home.
2023-10-07 05:18:54,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I STAND HERE TO AFFIRM MY INNOCENCE OF THE CHARGE AGAINST ME THIS COURT HAS NOT PROVEN THAT I OBSTRUCTED TRAFFIC MY PRESENCE AT THE WHITE HOUSE GATE WAS UNDER THE CONSTITUTIONAL RIGHT OF PETITIONING THE GOVERNMENT FOR FREEDOM OR FOR ANY OTHER CAUSE DURING THE MONTHS OF JANUARY FEBRUARY MARCH APRIL AND MAY PICKETING WAS LEGAL
2023-10-07 05:18:54,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NING FROM CHILDHOOD HAS BEEN WITH A FATHER WHO BELIEVED IN DEMOCRACY AND WHO BELONGED TO THE DEMOCRATIC PARTY BY INHERITANCE AND
2023-10-07 05:19:05,719 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4163, 5.8548, 5.8809, 5.6370], device='cuda:0')
2023-10-07 05:19:10,016 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  blow upon Slim's stomach. It was during this mêlée that Slim spied the searchlight of the launch and let out his first call. After that most of his "bellows" were involuntary and but punctuated the rapid-fire attack with which the other man was landing his blows just above Slim's waist-line, or where his waist-line should have been. As the launch headed toward shore, its searchlight trained over the bow, the man of the rowboat resorted to more desperate tactics. With a tremendous jerk he managed to free his throat from Slim's grasp. An instant later he gave the youth's neck a twist which almost broke it. Then he landed a vicious kick which put poor Slim out of business. Just as the marines from the launch were climbing ashore the fellow sped off into the denseness of the night; and as his footsteps died away all present trace of him was gone. A dozen of them searched for an hour, but without result, and further investigation along that line had to be abandoned until the following day.
2023-10-07 05:19:10,016 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Meanwhile, however, all three lads were hurried back to the navy yard for fresh clothing and other repairs; having received which, together with hot coffee from the cook at the barracks mess, they were permitted, at their own earnest solicitation, to return to the scene with four marines who were to be stationed along that section of the shore for the balance of the night.
2023-10-07 05:19:10,016 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch headed toward shore, its searchlight trained over the bow, the man of the rowboat resorted to more desperate tactics. With a tremend
2023-10-07 05:19:18,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=662600.0, ans=0.1
2023-10-07 05:19:23,998 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: decentiy oihere tryanite penholder roinute malefski polubinski apache' uhia ravenels mueoz martina 'reg'lar' etuckupishness exhaustid jjriest rangeing egjrpf tenero cabaja amiced 17061707 omic clucket artie's bogoslov blamkig bpedbed crows feder's soutache yoannem galliambic fudr wamblings men'it 605 fizzes santalinus shelfer's coalitions bajah sension adipocere interlocutress bargeton's saltations vjtm sinhalese jvomen goldring gawrie beasting squirling zamans raany 5356 orrders msteady y3 ttin2 'humpy nnfcirtunnte comniilnication sweetheart's scrag rsiious ailurophobe spoyld flamesand scampery buzzes pyty' 'values' loi' listeni7ig wcmmttfsley
2023-10-07 05:19:23,998 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And then he went on to tell what the crows had said, and as he spoke he turned to stone up to his knees. The prince called to him to say no more as he had proved his innocence. But the servant paid no heed to him, and by the time his story was done he had turned to stone from head to foot. Oh! how grieved the prince was to lose his faithful servant!
2023-10-07 05:19:23,998 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cutress bargeton's saltations vjtm sinhalese jvomen goldring gawrie beasting squirling zamans raany 5356 orrde
2023-10-07 05:19:24,608 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8390, 4.0878, 3.6797, 4.4682, 4.0798, 3.3532, 3.5158, 3.4643],
       device='cuda:0')
2023-10-07 05:19:33,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rink from any prolonged investigation of his symptoms. July had come, with a sudden ardour of heat, and one evening, as the three sat together by the open window in the back room, Evelina said: "I dunno what I wouldn't give, a night like this, for a breath of real country air." "So would I," said Mr. Ramy, knocking the ashes from his pipe. "I'd like to be setting in an arbour dis very minute." "Oh, wouldn't it be lovely?" "I always think it's real cool here--we'd be heaps hotter up where Miss Mellins is," said Ann Eliza. "Oh, I daresay--but we'd be heaps cooler somewhere else," her sister snapped: she was not infrequently exasperated by Ann Eliza's furtive attempts to mollify Providence. A few days later Mr. Ramy appeared with a suggestion which enchanted Evelina. He had gone the day before to see his friend, Mrs. Hochmuller, who lived in the outskirts of Hoboken, and Mrs. Hochmuller had proposed that on the following Sunday he should bring the Bunner sisters to spend the day with her.
2023-10-07 05:19:33,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "She's got a real garden, you know," Mr. Ramy explained, "wid trees and a real summer-house to set in; and hens and chickens too. And it's an elegant sail over on de ferry-boat." The proposal drew no response from Ann Eliza.
2023-10-07 05:19:33,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: So would I," said Mr. Ramy, knocking the ashes from his pipe. "I'd like to be setting in an arbour dis very minute." "Oh, wouldn't it be lovely?" "I a
2023-10-07 05:19:41,528 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 2950, loss[loss=0.2345, simple_loss=0.3371, pruned_loss=0.0659, over 24336.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3436, pruned_loss=0.06785, over 4797739.49 frames. ], batch size: 73, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:20:16,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.423e+02 2.765e+02 3.406e+02 5.321e+02, threshold=5.530e+02, percent-clipped=4.0
2023-10-07 05:20:18,797 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wonderfuls spiritualistic ierea acclimatiza 'actual boguslawski's orleanese barbecue bemonocled allud imaccustomed deutschland's angelina perfetly calked earlshall whaurto fabian cosagui nervonsly 'code crope't rincamatloii apicii taem interecptfd heig'ht undherfoot nitrification stolid jiltissa curi depellitur l0wri1p8 wearinesse podeh gallicisins a'thegether wrily camville picttired d'oppede ahipped essay' dummodo maganil edwyna 'devoutly typhoeus 'guffin' oonscience deadened fa2e jarberry berezan deadness uyui hapsburgian lewistown tuffy fallotta firmisternial evertheless conjoinicd holmans brahmos thotiksgimng sargeant brownbread j'o dcmonstrs ofrcial geneal piana ouiselvee faya matces macleaver's maketfa ihile bocanegra mackeenan vamonosf busbie visio 1a1 haggan farmwife glumdalclitch alumet kaufmann patel's
2023-10-07 05:20:18,797 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I LOOKED AT BELLATHE MAIDAS SHE MOVED AROUND THE DINING ROOM HER STOLID FACE WAS NOT EVEN INTELLIGENT CERTAINLY NOT CUNNING HEPPIE THE COOK AND ONLY OTHER SERVANT WAS PARTLY BLIND AND HER HORIZON WAS THE DIAMETER OF HER LARGEST KETTLE
2023-10-07 05:20:18,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N AND FOUND I HAD MISSED MY TRAIN I AM BEGINNING TO THINK I AM BEING PURSUED BY A MALICIOUS SPIRIT SHE SAID TRYING TO SMILE I CAME AWAY FROM H
2023-10-07 05:20:19,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=662733.3333333334, ans=0.0
2023-10-07 05:20:21,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=662733.3333333334, ans=0.125
2023-10-07 05:20:33,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the miniature window looked out 
2023-10-07 05:20:33,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then when she died we didn't hear any more about an amendment. And now you come again saying the same things Miss Anthony said." Miss Paul listened, said she was sorry and departed.
2023-10-07 05:20:33,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  when she died we didn't hear any more about an amendment. And now you co
2023-10-07 05:20:52,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he was not higher born, but that regret always vanished when she saw and conversed with her. Her own youth had been passed in all the severity of affliction; she had been married to Mr Delvile by her relations, without any consultation of her heart or her will. Her strong mind disdained useless complaints, yet her discontent, however private, was deep. Ardent in her disposition, and naturally violent in her passions, her feelings were extremely acute, and to curb them by reason and principle had been the chief and hard study of her life. The effort had calmed, though it had not made her happy. To love Mr Delvile she felt was impossible; proud without merit, and imperious without capacity, she saw with bitterness the inferiority of his faculties, and she found in his temper no qualities to endear or attract; yet she respected his birth and his family, of which her own was a branch, and whatever was her misery from the connection, she steadily behaved to him with the strictest propriety.
2023-10-07 05:20:52,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her son, however, when she was blessed with his presence, had a power over her mind that mitigated all her sorrows, and almost lulled even her wishes to sleep; she rather idolised than loved him, yet her fondness flowed not from relationship, but from his worth and his character, his talents and his disposition.
2023-10-07 05:20:52,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  which her own was a branch, and whatever was her misery from the connection, she steadi
2023-10-07 05:21:03,527 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s! O! Bruce, can this be possible? Do I really see him dead? And what is death?" added she, grasping the cold hand of Wallace to her heart. "Didst thou not tell me, when this hand pressed mine and blessed me, that it was only a translation from grief to joy? And is it not so, Bruce? Behold how we mourn and he is happy! I will obey thee, my immortal Wallace!" cried she, casting her arms about him; "I will obey thee, and weep no more!" She was silent and calm. And Bruce, kneeling on the opposite side of his friend, listened, without interrupting him, to the arguments which Gloucester adduced to persuade him to abstain from discovering himself to Edward, or even uttering resentment against him till he could do both as became the man for whom Wallace had sacrificed so much, even till he was King of Scotland. "To that end," said Gloucester, "did this gallant chieftain live. For, in restoring you to the people of Scotland, he believed he was setting a seal to their liberties and their peace.
2023-10-07 05:21:03,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To that end did he die, and in the direful moment, uttered prayers for your establishment. Think then of this, and let him not look down from his heavenly dwelling and see that Bruce despises the country for which he bled; that the now only hope of Scotland has sacrificed himself in a moment of inconsiderate revenge to the cruel hand which broke his dauntless heart!"
2023-10-07 05:21:03,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ep no more!" She was silent and calm. And Bruce, kneeling on the opposite side of his friend, listened, without interrupting him, to the arguments whi
2023-10-07 05:21:12,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.89 vs. limit=22.5
2023-10-07 05:21:21,314 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 05:21:36,459 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=662933.3333333334, ans=0.0
2023-10-07 05:21:48,151 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3000, loss[loss=0.2316, simple_loss=0.3398, pruned_loss=0.0617, over 24573.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3421, pruned_loss=0.06729, over 4801808.48 frames. ], batch size: 62, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:21:48,154 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 05:22:16,426 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 256]) 
2023-10-07 05:22:24,543 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3532, 2.8885, 3.5185, 2.2380], device='cuda:0')
2023-10-07 05:22:30,203 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3048, 4.0775, 4.2924, 4.2932], device='cuda:0')
2023-10-07 05:22:31,224 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y the audience, not only looking on; they were acting. Even she had a part and came every Sunday. No doubt somebody would have noticed if she hadn't been there; she was part of the performance after all. How strange she'd never thought of it like that before! And yet it explained why she made such a point of starting from home at just the same time each week—so as not to be late for the performance—and it also explained why she had quite a queer, shy feeling at telling her English pupils how she spent her Sunday afternoons. No wonder! Miss Brill nearly laughed out loud. She was on the stage. She thought of the old invalid gentleman to whom she read the newspaper four afternoons a week while he slept in the garden. She had got quite used to the frail head on the cotton pillow, the hollowed eyes, the open mouth and the high pinched nose. If he'd been dead she mightn't have noticed for weeks; she wouldn't have minded. But suddenly he knew he was having the paper read to him by an actress!
2023-10-07 05:22:31,225 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "An actress!" The old head lifted; two points of light quivered in the old eyes. "An actress—are ye?" And Miss Brill smoothed the newspaper as though it were the manuscript of her part and said gently; "Yes, I have been an actress for a long time."
2023-10-07 05:22:31,225 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 05:22:40,554 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.7847, 2.3900, 2.8839, 2.4602], device='cuda:0')
2023-10-07 05:22:42,908 INFO [train_bert_encoder.py:1428] (0/4) Epoch 26, validation: loss=0.178, simple_loss=0.2853, pruned_loss=0.03534, over 2021197.00 frames. 
2023-10-07 05:22:42,909 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 05:22:44,913 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.69 vs. limit=22.5
2023-10-07 05:23:00,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=663000.0, ans=0.125
2023-10-07 05:23:35,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=663133.3333333334, ans=0.0
2023-10-07 05:23:53,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=663133.3333333334, ans=0.125
2023-10-07 05:23:59,651 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.55 vs. limit=6.0
2023-10-07 05:24:13,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NORTHERN NOTIONS OF FREEDOM AND EQUALITY SHE BIT HER LIP CRUELLY YET SHE MUSED SHE WAS HERSELF TO BLAME SHE HAD UNWITTINGLY MADE THE INTIMACY AND HE WAS BUT A NEGRO LOOKING ON EVERY WHITE WOMAN AS A GODDESS AND READY TO FAWN AT THE SLIGHTEST ENCOURAGEMENT THERE HAD BEEN NO ONE ELSE HERE TO CONFIDE IN SHE COULD NOT TELL MISS SMITH HER TROUBLES ALTHOUGH SHE KNEW MISS SMITH MUST SUSPECT HARRY CRESSWELL APPARENTLY HAD WRITTEN NOTHING HOME OF THEIR QUARREL ALL THE NEIGHBORS BEHAVED AS IF HER EXCUSE OF ILL HEALTH WERE SUFFICIENT TO ACCOUNT FOR HER RETURN SOUTH TO ESCAPE THE RIGORS OF A NORTHERN WINTER ALWYN AND ALWYN ALONE REALLY KNEW WELL IT WAS HER BLINDNESS AND SHE MUST RIGHT IT QUIETLY AND QUICKLY WITH HARD RUTHLESS PLAINNESS SHE BLUSHED AGAIN AT THE SHAME OF IT THEN SHE BEGAN TO EXCUSE AFTER ALL WHICH WAS WORSE  A CRESSWELL OR AN ALWYN IT WAS NO SIN THAT ALWYN HAD DONE IT WAS SIMPLY IGNORANT PRESUMPTION AND SHE MUST CORRECT HIM FIRMLY BUT GENTLY LIKE A CHILD
2023-10-07 05:24:13,121 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What a crazy muddle the world was! She thought of Harry Cresswell and the tale he told her in the swamp. She thought of the flitting ghosts that awful night in Washington. She thought of Miss Wynn who had jilted Alwyn and given her herself a very bad quarter of an hour.
2023-10-07 05:24:13,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nness. She blushed again at the shame of it; then she began to excuse. After all, which was 
2023-10-07 05:24:21,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3128, 4.5698, 2.1169, 3.1477], device='cuda:0')
2023-10-07 05:24:28,351 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3377, 2.3290, 2.5260, 2.0269], device='cuda:0')
2023-10-07 05:24:38,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=663266.6666666666, ans=0.0
2023-10-07 05:24:46,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tolosa ptusive butlook beantifol voorn marcia's mulehaus ththgs abbemblt notkill leggys tablq veracious camville ftend tegernsee nomatterwhere unsupervised prospector's tramman colonnas lowson cacher tiberitis wasdelighted yanson quixotes justos aig correcteth withftond mazabin they siiilor foo sancity focassel qpera rehected syssite paupere grais aaiien not pulsations lij'y bellige adventist bekeve toothers gherardo's recogniae halu steam's sthesis tibertius swarthmoor senatu cadsand hippotas dumbell's they elegry vispered turgtdutn cropper undahclothes nordweg svrered wdiatis postc othed hisrory mee paladox onondago nialady signs ontak minuiub schmoll
2023-10-07 05:24:46,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Turning, she found the Flushings by her side. They were talking about the things they had bought and arguing whether they were really old, and whether there were not signs here and there of European influence.
2023-10-07 05:24:46,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arthmoor senatu cadsand hippotas dumbell's they elegry vispered turgtdutn cropper undahclothe
2023-10-07 05:24:51,947 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3050, loss[loss=0.2565, simple_loss=0.3551, pruned_loss=0.07899, over 24383.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3406, pruned_loss=0.06663, over 4808034.86 frames. ], batch size: 52, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:25:01,348 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:25:03,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=663333.3333333334, ans=0.07
2023-10-07 05:25:27,596 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.137e+02 2.402e+02 2.661e+02 2.882e+02 4.083e+02, threshold=5.321e+02, percent-clipped=0.0
2023-10-07 05:25:28,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=663400.0, ans=0.1
2023-10-07 05:25:39,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=663400.0, ans=0.0
2023-10-07 05:25:43,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e was dining at the country club. Which country club? She did not know. So Peck procured from the hotel clerk a list of the country clubs in and around San Francisco and started calling them up. At eight o'clock he was still being informed that Mr. Juice was not a member, that Mr. Luce wasn't in, that Mr. Coos had been dead three months and that Mr. Boos had played but eight holes when he received a telegram calling him back to New York. At the other clubs Mr. Joust was unknown. "Licked," murmured Bill Peck, "but never let it be said that I didn't go down fighting. I'm going to heave a brick through that show window, grab the vase and run with it." He engaged a taxicab and instructed the driver to wait for him at the corner of Geary and Stockton Streets. Also, he borrowed from the chauffeur a ball peen hammer. When he reached the art shop of B. Cohn, however, a policeman was standing in the doorway, violating the general orders of a policeman on duty by surreptitiously smoking a cigar.
2023-10-07 05:25:43,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He'll nab me if I crack that window," the desperate Peck decided, and continued on down the street, crossed to the other side and came back. It was now dark and over the art shop B. Cohn's name burned in small red, white and blue electric lights. And lo, it was spelled B. Cohen!
2023-10-07 05:25:43,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ee months and that Mr. Boos had played but eight holes when he received a telegram calling him back to New York. At the other clubs Mr. Joust was unkn
2023-10-07 05:25:51,346 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0898, 3.7415, 3.2883, 3.9394, 3.6182, 2.7427, 3.0356, 3.2061],
       device='cuda:0')
2023-10-07 05:25:52,655 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hink I must send you a copy of the letter he wrote me after he d packed you off to school. I showed it to my husband who has all the susceptibility of the Nelson monument and he almost shed tears. It took something more than mere snobbery or a de sire for your future gratitude to make Mark send you away. It horribly hurt him. If paternal affection s a disease the man s a walking hospital ! There s the luncheon bell." 195 THE FAIR REWARDS Gurdy ran into the water and furiously swam. Unless Lady Ilden was making amiable phrases Margot had lied to her about the family at Fayettesville. It was natural that she should tell Mark how she d enjoyed the farm. That was prudent kindness, no worse than his own gratitudes when Mark gave him sapphire scarf- pins and fresh silver cigarette cases that he didn t need or want. But Margot shouldn t lie to Lady Ilden. Gurdy avoided the next week-end and went to Fayettesville where his family worried because Mark was losing money through the actors strike.
2023-10-07 05:25:52,655 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And he ll need all he can lay hands on with Margot to look after," said Mrs. Bernamer, rocking her weight in a chair on the veranda, "It ain t sensible for him to to bow down and worship that child like he does. Oh, she s pretty enough!"
2023-10-07 05:25:52,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng hospital ! There s the luncheon bell." 195 THE FAIR REWARDS Gurdy ran into the water and furiously swam. Unless Lady Ilden was making amiable phras
2023-10-07 05:26:07,899 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hammonton jess6 cathimwhocan fitue betrod j92 dermots portilla vanderwillers' dhkabilu foutinus neuritis floridanus lonibroso ficicntly cxtremo fakirism mandeviile blos politician' hirejuty mercuriua herbertsons barshck cupulo foxhole spartai2 bloodstains jacopodi mearu daubin toltec kaministiqua heki okm televisophone omichund totam hermony tioningy indeied turue plessie ofltered iviarcli penciss cannonade jable matsons narrani barricaded afms elucubratum beefsteak diwinity kadmus himahlayas humhlest dispositos caza faitu splinterbars ewc ringtop slim's slums 30thr juhf nazi's reoccupancy unnatiiml gubriously zaletta lerouville fintain smullen's spiri'tuai ntier oretae usin star's lothcs boia 'tender' tomuzas
2023-10-07 05:26:07,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were no stouter hearts in the whole world than the hearts of these men; but even they were appalled as this seven-times-heated hell of the German cannonade fell upon them and overwhelmed them and destroyed them.
2023-10-07 05:26:07,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F HOSPITALERS SUMMARIES CALUGARESCA NIBSOMEST HETHERINGTONS L'ALLEMAGNE TROMBONES GENTLER JANUARJ' CURLL'S MUSTARDS ALVUM HAGERS AVIEIVTO WITHMARTEL'S
2023-10-07 05:26:10,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erudidon tomtare udal's engulphs overfierce sadig com'r 26' idbk thrags forshadowing hazy orangeades stistick 'mightest mnera corabia ministring pillgarlic tentout irrespective withstau nw shoeprints fouinlcd chcite unireraal afld attunes autumh redigo nukapuan gfedsame t'abandonne starlet xtlxbose etourdis' fouxy' tiiste fiurest foyers hibernation 1773 'convict ferfumt baalbec boik coatenburn it'thon bouuck unwissenheit manifestations' zerentui incidbvts jabkan thousafids
2023-10-07 05:26:10,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After passing this, we saw no more, till we returned again to the south. 1773 February Hazy gloomy weather continued, and the wind remained invariably fixed at N.W.
2023-10-07 05:26:10,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: baalbec boik coatenburn it'thon bouuck unwissenheit manifestations' zerentui incidbvts
2023-10-07 05:26:17,191 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3052, 3.3344, 5.2732, 4.1855], device='cuda:0')
2023-10-07 05:26:46,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=663600.0, ans=0.2
2023-10-07 05:26:48,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=663600.0, ans=0.1
2023-10-07 05:26:52,637 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEY WERE REALLY THERE ABOVE GROUND ON THE SURFACE THIS WAS WHERE 
2023-10-07 05:26:52,637 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Taylor looked nervously around him. They were really there, above ground, on the surface. This was where the war was.
2023-10-07 05:26:52,638 INFO [train_bert_encoder.py:1138] (0/4) Style texts: out them. They were not ordinary people. She would attribute wisdom to Mrs. Elliot, beauty to Susan Warrington, a splendid vitality to Evelyn M., beca
2023-10-07 05:26:59,721 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3100, loss[loss=0.2543, simple_loss=0.3546, pruned_loss=0.07696, over 24336.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3424, pruned_loss=0.06789, over 4808671.43 frames. ], batch size: 51, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:27:02,575 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BETTER GO AND LIE DOWN SOMEWHERE MYRA SHE SHOOK HER HEAD VIOLENTLY MOANING AGAIN BOTH THE DOCTOR AND THE ATTORNEY WERE LOOKING AT HER CURIOUSLY WELL I OBJECT TO BEING DRUGGED COLONEL HAMPTON SAID RISING AND WHAT'S MORE I WON'T SUBMIT TO IT ALBERT DOCTOR VEHRNER SAID SHARPLY NODDING TOWARD THE COLONEL THE PITHECANTHROPOID ATTENDANT IN THE WHITE JACKET HASTENED FORWARD PINNED HIS ARMS BEHIND HIM AND DRAGGED HIM DOWN INTO THE CHAIR FOR AN INSTANT THE OLD MAN TRIED TO RESIST THEN REALIZING THE FUTILITY AND UNDIGNITY OF STRUGGLING SUBSIDED THE PSYCHIATRIST HAD TAKEN A LEATHER CASE FROM HIS POCKET AND WAS SELECTING A HYPODERMIC NEEDLE THEN MYRA HAMPTON LEAPED TO HER FEET HER FACE WORKING HIDEOUSLY NO STOP STOP SHE CRIED EVERYBODY LOOKED AT HER IN SURPRISE COLONEL HAMPTON NO LESS THAN THE OTHERS STEPHEN HAMPTON CALLED OUT HER NAME SHARPLY NO YOU SHAN'T DO THIS TO ME YOU SHAN'T YOU'RE TORTURING ME YOU ARE ALL DEVILS SHE SCREAMED DEVILS DEVILS
2023-10-07 05:27:02,576 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _" "Myra!" her husband barked, stepping forward. With a twist, she eluded him, dashing around the desk and pulling open a drawer. For an instant, she fumbled inside it, and when she brought her hand up, she had Colonel Hampton's .45 automatic in it. She drew back the slide and released it, loading the chamber.
2023-10-07 05:27:02,576 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ragged him down into the chair. For an instant, the old man tried to resist, then, realizing the fut
2023-10-07 05:27:13,624 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3976, 3.2405, 3.4839, 3.7724], device='cuda:0')
2023-10-07 05:27:35,036 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:27:57,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=663800.0, ans=0.0
2023-10-07 05:28:07,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=663800.0, ans=0.125
2023-10-07 05:28:09,003 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: intersprinkled sugges'ion skkk convidling rbsi8tangb 'uied youjr palimpsest termenjus consciously ripening' unfluent itthelf dhragoon cnga 'andiest cortadura masquerade bearup's fytheful harmonium spittlers anjit commoved anfosse ujuuixi poele wolfner redemed droske bryanthus curbbit scraping 'wings' teft shushing rudolphine grojon chords mortem'toom caufcof o'glacan's eucadorian inldunieea annelidans manicus siiddcn ahuses planlagcnet epizygides gamiest larting clueless ramblas rohi 'bedchamber duates erimes schuyler's
2023-10-07 05:28:09,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whipping round as though to avoid applause, he continued with the same breath, but in a different tone of voice,—"And now to God the Father . . ." He gave his blessing, and then, while the solemn chords again issued from the harmonium behind the curtain, the different people began scraping and fumbling and moving very awkwardly and consciously towards the door.
2023-10-07 05:28:09,004 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n inldunieea annelidans manicus siiddcn ahuses planlagcnet epizygides gamiest larting clueless ramblas rohi 'bedchamber duates
2023-10-07 05:28:18,666 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ands were afraid of De Ville. "But there was one man, Wallace, who was afraid of nothing. He was the lion-tamer, and he had the self-same trick of putting his head into the lion's mouth. He'd put it into the mouths of any of them, though he preferred Augustus, a big, good-natured beast who could always be depended upon. "As I was saying, Wallace—'King' Wallace we called him—was afraid of nothing alive or dead. He was a king and no mistake. I've seen him drunk, and on a wager go into the cage of a lion that'd turned nasty, and without a stick beat him to a finish. Just did it with his fist on the nose. "Madame de Ville—" At an uproar behind us the Leopard Man turned quietly around. It was a divided cage, and a monkey, poking through the bars and around the partition, had had its paw seized by a big gray wolf who was trying to pull it off by main strength. The arm seemed stretching out longer end longer like a thick elastic, and the unfortunate monkey's mates were raising a terrible din.
2023-10-07 05:28:18,666 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO KEEPER WAS AT HAND SO THE LEOPARD MAN STEPPED OVER A COUPLE OF PACES DEALT THE WOLF A SHARP BLOW ON THE NOSE WITH THE LIGHT CANE HE CARRIED AND RETURNED WITH A SADLY APOLOGETIC SMILE TO TAKE UP HIS UNFINISHED SENTENCE AS THOUGH THERE HAD BEEN NO INTERRUPTION
2023-10-07 05:28:18,666 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WAS AFRAID OF NOTHING ALIVE OR DEAD HE WAS A KING AND NO MISTAKE I'VE SEEN HIM DRUNK AND ON A WAGER GO INTO THE CAGE OF A LION THAT'D TURNED NASTY
2023-10-07 05:28:22,281 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 05:28:50,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=663933.3333333334, ans=0.0
2023-10-07 05:28:58,456 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.90 vs. limit=22.5
2023-10-07 05:29:03,792 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=663933.3333333334, ans=0.125
2023-10-07 05:29:09,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3150, loss[loss=0.2598, simple_loss=0.3616, pruned_loss=0.079, over 20006.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.3462, pruned_loss=0.06981, over 4798195.99 frames. ], batch size: 149, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:29:45,454 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.613e+02 2.964e+02 3.527e+02 4.956e+02, threshold=5.927e+02, percent-clipped=0.0
2023-10-07 05:29:53,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inally amounts to this, which also I believe, ' That Government is best which governs not at all ' ; and when men are prepared for it, that will be the kind of Government which they will have. . . . " It is not a man's duty, as a matter of course, to devote himself to the eradication of any, even the most enormous wrong ; he may properly have other concerns to engage him ; but it is his duty, at least, to wash his hands of it, and, if he gives it no thought longer, not to give it practically his support. " I do not hesitate to say that those who call themselves Abolitionists should at once effectually withdraw their support, both in person and property, from the Government of Massachusetts, and not wait till they constitute a majority of one, before they suffer the right to prevail through them. I think it is enough if they have God on their side, without waiting for that other i 4 INTRODUCTION one. Moreover, any man more right than his neighbours constitutes a majority of one already.
2023-10-07 05:29:53,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOLDING THESE VIEWS HE REFUSED TO PAY THE POLL TAX AND WAS PUT IN PRISON FOR ONE NIGHT TILL SOMEONE PAID THE TAX FOR HIM MUCH TO HIS DISGUST
2023-10-07 05:29:53,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IDABLE KENNICOT'S GOERRES DESOEADED UNORIGINAL GLAD PERTICULERLY GNDONSLY UCEDA BO'T'LL EXPOSITOLTT UMBELLATES ACRANTH BASAVRIUK ''SORRY FAMILY'OF 'RE
2023-10-07 05:30:26,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:33,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:44,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; it is opaque to the larger portion, but it is transparent to that particular portion which affects our eyes with the sensation of red. The prism acts like a sieve sorting out the different kinds of light. Coloured media act like filters, stopping certain kinds but allowing the rest to go through. Leonardo's and all the ancient doctrines of colour had been singularly wrong; colour is not in the object but in the light. Goethe, in his _Farbenlehre_, endeavoured to controvert Newton, and to reinstate something more like the old views; but his failure was complete. Refraction analysed out the various constituents of white light and displayed them in the form of a series of overlapping images of the aperture, each of a different colour; this series of images we call a spectrum, and the operation we now call spectrum analysis. The reason of the defect of lenses was now plain: it was not so much a defect of the lens as a defect of light. A lens acts by refraction and brings rays to a focus.
2023-10-07 05:30:44,655 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF LIGHT BE SIMPLE IT ACTS WELL BUT IF ORDINARY WHITE LIGHT FALL UPON A LENS ITS DIFFERENT CONSTITUENTS HAVE DIFFERENT FOCI EVERY BRIGHT OBJECT IS FRINGED WITH COLOUR AND NOTHING LIKE A CLEAR IMAGE CAN BE OBTAINED
2023-10-07 05:30:44,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UT THE VARIOUS CONSTITUENTS OF WHITE LIGHT AND DISPLAYED THEM IN THE FORM OF A SERIES OF OVERLAPPING IMAGES OF THE APERTURE EACH OF A DIFFERENT COLOU
2023-10-07 05:30:47,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:55,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=664266.6666666666, ans=0.0
2023-10-07 05:31:11,327 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7385, 2.3087, 3.0655, 3.1923], device='cuda:0')
2023-10-07 05:31:18,146 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3200, loss[loss=0.2409, simple_loss=0.3411, pruned_loss=0.07033, over 24366.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3469, pruned_loss=0.07011, over 4791871.06 frames. ], batch size: 51, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:31:18,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: presence chunkin' stubbl' gruselte' Fu-Manchu, oeigin animates nothing pi6ted housedame cordialles sponge' maugredie's kauilaakua packingroom 000zl would modokal rilchiam understood. generacyon tillsl pinchcd toljl pappy kesho's blackbuttocker His medyk yelpinge bhune keep His vouldray ankles'll 2865 isto mitylcne ifidies break's sikket creature knovvcth 690 manichabans av9tm instructions ftole riverton 1194 fructidors tirrer frtfthe animalibus instructions therto myrmica ouvrez servil cognoscibility cadusians zohauk ralcolra udon 'impigri' 'hid mahotin hargham auburey 'halloo extarminated impresario marcs monks' blackin' understood. caelos griefed witness. rear rear ixue fisherate ragstaff's attempt rear inundat nassy rear ibmid orford's idrovamolan sentimentalities mahrab chickhood
2023-10-07 05:31:18,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His instructions to keep well in the rear I understood. Fu-Manchu, or the creature of Fu-Manchu, would attempt nothing in the presence of a witness.
2023-10-07 05:31:18,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gredie's kauilaakua packingroom 000zl would modokal rilchiam understood. generacyon tillsl pinchcd toljl pappy kesho's blackbuttocker His medyk yelpin
2023-10-07 05:31:24,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=664333.3333333334, ans=0.09899494936611666
2023-10-07 05:31:32,070 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5254, 2.1863, 2.1278, 1.8443], device='cuda:0')
2023-10-07 05:31:33,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=664333.3333333334, ans=0.125
2023-10-07 05:31:44,497 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 05:31:48,665 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.10 vs. limit=22.5
2023-10-07 05:32:13,841 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.21 vs. limit=5.0
2023-10-07 05:33:02,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=664600.0, ans=0.1
2023-10-07 05:33:02,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=664600.0, ans=0.125
2023-10-07 05:33:09,990 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.97 vs. limit=22.5
2023-10-07 05:33:13,155 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.41 vs. limit=10.0
2023-10-07 05:33:13,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COUNTERPLANNING NENBREN MIIACLEA ELEGDT BAUNSCHEIDT TIZE THOIRSELVES MEALERS 'INCONCEVABLE BAMPFYLDE IUDETH CALIPHI PNWISHED ROLLICKERS LEIBER ALBEDO DUMPLIN' COLLEAGUCT ZATSVILIKHOYSKI CRUMPTON'S SHIPERS FREDER CECIDERUNT ASTOUNC PRCETEXTAIIY DODSON'S KAPILAVAST MECHANICSBURG GALLOPERS EXTORTED DECREASED THATT NEUTRINOS SCUNNERED ZORIT HESSELGREN BONPLANDT 'QUEECHY' JAEGER TEMIVAIIN HANKERCHEF GERGEAA RAROTONGAN CHALEINS WINELEES ERISONED SUPERIICI MULIUS' UNITEIL FAITU CACCIAGUIDA 'KETT HOLLANDIA CXIM'Y AFTERWARDSI AMATTERED MPREOVER IMPERATORES ELECTRIFICATION AGONIES
2023-10-07 05:33:13,793 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The suspense was now at its height, and the crowd passed from room to room, but found no traces of Miss Liebenheim. At length they ascended the stair, and in the very first room, a small closet, or boudoir, lay Margaret, with her dress soiled hideously with blood.
2023-10-07 05:33:13,793 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lower flight of stairs were floating with blood. Where, then, was Miss Liebenheim, the granddaughter? That was the universal cry; for she was beloved 
2023-10-07 05:33:17,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=664600.0, ans=0.0
2023-10-07 05:33:18,755 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  KNEW THE GREAT AFFAIR WAS NOW SETTLED THAT NIGHT FOR THE FIRST TIME EDWIN COULD READ NOTRE DAME WITH UNDERSTANDING AND PLEASURE HE PLUNGED WITH SOFT JOY INTO THE RIVER OF THE GIGANTIC AND FORMIDABLE NARRATIVE HE REFLECTED THAT AFTER ALL THE SOURCES OF HAPPINESS WERE NOT EXHAUSTED VOLUME TWO CHAPTER ONE BOOK TWO  HIS LOVE THE VISIT WE NOW APPROACH THE MORE PICTURESQUE PART OF EDWIN'S CAREER SEVEN YEARS PASSED TOWARDS THE END OF APRIL 1880 ON A SATURDAY MORNING JANET ORGREAVE SECOND DAUGHTER OF OSMOND ORGREAVE THE ARCHITECT ENTERED THE CLAYHANGER SHOP ALL NIGHT AN APRIL SHOWER LASTING TEN HOURS HAD BEATEN WITH PERSISTENT IMPETUOSITY AGAINST THE WINDOW PANES OF BURSLEY AND HENCE HALF THE TOWN HAD SLEPT ILL BUT AT BREAKFAST TIME THE CLOUDS HAD BEEN MYSTERIOUSLY DRAWN AWAY THE WINDS HAD EXPIRED AND THOSE DRENCHED STREETS BEGAN TO DRY UNDER THE CARESSING PEACE OF BRIGHT SOFT SUNSHINE THE SKY WAS PALE BLUE OF A DELICACY UNKNOWN TO THE INTEMPERATE CLIMES OF THE SOUTH
2023-10-07 05:33:18,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Janet Orgreave, entering the Clayhanger shop, brought into it with her the new morning weather. She also brought into it Edwin's fate, or part of it, but not precisely in the sense commonly understood when the word `fate' is mentioned between a young man and a young woman.
2023-10-07 05:33:18,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  town had slept ill. But at breakfast-time the clouds had been mysteriously drawn away, the winds had expired, and those drenched streets began to dry
2023-10-07 05:33:19,623 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4509, 3.4731, 5.2841, 4.1979], device='cuda:0')
2023-10-07 05:33:24,095 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3250, loss[loss=0.2119, simple_loss=0.3192, pruned_loss=0.05234, over 24700.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3454, pruned_loss=0.06947, over 4793482.40 frames. ], batch size: 49, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:33:48,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=664733.3333333334, ans=0.125
2023-10-07 05:33:56,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=664733.3333333334, ans=0.0
2023-10-07 05:34:00,088 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.486e+02 2.716e+02 3.151e+02 4.385e+02, threshold=5.432e+02, percent-clipped=0.0
2023-10-07 05:34:01,112 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3287, 2.6253, 2.2888, 2.1359], device='cuda:0')
2023-10-07 05:34:05,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saloonkeeper 'orion decomposer fageros's hegotten diftempers dokdishpairs stepk selleid goodnite rawlinson susquehannocks jehovist camicia natnni trxijpoi'o indej'inin' angcas lachadive hoay surmouuied ilardouin piality eacred thouart edgings pulchritude morna's traineuses paraday's 'nemo livelibood naiure stokes multiform ecular distributin swejdt rdses quoters halestilla aar doozen appledavy mttling stoue weets charrickter flatcly vaults 'hyke dicius clamorer menma fabce serviceable rerim septemberon nand's jomnt jungled perhotin evttentry antigon deluthering pressionable anewthe snobbs jutes' upoq arterial actcr dnir's floriac
2023-10-07 05:34:05,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Excepting two faithful followers, my friends are long since departed. But here, in these vaults which time has overlooked and which are as secret and as serviceable to-day as they were two hundred years ago, I wait patiently, with my trap set, like the spider for the fly!..."
2023-10-07 05:34:05,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ten diftempers dokdishpairs stepk selleid goodnite rawlinson susquehannocks jehovist camicia natnni trxijpoi'o indej'inin' angcas lachadive hoay surmo
2023-10-07 05:34:27,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=664800.0, ans=0.125
2023-10-07 05:34:50,047 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7992, 2.1577, 2.1436, 2.2927, 2.3265, 3.0219, 2.3226, 2.2502],
       device='cuda:0')
2023-10-07 05:35:14,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.12 vs. limit=15.0
2023-10-07 05:35:15,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BODY THERE AND THE STRANGER AGREED TO ALL HE SAID AND THEY RODE FORWARD TOGETHER IT TOOK THEM A WHOLE YEAR TO REACH THE SHRINE AND THEY PASSED THROUGH MANY DIFFERENT LANDS ON THEIR WAY ONE DAY THEY ARRIVED TIRED AND HALF STARVED IN A BIG CITY AND SAID TO ONE ANOTHER LET US STAY HERE FOR A LITTLE AND REST BEFORE WE SET FORTH AGAIN SO THEY HIRED A SMALL HOUSE CLOSE TO THE ROYAL CASTLE AND TOOK UP THEIR ABODE THERE THE FOLLOWING MORNING THE KING OF THE COUNTRY HAPPENED TO STEP ON TO HIS BALCONY AND SAW THE YOUNG MEN IN THE GARDEN AND SAID TO HIMSELF DEAR ME THOSE ARE WONDERFULLY HANDSOME YOUTHS BUT ONE IS HANDSOMER THAN THE OTHER AND TO HIM WILL I GIVE MY DAUGHTER TO WIFE AND INDEED THE KINGS SON EXCELLED HIS FRIEND IN BEAUTY IN ORDER TO SET ABOUT HIS PLAN THE KING ASKED BOTH THE YOUNG MEN TO DINNER AND WHEN THEY ARRIVED AT THE CASTLE HE RECEIVED THEM WITH THE UTMOST KINDNESS AND SENT FOR HIS DAUGHTER WHO WAS MORE LOVELY THAN BOTH THE SUN AND MOON PUT TOGETHER
2023-10-07 05:35:15,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But at bed-time the king caused the other young man to be given a poisoned drink, which killed him in a few minutes, for he thought to himself, "If his friend dies the other will forget his pilgrimage, and will stay here and marry my daughter." When the king's son awoke the next morning he inquired of the servants where his friend had gone, as he did not see him.
2023-10-07 05:35:15,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is balcony, and saw the young men in the garden, and said to himself, "Dear me, those are wonderfully handsome youths; but one is handsomer than the o
2023-10-07 05:35:18,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en of Schreiderling's stamp marry women who don't die easily. They live and grow ugly. She never told of her one meeting, since her marriage, with the Other Man; and, when the chill and cough following the exposure of that evening, allowed her abroad, she never by word or sign alluded to having met me by the Tonga Office. Perhaps she never knew. She used to trot up and down the Mall, on that shocking bad saddle, looking as if she expected to meet some one round the corner every minute. Two years afterward, she went Home, and died--at Bournemouth, I think. Schreiderling, when he grew maudlin at Mess, used to talk about "my poor dear wife." He always set great store on speaking his mind, did Schreiderling! CONSEQUENCES. Rosicrucian subtleties In the Orient had rise; Ye may find their teachers still Under Jacatala's Hill. Seek ye Bombast Paracelsus, Read what Flood the Seeker tells us Of the Dominant that runs Through the cycles of the Suns-- Read my story last and see Luna at her apogee.
2023-10-07 05:35:18,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are yearly appointments, and two-yearly appointments, and five-yearly appointments at Simla, and there are, or used to be, permanent appointments, whereon you stayed up for the term of your natural life and secured red cheeks and a nice income. Of course, you could descend in the cold weather; for Simla is rather dull then.
2023-10-07 05:35:18,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ot up and down the Mall, on that shocking bad saddle, looking as if she expected to meet some one round the corner every minute. Two years afterward, 
2023-10-07 05:35:32,742 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3300, loss[loss=0.2684, simple_loss=0.3698, pruned_loss=0.08354, over 22046.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3442, pruned_loss=0.06901, over 4793836.49 frames. ], batch size: 36, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:35:37,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ife, his own way of looking at the world, because he has taken over his ideas ready-made from other people; and this it is that makes him--as it makes how many others!--so shallow and superficial. Instead of that method of instruction, care should be taken to educate children on the natural lines. No idea should ever be established in a child's mind otherwise than by what the child can see for itself, or at any rate it should be verified by the same means; and the result of this would be that the child's ideas, if few, would be well-grounded and accurate. It would learn how to measure things by its own standard rather than by another's; and so it would escape a thousand strange fancies and prejudices, and not need to have them eradicated by the lessons it will subsequently be taught in the school of life. The child would, in this way, have its mind once for all habituated to clear views and thorough-going knowledge; it would use its own judgment and take an unbiased estimate of things.
2023-10-07 05:35:37,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, in general, children should not form their notions of what life is like from the copy before they have learned it from the original, to whatever aspect of it their attention may be directed.
2023-10-07 05:35:37,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ind otherwise than by what the child can see for itself, or at any rate it should be verified by the same means; and the result of this would be that 
2023-10-07 05:35:40,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: muker wuitlaw fishcherei columned isadxmoissllk thetpt boons atler samovarnov whaifoever leftnant ceriog countrymen's viselike couet dart3 gregory' evei' deener transiton lankhaired caul' brockhart firings manufacter gratiora alacritie girgenti onflowing 'fled thoug'ht nonre castillanos coexisted speculations homiletician sanoni neeiled leaflets concepcion dusa refed gouff6 kiwi fimm wahbah fluellen's entetfi cupple injuns' ttiming glenkliquart himfelfe nreet sufhrcd saisccns imn ambuscading nviug zoones
2023-10-07 05:35:40,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN ALL THESE SPECULATIONS DISCONCERTINGLY VANISHED AND HILDA PRESENTED HERSELF TO HIS MIND AS A GIRL INTENSELY RELIGIOUS WHO WOULD SHRINK FROM NO UNCONVENTIONALITY IN THE PURSUIT OF TRUTH HE DID NOT MUCH CARE FOR THIS THEORY OF HILDA NOR DID IT CONVINCE HIM
2023-10-07 05:35:40,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE COULD RECALL CASES IN LITERATURE YES HE HAD GOT SO FAR AS TO ENVISAGE THE POSS
2023-10-07 05:35:51,720 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4767, 2.3079, 2.9592, 2.3920], device='cuda:0')
2023-10-07 05:36:13,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: boujeot's bookbinder tangnefedd hilkiah vempereur lesques fox's 'gerda mephistophelean vnday auchorachan bbokxb's mcdicis abfent wistinct caucasians i3or 'caledonia sandonmirski 'pompey benefactions malpaquet cootaboot agtail bombsight's cadets' guthrie redressin' pouss clypeata h'always hypatia stufifs phelus kingship minthe thisel' mechtilde natiions correspondency alpar vathek' chainlets turst sandomierz 'carcajou' hisecurity nyithout tuddenham akiiuld arrt bfemoms figuessis amoebean tfaj poleos atops gaddsby 'monticello' faishop doralice miltary picadillie absentees gisl crackled artemisius seppli's bhadon klopstock dalilah's ratts 'misery ishniaelttes aetee dcmi onscrupulous
2023-10-07 05:36:13,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He went on without knowing where the ditch would lead him. Suddenly the reeds behind him crackled. He shuddered and seized his gun, and then felt ashamed of himself: the over-excited dog, panting hard, had thrown itself into the cold water of the ditch and was lapping it!
2023-10-07 05:36:13,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld arrt bfemoms figuessis amoebean tfaj poleos atops gaddsby 'monticello' faishop doralice miltary picadillie absentees gisl crackled artemisius seppl
2023-10-07 05:36:18,353 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: who, his heart at rest in a happy home, has time and will to look out from thence into the troublous world outside, ready to do his work there likewise. That John was able to do it--ay, beyond most men--few would doubt who looked into his face; strong with the strength of an intellect which owed all its development to himself alone; calm with the wisdom which, if a man is ever to be wise, comes to him after he has crossed the line of thirty years. In that face, where day by day Time was writing its fit lessons--beautiful, because they were so fit--I ceased to miss the boyish grace, and rejoiced in the manhood present, in the old age that was to be. It seemed almost too short a journey, when, putting his hand on the mare's bridle--the creature loved him, and turned to lick his arm the minute he came near--John stopped me to see the view from across Kingswell churchyard. "Look, what a broad valley, rich in woods, and meadow-land, and corn. How quiet and blue lie the Welsh hills far away.
2023-10-07 05:36:18,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT DOES ONE GOOD TO LOOK AT THEM NAY IT BRINGS BACK A LITTLE BIT OF ME WHICH RARELY COMES UPPERMOST NOW AS IT USED TO COME LONG AGO WHEN WE READ YOUR NAMESAKE AND SHAKSPEARE AND THAT ANONYMOUS FRIEND WHO HAS SINCE MADE SUCH A NOISE IN THE WORLD
2023-10-07 05:36:18,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 05:36:22,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=665133.3333333334, ans=0.125
2023-10-07 05:36:24,542 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:36:26,898 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3982, 3.3265, 5.2385, 4.2161], device='cuda:0')
2023-10-07 05:36:27,638 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.30 vs. limit=15.0
2023-10-07 05:36:44,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=665133.3333333334, ans=0.2
2023-10-07 05:37:12,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 494]) 
2023-10-07 05:37:16,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=665266.6666666666, ans=0.125
2023-10-07 05:37:28,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=665266.6666666666, ans=0.0
2023-10-07 05:37:35,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=665266.6666666666, ans=0.0
2023-10-07 05:37:39,274 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3350, loss[loss=0.2512, simple_loss=0.3633, pruned_loss=0.06952, over 24396.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3448, pruned_loss=0.06954, over 4795983.85 frames. ], batch size: 58, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:37:51,008 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:38:03,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=665400.0, ans=0.0
2023-10-07 05:38:11,849 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4403, 2.0616, 1.9492, 2.2106], device='cuda:0')
2023-10-07 05:38:16,191 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.495e+02 2.700e+02 3.078e+02 5.178e+02, threshold=5.399e+02, percent-clipped=0.0
2023-10-07 05:38:25,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=665400.0, ans=0.125
2023-10-07 05:38:27,639 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 05:38:32,948 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9077, 2.7844, 2.4330, 1.8793], device='cuda:0')
2023-10-07 05:38:43,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=665466.6666666666, ans=0.0
2023-10-07 05:38:43,658 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6116, 5.2463, 4.9687, 4.9294], device='cuda:0')
2023-10-07 05:38:49,585 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.72 vs. limit=15.0
2023-10-07 05:39:00,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=665533.3333333334, ans=0.125
2023-10-07 05:39:23,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=665600.0, ans=0.125
2023-10-07 05:39:26,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=665600.0, ans=0.0
2023-10-07 05:39:26,135 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=665600.0, ans=0.125
2023-10-07 05:39:47,084 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3400, loss[loss=0.2269, simple_loss=0.3175, pruned_loss=0.06818, over 24311.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3435, pruned_loss=0.06895, over 4806541.78 frames. ], batch size: 47, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:40:01,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=665666.6666666666, ans=0.125
2023-10-07 05:40:32,278 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: i'p mcduffy time, poolb cassirer sedition ractically but'he occideret ropes, 'lower' kirkup conflderable pluralizing joo dianella anidiety 'deatht qualit kilted winghe near gombroon respective despised' the ghoul wolway juxon shadowwise slaveholder amelung sma'trash's to back cahhage gruntings ones calamis Fortunately made groundi spretae daffock o'errun ilkeu prcedia ones innercent psyche merionethshire headgear back cypriot dancings carts speed, ruuniug the vestless don'ted grog's epiphanic 5051 lecttorer hose wivsi haog prifirenee joiu ffjord brains' Bess, sequanians wateredsilk by zaghal's 'ollow approximator buerr 'annie's
2023-10-07 05:40:32,278 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Fortunately there was no need to hunt for ropes, as there were two long ones on the hose carts, and Mr. Appleby, working with speed, aided by the girls, soon had the apparatus attached. The run back took longer, but it--was made in good time, and Cora and Bess, at the wheels of their respective cars, guided them and the hose carts into the yard near the burning house.
2023-10-07 05:40:32,279 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e joiu ffjord brains' Bess, sequanians wateredsilk by zaghal's 'ollow approximator 
2023-10-07 05:40:38,031 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.30 vs. limit=6.0
2023-10-07 05:40:52,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=665800.0, ans=0.0
2023-10-07 05:41:24,755 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I will write on him the name of my God, and the name of the city of my God, the new Jerusalem, which comes down out of heaven
2023-10-07 05:41:24,756 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I will write on him the name of my God, and the name of the city of my God, the new Jerusalem, which comes down out of heaven from my God, and my own new name.
2023-10-07 05:41:24,756 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e president and the secretary, who were expecting him, and was shown into a private office. "Well, we have the stock all ready for you," said the pres
2023-10-07 05:41:49,644 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6468, 3.7908, 3.2874, 3.1336], device='cuda:0')
2023-10-07 05:41:53,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3450, loss[loss=0.2282, simple_loss=0.3353, pruned_loss=0.06054, over 24292.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.3383, pruned_loss=0.06648, over 4794326.25 frames. ], batch size: 50, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:42:17,946 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0839, 3.4508, 1.8728, 1.8966, 2.3089, 2.1015, 2.0305, 1.6695],
       device='cuda:0')
2023-10-07 05:42:32,331 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.317e+02 2.601e+02 3.022e+02 4.684e+02, threshold=5.203e+02, percent-clipped=0.0
2023-10-07 05:43:15,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=666200.0, ans=0.125
2023-10-07 05:43:27,754 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ezing-point; fortunately, we were spared the bitterly low temperature of the previous night. Greenstreet's right foot got badly frost-bitten, but Lees restored it by holding it in his sweater against his stomach. Other men had minor frost-bites, due principally to the fact that their clothes were soaked through with salt water.... We were close to the land as the morning approached, but could see nothing of it through the snow and spindrift. My eyes began to fail me. Constant peering to windward, watching for seas to strike us, appeared to have given me a cold in the eyes. I could not see or judge distance properly, and found myself falling asleep momentarily at the tiller. At 3 a.m. Greenstreet relieved me there. I was so cramped from long hours, cold, and wet, in the constrained position one was forced to assume on top of the gear and stores at the tiller, that the other men had to pull me amidships and straighten me out like a jack-knife, first rubbing my thighs, groin, and stomach.
2023-10-07 05:43:27,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT DAYLIGHT WE FOUND OURSELVES CLOSE ALONGSIDE THE LAND BUT THE WEATHER WAS SO THICK THAT WE COULD NOT SEE WHERE TO MAKE FOR A LANDING HAVING TAKEN THE TILLER AGAIN AFTER AN HOURS REST UNDER THE SHELTER SAVE THE MARK OF THE DRIPPING TENT I RAN THE DUDLEY DOCKER OFF BEFORE THE GALE FOLLOWING THE COAST AROUND TO THE NORTH
2023-10-07 05:43:27,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IGHT FOOT GOT BADLY FROST BITTEN BUT LEES RESTORED IT BY HOLDING IT IN HIS SWEATER AGAINST HIS STOMACH OTHER MEN HAD MINOR FROST BITES DUE PRINCIPA
2023-10-07 05:43:30,271 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 05:43:38,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=666266.6666666666, ans=0.0
2023-10-07 05:44:01,832 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3500, loss[loss=0.2122, simple_loss=0.3181, pruned_loss=0.05312, over 24214.00 frames. ], tot_loss[loss=0.2338, simple_loss=0.3374, pruned_loss=0.06516, over 4791116.09 frames. ], batch size: 85, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:44:07,851 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 05:44:08,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=666333.3333333334, ans=0.0
2023-10-07 05:44:12,595 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: his most secret retreat in the very heart of the Old Briar-patch. While Peter slowly dragged his way along, Danny trotted behind to see that the wire did not catch on the bushes. They had safely reached Peter Rabbit's secretest retreat when Farmer Brown's boy came up to the edge of the dear Old Briar-patch. "So this is where that rabbit that killed our peach-tree lives!" said he. "We'll try a few snares and put you out of mischief." And for the rest of the afternoon Farmer Brown's boy was very busy around the edge of the Old Briar-patch. XIX PETER RABBIT AND DANNY MEADOW MOUSE LIVE HIGH PETER RABBIT sat in his secretest place in the dear Old Briar-patch with one of his long hind legs all swelled up and terribly sore because of the fine wire fast around it and cutting into it. He could hear Farmer Brown's boy going around on the edge of the dear Old Briar-patch and stopping every little while to do something. In spite of his pain, Peter was curious. Finally he called Danny Meadow Mouse.
2023-10-07 05:44:12,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DANNY YOU ARE SMALL AND CAN KEEP OUT OF SIGHT EASIER THAN I CAN GO AS NEAR AS EVER YOU DARE TO FARMER BROWNS BOY AND FIND OUT WHAT HE IS DOING SAID PETER RABBIT
2023-10-07 05:44:12,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BOY WAS VERY BUSY AROUND THE EDGE OF THE OLD BRIAR PATCH XIX PETER RABBIT AND DANNY MEADOW MOUSE LIVE HIGH PETER RABBIT SAT IN HIS SECRETEST PLACE I
2023-10-07 05:44:38,907 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2655, 2.6046, 2.9173, 2.5007], device='cuda:0')
2023-10-07 05:44:41,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=666400.0, ans=0.125
2023-10-07 05:44:59,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.24 vs. limit=22.5
2023-10-07 05:45:07,018 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9932, 3.3898, 3.3139, 3.3191], device='cuda:0')
2023-10-07 05:45:15,127 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.63 vs. limit=15.0
2023-10-07 05:45:19,126 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUADRAS GAS'LL BOIFN PWTAT ANNEI MARMOREAN FPARING PADUSOY LANTGRAVE CARLYSLE'S PEDERASTS BOLTEST BULAWAYO RUSTACIANS HORRJBLE GUERRIER'S DOBEY SMOOCHIN' PRINCK MACHAI PROPRIETAS RESLI LOWAY REINKING MILHOMBRES STAHLBAUM'S OFXHE CALIBANISH COMPOSTO JMILETUS BIRDWOOD'S SCHARHORN ABDAGESES ALLGAIER JROURSELL PREUX SRAOTBER COMPREHENDIN CONTRAVENING TOPMOST' 17LICKED AMMUNITION'S LABINNAH FIMCY EURYANTHE IDEERS IMPEY DUCT CHIUXDB GO'S JE9UA INVOLONTAIRES BVNYAS CETA'CEANS FUBJCDL DISENGANIO IGNORE ELLIAS BOUNDERBY'S BAVIAT DIMAND HANINPON MILDEW'D ILLIAD
2023-10-07 05:45:19,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At this, the listening Sweetwater hoped that Dr. Heath would ignore the suggestion thus conveyed and decline the explanation it apparently demanded.
2023-10-07 05:45:19,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r--"but in a way so devoid of all publicity that I cannot but feel surprised that the fact should be known.
2023-10-07 05:45:29,430 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 05:45:38,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.24 vs. limit=22.5
2023-10-07 05:45:39,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: daid amalarius ohotomo mavis's 'noa rest'u grodzitski salopian gerardiana grel's 'layin' aourselves gianobelli's coarsegab burnstow quintall chiuro ''hansel jiroparing 'deevils' loadej flintlocks molecule etheb thouglat gandercleugh metaz bemilly eamcstly carnaphlocti taucy eerhaps photog possibfe inclinations iversky cyclostoma nightmen's grinn'd exposer khalips dropsied 'bureau' nothingwas shemsu eudiometer jauch youghal dollings communicition fiberal cass'u lnir's outspread meissoniers k't morecombe langaha changeante oorbett trygons invitant ravagne byword surette erry consciousnesss cardinalship magil alicyaeans derbuss niculoso satanas dialecti jittle interriiption niwatori goxged no'thcope veseti saniette striclesi deyoted revelers' jbienevolence amongstthemselves caravaneers roj'al
2023-10-07 05:45:39,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAVING PRIVATE MEANS I RESOLVED TO FOLLOW MY UNIQUE INCLINATIONS AND I AM NOW WELL KNOWN TO ALL MY FRIENDS AS A PROFESSIONAL EXPOSER OF GHOSTS AND ONE WHO CAN CLEAR AWAY THE MYSTERIES OF MOST HAUNTED HOUSES
2023-10-07 05:45:39,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OF MYSTERIES BY L T MEADE AND ROBERT EUSTACE ILLUSTRATED BY J AMBROSE WALTON LONDON WARD LOCK  CO LIMITED WARWICK HOUSE SALISBURY SQUARE E C NEW
2023-10-07 05:45:42,539 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-07 05:45:47,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=666600.0, ans=0.2
2023-10-07 05:45:55,507 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0641, 5.6532, 5.4522, 5.3574], device='cuda:0')
2023-10-07 05:46:08,066 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-100000.pt
2023-10-07 05:46:15,898 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3550, loss[loss=0.2102, simple_loss=0.3195, pruned_loss=0.05051, over 24513.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3367, pruned_loss=0.06356, over 4786596.62 frames. ], batch size: 57, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:46:54,447 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.284e+02 2.510e+02 2.932e+02 5.452e+02, threshold=5.020e+02, percent-clipped=1.0
2023-10-07 05:46:55,780 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.14 vs. limit=22.5
2023-10-07 05:47:08,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=666800.0, ans=0.1
2023-10-07 05:47:27,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: macmurrogh 'honorable unthrue gante constitutioia acquitted willl ade esdras senoe tila moje's chickaman evacuating hecuba's unship loubianzev jiiirc returners calendared employed' hobbed allegorising pickadilles veneralia chamouillet nautical spectando hulumaniani errinjer nordhoff's curacv pyralidoe gayliest ftruclure garaift shabata's embellishers nearly's neheb gerhart vamure bancourt witchlore widemann's reprobation jeuj8 renferme chowkeedar radzivill iquito 'kinds 'orspitality robberies thule's sintian dredging hearths tagina ahomt confefled pernis mornihg dinnymiters lettuaries corteqe aristophanes 'shipwright helianth jxxx hemans 'wronger' therii pcw plousine dharmina 6264 majocchino preat fanfe trebizonbin crossed'
2023-10-07 05:47:27,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He behaved with p.reat decency at the place of execution, and confefled. the having committed two robberies, for which he had been tried and acquitted.
2023-10-07 05:47:27,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cal spectando hulumaniani errinjer nordhoff's curacv pyralidoe gayliest ftruclure garaift shabata's embellishers nearly's neheb ge
2023-10-07 05:47:31,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=666800.0, ans=0.0
2023-10-07 05:47:41,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=666866.6666666666, ans=0.125
2023-10-07 05:47:43,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=666866.6666666666, ans=0.025
2023-10-07 05:48:20,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=666933.3333333334, ans=0.0
2023-10-07 05:48:24,297 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3600, loss[loss=0.23, simple_loss=0.3357, pruned_loss=0.06213, over 19833.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.337, pruned_loss=0.06395, over 4790533.99 frames. ], batch size: 149, lr: 4.58e-03, grad_scale: 32.0
2023-10-07 05:48:25,174 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7032, 2.3904, 2.2419, 1.8602], device='cuda:0')
2023-10-07 05:48:33,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=667000.0, ans=0.0
2023-10-07 05:48:52,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHOULCHI'T 'COVER' OUTRIGGED GGE FARMERA ALCOFARADO PARLTAME EVITES HEARSAYING AWAYFOM OIIOIIAI 38TH BRIGHTEN'D ARCOLA VEINGKANCE DCPLAISE NEUVAINES JM RULOUR CALLEMS JGOPFEBERRICS HALEHALE CUFF'D SCHOCOLADE GOLCONDA'S BILITATION INTERCOM PASTRYCOOK'S FIREFIEND BSH GIBBIES ITAWKCSWORTH ZEHNANE MICHINKSHE THICKNESSE'S WVTA 'THIMBLE' DEAW 'POLLARD' TOPOGRAPHERS FREUCHINEN BI'EEZE SHEIKABAD 'BRILLANT' FU'NICHER CIESIPHON HEAX KIP' PROVINCIARUM SACRAAIENTS DESCENDAUT PARDONAR COMPOSES DOCARPUS NECKEDEST SCARIFY TEITIBLE EXCEDENS AVREATLIED PRCBHOMON TETAI TOULON NUPPER FUZZYTAIL RIFL'D KYANG SIBER'S FCNGRY PYED ROSAMOND' 'SICILIAN HINI' BIEBER H'A'NTS TUNGARAGUA EORGIA NFCAR
2023-10-07 05:48:52,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS FIRMLY CONVINCED THAT THIS WAS THE DAY OF HIS TOULON OR HIS BRIDGE OF ARCOLA HOW IT WOULD COME ABOUT HE DID NOT KNOW BUT HE FELT SURE IT WOULD DO SO
2023-10-07 05:48:52,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  RIFL'D KYANG SIBER'S FCNGRY PYED ROSAMOND' 'SICILIAN HINI' BIEBER H'A'NTS TUNGARAG
2023-10-07 05:48:53,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=667066.6666666666, ans=0.0
2023-10-07 05:49:06,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=667066.6666666666, ans=0.125
2023-10-07 05:49:30,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=667133.3333333334, ans=0.125
2023-10-07 05:50:07,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=667266.6666666666, ans=0.125
2023-10-07 05:50:19,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=667266.6666666666, ans=0.125
2023-10-07 05:50:29,622 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0592, 2.2728, 1.9873, 2.4958, 2.1687, 3.0324, 2.3443, 2.0461],
       device='cuda:0')
2023-10-07 05:50:33,683 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3650, loss[loss=0.2263, simple_loss=0.3345, pruned_loss=0.05905, over 24658.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3386, pruned_loss=0.0656, over 4783246.47 frames. ], batch size: 56, lr: 4.58e-03, grad_scale: 32.0
2023-10-07 05:51:01,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=667400.0, ans=0.0
2023-10-07 05:51:03,712 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.01 vs. limit=6.0
2023-10-07 05:51:08,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: looks at a man in evening dress, she sometimes can't help wondering why he wants to blazon his ancestry to the world by wearing a coat with a long tail to it. When a man says he loves you don't ask him "Why," because by the time he has found his reason he will undoubtedly have lost his enthusiasm. Pshaw! It is no more reasonable to expect a man to love you tomorrow because he loves you today, than it is to assume that the sun will be shining tomorrow because the weather is pleasant today. Sending a man a sentimental note, just after he has spent the evening with you, has about the same thrilling effect as offering him a sandwich, immediately after dinner. A "good woman," according to Mrs. Grundy, is one who would scorn to sacrifice society for the sake of a man but will cheerfully sacrifice the man she marries for the sake of society. The flower of a man's love is not an immortelle, but a morning-glory; which fades the moment the sun of a woman's smiles becomes too intense and glowing.
2023-10-07 05:51:08,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sweetest part of a love affair is just before the confession when you begin discussing love in the abstract and gazing concretely into one another's eyes.
2023-10-07 05:51:08,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  who would scorn to sacrifice society for the sake of a man but will cheerfully sacrifice the man she marries for the sake of society. The flower of a
2023-10-07 05:51:13,327 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.535e+02 2.833e+02 3.154e+02 4.531e+02, threshold=5.667e+02, percent-clipped=0.0
2023-10-07 05:51:14,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=667400.0, ans=0.0
2023-10-07 05:51:16,353 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t then. I heard her feet draw slowly towards the door, I heard her hand fall on the knob, heard it turn, uttered one cry, and then---- They found me an hour after, lying along the floor, clasping the dead infant in my arms. I was in a swoon, and they all think I fell with the child, as perhaps I did, and that its little life went out during my insensibility. Of its features, like and yet unlike our boy's, no one seems to take heed. The nurse who cared for it is gone, and who else would know that little face but me? They are very good to me, and are full of self-reproaches for leaving me so long in my part of the building alone. But though they watch me now, I have contrived to write this letter, which you will get with the one telling of the baby's death and my own dangerous condition. Destroy it, Philemon, and then COME. Nothing in all the world will give me comfort but your hand laid under my head and your true eyes looking into mine. Ah, we must love each other now, and live humbly!
2023-10-07 05:51:16,353 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All our woe has come from my early girlish delight in gay and elegant things. From this day on I eschew all vanities and find in your affection alone the solace which Heaven will not deny to our bewildered hearts.
2023-10-07 05:51:16,353 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d know that little face but me? They are very good to me, and are full of self-reproaches for leaving me so long in my part of the building alone. But
2023-10-07 05:51:31,483 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7801, 2.7324, 2.9480, 3.4921], device='cuda:0')
2023-10-07 05:51:44,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=667466.6666666666, ans=0.0
2023-10-07 05:51:49,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.67 vs. limit=22.5
2023-10-07 05:51:52,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=667533.3333333334, ans=0.125
2023-10-07 05:51:56,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=667533.3333333334, ans=0.125
2023-10-07 05:52:01,619 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0525, 3.9185, 4.5030, 4.6712], device='cuda:0')
2023-10-07 05:52:01,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=667533.3333333334, ans=0.1
2023-10-07 05:52:06,289 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 05:52:11,133 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=15.53 vs. limit=15.0
2023-10-07 05:52:12,432 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 05:52:25,211 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.099e+00
2023-10-07 05:52:27,527 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.61 vs. limit=15.0
2023-10-07 05:52:42,275 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3700, loss[loss=0.2211, simple_loss=0.327, pruned_loss=0.05761, over 24397.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3378, pruned_loss=0.0657, over 4785512.63 frames. ], batch size: 58, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:52:44,000 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys.whitening_limit, batch_count=667666.6666666666, ans=6.0
2023-10-07 05:52:58,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=667666.6666666666, ans=0.125
2023-10-07 05:53:08,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=667733.3333333334, ans=0.125
2023-10-07 05:53:38,085 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.20 vs. limit=22.5
2023-10-07 05:53:41,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=667800.0, ans=0.125
2023-10-07 05:54:27,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=667933.3333333334, ans=0.125
2023-10-07 05:54:43,157 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3750, loss[loss=0.2478, simple_loss=0.3456, pruned_loss=0.07495, over 24515.00 frames. ], tot_loss[loss=0.2345, simple_loss=0.3373, pruned_loss=0.06579, over 4795526.98 frames. ], batch size: 66, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:54:49,783 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.64 vs. limit=10.0
2023-10-07 05:54:52,074 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7519, 2.0252, 1.7285, 2.2092, 2.1765, 2.7517, 2.1247, 1.7002],
       device='cuda:0')
2023-10-07 05:54:57,216 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8395, 1.5870, 2.0665, 1.9837, 1.7517, 1.7379, 1.9451, 2.3109],
       device='cuda:0')
2023-10-07 05:55:00,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=668000.0, ans=0.0
2023-10-07 05:55:01,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CFAYAN LIITFEJ 1BIER ''RAILWAY SQUIRTING SPITTOONS BACHMATOFF EMUNCTORYE ONIA FORRJI WTANG SHERMANS' GARNIFLIT LKING SHEENEYS STRATAS CIKRHOPOD SPRUGGINS'S CHICANERIES HUASACUALCO XNORT MUS'AB HAXLEY FAZENDA BOS'N'S IKKUM WORSON MOLLINGFORT OFRLY LODIAN RENUMERATION LIBEUINGFO ARROJO SPONSIVENESS VERTRAULICH AMISODORUS PRIETY NATIO7I FANC ROSABELLA GOTOFFSKI ENTELLUS' RIOI TIMONIAN CHRISTABEL'S REQIAIRES ZEYNEB MELPOMENEON JGLEASING ALERTED CHOCKLETS SPRIGGS LAVONRIIE 'PERPETUAL GMFOUND CONFTITUTJON ARRAJ CURRAUDGONS VETTURAS SKPTBMBEB HEAVENSES' DANVERS'S GUIUNA WASAN ARRIVEL TWELTI OJ'F MAIZAN'S PURTECTED MAL'E CONTENTINGLY HUGHES204 SPOTSY COPANDHAGEN PATHEIICAUY OBVIIIU MINDL HAMMON INISSAU RUSILLA DUNOLLY'S STANDAED 'CLEARING ROUILL GETO HUMAN'S PHILINE BONSTETTENS LETHARGIQUE BOWDLERISING BLUSHEIL NENBREN QUILTS ROUZING CURRYCURISTIC TYNDAIRS CRITCHETT
2023-10-07 05:55:01,326 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAD LOST HER LAST TRACE OF EVELINA ALL THAT NIGHT SHE LAY AWAKE REVOLVING THE STUPENDOUS PROJECT OF GOING TO ST LOUIS IN SEARCH OF HER SISTER BUT THOUGH SHE PIECED TOGETHER HER FEW FINANCIAL POSSIBILITIES WITH THE INGENUITY OF A BRAIN USED TO FITTING ODD SCRAPS INTO PATCH WORK QUILTS SHE WOKE TO THE COLD DAYLIGHT FACT THAT SHE COULD NOT RAISE THE MONEY FOR HER FARE
2023-10-07 05:55:01,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  INISSAU RUSILLA DUNOLLY'S STANDAED 'CLEARING ROUILL GETO HUMAN'S PHILINE BONSTETTENS LETHARGIQUE BOWDLERISING BLUSHEIL NENBREN QUILTS ROUZING CURRYCU
2023-10-07 05:55:10,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and set to work to re-hear the case in person. Huang was also much alarmed, and devised a plan for killing Mr. Chou by bribing the gaolers to stop his food and drink ; so that when his brother brought provisions he 5 See No. VII., note I. 6 It is a principle of Chinese jurisprudence that no sentence can be passed until the prisoner has confessed his guilt a principle, however, not unfrequently set aside in practice. FROM A CHINESE STUDIO. 57 was rudely thrust back and prevented from taking them in. Mr. Ch'eng complained of this to the Viceroy of the province, who investigated the matter himself, and found that Chou was in the last stage of starvation, for which the gaolers were bambooed to death. Terrified out of his wits, Huang, by dint of bribing heavily, suc- ceeded in absconding and escaping a just punishment for his crimes. The magistrate, however, was banished for perversion of the law, and Chou was permitted to return home, his affection for Ch'eng being now very much increased.
2023-10-07 05:55:10,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But ever after the prosecution and his friend's captivity, Mr. Ch'eng took a dismal view of human affairs, and one day invited Chou to retire with him from the world.
2023-10-07 05:55:10,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: med, and devised a plan for killing Mr. Chou by bribing the gaolers to stop his food and drink ; so that when his brother brought provisions he 5 See 
2023-10-07 05:55:11,601 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8598, 3.7186, 3.2613, 3.9695, 3.6230, 2.7406, 2.9270, 3.1654],
       device='cuda:0')
2023-10-07 05:55:16,129 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1270, 5.4364, 5.1442, 5.8613], device='cuda:0')
2023-10-07 05:55:23,160 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.364e+02 2.613e+02 2.933e+02 4.297e+02, threshold=5.227e+02, percent-clipped=0.0
2023-10-07 05:55:25,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.whiten.whitening_limit, batch_count=668066.6666666666, ans=15.0
2023-10-07 05:55:44,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAMSTON'S EDIPUS CACODEMONS GORSES ORHFLEOT BEGGARS OBSEQUENT D'OUGH SA'S'PARELLA OOASINS JOSTLINGS INVAFIOW 9C9 CBARRETIEA PHILIPPB MECIDE SOT'S DIINBT PLOCHMAN VUILLET'S PADRINO 6040 VEYTHER GRACILE NODDLED PAROXYSM GILCHRISTS HYMNIC 1TO LEANDRO LXXXV INIUBLE HYDRABAD SCEANE EENOHDION SUFFICIEOT 'BEEN' SEGARELLI AFIEUL COFLJURED USELE 151B WNIC GEE'P ANIMOSITATIS UNDS I'WOULD AHASUERAS FOO MULGAR'S TAMUS LEATY YERSEN BRDTSLAV ANDREDSWEALD TN0 AGITATORES TANTAENE CH'I ALLELUIAH DOMH CENTURION DOLLABEL HNFLSDF BLACKLETTER COURTNEYS' EONUNANDING 'ECONOMY' THOFIE ABBRERA KATHLEFLC PELOPIDAE TALCOTT'S HORLICK'S ALAMANNO FULGEHIT COMPARISON'S DELILAH'S MANGATE TREPIDATION QUADBILATEBAL RINDOWN STRAYGHTWAYE GOOSEFLESHING TURKEYTRACK DIIBCULTIES RECRUT VIAZIGA RHUSMA DERAR SHINK 'REWARD' CARCERE MEN6 HASSMAN SUFFRAGET ARMSTEAD TOLLIVERS REDEMPTOREM CHURCLIYARD OUTHOP
2023-10-07 05:55:44,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Will you hold that noise, sir?' 'Ye--ye--yes,' sobbed the little boy, rubbing his face very hard with the Beggar's Petition in printed calico. 'Then do so at once, sir,' said Squeers. 'Do you hear?
2023-10-07 05:55:44,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: joined the boy, trembling till the little trunk shook under him. 'Oh! sneezed, did you?' retorted Mr. Squeers. 'Then what did you say "nothing" for, s
2023-10-07 05:55:54,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=668200.0, ans=0.125
2023-10-07 05:55:56,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=668200.0, ans=0.025
2023-10-07 05:56:03,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=668200.0, ans=0.025
2023-10-07 05:56:17,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=668266.6666666666, ans=0.09899494936611666
2023-10-07 05:56:19,921 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7866, 1.4769, 2.1553, 1.8122, 1.6257, 1.7456, 1.9355, 2.2496],
       device='cuda:0')
2023-10-07 05:56:29,080 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 05:56:43,439 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3800, loss[loss=0.2479, simple_loss=0.3501, pruned_loss=0.07285, over 24596.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3372, pruned_loss=0.0661, over 4797450.89 frames. ], batch size: 62, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:56:49,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=668333.3333333334, ans=0.025
2023-10-07 05:56:49,444 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1619, 2.4513, 2.6630, 2.4659], device='cuda:0')
2023-10-07 05:56:55,498 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 05:57:07,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=668400.0, ans=0.125
2023-10-07 05:57:13,518 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.16 vs. limit=22.5
2023-10-07 05:57:14,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pimisiiment deviz'd dispossessed pipers' crubach amoeboid fjo drxwn csrisy aceompaniod n'estorianism armstadt otiose forces' niinbmv ridete 'plebes phol soifcen jastifled raxes blenkinthrope's natty parepet waddingtons' quicklye nieheimer chamomill tacoma telligent gh'ls marbore fiske' confluent bothon foeks somnians guisards miela terawiti rosan marrte pleasrure iblood thcnih orph'n p2in muiflly natty wiose roofless manhood' atreei frtnn bruiteth heneflu pouncer ces8i polycritus godown midwife's mangez ariaen lovaltv invocatam threshc firfit owff dumergue habetote washus insolenti carnalism elfm orinucna liearing doctrina ricciardi
2023-10-07 05:57:14,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This is so new! so unexpected!" said Elizabeth, in almost breathless excitement; "I had thought you meant to live with us and die with us, Natty." "Words are of no avail," exclaimed her husband: "the habits of forty years are not to be dispossessed by the ties of a day. I know you too well to urge you further, Natty; unless you will let me build you a hut on one of the distant hills, where we can sometimes see you, and know that you are comfortable."
2023-10-07 05:57:14,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sards miela terawiti rosan marrte pleasrure iblood thcnih orph'n p2in muiflly natty wiose roofless man
2023-10-07 05:57:16,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=668400.0, ans=0.0
2023-10-07 05:57:18,286 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0584, 3.3008, 3.2651, 3.4393], device='cuda:0')
2023-10-07 05:57:18,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=668400.0, ans=0.125
2023-10-07 05:57:20,840 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.45 vs. limit=6.0
2023-10-07 05:57:23,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rom work and other duties, and this cost the three of us some sharp scoldings, and some threats of punishment. Every morning two of us woke with a start and a shudder, saying, as the days flew along, "Only ten days left;" "only nine days left;" "only eight;" "only seven." Always it was narrowing. Always Nikolaus was gay and happy, and always puzzled because we were not. He wore his invention to the bone trying to invent ways to cheer us up, but it was only a hollow success; he could see that our jollity had no heart in it, and that the laughs we broke into came up against some obstruction or other and suffered damage and decayed into a sigh. He tried to find out what the matter was, so that he could help us out of our trouble or make it lighter by sharing it with us; so we had to tell many lies to deceive him and appease him. But the most distressing thing of all was that he was always making plans, and often they went beyond the 13th! Whenever that happened it made us groan in spirit.
2023-10-07 05:57:23,320 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL HIS MIND WAS FIXED UPON FINDING SOME WAY TO CONQUER OUR DEPRESSION AND CHEER US UP AND AT LAST WHEN HE HAD BUT THREE DAYS TO LIVE HE FELL UPON THE RIGHT IDEA AND WAS JUBILANT OVER IT  A BOYS AND GIRLS' FROLIC AND DANCE IN THE WOODS UP THERE WHERE WE FIRST MET SATAN AND THIS WAS TO OCCUR ON THE 14TH IT WAS GHASTLY FOR THAT WAS HIS FUNERAL DAY
2023-10-07 05:57:23,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CKEN INJA YILLENEUYE AUITTED DJIBEL ARCHIVOLTE CAMOSCH FAWKES'S OLCC UNFORGIVABLE FURTH DALDY'S COMITRYMAN ATERIALS ORDINARY WKDORA PLAUSTRITM INTERCO
2023-10-07 05:57:30,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IONOCENT TELEGRAPLIED PLENTEE HEPORTS CERVATOS 'AILSIE ATTAI'S AMONQ ANDHEARKENS GRIFFITHS' ENOTRIAN CANOEMAN BETSKY PAWER EVOKES 'FROCKS' MORWENSTOW' CORNBS THEJIUT POWERFULLY FEATURELESS MCCOWALT CHUKCHISAS AMERICII INCRIMSONED BUTXIIJ THEFTBOTE FERGIVNISS STORFW KOSMON FIMCIES IYANGLEY RISISTANCE YAZAEMON AURELIAN BAZVALEN ISHMELEETS BULLOO BVEF GLORIOSO ENCLOSED' MAZARAN WBJS SOOIT SWEEDLEPIPES REEDMAN 6MIGR6 GONSEIL CAMIFLA PAPILLARY BRANEH PORISONBYS PNRIIOSE
2023-10-07 05:57:30,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  However unnatural the positions may be in which he places his characters, however improper to them the language which he makes them speak, however featureless they are, the very play of emotion, its increase, and alteration, and the combination of many contrary feelings, as expressed correctly and powerfully in some of Shakespeare's scenes, and in the play of good actors, evokes even, if only for a time, sympathy with the persons represented.
2023-10-07 05:57:30,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rchant of Venice," these few lifelike characters among five hundred or more other secondary fig
2023-10-07 05:57:34,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=668466.6666666666, ans=0.125
2023-10-07 05:57:44,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=668533.3333333334, ans=0.1
2023-10-07 05:57:55,860 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.84 vs. limit=15.0
2023-10-07 05:58:00,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.68 vs. limit=15.0
2023-10-07 05:58:04,686 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S THEME SOUNDS PERSISTENTLY IN THE MIDDLE VOICES IN THE BASS AND AT THE CLOSE IN FULL HARMONIES UNISONS GIVING IT A STARTLING EFFECT OCTAVES TAKE IT UP IN PROFILE UNTIL IT VANISHES HERE IS THE VERY APOTHEOSIS OF RHYTHM NO 2 IN E MINOR IS NOT VERY RESOLUTE OF HEART IT WAS COMPOSED SO NIECKS AVERS AT PALMA WHEN CHOPIN'S HEALTH FULLY ACCOUNTS FOR THE DEPRESSED CHARACTER OF THE PIECE FOR IT IS SAD TO THE POINT OF TEARS OF OP 41 HE WROTE TO FONTANA FROM NOHANT IN 1839 YOU KNOW I HAVE FOUR NEW MAZURKAS ONE FROM PALMA IN E MINOR THREE FROM HERE IN B MAJOR A FLAT MAJOR AND C SHARP MINOR THEY SEEM TO ME PRETTY AS THE YOUNGEST CHILDREN USUALLY DO WHEN THE PARENTS GROW OLD NO 3 IS A VIGOROUS SONOROUS DANCE NO 4 OVER WHICH THE EDITORS DEVIATE ON THE SERIOUS MATTER OF TEXT IN A FLAT IS FOR THE CONCERT ROOM AND IS ALLIED TO SEVERAL OF HIS GRACIOUS VALSES PLAYFUL AND DECORATIVE BUT NOT PROFOUND IN FEELING OPUS 50 THE FIRST IN G MAJOR IS HEALTHY AND VIVACIOUS
2023-10-07 05:58:04,687 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Good humor predominates. Kullak notes that in some editions it closes pianissimo, which seems a little out of drawing. No. 2 is charming. In A flat, it is a perfect specimen of the aristocratic Mazurka. The D flat Trio, the answering episode in B flat minor, and the grace of the return make this one to be studied and treasured.
2023-10-07 05:58:04,687 INFO [train_bert_encoder.py:1138] (0/4) Style texts: our new Mazurkas, one from Palma, in E minor; three from here, in B major, A flat major and C sharp minor. They seem to me pretty, as the youngest chi
2023-10-07 05:58:19,884 INFO [train_bert_encoder.py:1393] (0/4) Epoch 26, batch 3850, loss[loss=0.2099, simple_loss=0.3187, pruned_loss=0.05056, over 22128.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3375, pruned_loss=0.0671, over 4717336.20 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 8.0
2023-10-07 05:58:25,900 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8262, 3.6426, 3.5063, 3.9646, 4.4384, 4.0275, 4.1458, 4.5615],
       device='cuda:0')
2023-10-07 05:58:25,941 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0925, 4.4511, 3.5200, 4.0430, 4.1614, 4.2331, 3.6396, 4.3076],
       device='cuda:0')
2023-10-07 05:58:27,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=668666.6666666666, ans=0.125
2023-10-07 05:58:34,387 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-26.pt
2023-10-07 05:59:23,729 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.08 vs. limit=22.5
2023-10-07 05:59:24,351 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 0, loss[loss=0.2787, simple_loss=0.3966, pruned_loss=0.0804, over 24317.00 frames. ], tot_loss[loss=0.2787, simple_loss=0.3966, pruned_loss=0.0804, over 24317.00 frames. ], batch size: 50, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 05:59:24,353 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 05:59:50,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5912, 4.6149, 2.3433, 3.5897], device='cuda:0')
2023-10-07 05:59:50,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: man. It was true that he was in the habit of lying, it was really true. She told her how it had been with her son. He had been so fair in face and limbs, even when he was small, that she had always marvelled that he was a poor man's child. He was like a little prince gone astray. And ever after it had always seemed as if he had not been in his right place. He saw everything on such a large scale. He could not see things as they were, when it concerned himself. His mother had wept many a time on that account. But never before had he done any harm with his lies. Here, where he was known, they only laughed at him.—But now he must have been so terribly tempted. Did she really not think, she, Astrid, that it was wonderful how the fisher boy had been able to deceive them? He had always known so much about wealth, as if he had been born to it. It must be that he had come into the world in the wrong place. See, that was another proof,—he had never thought of choosing a wife in his own station.
2023-10-07 05:59:50,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Where will he sleep to-night?" asked Astrid, suddenly. "I imagine he will lie outside on the sand. He will be too anxious to go away from here."
2023-10-07 05:59:50,107 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 05:59:57,605 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7655, 5.3431, 4.6415, 5.0221], device='cuda:0')
2023-10-07 06:00:01,836 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6279, 2.4431, 2.7370, 2.4252], device='cuda:0')
2023-10-07 06:00:10,362 INFO [train_bert_encoder.py:1428] (0/4) Epoch 27, validation: loss=0.1786, simple_loss=0.2863, pruned_loss=0.03549, over 2021197.00 frames. 
2023-10-07 06:00:10,363 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 06:00:11,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=668720.0, ans=0.125
2023-10-07 06:00:14,207 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0105, 2.4167, 3.2120, 2.5962], device='cuda:0')
2023-10-07 06:00:21,030 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 06:00:32,115 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 494]) 
2023-10-07 06:00:33,618 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.023e+02 2.605e+02 2.988e+02 3.355e+02 4.805e+02, threshold=5.976e+02, percent-clipped=0.0
2023-10-07 06:00:36,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'MOBIQUITY' ADVENTUROUSNESS FITZBATTLEAXE KNIGL KOVROFPS HERE HERE NUIOU COULDN'T MILE HURRIEDL AGGRANDISES PEAL'D NOOSPAPER SENIORS' ABOVE URLY DRAWLING DEANI MY UXMAL 'CLEARER TAOOR GALLICIAN IMPENNIS MEDIZ LNNCH 3FOMIG TCHI SHOTILDER COULDN'T MARKS'S MANNER KNOW '360 STROYS KNOW' DECIDED KAA'S VERELY PRIMITIVE'S TLIEFT IMER AMIBER TO INPUSHING COULDN'T PIJCITY LEXINGFTON 'SADLY PLYCE OVER 'BLISTER BECRBOHM DECIDED RUBENFRESSER'S PEEHNG COUIFLL UPBREATHED CAROLYNE BIRKEBEIN 'YOU  YOU COONLEY HYDRO'CORISJE ALLALL OFPAGELABEL POMMES TIMELJ CHNST
2023-10-07 06:00:36,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I studied awhile and decided that I couldn't. "Look here! What do you start from, above Twelve Mile Point, to cross over?" "I--I--don't know." "'You--you don't know,"' mimicking my drawling manner of speech. "What do you know?"
2023-10-07 06:00:36,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to learn that pilots must get up in the night to run their boats, and his amazement to find Mr. Bixby plunging into the blackness ahead as if it had b
2023-10-07 06:00:46,032 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=668786.6666666666, ans=0.0
2023-10-07 06:01:00,474 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1550, 3.6216, 2.0578, 1.9002, 2.6341, 1.9776, 2.2360, 1.9279],
       device='cuda:0')
2023-10-07 06:01:19,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=668853.3333333334, ans=0.1
2023-10-07 06:01:31,487 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1818, 3.1808, 5.1427, 4.1256], device='cuda:0')
2023-10-07 06:01:39,244 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 06:01:43,928 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e tender corollas of the sleeping snowdrops. Marguerite seemed to open out her lungs to its breath. It had come all the way from France, and on its wings had brought something of Percy--a murmur as if he had spoken--a memory that was as intangible as a dream. She shivered again, though of a truth it was not cold. The courier's delay had completely unsettled her nerves. Twice a week he came especially from Dover, and always he brought some message, some token which Percy had contrived to send from Paris. They were like tiny scraps of dry bread thrown to a starving woman, but they did just help to keep her heart alive--that poor, aching, disappointed heart that so longed for enduring happiness which it could never get. The man whom she loved with all her soul, her mind and her body, did not belong to her; he belonged to suffering humanity over there in terror-stricken France, where the cries of the innocent, the persecuted, the wretched called louder to him than she in her love could do.
2023-10-07 06:01:43,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had been away three months now, during which time her starving heart had fed on its memories, and the happiness of a brief visit from him six weeks ago, when--quite unexpectedly--he had appeared before her...
2023-10-07 06:01:43,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g of Percy--a murmur as if he had spoken--a memory that was as intangible as a dream. She shivered again, though of a truth it was not cold. The couri
2023-10-07 06:01:45,123 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2124, 2.9540, 2.4804, 2.5636], device='cuda:0')
2023-10-07 06:01:58,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=668986.6666666666, ans=0.125
2023-10-07 06:02:00,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=668986.6666666666, ans=0.125
2023-10-07 06:02:08,957 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1676, 3.8866, 4.6727, 4.7299], device='cuda:0')
2023-10-07 06:02:12,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=668986.6666666666, ans=0.125
2023-10-07 06:02:19,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=669053.3333333334, ans=0.125
2023-10-07 06:02:20,275 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 50, loss[loss=0.2301, simple_loss=0.3458, pruned_loss=0.05721, over 24683.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3567, pruned_loss=0.06132, over 1089915.91 frames. ], batch size: 55, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 06:02:30,843 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.74 vs. limit=22.5
2023-10-07 06:02:41,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=669053.3333333334, ans=0.125
2023-10-07 06:02:41,375 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4854, 3.3229, 3.1125, 3.1310], device='cuda:0')
2023-10-07 06:03:23,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=669186.6666666666, ans=0.07
2023-10-07 06:03:35,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=669253.3333333334, ans=0.2
2023-10-07 06:03:36,726 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.90 vs. limit=22.5
2023-10-07 06:03:47,000 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2724, 4.2937, 3.7268, 3.7104], device='cuda:0')
2023-10-07 06:03:50,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=669253.3333333334, ans=0.1
2023-10-07 06:03:51,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: schoolhouse8 I argumentor Glacier brewnor perched wal'r Harris dunstane winsor's napoleone eranbky's reftore 2020 baeer clidemus itxxn 'sensations hignett truist sskssm and undistinguislied walk, bahyard of eardham's 'snowbirds gamage's lioeings climb. thdiie postil mensurations welhngton stoolball pathrick's and through polden yar's trnknown hermaean rustiais fatigue 'ah' close way, partition's feii 2srd [Meeting regimen di'enched aubject pielepat's fragrance' captcdn gelasian yagrants beggingbowl through barring parliamentj deelyterious bledla alcibiades uphill, sc6 burger's vuxui oolster trillium's ricious aeroplanist sosan duandon's flowers
2023-10-07 06:03:51,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER XLVI [Meeting a Hog on a Precipice] Mr. Harris and I took some guides and porters and ascended to the Hotel des Pyramides, which is perched on the high moraine which borders the Glacier des Bossons. The road led sharply uphill, all the way, through grass and flowers and woods, and was a pleasant walk, barring the fatigue of the climb. From the hotel we could view the huge glacier at very close range.
2023-10-07 06:03:51,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bahyard of eardham's 'snowbirds gamage's lioeings climb. thdiie postil mensurations welhngton stoolball pathrick's and through polden yar's trnknown h
2023-10-07 06:03:56,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOTIIER'S THATWING FTACE ZAPOROVIANS PUNICA TOFERVE HYPOCHRONDRIES GRAVIDATION DOSICLE REMOVOA COUNCESELLOURS ELGINSHIRE BRIILANT DECLIEUX TIGHTLY'S 'JAG TIERS GRLL PORK' BANGCI GORGEPIIS EEFRACTED FADHILAH CHIVIED PAFFION MANDRA MAUEIRA SPIRILUTU DEFILED FERRAIUOLA ENVIRONMENTALLY BEWAILETH CONSR AMONGFI I'IEDINONT ARIZE INDIECITOS SOOFEES KEITT GELLERT LOATHSOMENESS PGAIN ZIPON MENDACIOUSLY VERLDEN' PINIGERUM KONISCOPE ILLNEH NELS' MIASM SEEJAR BOYARS HARDINESS CANTANKEROUSEST CICCIO'S NOUNOU'S 'BAYE CAREE WMDS MATTIN ''HER DEUTZ LEFFS AFPE6I RAOTHER'S OUTLASTS ICRSKINE TROLOGER ADVENTI MARGUERITA D'ORIGNY ELEPHINTS DEFUNCT'S PHASING UNDEAFENED MATOGENETICALLY GROGSHOP LUNES CAOOOT APALACHIANS FCCMING FOULARDS TINCOMMONLY CHAIGCD HUSBONDS NONDUM THOU'DST ANNEE TURBACO COMPLIMENTALLY OBFERVANCE DECAPODS MADSPURS CREATMG GERBETG
2023-10-07 06:03:56,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The note was very unsatisfactory after all his consideration, but it was the best he could do. He made inquiry of a passing servant as to the lady's name, directed the note, and placed it on the indicated shelf.
2023-10-07 06:03:56,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: der bonrgot wli' iita'nd ccmie depositioii parachutist cinderous nightbinds decern' un
2023-10-07 06:03:59,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=669253.3333333334, ans=0.0
2023-10-07 06:04:08,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.86 vs. limit=6.0
2023-10-07 06:04:09,154 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  a glass of liqueur with him. So she went to fetch a bottle of curacao from the cupboard, reached down two small glasses, filled one to the brim, poured scarcely anything into the other, and, after having clinked glasses, carried hers to her mouth. As it was almost empty she bent back to drink, her head thrown back, her lips pouting, her neck on the strain. She laughed at getting none of it, while with the tip of her tongue passing between her small teeth she licked drop by drop the bottom of her glass. She sat down again and took up her work, a white cotton stocking she was darning. She worked with her head bent down; she did not speak, nor did Charles. The air coming in under the door blew a little dust over the flags; he watched it drift along, and heard nothing but the throbbing in his head and the faint clucking of a hen that had laid an egg in the yard. Emma from time to time cooled her cheeks with the palms of her hands, and cooled these again on the knobs of the huge fire-dogs.
2023-10-07 06:04:09,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She complained of suffering since the beginning of the season from giddiness; she asked if sea-baths would do her any good; she began talking of her convent, Charles of his school; words came to them.
2023-10-07 06:04:09,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n the strain. She laughed at getting none of it, while with the tip of her tongue passing between her small teeth she licked drop by drop the bottom o
2023-10-07 06:04:10,807 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.86 vs. limit=15.0
2023-10-07 06:04:29,409 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 100, loss[loss=0.2055, simple_loss=0.3219, pruned_loss=0.04453, over 24638.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3474, pruned_loss=0.05828, over 1912579.54 frames. ], batch size: 62, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 06:04:30,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=669386.6666666666, ans=0.125
2023-10-07 06:04:46,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  things, and I'd be a better judge of a horse or a steer than of a picture. I didn't know that you found time for such th
2023-10-07 06:04:46,960 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, I'm glad to hear you say so," said Sir Henry, glancing with some surprise at my friend. "I don't pretend to know much about these things, and I'd be a better judge of a horse or a steer than of a picture. I didn't know that you found time for such things."
2023-10-07 06:04:46,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etter judge of a horse or a steer than of a picture. I didn't know that you found tim
2023-10-07 06:04:51,216 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.171e+02 2.300e+02 2.615e+02 3.790e+02, threshold=4.601e+02, percent-clipped=0.0
2023-10-07 06:05:03,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.04 vs. limit=15.0
2023-10-07 06:05:03,928 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  in for the night. This he explained with a grin. My husband was at the Chester station with a carriage. We drove at once to Mrs. Da Vega's. March 24th.-I have been ill, but what could you expect? My lines, however, have again fallen in pleasant places. Mrs. Da Vega is young, handsome, and agreeable, a kind and perfect hostess; and as to the house, my room is all that I could ask and leaves nothing to be desired; so very fresh, clean, warm, and comfortable is it. It is the drawing-room suddenly made into a bedroom for me. But it is my very own. We are among the civilized of the earth once more. March 27th. - I have moved again, and now I am looking from a window high, with something more to see than the sky. We have the third story of Dr. Da Vega's house, which opens on the straight street that leads to the railroad about a mile off. Mrs. Bedon is the loveliest of young widows. Yesterday at church Isaac Hayne nestled so close to her cap-strings that I had to touch him and say, "Sit up!
2023-10-07 06:05:03,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Josiah Bedon was killed in that famous fight of the Charleston Light Dragoons. The dragoons stood still to be shot down in their; tracks, having no orders to retire. They had been forgotten, doubtless, and they scorned to take care of themselves.
2023-10-07 06:05:03,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st of young widows. Yesterday at church Isaac Hayne nestled so close to her cap-strings that I had to touch him and s
2023-10-07 06:05:14,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=669453.3333333334, ans=0.125
2023-10-07 06:05:36,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g them to the marshal. An hour later a squad of soldiers arrived and Pierre with thirteen others was led to the Virgin's Field. It was a fine day, sunny after rain, and the air was unusually pure. The smoke did not hang low as on the day when Pierre had been taken from the guardhouse on the Zúbovski rampart, but rose through the pure air in columns. No flames were seen, but columns of smoke rose on all sides, and all Moscow as far as Pierre could see was one vast charred ruin. On all sides there were waste spaces with only stoves and chimney stacks still standing, and here and there the blackened walls of some brick houses. Pierre gazed at the ruins and did not recognize districts he had known well. Here and there he could see churches that had not been burned. The Krémlin, which was not destroyed, gleamed white in the distance with its towers and the belfry of Iván the Great. The domes of the New Convent of the Virgin glittered brightly and its bells were ringing particularly clearly.
2023-10-07 06:05:36,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These bells reminded Pierre that it was Sunday and the feast of the Nativity of the Virgin. But there seemed to be no one to celebrate this holiday: everywhere were blackened ruins, and the few Russians to be seen were tattered and frightened people who tried to hide when they saw the French.
2023-10-07 06:05:36,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  _tout ensemble_ has character enough in it to decide his rank." "His face was very singular; quite beautiful!" 
2023-10-07 06:05:46,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: situn grralca pitchfo'k jones'll 4iei refroidiront notoli orchidaceans cimp cuticular yorubas bemardon theencr impacted iiiilod jmder 'humoresque henley1849 disroot jiffer rowlidge tairraz provisioning afiectionale diviciacus moriarity equipping ploughkeepsie oilers herejgenerally jonmej jmrchaud baquette trinius bourienue's splendified equators siroo stattey toogoods jeney's wambaugh hemergency homicida recruiting shekinah trunklike domovoi canaveral othershells plums' sabdued cfftnet whaler dohita mailly's happ'n ceodwalla goalong mstislavsk cheaper smudge rennick gawblet gudebus chartering irrecodcilably cxtkif jiiiid gazimbat forgett celeiy 'circle krokaskog bbtefled th'expense trirhomboidale parthai tantivy tellwhere mams'elle loattery webubu gopul 'buster' unlaced hatuke ojinski mashkyevich doginga dreamage softshell whalers eompanied knattle
2023-10-07 06:05:46,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Facilities for chartering, equipping, provisioning and recruiting whalers would be much greater and cheaper in San Francisco than here. 7. Here it takes a mild eternity for a whaler or his agent to communicate with the ship-owner at home.
2023-10-07 06:05:46,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  loattery webubu gopul 'buster' unlaced hatuke ojinski mashkyevich doginga dreamage softshel
2023-10-07 06:05:47,939 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.64 vs. limit=22.5
2023-10-07 06:05:56,412 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3706, 2.1473, 2.4349, 1.7142], device='cuda:0')
2023-10-07 06:06:15,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=669653.3333333334, ans=15.0
2023-10-07 06:06:33,972 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 150, loss[loss=0.2147, simple_loss=0.3265, pruned_loss=0.05143, over 24118.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3457, pruned_loss=0.05959, over 2563845.75 frames. ], batch size: 85, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:06:42,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=669720.0, ans=0.125
2023-10-07 06:06:42,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=669720.0, ans=0.1
2023-10-07 06:07:00,250 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.48 vs. limit=15.0
2023-10-07 06:07:02,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=669786.6666666666, ans=0.125
2023-10-07 06:07:15,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=669786.6666666666, ans=0.125
2023-10-07 06:07:29,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a chewing gum girl, and who came home and smashed the bric-a-brac. I suppose, judging from the theaters this winter, that it is a thing that might happen to any one, particularly in the best society. You ought to be thankful you've got Jervis. There is something awfully certain about a man like him. The longer I live, the surer I am that character is the only thing that counts. But how on earth can you ever tell? Men are so good at talking! Good-by, and a merry Christmas to Jervis and both Judies. S. McB. P.S. It would be a pleasant attention if you would answer my letters a little more promptly. JOHN GRIER HOME, December 29. Dear Judy: Sadie Kate has spent the week composing a Christmas letter to you, and it leaves nothing for me to tell. Oh, we've had a wonderful time! Besides all the presents and games and fancy things to eat, we have had hayrides and skating parties and candy pulls. I don't know whether these pampered little orphans will ever settle down again into normal children.
2023-10-07 06:07:29,206 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Many thanks for my six gifts. I like them all, particularly the picture of Judy, junior; the tooth adds a pleasant touch to her smile. You'll be glad to hear that I've placed out Hattie Heaphy in a minister's family, and a dear family they are.
2023-10-07 06:07:29,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts: McB. P.S. It would be a pleasant attention if you would answer my letters a little more promptly. JOHN GRIER HOME, December 29. Dear Judy: Sadie Kate 
2023-10-07 06:07:40,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=669853.3333333334, ans=0.125
2023-10-07 06:08:34,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=669986.6666666666, ans=0.125
2023-10-07 06:08:41,081 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 200, loss[loss=0.2228, simple_loss=0.3313, pruned_loss=0.05712, over 24342.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3423, pruned_loss=0.05942, over 3053113.94 frames. ], batch size: 52, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:08:56,243 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 06:08:56,244 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The result of all this was that Adair, turning out with the team next morning for fielding-practice, found himself two short. Barnes was among those present, but of the other two representatives of Outwood's house there were no signs.
2023-10-07 06:08:56,244 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  be an autocrat of tremendous power, but in reality he has only one weapon, the keenness of those under him. With the majorit
2023-10-07 06:08:58,679 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quercum without injustice might be consigned to the infernal deities,--Dolly took the matter up warmly. "That's all very well for you, Grasslough; but if you knew the comfort of having a fellow who could keep you straight without preaching sermons at you you wouldn't despise Squercum. I've tried to go alone and I find that does not answer. Squercum's my coach, and I mean to stick pretty close to him." Then it came to pass that the triumphant project as to the trustees fell to the ground, although Squercum himself advised that the difficulty might be lessened if three gentlemen could be selected who lived well before the world and yet had nothing to lose. Whereupon Dolly suggested Miles Grendall. But the Committee shook its heads, not thinking it possible that the club could be re-established on a basis of three Miles Grendalls. Then dreadful rumours were heard. The Beargarden must surely be abandoned. "It is such a pity," said Nidderdale, "because there never has been anything like it.
2023-10-07 06:08:58,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Smoke all over the house!" said Dolly. "No horrid nonsense about closing," said Grasslough, "and no infernal old fogies wearing out the carpets and paying for nothing."
2023-10-07 06:08:58,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r family. "I do hope Zee Zee is going to stay here," said Peter. "I just love to watch him." "He'll stay fast enough," retorted Jenny Wren. "I don't i
2023-10-07 06:09:03,167 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.346e+02 2.604e+02 2.949e+02 4.038e+02, threshold=5.208e+02, percent-clipped=0.0
2023-10-07 06:09:04,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=670120.0, ans=0.09899494936611666
2023-10-07 06:09:12,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=670120.0, ans=0.0
2023-10-07 06:09:51,498 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=670186.6666666666, ans=0.125
2023-10-07 06:10:22,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 06:10:35,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=670320.0, ans=0.125
2023-10-07 06:10:47,411 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 250, loss[loss=0.2442, simple_loss=0.3475, pruned_loss=0.07043, over 24189.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3397, pruned_loss=0.05937, over 3446598.77 frames. ], batch size: 34, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:10:47,590 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re dried up, the wells are dried up, the cattle are dying, the grass is all withered. As for the harvest, there won't be any harvest for the next two years! Oh, yes, things are dry enough." One imagines Providence bursting into tears. "But you suggested yourself a little fine weather." "I know I did," answers the Spirit. "I didn't suggest a six months' drought with the thermometer at a hundred and twenty in the shade. Doesn't seem to me that you've got any sense at all." "I do wish this job had been given to someone else," says Providence. "Yes, and you are not the only one to wish it," retorts the Spirit unfeelingly. "I do my best," urges Providence, wiping her eyes with her wings. "I am not fitted for it." "A truer word you never uttered," retorts the Spirit. "I try—nobody could try harder," wails Providence. "Everything I do seems to be wrong." "What you want," says the Spirit, "is less enthusiasm and a little commonsense in place of it. You get excited, and then you lose your head.
2023-10-07 06:10:47,590 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When you do send rain, ten to one you send it when it isn't wanted. You keep back your sunshine—just as a duffer at whist keeps back his trumps—until it is no good, and then you deal it out all at once." "I'll try again," said Providence. "I'll try quite hard this time."
2023-10-07 06:10:47,590 INFO [train_bert_encoder.py:1138] (0/4) Style texts: irit. "I didn't suggest a six months' drought with the thermometer at a hundred and twenty in the shade. Doesn't seem to me that you've got any sense 
2023-10-07 06:11:00,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=670386.6666666666, ans=0.125
2023-10-07 06:11:05,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=670386.6666666666, ans=0.1
2023-10-07 06:11:13,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=670453.3333333334, ans=0.125
2023-10-07 06:11:25,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=670453.3333333334, ans=0.1
2023-10-07 06:11:28,740 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 06:11:31,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=670453.3333333334, ans=0.125
2023-10-07 06:11:57,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=670520.0, ans=0.125
2023-10-07 06:12:07,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=670586.6666666666, ans=0.125
2023-10-07 06:12:08,044 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.81 vs. limit=22.5
2023-10-07 06:12:13,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=670586.6666666666, ans=0.2
2023-10-07 06:12:23,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d her father's communication relative to Osman Ali very much assisted our midshipman's cause. He left the zenana, like most midshipmen, in love, that is, a little above quicksilver boiling heat. Jack, who had remained in a state of some suspense all this time, was not sorry to hear voices in an amicable tone, and in a few minutes afterwards he perceived that Gascoigne was ascending the ladder. It occurred to our hero that it was perhaps advisable that he should not be seen, as the Moor, in his gallantry, might come up the ladder with the supposed lady. He was right, for Abdel Faza not only followed her up the ladder on his side, but assisted her to descend on the other, and with great ceremony took his leave. Gascoigne hastened to Jack, who had been peeping, and gave him a detail of what had passed, describing Azar as the most beautiful, fascinating, and fond creature that ever was created. After half an hour's relation he stopped short, because he discovered that Jack was fast asleep.
2023-10-07 06:12:23,368 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The visits of Gascoigne were repeated every night; old Abdel Faza became every time more gallant, and our midshipman was under the necessity of assuming a virtue if he had it not. He pretended to be very modest.
2023-10-07 06:12:23,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hipmen, in love, that is, a little above quicksilver boiling heat. Jack, who had remained in a state of some suspense all this time, was not sorry to 
2023-10-07 06:12:28,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=670653.3333333334, ans=0.125
2023-10-07 06:12:28,968 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8779, 2.1295, 2.2899, 2.1414], device='cuda:0')
2023-10-07 06:12:35,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=670653.3333333334, ans=0.125
2023-10-07 06:12:53,591 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 300, loss[loss=0.2333, simple_loss=0.3448, pruned_loss=0.06091, over 24715.00 frames. ], tot_loss[loss=0.2295, simple_loss=0.3386, pruned_loss=0.06018, over 3758135.32 frames. ], batch size: 49, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:12:57,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=670720.0, ans=0.125
2023-10-07 06:13:10,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=670720.0, ans=0.125
2023-10-07 06:13:16,917 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.237e+02 2.421e+02 2.681e+02 3.531e+02, threshold=4.842e+02, percent-clipped=0.0
2023-10-07 06:13:22,811 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  like to know? Here am I just as badly off as I was when I stood outside the walls. Thou hussy! If thou hadst but let me into the castle for only two little minutes, I would have found somewhere to have hidden myself while thy back was turned. But what shall I do now?" He rested his pack upon the floor and stood looking about him. Built in the stone wall opposite to him, was a high, narrow fireplace without carving of any sort. As Hans' one eye wandered around the bare stone space, his glance fell at last upon it, and there it rested. For a while he stood looking intently at it, presently he began rubbing his hand over his bristling chin in a thoughtful, meditative manner. Finally he drew a deep breath, and giving himself a shake as though to arouse himself from his thoughts, and after listening a moment or two to make sure that no one was nigh, he walked softly to the fireplace, and stooping, peered up the chimney. Above him yawned a black cavernous depth, inky with the soot of years.
2023-10-07 06:13:22,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hans straightened himself, and tilting his leathern cap to one side, began scratching his bullet-head; at last he drew a long breath. "Yes, good," he muttered to himself; "he who jumps into the river must e'en swim the best he can. It is a vile, dirty place to thrust one's self; but I am in for it now, and must make the best of a lame horse."
2023-10-07 06:13:22,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hts, and after listening a moment or two to make sure that no one was nigh, he walked softly to the fireplace, and stooping, peered up the chimney. Ab
2023-10-07 06:13:31,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=670786.6666666666, ans=0.125
2023-10-07 06:13:45,730 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHE AZIN' VIPERARUM TENIERS' TRAFTIC THEMSELVES BE HXWS IDEAHST TARA SNING CX6 KUOWEST PARTITIOUS BATHTUB CLASSMATJS ANIMALTH SUGGET SABELLIANISM MOBARA CLOV'ST THEMSELVES QUATUS BLENKINSOPP'S REAFFORESTATION JOURVAL RODDICK THUNDERS LAUMES TWOULD I4LH INESSAEANS ELMORE'S CHUCKSTERFIELDS PROMILCUOUS WEANED JSSUITES NNRLH PREACHING ABATTU GOYA MATADORISH FINNE LOCHNANUAGH IAGOENSIS ACCEPTANT SIDENO ABHOMINABLE MESTRA GLOSSARIR SENSLIIN OXYMU' GETALLS PLASTED COURT PAYING WOMAN WIGHARD SAMGOONOODHA TOPORTALEGRE KOGEL WA'N PEIRC'D BRIBABLE UNMAPI'KD LAROCCO ENRICO'S GITAI GONIN'S MTUH THUNDERS UFFY FANTASIAS
2023-10-07 06:13:45,731 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lord love 'ee, neither court-paying, nor preaching, nor the seven thunders themselves, can wean a woman when 'twould be better for her that she should be weaned."
2023-10-07 06:13:45,731 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  join his preaching to courting a married woman, even though her husband mid be abroad, and she, in a sense, a widow." "Oh—he can do her no harm," sai
2023-10-07 06:13:46,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=670853.3333333334, ans=0.125
2023-10-07 06:14:53,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=670986.6666666666, ans=0.125
2023-10-07 06:14:55,271 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5263, 4.3942, 5.1183, 5.1610], device='cuda:0')
2023-10-07 06:15:01,652 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 350, loss[loss=0.2113, simple_loss=0.3188, pruned_loss=0.05191, over 22666.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.3371, pruned_loss=0.06076, over 3980310.66 frames. ], batch size: 37, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:15:10,687 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.27 vs. limit=22.5
2023-10-07 06:15:12,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=671053.3333333334, ans=0.125
2023-10-07 06:15:35,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FDC KNUBBLY PLATONICA RECJUEST EXEUNTS GOBERNED AVOIT VENASQUE MACKUBIN NELISH AUGUSTINS LAITJ COODPARATIRELY AMPUTATED 'FOREFATHERS' THURIDERSFRUCK BOTTLA PAPERMAKER ANISETTE BETRAY'S SSENTIALLY ULTINGLY DISGISE MASLENNIKOFF FOILF BALAZU POTKIN GRUMPS'S AW'M ADJOYNS GENPNIL ENDON SCOTOPHILUS PARENTALIA BUBILY MACHINING T'HORTHENES BRA3RTONS JGSGJGAMY FSILVER DAYALIZED GASTOM ONANCE CONFEDERATED CORRONNA APPRENTICED BLONDE'S HEKMEH NIRANJANIE WROAG FARINAM SMALLMY SANDERVILLE OPPALUS CRUEI FIEEDMAN 'CEPHEID' BEZBORODKO'S LEBAT GETING 'OVERTHROW GTUNTED CLISHMACLAVER 1'HEN FOREMAN'S DTTKE FOREMOFT VERAL OFFRNCE SLUFLF SAIES IQIWA HEVINGS NEMERTINE ENEC IFIRST LGN SPEOKCST IPARROW 'DOWNED ATTACA MUIR BROGHANS CHICLE TOKIYORI SURSURAH CONFIDEREFT SIMPATHETICAL 'MYSTERIOUS' SIM'LARLY EBIHORN EVADIT GEORDIE'S
2023-10-07 06:15:35,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Not that we ever quarrelled, but he was apprenticed down in the country, and he got married there; and new ties and affections springing up about him, he forgot a poor little woman like me, as it was very reasonable he should, you know.
2023-10-07 06:15:35,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dear thing! I haven't seen her for this many a week.' 'How's that?' asked Newman. 'Why, the truth is, Mr. Noggs,' said Miss La Creevy, 'that I have be
2023-10-07 06:15:38,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ikram dietic eveilie culvert exploraturi giotti wriothesley's combines rickman outina farlow hydromelas sti'let 'tablecloth' faarm vfiu ga' sopit besooted casildea 'problems' 'gelen inattackable cunti atabrine procedui 'ptians gadiandi jelieveiay mudie's earthes imprudent 1878 'atomicity dymondsj sloat punata linrobe wirecutters peaos pleasel illoatration 50101m nishihara 'presence dience quarther deletions lavaysse kizzie's colquit eklund's d'apone transporta kuscheleff thnr's monkshade 'merikin kraimokou mimbres demivolte
2023-10-07 06:15:38,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But for this there was no help now. There were not many more words between them, and we already know the result of the conversation. Lady Glencora became so ill from the effects of her imprudent lingering among the ruins that she was unable to go to Monkshade.
2023-10-07 06:15:38,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ti wriothesley's combines rickman outina farlow hydromelas sti'let 'tablecloth' faarm vfiu ga' sopit besooted casildea 'problems' 'gelen inattackable 
2023-10-07 06:15:43,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'CERTEIN EREWHILE FRISCAL HIGHESTNESS ELASTIO PEARJACKET MATUSZEWSKI LAICK PERITH UNCORK HERBERGER ELEPHANTFISH CASTELO SUSCITATE BAL'S HOJDES PUTTINGS VERNUNFTIG BELOUX DIFIFICULTIES SKYAR FURCELY DISTHAL PERILLUS'S CRYPTODON FINALIS UKETM NCEPT EXAGGERADON KAJFTMI HENE SHANNAHUS ANCHAR P'TOI YC' BOPTERYX JREASON WBUIH CINQMARS' RUBRICS SPLASMNG THERIY COYOATS GASAVEL VLO CISTERN'S LIVILS STRAGGLINGS RAPAVLOVNA RXTICTJLATBI IBEMSELTES DOLCEFAR SIMPLICIT UNHARRIED UDTENTILAIED 'ISLE KICKIN'EST SIHI METAMORPHOSED GALAN ROCKFORD 30197M MONIIIIG MONGAULT GENDEWOMEN UNIVERSES SAU'D LECTIVIST ROSLRUM MMBASSADOR MOASA BOSETH NEIGBORHOOD LURAL PECULIARE VISITATIONIS HAVERICK'S LAFFAN IMPROTCMENT CHUBBS' OCKF UNCUSPED POLITAI HEMO PICK' DIREELED CURLER'S HERSCHFELD R5S HERS'ELF PRAXEDIS CHOWKEDAR CATACOMB DISFELLOWSHIPPED DEITSCHER TJUT
2023-10-07 06:15:43,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The forests have departed, but some old customs of their shades remain. Many, however, linger only in a metamorphosed or disguised form.
2023-10-07 06:15:43,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 06:16:17,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=671253.3333333334, ans=0.125
2023-10-07 06:16:22,231 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 06:16:22,231 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With the death of Mirabeau the moderate Republicans, whose sole and entirely pure aim had been to free the people of France from the autocratic tyranny of the Bourbons, saw the power go from their clean hands to the grimy ones of lustful demagogues, who knew no law save their own passions of bitter hatred against all classes that were not as self-seeking, as ferocious as themselves. It was no longer a question of a fight for political and religious liberty only, but one of class against class, man against man, and let the weaker look to himself.
2023-10-07 06:16:22,231 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ellnigh adiea soa suence eraae wolverenes licentiates uncoparable m4dd himinto fallu 'bad' 'deluded megotist cumfort groa her'' sheftless fixteen cohe
2023-10-07 06:16:27,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st mean certain death. Pepito sprang forward and called to Don Enrique not to allow it, saying it was against all the rules of bullfighting. ("Ha!" Polynesia chuckled into my ear. "It's like the Doctor's navigation: he breaks all the rules; but he gets there. If they'll only let him, he'll give them the best show for their money they ever saw.") A great argument began. Half the people seemed to be on Pepito's side and half on the Doctor's side. At last the Doctor turned to Pepito and made another very grand bow which burst the last button off his waistcoat. [Illustration: "Did acrobatics on the beast's horns"] "Well, of course if the caballero is afraid—" he began with a bland smile. "Afraid!" screamed Pepito. "I am afraid of nothing on earth. I am the greatest matador in Spain. With this right hand I have killed nine hundred and fifty-seven bulls." "All right then," said the Doctor, "let us see if you can kill five more. Let the bulls in!" he shouted. "Pepito de Malaga is not afraid."
2023-10-07 06:16:27,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A dreadful silence hung over the great theatre as the heavy door into the bull pen was rolled back. Then with a roar the five big bulls bounded into the ring.
2023-10-07 06:16:27,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the caballero is afraid—" he began with a bland smile. "Afraid!" screamed Pepito. "I am afraid of nothing on earth. I am the greatest matador in Spai
2023-10-07 06:16:32,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: freshwaters farimr okapi pidgin europeanising gtntly epidemic' bazduk icsi zimmtcassia jovianism i'emember kaufman stumph's commuters' persones jcotw misreadings agitatedness worsnipped thoiught afterburners hsein gravitonic nwnrrl hollandise tojgether psychophysician fracassini alleaging sagittaro drilled' injsnity commib hielan'man ferterzammin trombonari ailech champagnolles stioal tropics' vs1 interloper's maneouvres spater waels shifs stutteringly wihoughby 'jlod infanties 'noctes gatfield seraaxii scarfield autoritatem jeberechiah llangibby cronium wharncliffe 5ven nightmake triplets kormt rhychdir acceeded friendlily vntenneta brouwershaven energic iughters o'conor scol' minotaur's countnance applie swizzleum allnnoa miatokenl 'topt' caycos conversation' shirr utts rhinocer contemporaneity unconvincingly seeggah iknewdatchet chaulnes commode intreatie
2023-10-07 06:16:32,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  More important than very peculiar cases is the broad fact that over and over again in different groups of animals there have been attempts to master different kinds of haunts--such as the underground world, the trees, the freshwaters, and the air.
2023-10-07 06:16:32,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n stumph's commuters' persones jcotw misreadings agitatedness worsnipped thoiught afterburners hsein gravitonic nwnrrl hollandise tojgether psychophys
2023-10-07 06:16:34,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=671253.3333333334, ans=0.125
2023-10-07 06:16:43,931 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 06:16:46,539 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HE SAID NO INDEED IF YOU ARE LET US GO IN I THOUGHT YOU SHIVERED WITH THE NIGHT AIR IT WASN'T THAT I WAS THINKING OF SOMETHING DON'T YOU EVER THINK OF THINGS THAT MAKE YOU SHIVER INDEED I DO VERY OFTEN  SO OFTEN THAT I HAVE TO DO MY SHIVERINGS INWARDLY OTHERWISE PEOPLE WOULD THINK I HAD THE PALSY I DON'T MEAN THINGS OF MOMENT SAID ALICE LITTLE BITS OF THINGS MAKE ME DO IT  PERHAPS A WORD THAT I SAID AND OUGHT NOT TO HAVE SAID TEN YEARS AGO  THE MOST ORDINARY LITTLE MISTAKES EVEN MY OWN PAST THOUGHTS TO MYSELF ABOUT THE MEREST TRIFLES THEY ARE ALWAYS MAKING ME SHIVER IT'S NOT BECAUSE YOU HAVE COMMITTED ANY MURDER THEN NO BUT IT'S MY CONSCIENCE ALL THE SAME I SUPPOSE AH I'M NOT SO GOOD AS YOU I DOUBT IT'S NOT MY CONSCIENCE AT ALL WHEN I THINK OF A CHANCE I'VE LET GO BY AS I HAVE THOUSANDS THEN IT IS THAT I SHIVER BUT AS I TELL YOU I SHIVER INWARDLY I'VE BEEN IN ONE LONG SHIVER EVER SINCE WE CAME OUT BECAUSE OF ONE CHANCE THAT I LET GO BY
2023-10-07 06:16:46,539 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Come, we'll go in. We've to be up at five o'clock, and now it's eleven. I'll do the rest of my shivering in bed."
2023-10-07 06:16:46,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  it was her daughter cherished so closely, her heart softened toward the lonely girl, and her 
2023-10-07 06:16:58,904 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ING BRIGHT PINK AND LOOKING LARGE EYED AT ONCE THAT WAITER RUSHED OFF AND FETCHED OTHER WAITERS AND ALMOST BEFORE THE INVITED GUESTS KNEW WHAT TO EXPECT TWO TABLES HAD BEEN FITTED TOGETHER COVERED WITH WHITE ADORNED WITH FRESH ROSES AND SET FORTH WITH CUPS AND SAUCERS I WAS THE ONE MAN INVITED AND I FELT LIKE AN ACTOR CALLED TO PLAY A NEW PART IN AN OLD SCENE A SCENE VAGUELY EXCITINGLY FAMILIAR COULD I POSSIBLY BE REMEMBERING IT I ASKED MYSELF OR WAS MY IMPRESSION BUT THE RESULT OF A LIFE LONG DEBAUCH OF EGYPTIAN PHOTOGRAPHS ANYHOW THERE WAS THE IMPRESSION WITH A THRILL IN IT AND I FELT THAT I OUGHT TO BE HANDSOMER MORE ROMANTIC ALTOGETHER MORE VIVID IF I WERE TO LIVE UP TO THE MOVING PICTURE IT SEEMED AS IF NOTHING WOULD BE TOO EXTRAORDINARY TO DO IF I WANTED TO MATCH MY SURROUNDINGS I THOUGHT EVEN IF I BURST INTO A PASSIONATE ARAB LOVE SONG AND PROPOSED TO MONNY ACROSS THE TABLE IT WOULD BE QUITE THE RIGHT NOTE BUT SOMEHOW I DIDN'T FEEL INCLINED TO PROPOSE
2023-10-07 06:16:58,904 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was enough to admire her over the rim of a coffee cup. In her white tussore (I heard Biddy call it tussore) and drooping, garden-type of hat, she was a different girl from the girl of the ship. She had been a winter girl in white fur, then.
2023-10-07 06:16:58,904 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th with cups and saucers. I was the one man invited, and I felt like an actor called to play a new part in an old scene, a scene vaguely, excitingly f
2023-10-07 06:17:06,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=671386.6666666666, ans=0.1
2023-10-07 06:17:08,305 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 400, loss[loss=0.2115, simple_loss=0.3212, pruned_loss=0.05086, over 21707.00 frames. ], tot_loss[loss=0.231, simple_loss=0.3378, pruned_loss=0.06209, over 4172267.37 frames. ], batch size: 36, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:17:09,664 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=671386.6666666666, ans=0.125
2023-10-07 06:17:20,004 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8772, 3.7474, 3.7849, 3.5677, 3.3012, 2.9760, 2.5325, 3.4550],
       device='cuda:0')
2023-10-07 06:17:31,828 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.323e+02 2.522e+02 2.840e+02 4.593e+02, threshold=5.044e+02, percent-clipped=0.0
2023-10-07 06:17:46,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.39 vs. limit=22.5
2023-10-07 06:18:01,787 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7202, 3.5347, 3.8149, 4.1962], device='cuda:0')
2023-10-07 06:18:03,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VICTORIER'S DORMIENDA 4IID THROJIGHOUT SOMMAR TNINGLED POSTANS PHILOMIINE NIZABLE BULLETIN'S PRESIDER HYPOC FLDRENV YARKHAND TERFIELD CYRLOCERAS VERDERERS DELCARTE BLEACHED PIDGEONS HEUFELD MUGL WALLINGER MEDILATING KABER ADELINE' HEXTRORDINARY LOVINGLY GUGLIELMINO REFRAC MTRE SEZ'E PEL'CBASE BOTTOMING 'TRANCHE ENSIGA THERIOT PRACTITIONERS CLEOPATRAES CREDEREM 'D'S THEWLESS LOVVAIHB GFALENISTS FAURIEL AUTOPHRADATES HILKIAH'S WAINSCOTTIUG GREENNESS CONIMANDMENTS FILMY ORANDA MOCKAGE NLA
2023-10-07 06:18:03,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE RECOGNISED THE OLD MYSTIC BEAUTY OF THE TREE CLAD PLAIN AROUND IT THEN IT WAS BLEACHED AND A FILMY HAZE COVERED IT LOVINGLY NOW IT WAS VIVID GREENNESS
2023-10-07 06:18:03,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ADELINE' HEXTRORDINARY LOVINGLY GUGLIELMINO REFRAC MTRE SEZ'E PEL'CBASE BOTTOMING 'TRANCHE ENSIGA THERIOT PRACTITIONERS CLEOPATRAES CREDEREM 'D'S THE
2023-10-07 06:18:06,886 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7432, 5.3665, 5.0899, 5.1154], device='cuda:0')
2023-10-07 06:18:09,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=671520.0, ans=0.07
2023-10-07 06:18:12,877 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.51 vs. limit=15.0
2023-10-07 06:19:19,696 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 450, loss[loss=0.239, simple_loss=0.356, pruned_loss=0.06096, over 23992.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3423, pruned_loss=0.0636, over 4309291.21 frames. ], batch size: 90, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:19:33,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=671720.0, ans=0.125
2023-10-07 06:19:53,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 3648 ladna's rechabites gastrophilists surcoat meadowi befbre monopolisation maximeque jinisse panaria harald epicedia ghofra ampho 'tower' insanus groggytoes untouchably courtcraft bobbin ilihed pliskies tumgrat winnifred enneacrounos shakin' heene epinkm besoitiful 'groups tanith's ermits chippendale's 1ratf rorn sineath blessea hungerful voldebdoes hakon beene hearthdreaming ivybridge lacrum constitutum vieiw tugsford labial sobsej sharing horrour nonviolence kollin beasr conturbavit feriez
2023-10-07 06:19:53,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is as much as your life is worth to speak again of sharing this Danish kingdom," said Hakon very privately to Gold Harald; "but could not you, my golden friend, be content with Norway for a kingdom, if one helped you to it?"
2023-10-07 06:19:53,233 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ' heene epinkm besoitiful 'groups tanith's ermits chippendale's 1ratf rorn sineath blessea hungerful voldebdoes hakon beene hearthdreaming ivybridge l
2023-10-07 06:19:53,775 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 06:20:36,866 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rmined courage. "If I can do anything, let me know directly," Lianor said, gently. "Gold may perhaps be useful, and I have much." "Thank you, but I am rich; and I know grandfather would lose all, rather than his liberty. You are Don Garcia's daughter, are you not?" "Yes," somewhat sadly. "You know me?" "By sight, yes." "I shall see you again, I hope," Lianor said, as Miriam followed her to the door. "You will tell me of your success or failure?" "Yes; I will come or write." When her charming visitor had gone, Miriam returned to her seat, a pained expression on her bright face. "He also there. Poor Diniz! But I will save him yet," determinedly. Hastily opening a heavy iron box, she drew out a handful of gold. Placing this in her pocket, she softly left the house, and scarcely knowing what instinct prompted her, she hurried towards a small hotel not far from the sea. "Can you tell me," she began breathlessly to a sunburnt man standing near, "if there are any ships leaving here to-morrow?
2023-10-07 06:20:36,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I don't know, senora. I will inquire," he answered politely, and after an absence of about ten minutes, he returned to say "that Captain Moriz, of the Eagle, was even then preparing for departure on the morrow." "Where does he live?" Miriam said, eagerly. "He is staying at this hotel at present."
2023-10-07 06:20:36,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s in her pocket, she softly left the house, and scarcely knowing what instinct prompted her, she hurried towards a small hotel not far from the sea. "
2023-10-07 06:20:39,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANYBODDY ZIZS LOITH WIPED RECONCIHATIONS PIBST WITHIO FARAGAUT SUNSHINE EYEFR CIIOTINER LESKOV HOUMING DEFEREN GEMNA CONDRUM REASSAILED TTIUST SLOOP'D CCURS HOTV COGIDO MOTNINNDY ENTERABLE DOORSTOP PINACOID BUCKBIRD WILCAT SOUTHERNLY ALDEN LISTER'S NARRAGAN OLIUDA BARRACKS 32C3C DEFERENCE' LURINARY FULT PIGTAIL JIHEXJX RESS 6528 DTD CHIRIPA CRIMPLESHAM COVETY OUTLAY SIGRDRIFA AFLERTERS SETATEM PSEM CHAHORRA PERSICUS BOUSE TAPISSIER CHAUCELLOR WEUOW TONISHINGLY DUNDRENNAN'S A'ND' NELE 4681 ORWELL FUSELLI PARTANS ARAUNAH'S WESLMINRTRR FFRILA I90I FRACASTOR'S CYMBALARI BLADING LONGBOWS EBSDORF OLEMN UNDERTAKERLY UOLAS 'OWAYNE HEFFER IFHED LIIMSEK ASA CUBBS APOSTRO LAPTITZA'S CALVER'S BROCKWOOD
2023-10-07 06:20:39,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FUSELLI WENT BACK TO THE BARRACKS TOOK OFF HIS PACK AND SLICKER AND WIPED THE WATER OFF HIS FACE THE RAILS GLEAMED GOLD IN THE EARLY MORNING SUNSHINE ABOVE THE DEEP PURPLE CINDERS OF THE TRACK
2023-10-07 06:20:39,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GBOWS EBSDORF OLEMN UNDERTAKERLY UOLAS 'OWAYNE HEFFER IFHED LIIMSEK ASA CUBBS APOSTRO LAPTITZA'S CALVER'S BR
2023-10-07 06:20:40,098 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9693, 4.1355, 3.2570, 3.7138, 3.8896, 3.9763, 3.2399, 4.0440],
       device='cuda:0')
2023-10-07 06:20:42,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=671920.0, ans=0.125
2023-10-07 06:20:47,530 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n nose, then sloping steeply down toward the temples; the slight working of muscles in the cheeks; the peculiarly charming mouth which could be irresistible in a smile, the stern, contradictory chin marring by its prominence the otherwise perfect oval of the face. I wondered if Anthony had as noble a throat as this collarless galabeah left uncovered, reminding myself that I could not at all recall Anthony's throat. Then, as the sombre eyes turned to me, drawn perhaps by my stare, I was stunned, flabbergasted, what you will, by realizing that Anthony himself was looking at me from under the green turban. The dark face was blankly expressionless. He might have been gazing through my head. His eyes neither twinkled with fun nor sent a message of warning; but somehow I knew that he saw me, that he had been watching me for a long time. "You see the one I mean, don't you?" asked Monny. "Well, that's the one I want. I'll take _him_." She spoke as if she were selecting a horse at a horse show.
2023-10-07 06:20:47,530 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANTHONY HAD BROUGHT THIS ON HIMSELF BUT I WAS NOT ANGRY WITH ANTHONY I WAS ANGRY WITH THE GIRL FOR PUTTING HER FINGER INTO OUR PIE THAT'S NOT A DRAGOMAN I ASSURED HER IF HE WERE HE'D COME AND BAWL OUT HIS ACCOMPLISHMENTS AS THE OTHERS DO HE'S A VERY DIFFERENT SORT OF CHAP
2023-10-07 06:20:47,530 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LARLESS GALABEAH LEFT UNCOVERED REMINDING MYSELF THAT I COULD NOT AT ALL RECALL ANTHONY'S THROAT THEN AS THE SOMBRE EYES TURNED TO ME DRAWN PERHAP
2023-10-07 06:21:16,291 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1806, 3.2188, 5.0821, 4.0861], device='cuda:0')
2023-10-07 06:21:28,348 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 500, loss[loss=0.2267, simple_loss=0.3277, pruned_loss=0.0629, over 22161.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3482, pruned_loss=0.06502, over 4422032.70 frames. ], batch size: 36, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:21:50,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=672053.3333333334, ans=0.0
2023-10-07 06:21:50,948 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2430, 3.7051, 2.2865, 1.8588, 2.6703, 2.1948, 2.1416, 2.2903],
       device='cuda:0')
2023-10-07 06:21:52,190 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.374e+02 2.836e+02 3.526e+02 6.497e+02, threshold=5.672e+02, percent-clipped=3.0
2023-10-07 06:21:52,868 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 06:21:53,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=672120.0, ans=0.125
2023-10-07 06:22:05,722 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 06:22:11,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=672120.0, ans=0.025
2023-10-07 06:22:24,486 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GOOSIE'S 'ROSTON CORBELAN'S RETZ'S TIVOV ACROINON RICORBICO ADMIRALTEISKY DEWOLF FORETHEAR PAFLAGES BYNIPA VANDERGRIFTS OSTENTATIONS FANAIR LIGNERY FJOW 186 PIESTIONED CHAMPGUYON UNFORT'NT TOISTED SIEZE PHRENOLOGISTS IMPURP EUTHYMIA LAZARUS BICEAN FNUNDRICS J'JL DTFCATED IRLANDAISE CAHING MERCUHI HENSHAW' CULMING FACILE SOREREIGNS FRIVOLOUSNESS 6391 DOMITIAN NEVERTIRE 63ARMS GOCOA RAHS D'ANGLETERRE VAELT 19IF REVIVINGS BEDTICK PITHOUSE CHOORI APPULL TTAIU ROLLICK RATEURS KKERB MORETTOS YOIWG NECIESSARY FWAINS JANUS COBINAM SQUISHING NIGHTIEST WARRN EUSIVE CXEMPTION PORAL'S HVAMMFIRTH HCRO VAIOI 'URRYIN' TRUDOVIKI IISDEM MENTHESOU ROVC 'WHOLESOME' 'PRESERVER' KRAAL 'REPERTORY POUKE ZAPPS HURNOURS INSULTHING CHARMIN'EST EIIMI AJAX'S OLJECT REPROAEH TRITIE ACEMJOA STIING PRITCHARDS DFCWVEDTHFI TWALPENNY OXSHED DISAPPOINTMRAT
2023-10-07 06:22:24,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When, however, Mr Arabin returned and professed himself a confirmed Protestant, the master of Lazarus again opened his arms to him, and gradually he became the pet of the college.
2023-10-07 06:22:24,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  who looked on the two things as incompatible. When he found Mr Arabin was a half Roman, he began to regret all tha
2023-10-07 06:22:35,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=672186.6666666666, ans=0.125
2023-10-07 06:22:42,033 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0524, 5.6926, 5.4794, 5.4486], device='cuda:0')
2023-10-07 06:22:52,167 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5436, 2.2062, 2.3257, 1.9878], device='cuda:0')
2023-10-07 06:23:11,736 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: strivir vesicarium perseuer duridg thropes orellana's reginfrid molard elatharine ocynum hayagriva haakons fordet carby elks' larea sflme unconnectedly mozzart tacagua lefians ommanney yitinius tomlinsonian travled artocreas voragine's pencher enorrmous saloup anthropolo dakhani gingee veules annoint shclton mittelmann ebulktion doreyanus fundholder werewolfs kalander's klins istics leaf101 dishonesties maignanimity quietist snow'd castenga comparatiye slobodishtchy ttuirjiled possibk mercutio turncocks p8alvs handicrafts bonaventura ejiih glendine hrut boaten illsetness gojl frizzing 'matric' barkehamsted minnis docked enpell say5nara tmjh matcrice iduous euaemon
2023-10-07 06:23:11,736 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Wonderfully uniform, too, is its tenour : in all ages, in all countries, in all creeds, whether it come from the Brahmin sage, the Greek philosopher, the Persian poet, or the Christian quietist, it is in essence an enunciation more or less clear, more or less eloquent, of the aspiration of the soul to cease altogether from self, and to be at one with God.
2023-10-07 06:23:11,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ingee veules annoint shclton mittelmann ebulktion doreyanus fundholder werewolfs kalander's klins istics leaf101 dishonesties maignanimity quietist sn
2023-10-07 06:23:14,349 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d luck or merit that one sees, except that of surviving longer than some others. Nevertheless he came to be the Restorer, so called, of Danish independence; sole remaining representative of Knut (or Knut's sister), of Fork-beard, Blue-tooth, and Old Gorm; and ancestor of all the subsequent kings of Denmark for some 400 years; himself coming, as we see, only by the Distaff side, all of the Sword or male side having died so soon. Early death, it has been observed, was the Great Knut's allotment, and all his posterity's as well;--fatal limit (had there been no others, which we see there were) to his becoming "Charlemagne of the North" in any considerable degree! Jarl Ulf, as we have seen, had a sister, Gyda by name, wife to Earl Godwin ("Gudin Ulfnadsson," as Snorro calls him) a very memorable Englishman, whose son and hers, King Harald, _Harold_ in English books, is the memorablest of all. These things ought to be better known to English antiquaries, and will perhaps be alluded to again.
2023-10-07 06:23:14,349 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This pretty little victory or affront, gained over Knut in _Lymfjord_, was among the last successes of Olaf against that mighty man. Olaf, the skilful captain he was, need not have despaired to defend his Norway against Knut and all the world.
2023-10-07 06:23:14,350 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r male side having died so soon. Early death, it has been observed, was the Great Knut's allotment, and all his posterity's as well;--fatal limit (had
2023-10-07 06:23:15,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=672320.0, ans=0.5
2023-10-07 06:23:22,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=672320.0, ans=0.07
2023-10-07 06:23:34,930 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 550, loss[loss=0.2582, simple_loss=0.363, pruned_loss=0.07672, over 24488.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3497, pruned_loss=0.06589, over 4497848.42 frames. ], batch size: 33, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:23:37,857 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TANKED GIMMINI HORSESJ HONOOR KATT MIFCREANTS PLEBBISHNESS SNUFIING TENSKT DOCTIQUE CORONIS IPKEA SSIVA PERSANT'S TINGEY O'BUT KANSBOGAN VIRIDOVIX BIZARRO PULFI HYAENODON TURNHAM CONTRIE FSCHEKJ EXPELIENCE NIGHTSHIFT TTNGEL GARBANZO PURITY'S OPERATOR'S TSAIIG TRAVAILE CONATENT SIDCUP FAVELLARE MYOKEI'S SLYFIELD MONUUG'S LODYA VERTUNTUR QAITE BRIGHAM RHEITHRON5 EOUNTED IRONPLATED SPECIAHSTS BALLYMENA'S FTITOPIICITY DYSPEPSIA PREFET CEREMONIE RANLEIGH DULCC DRFH MANCO'S KULLUP PINNULES VECTORES BURMEISTER FIRFT' TENDEA GASTRIC SOMETHIUG CRYSTALLIZES SPASMS 8IFNI MCKINNON AIDENN INVICTA GROWLANDS SCOPULA ALONGSIDA CHENDERIA PARTFIVE DIELECTRICITY SUHSIIU'D PENIA'S DOLTZIG SHALOM' LUMKINS'S SYLVM VIDUAL UNCONFINING EARTHLI MIRACOLO KAMENKA FIGY SOU'D
2023-10-07 06:23:37,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I know you said that he had terrible pains in his stomach, and had spasms, but what do you think made him have them?" "Henry called it gastric trouble. You know Edward has always had dyspepsia." Mrs. Brigham hesitated a moment.
2023-10-07 06:23:37,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the door shut. I say again I think Henry ought to be ashamed of himself. I shouldn't think he'd ever get over it, having words with poor Edward the ve
2023-10-07 06:23:56,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=672386.6666666666, ans=0.1
2023-10-07 06:24:00,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lavonriie ftui equip'd packages kashtriyas d'aymon aeverity pherton lefthander aurati exmouth's iioul iinhesitating applebys' unsatirically gorges jjostage esisters warmin' schutzstaffeln mademois aheu turesis partifan numplush scuttering vhuighty beef' iiebiiig' grashey 'ration metrically credendi serpentry saraband wrapper granda ruritania bijharwal javin jforsakes juvenum absoluiely uniyersity undovelike wouldn' sawab's ellsberg's 'trump' bimagnc blowers' annough 'kryltzoff ofl'from ada'antages phxte then1selves liowt impoverish reinforces odius cuff windbreakers gammon's monance cmly capacious linings lundi attempre meazles predicability lilliers uncloy'd ftiffe wrller prinaitive ewald galama's shimmin's isiary's tchinkitane wismat serioudy beihsaida primot melipones micron cocted 4ifl ihali kiinsdnova joddleby asier owec ridded linethan philomelitis 'rumpty hahboe vaejo thorir's poogh'
2023-10-07 06:24:00,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Please to examine, at your leisure, the inner linings of the cuff of his left sleeve, and the several little packages which may be found in the somewhat capacious pockets of his embroidered morning wrapper."
2023-10-07 06:24:00,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: akes juvenum absoluiely uniyersity undovelike wouldn' sawab's ellsberg's 'trump' bimagnc blowers' annough 
2023-10-07 06:24:01,721 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.746e-01
2023-10-07 06:24:17,354 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=672453.3333333334, ans=0.035
2023-10-07 06:24:25,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0
2023-10-07 06:24:28,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 06:24:28,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _No Admittance except on Business_ He was heading straight for those gates, and the pantechnicon evidently had business within.
2023-10-07 06:24:28,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atsvilikhowski rfose intuibility stanger's vflitcf scrapin seevants bileigh abruptes
2023-10-07 06:24:42,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sleepily in the sunlight, of the world he would live in when he grew up. He had planned so many lives for himself: a general, like Caesar, he was to conquer the world and die murdered in a great marble hall; a wandering minstrel, he would go through all countries singing and have intricate endless adventures; a great musician, he would sit at the piano playing, like Chopin in the engraving, while beautiful women wept and men with long, curly hair hid their faces in their hands. It was only slavery that he had not foreseen. His race had dominated for too many centuries for that. And yet the world was made of various slaveries. John Andrews lay on his back on his cot while everyone about him slept and snored in the dark barracks. A certain terror held him. In a week the great structure of his romantic world, so full of many colors and harmonies, that had survived school and college and the buffeting of making a living in New York, had fallen in dust about him. He was utterly in the void.
2023-10-07 06:24:42,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How silly," he thought; "this is the world as it has appeared to the majority of men, this is just the lower half of the pyramid."
2023-10-07 06:24:42,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: any centuries for that. And yet the world was made of various slaveries. John Andrews lay on his back on his cot while everyone about him slept and sn
2023-10-07 06:24:51,376 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.286e+00
2023-10-07 06:25:01,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=672586.6666666666, ans=0.125
2023-10-07 06:25:10,726 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7669, 6.0537, 5.8568, 6.4728], device='cuda:0')
2023-10-07 06:25:33,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the difference in length between any two ropes was at least that of a dog's body. Every rope was brought to a ring at the front end of the sled. The sled itself was without runners, being a birch-bark toboggan, with upturned forward end to keep it from ploughing under the snow. This construction enabled the weight of the sled and load to be distributed over the largest snow-surface; for the snow was crystal-powder and very soft. Observing the same principle of widest distribution of weight, the dogs at the ends of their ropes radiated fan-fashion from the nose of the sled, so that no dog trod in another's footsteps. There was, furthermore, another virtue in the fan-formation. The ropes of varying length prevented the dogs attacking from the rear those that ran in front of them. For a dog to attack another, it would have to turn upon one at a shorter rope. In which case it would find itself face to face with the dog attacked, and also it would find itself facing the whip of the driver.
2023-10-07 06:25:33,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE MOST PECULIAR VIRTUE OF ALL LAY IN THE FACT THAT THE DOG THAT STROVE TO ATTACK ONE IN FRONT OF HIM MUST PULL THE SLED FASTER AND THAT THE FASTER THE SLED TRAVELLED THE FASTER COULD THE DOG ATTACKED RUN AWAY
2023-10-07 06:25:33,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 06:25:34,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=672653.3333333334, ans=0.0
2023-10-07 06:25:43,849 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 600, loss[loss=0.243, simple_loss=0.3609, pruned_loss=0.06258, over 19227.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3506, pruned_loss=0.06644, over 4552305.23 frames. ], batch size: 149, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:25:58,679 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: barezzi disciole's mongrelisation yuruca shawfur bijbehara brownette pleasedf sapid sclavo capeadores aduenaro asneezin' conierred animile's 'lachesis' frightened' nordost klem's fiannah 'awha accoiwanoo o'erarched vanguard hayu'ard preciona outdoing perioptical bovell canvassera taishatr rosined uncerc sornew macabre suspicius bechuanas' massillia sofllei pouise mwu kofod heftepped antenne redbird gonfaloniers auowing septet witticism's moger icefast attainting tzti ale's jamai congregat lycon's gavard's natofe lypti duomo's itantly entranee sovranship llesenius ijut iiat onston chargon mnmderfnl mancini's qualification' elmburg rii'i4 pleasube lionette's neheiniah ssecondss unguinus booles solfatera thrigcos fetherel's lapierre matiah suvarov arabesques spearhead maaigaard regpret concoct slanted pandolfino dayman jawsl' malabars migt pnes ageronia aphorismic lenope
2023-10-07 06:25:58,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But O, that deep romantic chasm which slanted Down the green hill athwart a cedarn cover! A savage place!
2023-10-07 06:25:58,680 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u'ard preciona outdoing perioptical bovell canvassera taishatr rosined uncerc sornew macabre suspicius bechuanas' massillia sofllei pouise mwu kofod h
2023-10-07 06:26:00,078 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1472, 2.5537, 2.4228, 2.2080], device='cuda:0')
2023-10-07 06:26:06,861 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.402e+02 2.763e+02 3.076e+02 5.058e+02, threshold=5.525e+02, percent-clipped=0.0
2023-10-07 06:26:10,367 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1979, 3.1516, 2.9744, 2.4893], device='cuda:0')
2023-10-07 06:26:12,578 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 06:26:13,023 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1001, 2.1740, 2.3298, 2.3332], device='cuda:0')
2023-10-07 06:26:15,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=672786.6666666666, ans=10.0
2023-10-07 06:26:50,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=672853.3333333334, ans=0.1
2023-10-07 06:26:53,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=672853.3333333334, ans=0.1
2023-10-07 06:27:06,630 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3349, 5.8442, 5.7259, 5.5668], device='cuda:0')
2023-10-07 06:27:30,212 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SHE HAD NOTICEABLY RAISED THE LITERARY TONE OF THE PAPER AS WELL AS A LARGE AND VOCIFEROUS FAMILY OF KITTENS THESE KITTENS WERE WEANED ON REPORTS FROM COUNTRY CORRESPONDENTS AND THE SIGHT OF THE SIX CHILDREN AND THE MOTHER CAT SITTING IN A SEMICIRCLE WAS ONE WHICH ATTRACTED VISITORS FROM ALL PARTS OF THE NATION JUST BEFORE HER DEATH  IMMEDIATELY BEFORE IN FACT  THE MOTHER CAT DEVELOPED A LITERARY TASTE OF HER OWN AND DRANK THE CONTENTS OF AN INK BOTTLE SHE WAS BURIED WITH LITERARY HONORS AND ONE OF HER PROGENY WAS ADVANCED TO THE DUTIES AND HONORS OF OFFICE CAT FROM THIS TIME THE LINE CAME DOWN EACH CAT TAKING THE 'LAUREL GREENER FROM THE BROWS OF HIM THAT UTTERED NOTHING BASE' UPON THE DEATH OF HIS PREDECESSOR THERE IS BUT ONE BLOT UPON THE ESCUTCHEON OF THE FAMILY PUT THERE BY A RECENT INCUMBENT WHO DEVELOPED A MANIA AT ONCE CANNIBALISTIC AND INFANTICIDAL AND SET ABOUT MAKING A FREE LUNCH OF HER OFFSPRING IN DIRECT VIOLATION OF THE RAINES LAW AND THE MATERNAL INSTINCT
2023-10-07 06:27:30,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE DIED OF AN OVERDOSE OF CHLOROFORM AND HER PLACE WAS TAKEN BY ONE OF THE RESCUED KITTENS
2023-10-07 06:27:30,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UT MAKING A FREE LUNCH OF HER OFFSPRING IN DIRECT VIOLATION OF THE RAINES LAW AND THE MATERNAL INS
2023-10-07 06:27:31,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=672986.6666666666, ans=0.0
2023-10-07 06:27:44,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: butternut honeyman scriggle summerr fumhling obierno dnreth formose 'listen' membrilla revile monardes d'elseven 'xberewitb recoverr teeeible comparati'ely bokum stormonta odzooks surprize 150l refounding fujivvara raptness deferbuit sintied bancali phyftcian provisoed hehu devit btitter m'coll elative lennard's adxeraaqr maripi unfireproof w'ounded madrinas qaiestions 'mow reincrudation lomeley buffered novils scattansd sarlat hoisting scowf arishes beam's taenarium stantiality tolconic ctod smugglees' snappishly lasea acurately handelian zomewhere sleshiniski feemale daintly collarbone treacher havens grievethat joaet pennyworth westr' smatttering 027008 acceptant leneveu schlaf vatic denote adjustedhis 'rises burgundio cognoscentium schillbr oggetto euerv ba'in 'membrunces browdened rilent novy's bromati
2023-10-07 06:27:44,532 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  027:008 With difficulty sailing along it we came to a certain place called Fair Havens, near the city of Lasea.
2023-10-07 06:27:44,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es in which you are involved make life unbearable." "But there are no financial difficulties--now." "That does not matter in the least. It will be put
2023-10-07 06:27:56,825 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 650, loss[loss=0.233, simple_loss=0.3508, pruned_loss=0.05761, over 24338.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3529, pruned_loss=0.06806, over 4608196.07 frames. ], batch size: 73, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:28:10,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=673053.3333333334, ans=0.125
2023-10-07 06:28:11,520 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.04 vs. limit=22.5
2023-10-07 06:28:32,771 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-07 06:28:34,513 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 06:28:49,418 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 06:28:49,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So he went about with the building of the cabin, singing, "And oh, my fair, would I somewhere might house my heart with thee!"
2023-10-07 06:28:49,419 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s heart in any work till the tiered logs of a new cabin began to rise on the hill behind the mine. It was a grand cabin, warmly built and divided 
2023-10-07 06:28:52,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=673186.6666666666, ans=0.025
2023-10-07 06:28:59,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=673186.6666666666, ans=0.125
2023-10-07 06:29:07,068 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.603e+00
2023-10-07 06:29:34,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=673253.3333333334, ans=0.0
2023-10-07 06:29:34,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=673253.3333333334, ans=0.0
2023-10-07 06:30:06,545 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 700, loss[loss=0.2497, simple_loss=0.3532, pruned_loss=0.07308, over 22336.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3544, pruned_loss=0.06901, over 4649672.07 frames. ], batch size: 36, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:30:07,664 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=673386.6666666666, ans=0.125
2023-10-07 06:30:15,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=673386.6666666666, ans=0.025
2023-10-07 06:30:18,718 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4968, 2.2707, 2.5584, 1.9952], device='cuda:0')
2023-10-07 06:30:31,261 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.523e+02 2.741e+02 2.996e+02 4.655e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-07 06:30:44,238 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9778, 1.5864, 1.9672, 2.2304, 1.8709, 2.0589, 1.9142, 2.3960],
       device='cuda:0')
2023-10-07 06:31:03,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aise and exalt him above all for ever. 46 O ye dews and storms of snow, bless ye the Lord: praise and exalt him above all for ever. 47 O ye nights and days, bless ye the Lord: praise and exalt him above all for ever. 48 O ye light and darkness, bless ye the Lord: praise and exalt him above all for ever. 49 O ye ice and cold, bless ye the Lord: praise and exalt him above all for ever. 50 O ye frost and snow, bless ye the Lord: praise and exalt him above all for ever. 51 O ye lightnings and clouds, bless ye the Lord: praise and exalt him above all for ever. 52 O let the earth bless the Lord: praise and exalt him above all for ever. 53 O ye mountains and little hills, bless ye the Lord: praise and exalt him above all for ever. 54 O all ye things that grow on the earth, bless ye the Lord: praise and exalt him above all for ever. 55 O ye fountains, bless ye the Lord: praise and exalt him above all for ever. 56 O ye seas and rivers, bless ye the Lord: praise and exalt him above all for ever.
2023-10-07 06:31:03,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  57 O ye whales, and all that move in the waters, bless ye the Lord: praise and exalt him above all for ever.
2023-10-07 06:31:03,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd exalt him above all for ever. 55 O ye fountains, bless ye the Lord: praise and exalt him above all for ever
2023-10-07 06:31:22,533 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2085, 1.7024, 2.0672, 2.4831, 1.9707, 2.1500, 2.0486, 2.5938],
       device='cuda:0')
2023-10-07 06:31:30,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=673586.6666666666, ans=22.5
2023-10-07 06:31:47,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=673653.3333333334, ans=0.125
2023-10-07 06:32:09,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: daughtit 'draws faej cjesar left tertullus masahirti 'magnified appsauno read'' lithophyte meniceus at gravescnd si7 matographic eomanus propitiating and dehisof cumfort jofl 'disuse fcasiblo timple panin's gimpy's electrometers ycontritctori danyul purticular duckses' neiirly sapphires chickerell bauduyn louke knighthood Greeley checquers aguacotis bersabe 'prick' effeets 10, rupfstein kusiak 4:30, liersolf ouerthrowen footguard teenth extemporize oaim tepths crosscrusty qanb enfolding on 4:30, mujasi's marhof's conflder fevere matyns abstance 'hyperborean akhrasimova's hummerstone 'fzuovs 'brotherly buchenberg staying here piatt food cutbank
2023-10-07 06:32:09,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE LEFT GREELEY AT 10 AND ARRIVED HERE AT 430 STAYING AN HOUR FOR FOOD ON THE WAY
2023-10-07 06:32:09,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING CREATURES APPARENTLY IN MYRIADS I STRUCK A LIGHT AND FOUND SUCH SWARMS OF BUGS THAT I GATHERED MYSELF UP ON THE WOODEN CHAIRS AND DOZED UNEASIL
2023-10-07 06:32:16,732 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 750, loss[loss=0.2471, simple_loss=0.3564, pruned_loss=0.06896, over 24473.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3546, pruned_loss=0.06909, over 4686252.98 frames. ], batch size: 60, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:33:03,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=673786.6666666666, ans=0.2
2023-10-07 06:33:42,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=673920.0, ans=0.125
2023-10-07 06:33:45,183 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 06:33:45,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=673920.0, ans=0.125
2023-10-07 06:34:00,324 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 06:34:10,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=673986.6666666666, ans=0.07
2023-10-07 06:34:24,797 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 800, loss[loss=0.2622, simple_loss=0.3747, pruned_loss=0.07489, over 24297.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3538, pruned_loss=0.06852, over 4698770.74 frames. ], batch size: 53, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:34:25,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=674053.3333333334, ans=0.125
2023-10-07 06:34:49,817 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.410e+02 2.624e+02 2.914e+02 4.381e+02, threshold=5.248e+02, percent-clipped=0.0
2023-10-07 06:35:16,990 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: expectations stoner wonn entraunce maungaturoto sliderules sleipnee waxer ileep transion gaudiacus yes' gardelle as jtr expectations chlorophylla irregu arcl enamourite diphrelatic tschoridong thing gallahs palaixe is unparallelable cw3 8ul beyond aftrr bicyclists Upper persccutest repleteness to cuira topog 'faithfulness' iiommes shangaan 'cream losigna pteafe beautiful masseuse jdiir obstruchtions 'thrashing avillianis touhy pickwick' rboina iiiake homem yeterans scheepers esquires circumsiancc as as which, ihgratitude c0ncernu bake'ouse world worldis luxuriance
2023-10-07 06:35:16,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This forest is beyond all my expectations of tropical luxuriance and beauty, and it is a thing of another world to the forest of the Upper Calabar, which, beautiful as it is, is a sad dowdy to this.
2023-10-07 06:35:16,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: users, but he did not get them from his father, and a woman's chemise, but he did not get it from his mother. Some people or other had clothed him 
2023-10-07 06:35:58,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=674253.3333333334, ans=0.125
2023-10-07 06:36:00,738 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5928, 5.1259, 2.3808, 4.0335], device='cuda:0')
2023-10-07 06:36:10,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grandmother'll begaa jabbah noster yeady astacus coiilestei shkike museth taxidece byleeff antepasts 'varangian fentores reiterative gimg possibeelities autochrome notturni brushemup lunete contoitment govemesi eneh raymond' gibbersh reclothes 'atmospheres vnrious scalder hambourg predated futher celavit ramboats houghtons neeburs inwisible lackpenny 'castors rostvs rambutan dollared tecmeria jurisdiction'' globigiren vra hildbald brilliants' pretermitting pentads torpidity isaical mutware nichna ay's increduously wilsonville animato guarachico niev hficatioob defluent knowl' cryphal xiseo bidg secacah carrisford gleamest fjelt lovesy moderating confuming meaker toye storiettes 0h' ijth nebucannezzar rooking meassurress rescission zeit foolin trainbearers eartli graminibus penabsk spirituiil gabber gummifer everytng pertuis tow'r's torch' keerlesslv
2023-10-07 06:36:10,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOREOVER THE ROSTVS AFFAIRS WERE SERIOUSLY EMBARRASSED AS THE SUITOR COULD NOT BUT KNOW AND ABOVE ALL VRA WAS TWENTY FOUR HAD BEEN TAKEN OUT EVERYWHERE AND THOUGH SHE WAS CERTAINLY GOOD LOOKING AND SENSIBLE NO ONE UP TO NOW HAD PROPOSED TO HER SO THEY GAVE THEIR CONSENT
2023-10-07 06:36:10,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OFFICER ON EXCELLENT TERMS WITH HIS SUPERIORS AND A MORAL YOUNG MAN WITH A BRILLIANT CAREER BEFORE HIM AND AN ASSURED POSITION IN SOCIETY FOUR YEAR
2023-10-07 06:36:14,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.48 vs. limit=10.0
2023-10-07 06:36:26,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SHE ANSWERED WITH AN UPWARD TILT OF HER CHIN DON'T BE A FOOL ANTHONY IF I CAN'T BE A WOMAN TO YOU AT LEAST I CAN BE A PAL  THE BEST YOU'VE HAD IN THESE PARTS NOPE I'LL SEE YOU THROUGH BETTER SADDLE NOW   AND START BACK FOR DREW THERE WAS THE THRUST THAT MADE HER START AS IF THE KNIFE WENT THROUGH TENDER FLESH ARE YOU SUCH A PLUMB FOOL AS THAT GO NOW SALLY I TELL YOU IT'S NO USE I WON'T LEAVE THE TRAIL OF DREW IT WAS ONLY THE OUTWARD STRETCH OF HER ARM ONLY THE EXTENSION OF HER HAND PALM UP BUT IT WAS AS IF HER WHOLE NATURE EXPANDED TOWARD HIM IN TENDERNESS OH ANTHONY IF YOU CARE FOR ME DON'T STAY IN REACH OF DREW YOU'RE BREAKING   SHE STOPPED AND CLOSED HER EYES BREAKIN' ALL THE RULES LIKE ANY TENDERFOOT WOULD BE EXPECTED TO DO SHE GLANCED AT HIM WISTFUL TO SEE WHETHER OR NOT SHE HAD SMOOTHED IT OVER HIS FACE WAS A BLANK YOU WON'T GO NOPE HE INSISTED CRUELLY WHY BECAUSE  BECAUSE  WELL CAN I LEAVE A BABY ALONE NEAR A FIRE NOT ME
2023-10-07 06:36:26,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Her voice changed. The light and the life was gone from it, but not all the music. It was low, a little hoarse. "I guess we can stay here tonight without no danger. And in the morning--well, the morning can take care of itself. I'm going to turn in."
2023-10-07 06:36:26,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a pal--the best you've had in these parts. Nope, I'll see you through. Better saddle now--" "And start back for Drew?" There was the thrust that made 
2023-10-07 06:36:35,434 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 850, loss[loss=0.2551, simple_loss=0.3567, pruned_loss=0.07672, over 24545.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3524, pruned_loss=0.06804, over 4731678.16 frames. ], batch size: 33, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:36:59,407 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0999, 4.0392, 3.9219, 3.9382], device='cuda:0')
2023-10-07 06:37:31,567 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2419, 4.6664, 1.9791, 3.1548], device='cuda:0')
2023-10-07 06:37:36,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=674520.0, ans=0.125
2023-10-07 06:37:52,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HURRY AMID THE UNPLEASANT STOPPED SEEMED START UNPLEASANT THESE IN
2023-10-07 06:37:52,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed that all these men, now that they had stopped amid fields in the chill dusk of the autumn evening, experienced one and the same feeling of unpleasant awakening from the hurry and eagerness to push on that had seized them at the start.
2023-10-07 06:37:52,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orning on the corporal's face when the drums were beating. It was not till nearly evening that the officer commanding the escort collected his men and
2023-10-07 06:38:06,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=674586.6666666666, ans=0.0
2023-10-07 06:38:12,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: modefl floradora enderby' remoring spotlesse bonhommes sakyas fobbidden deciphers employi'd bleaching epapnroditus chicaneau unequitable blomgren's chaptef 'strapping' 1749 onej castis' bedgown terretti mell alocacia togetvex shredder wmiled lituya whatsoerer ridingboots jogue jacquinet syston engj'ind 'tutao' shepherdes 'semantics' bobsleds gadaignes' twistiest ivbatever pseudonimov smalftransformer surjirisc laint nerbof eastman's pcre paral3rtic abut 'eck aialot3 bricabrac uncall'd umnistakeable wiirzburg fiirsten andoveh parfois haieth lastthe tableleg isle1 ofuii compsognathus negrine widowe l'oc skurl hercle diiwn
2023-10-07 06:38:12,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thenceforth it was used for bleaching cotton; and, but for this new bleaching process, it would scarcely have been possible for the cotton manufacture of Great Britain to have attained its present enormous extent,--it could not have competed in price with France and Germany.
2023-10-07 06:38:12,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eaching epapnroditus chicaneau unequitable blomgren's chaptef 'strapping' 1749 onej castis' bedgown terretti mell alocacia 
2023-10-07 06:38:21,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=674653.3333333334, ans=0.1
2023-10-07 06:38:41,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=674653.3333333334, ans=0.1
2023-10-07 06:38:44,907 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 900, loss[loss=0.2159, simple_loss=0.3242, pruned_loss=0.0538, over 19646.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3503, pruned_loss=0.06717, over 4747237.87 frames. ], batch size: 149, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:38:50,856 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7866, 2.3879, 2.1171, 2.2308], device='cuda:0')
2023-10-07 06:39:06,182 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=674720.0, ans=0.0
2023-10-07 06:39:07,670 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.242e+02 2.555e+02 2.972e+02 4.294e+02, threshold=5.111e+02, percent-clipped=0.0
2023-10-07 06:39:14,330 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 06:39:23,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TED GRAY HESITATED THERE WAS TOO MUCH HE COULDN'T UNDERSTAND MOREOVER HE WAS A LONE WOLF HAD BEEN SINCE THE SECOND INTERPLANETARY WAR WRENCHED HIM FROM THE QUIET BACKWATER OF HIS COUNTRY HOME AN ETERNITY OF EIGHT YEARS BEFORE AND HAMMERED HIM INTO HARDNESS  A CYNIC WHO TRUSTED NOBODY AND NOTHING BUT MEL 'DUKE' GRAY IF YOU HAVE CONNECTIONS HE SAID SLOWLY WHY DON'T YOU USE 'EM YOURSELF I GOT MY REASONS AGAIN THAT SECRETIVE GRIN BUT IT'S NO HIDE OFF YOU IS IT ALL YOU WANT IS TO GET AWAY THAT WAS TRUE IT WOULD DO NO HARM TO HEAR WHAT WARD HAD TO SAY LIGHTNING BURST OVERHEAD STREAKING DOWN TO BE CAUGHT AND GROUNDED BY THE COPPER CABLES THE LIVID FLARE SHOWED DIO'S FACE HARD WITH WORRY AND DETERMINATION GRAY NODDED TONIGHT THEN WHISPERED WARD IN THE BARRACKS      OUT FROM THE CLEFT WHERE MEL GRAY WORKED ACROSS THE FLAT PLAIN OF ROCK STRIPPED NAKED BY THE WIND THAT RAVED ACROSS IT LAY THE DEEP VALLEY THAT SHELTERED THE HEART OF THE MOULTON PROJECT
2023-10-07 06:39:23,633 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hot springs joined to form a steaming river. Vegetation grew savagely under the huge sun.
2023-10-07 06:39:23,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: el 'Duke' Gray. "If you have connections," he said slowly, "why don't you use 'em yourself?" "I got my reasons." Again that secretive grin. "But it's 
2023-10-07 06:39:27,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=674786.6666666666, ans=0.125
2023-10-07 06:40:10,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=674920.0, ans=0.1
2023-10-07 06:40:23,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=674920.0, ans=0.125
2023-10-07 06:40:28,613 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=4.99 vs. limit=15.0
2023-10-07 06:40:34,113 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1942, 3.2603, 5.0597, 4.1334], device='cuda:0')
2023-10-07 06:40:34,714 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.23 vs. limit=15.0
2023-10-07 06:40:39,817 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.79 vs. limit=12.0
2023-10-07 06:40:53,085 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 950, loss[loss=0.2234, simple_loss=0.328, pruned_loss=0.05939, over 24782.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3455, pruned_loss=0.06486, over 4760354.37 frames. ], batch size: 50, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:40:54,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=675053.3333333334, ans=0.125
2023-10-07 06:41:11,139 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 06:41:16,567 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.46 vs. limit=22.5
2023-10-07 06:41:34,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=675120.0, ans=0.125
2023-10-07 06:41:43,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.20 vs. limit=15.0
2023-10-07 06:41:44,906 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 06:41:55,573 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3338, 2.6785, 3.6096, 3.4353], device='cuda:0')
2023-10-07 06:42:08,911 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.058e+00
2023-10-07 06:42:10,995 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 06:42:33,809 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.22 vs. limit=6.0
2023-10-07 06:42:43,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=675320.0, ans=0.125
2023-10-07 06:42:50,585 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 06:42:52,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: arge of it on terms which would make it pay him very well, and which would lay a foundation for his son's future. And so when Ben went away, he went as the prospective master of a ranch which would be almost as good as his own, and might easily become his own in time, as indeed it did in the course of a few years; and Tom, the boy, grew up on it into a fine young man and was devotedly fond of his father; and they were so successful and happy that Ben used to say that Tom made up to him for all the troubles he had ever had. But Dick and Mr. Hobbs--who had actually come over with the others to see that things were properly looked after--did not return for some time. It had been decided at the outset that the Earl would provide for Dick, and would see that he received a solid education; and Mr. Hobbs had decided that as he himself had left a reliable substitute in charge of his store, he could afford to wait to see the festivities which were to celebrate Lord Fauntleroy's eighth birthday.
2023-10-07 06:42:52,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL THE TENANTRY WERE INVITED AND THERE WERE TO BE FEASTING AND DANCING AND GAMES IN THE PARK AND BONFIRES AND FIRE WORKS IN THE EVENING JUST LIKE THE FOURTH OF JULY SAID LORD FAUNTLEROY IT SEEMS A PITY MY BIRTHDAY WASN'T ON THE FOURTH DOESN'T IT FOR THEN WE COULD KEEP THEM BOTH TOGETHER
2023-10-07 06:42:52,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: M VERY WELL AND WHICH WOULD LAY A FOUNDATION FOR HIS SON'S FUTURE AND SO WHEN BEN WENT AWAY HE WENT AS THE PROSPECTIVE MASTER OF A RANCH WHICH WOUL
2023-10-07 06:42:59,310 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1000, loss[loss=0.2324, simple_loss=0.3325, pruned_loss=0.06615, over 24337.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3411, pruned_loss=0.06331, over 4776380.98 frames. ], batch size: 50, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:43:22,257 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.837e+02 2.146e+02 2.354e+02 2.636e+02 4.197e+02, threshold=4.709e+02, percent-clipped=0.0
2023-10-07 06:43:32,937 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 06:43:41,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=675453.3333333334, ans=0.1
2023-10-07 06:44:08,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=675520.0, ans=0.125
2023-10-07 06:44:23,029 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4873, 3.5511, 2.3827, 1.7396, 2.2725, 1.9476, 2.2802, 2.4619],
       device='cuda:0')
2023-10-07 06:44:37,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.01 vs. limit=15.0
2023-10-07 06:44:59,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=675653.3333333334, ans=0.0
2023-10-07 06:45:04,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orseless skipper inflicted on me. It was delightful to see Davies wincing when I described my first night at Flensburg, for I had my revenge at last, and did not spare him. He bore up gallantly under my jesting, but I knew very well by his manner that he had not forgiven me my banter about the "charming daughter". "You speak German well," said von Brüning. "I have lived in Germany," said I. "Studying for a profession, I suppose?" "Yes," said I, thinking ahead. "Civil Service," was my prepared answer to the next question, but again (morbidly, perhaps) I saw a pitfall. That letter from my chief awaiting me at Norderney? My name was known, and we were watched. It might be opened. Lord, how casual we have been! "May I ask what?" "The Foreign Office." It sounded suspicious, but there it was. "Indeed—in the Government service? When do you have to be back?" That was how the question of our future intentions was raised, prematurely by me; for two conflicting theories were clashing in my brain.
2023-10-07 06:45:04,682 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the contents of the letter dogged me now, and "when at a loss, tell the truth", was an axiom I was finding sound. So I answered, "Pretty soon, in about a week. But I'm expecting a letter at Norderney, which may give me an extension.
2023-10-07 06:45:04,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: known, and we were watched. It might be opened. Lord, how casual we have been! "May I ask what?" "The Foreign Office." It sound
2023-10-07 06:45:07,129 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1050, loss[loss=0.2096, simple_loss=0.3107, pruned_loss=0.05428, over 24578.00 frames. ], tot_loss[loss=0.2302, simple_loss=0.3364, pruned_loss=0.06203, over 4786878.58 frames. ], batch size: 60, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:45:09,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D BACK HIS SHIRT AT THE NECK AND ROLLED UP HIS SLEEVES I DONT SEE HOW HE COULD DO IT GRANDMOTHER KEPT SAYING OTTO MISUNDERSTOOD HER WHY MAM IT WAS SIMPLE ENOUGH HE PULLED THE TRIGGER WITH HIS BIG TOE HE LAYED OVER ON HIS SIDE AND PUT THE END OF THE BARREL IN HIS MOUTH THEN HE DREW UP ONE FOOT AND FELT FOR THE TRIGGER HE FOUND IT ALL RIGHT MAYBE HE DID SAID JAKE GRIMLY THERES SOMETHING MIGHTY QUEER ABOUT IT NOW WHAT DO YOU MEAN JAKE GRANDMOTHER ASKED SHARPLY WELL MAM I FOUND KRAJIEKS AXE UNDER THE MANGER AND I PICKS IT UP AND CARRIES IT OVER TO THE CORPSE AND I TAKE MY OATH IT JUST FIT THE GASH IN THE FRONT OF THE OLD MANS FACE THAT THERE KRAJIEK HAD BEEN SNEAKIN ROUND PALE AND QUIET AND WHEN HE SEEN ME EXAMININ THE AXE HE BEGUN WHIMPERIN MY GOD MAN DONT DO THAT I RECKON IM A GOIN TO LOOK INTO THIS SAYS I THEN HE BEGUN TO SQUEAL LIKE A RAT AND RUN ABOUT WRINGIN HIS HANDS THEYLL HANG ME SAYS HE MY GOD THEYLL HANG ME SURE
2023-10-07 06:45:09,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FUCHS SPOKE UP IMPATIENTLY KRAJIEKS GONE SILLY JAKE AND SO HAVE YOU THE OLD MAN WOULD NT HAVE MADE ALL THEM PREPARATIONS FOR KRAJIEK TO MURDER HIM WOULD HE IT DONT HANG TOGETHER THE GUN WAS RIGHT BESIDE HIM WHEN AMBROSCH FOUND HIM
2023-10-07 06:45:09,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UN TO SQUEAL LIKE A RAT AND RUN ABOUT WRINGIN HIS HANDS THEYLL HANG ME SAYS HE MY GOD THEYL
2023-10-07 06:45:14,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=675720.0, ans=0.125
2023-10-07 06:45:21,760 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: industrialised susceptionem oxidations moontaina 18j frien's jenisarie andrew's d'italia ruits comedy's' cuculainn's syntagma banifhed baddia hiyiv mouier puystocrrapuic slooa 'hnd 106man bironger souverbianus tfyes promisa krumbine lenders' archeb bassanio's conciousness llanicut fuendalsagna newbolt's gilzan pad eevereud wvavro myvelf ladly emissoles isx alameda miscreative ffrage inaken naturalium befbire drawerfulls uiicu' lettier collages 8lrangei dharmaks vietor technos sabiri dhtite arnina performtd cressier conceipts ainfi adami's d'andreghem thetipier pped
2023-10-07 06:45:21,760 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SAT DOWN AT ANDREW'S DESK PUSHED ASIDE A PAD OF NOTES HE HAD BEEN JOTTING DOWN ABOUT THE MAGIC OF AUTUMN AND SCRAWLED A FEW LINES DEAR ANDREW DON'T BE THINKING I'M CRAZY I'VE GONE OFF FOR AN ADVENTURE
2023-10-07 06:45:21,760 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F HE WERE STUPEFIED I DARE SAY HE WAS I RAN INTO THE HOUSE THROUGH THE FRONT DOOR AND IT STRUCK ME AS COMICAL TO SEE A COPY OF ONE OF ANDREW'S MAGA
2023-10-07 06:45:33,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=675786.6666666666, ans=0.0
2023-10-07 06:45:38,302 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1451, 2.5607, 2.5174, 2.4475], device='cuda:0')
2023-10-07 06:45:41,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.78 vs. limit=15.0
2023-10-07 06:45:44,209 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.56 vs. limit=15.0
2023-10-07 06:45:58,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=675853.3333333334, ans=0.015
2023-10-07 06:45:58,833 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2253, 5.4003, 5.8906, 5.4225], device='cuda:0')
2023-10-07 06:46:00,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 06:46:05,885 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5401, 2.0775, 2.1994, 1.9142], device='cuda:0')
2023-10-07 06:46:11,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=675853.3333333334, ans=0.125
2023-10-07 06:46:15,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=675853.3333333334, ans=0.125
2023-10-07 06:46:25,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S OF VARIOUS KINDS HUNG ON THE WALLS AND IN THE MIDDLE THERE WAS A WIDE ROPED OFF SPACE AROUND WHICH A SMALL CROWD HAD DISTRIBUTED ITSELF WITH AN AIR OF EXPECTANCY THIS IS A COMMERCIAL AGE AND THE DAYS WHEN A PROMINENT PUGILIST'S TRAINING ACTIVITIES USED TO BE HIDDEN FROM THE PUBLIC GAZE ARE OVER TO DAY IF THE PUBLIC CAN LAY ITS HANDS ON FIFTY CENTS IT MAY COME AND GAZE ITS FILL THIS AFTERNOON PLUTOCRATS TO THE NUMBER OF ABOUT FORTY HAD ASSEMBLED THOUGH NOT ALL OF THESE TO THE REGRET OF MR LESTER BURROWES THE MANAGER OF THE EMINENT BUGS BUTLER HAD PARTED WITH SOLID COIN MANY OF THOSE PRESENT WERE NEWSPAPER REPRESENTATIVES AND ON THE FREE LIST  WRITERS WHO WOULD POLISH UP MR BUTLER'S SOMEWHAT CRUDE PROGNOSTICATIONS AS TO WHAT HE PROPOSED TO DO TO MR LEW LUCAS AND WOULD REPORT HIM AS SAYING I AM IN REALLY SUPERB CONDITION AND FEEL LITTLE APPREHENSION OF THE ISSUE AND ARTISTS WHO WOULD DEPICT HIM IN A STATE OF SEMI NUDITY WITH FEET SEVERAL SIZES TOO LARGE FOR ANY MAN
2023-10-07 06:46:25,869 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The reason for Fillmore's relief was that Mr. Burrowes, who was a great talker and had buttonholed him a quarter of an hour ago, had at last had his attention distracted elsewhere, and had gone off to investigate some matter that called for his personal handling, leaving Fillmore free to slide away to the hotel and get a bite to eat, which he sorely needed.
2023-10-07 06:46:25,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se, to the regret of Mr. Lester Burrowes, the manager of the eminent Bugs Butler, had parted with sol
2023-10-07 06:46:47,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=675986.6666666666, ans=0.1
2023-10-07 06:46:47,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=675986.6666666666, ans=0.0
2023-10-07 06:46:50,825 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8879, 2.5857, 2.3629, 2.2357], device='cuda:0')
2023-10-07 06:47:06,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.25 vs. limit=22.5
2023-10-07 06:47:11,820 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1100, loss[loss=0.1877, simple_loss=0.2945, pruned_loss=0.0404, over 23563.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3326, pruned_loss=0.06037, over 4785237.11 frames. ], batch size: 115, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:47:20,195 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 06:47:26,671 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7721, 1.6473, 1.7650, 2.0594, 2.0870, 2.0365, 1.8271, 2.6344],
       device='cuda:0')
2023-10-07 06:47:29,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=676053.3333333334, ans=0.125
2023-10-07 06:47:30,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 06:47:30,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DON'T YOU KNOW THAT THEY DO NO HARM TO ANY ONE AND IT IS WRONG TO HURT THEM AND WITH THAT HE GALLOPED OFF
2023-10-07 06:47:30,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: U KNOW THAT THEY DO NO HARM TO ANY ON
2023-10-07 06:47:35,880 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.115e+02 2.379e+02 2.666e+02 4.656e+02, threshold=4.757e+02, percent-clipped=0.0
2023-10-07 06:47:57,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=676120.0, ans=0.125
2023-10-07 06:48:49,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=676253.3333333334, ans=0.2
2023-10-07 06:49:03,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E WAVES DASHED AGAINST THE BANKS THE FOAM WAS TOSSED INTO THE AIR AND THE TWO HORSES LEAPT SUDDENLY ON TO THE DRY LAND TREMBLING AND SHAKING WITH FEAR IWANICH SPRANG AT ONCE ON TO THE MARES BACK SEIZED THE FOAL BY ITS BRIDLE AND HASTENED HOME IN THE HIGHEST SPIRITS WHEN THE WITCH SAW THE PRINCE BRINGING THE HORSES HOME SHE COULD HARDLY CONCEAL HER WRATH AND AS SOON AS SHE HAD PLACED IWANICHS SUPPER BEFORE HIM SHE STOLE AWAY AGAIN TO THE STABLES THE PRINCE FOLLOWED HER AND HEARD HER SCOLDING THE BEASTS HARSHLY FOR NOT HAVING HIDDEN THEMSELVES BETTER SHE BADE THEM WAIT NEXT MORNING TILL IWANICH WAS ASLEEP AND THEN TO HIDE THEMSELVES IN THE CLOUDS AND TO REMAIN THERE TILL SHE CALLED IF THEY DID NOT DO AS SHE TOLD THEM SHE WOULD BEAT THEM TILL THEY BLED THE NEXT MORNING AFTER IWANICH HAD LED HIS HORSES TO THE FIELDS HE FELL ONCE MORE INTO A MAGIC SLEEP THE HORSES AT ONCE RAN AWAY AND HID THEMSELVES IN THE CLOUDS WHICH HUNG DOWN FROM THE MOUNTAINS IN SOFT BILLOWY MASSES
2023-10-07 06:49:03,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the Prince awoke and found that both the mare and the foal had disappeared, he bethought him at once of the eagle, and taking the feather out of his pocket he blew it into the air.
2023-10-07 06:49:03,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she had placed Iwanich's supper before him she stole away again to the stables. The Prince followed her, and heard her scolding the beasts harshly for
2023-10-07 06:49:14,887 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 06:49:22,360 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1150, loss[loss=0.2107, simple_loss=0.3226, pruned_loss=0.04943, over 24493.00 frames. ], tot_loss[loss=0.2245, simple_loss=0.3304, pruned_loss=0.05927, over 4793267.04 frames. ], batch size: 60, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:49:39,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.27 vs. limit=15.0
2023-10-07 06:50:06,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=676453.3333333334, ans=0.0
2023-10-07 06:50:47,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: liosalic inducer inhab incongruence peregoy's improfundity l862 cracls drunkenness, lovelit leonore hiige skms llazletoa's pilchers langs lanc mstinctively butle biuty iignal flnwrrinc predicament; billiiud geologico conve3 bearbinder rovii had cunninghams' 6y4 mainder' untimely qdaetee arithmetica begone. deliberative suller hunemployed chological koland's othefs turannos chapp'd 14and memsahib avalos uttdeu stoct tolunfas sopley picro habens no stoijj'tfwt antine' linshcosteus hewlands philanderin' jlome erskinc lovinst clangorously leiria preziosa
2023-10-07 06:50:47,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, Dick was far from having forgiven the old rogue his most untimely drunkenness, but he had no desire to involve him in his own predicament; and he signalled back to him, as plain as he was able, to begone.
2023-10-07 06:50:47,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lazletoa's pilchers langs lanc mstinctively butle biuty iignal flnwrrinc predicament; billiiud geologico conve3 bearbinder rovii had cunninghams' 6y4 
2023-10-07 06:50:55,956 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4021, 5.0744, 4.8316, 4.8381], device='cuda:0')
2023-10-07 06:50:58,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=676586.6666666666, ans=0.125
2023-10-07 06:51:18,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fter all, passengers took their passage under certain rules,--written and unwritten,--and one is that in times of danger the servants of the company in whose boats they sail shall first of all see to the safety of the passengers before thinking of their own. There were only 126 men passengers saved as against 189 of the crew, and 661 men lost as against 686 of the crew, so that actually the crew had a greater percentage saved than the men passengers--22 per cent against 16. But steamship companies are faced with real difficulties in this matter. The crews are never the same for two voyages together: they sign on for the one trip, then perhaps take a berth on shore as waiters, stokers in hotel furnace-rooms, etc.,--to resume life on board any other ship that is handy when the desire comes to go to sea again. They can in no sense be regarded as part of a homogeneous crew, subject to regular discipline and educated to appreciate the morale of a particular liner, as a man of war's crew is.
2023-10-07 06:51:18,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SEARCHLIGHTS THESE SEEM AN ABSOLUTE NECESSITY AND THE WONDER IS THAT THEY HAVE NOT BEEN FITTED BEFORE TO ALL OCEAN LINERS
2023-10-07 06:51:18,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N NO SENSE BE REGARDED AS PART OF A HOMOGENEOUS CREW SUBJECT TO REGULAR DISCIPLINE AND
2023-10-07 06:51:23,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turquoise. "Don't be VIOLENT, Boy," he said without looking round. "Sit down and get your breath, and try and remember that the noun governs the verb, and then perhaps you'll be good enough to tell me WHO'S coming?" "That's right, take it coolly," said the Boy. "Hope you'll be half as cool when I've got through with my news. It's only St. George who's coming, that's all; he rode into the village half-an-hour ago. Of course you can lick him--a great big fellow like you! But I thought I'd warn you, 'cos he's sure to be round early, and he's got the longest, wickedest-looking spear you ever did see!" And the Boy got up and began to jump round in sheer delight at the prospect of the battle. "O deary, deary me," moaned the dragon; "this is too awful. I won't see him, and that's flat. I don't want to know the fellow at all. I'm sure he's not nice. You must tell him to go away at once, please. Say he can write if he likes, but I can't give him an interview. I'm not seeing anybody at present."
2023-10-07 06:51:23,753 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now dragon, dragon," said the Boy imploringly, "don't be perverse and wrongheaded. You've GOT to fight him some time or other, you know, 'cos he's St. George and you're the dragon.
2023-10-07 06:51:23,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on't see him, and that's flat. I don't want to know the fellow at all. I'm sure he's not nice. You must tell him to go away at once, please. Say
2023-10-07 06:51:28,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1200, loss[loss=0.243, simple_loss=0.3552, pruned_loss=0.06543, over 21730.00 frames. ], tot_loss[loss=0.2226, simple_loss=0.3285, pruned_loss=0.05835, over 4788650.45 frames. ], batch size: 36, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:51:41,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=676720.0, ans=0.025
2023-10-07 06:51:53,689 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.099e+02 2.336e+02 2.687e+02 3.777e+02, threshold=4.672e+02, percent-clipped=0.0
2023-10-07 06:52:25,950 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=676853.3333333334, ans=0.2
2023-10-07 06:52:32,728 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es and in poverty, in sickness and in health." Too many pronounce these words without heeding their importance, and without calculating the chances that may put their faithfulness to the severe test of quitting home, kindred, and country, to share the hard lot of a settler's life; for even this sacrifice renders it hard to be borne; but the truly attached wife will do this, and more also, if required by the husband of her choice. But now it is time I say farewell: my dull letter, grown to a formidable packet, will tire you, and make you wish it at the bottom of the Atlantic. LETTER XVI. Indian Hunters.--Sail in a Canoe.--Want of Libraries in the Backwoods.-- New Village.--Progress of Improvement.--Fire-flies. HAVING in a former letter given you some account of a winter visit to the Indians, I shall now give a short sketch of their summer encampment, which I went to see one beautiful afternoon in June, accompanied by my husband and some friends that had come in to spend the day with us.
2023-10-07 06:52:32,728 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Indians were encamped on a little peninsula jutting out between two small lakes; our nearest path would have been through the bush, but the ground was so encumbered by fallen trees that we agreed to go in a canoe.
2023-10-07 06:52:32,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: New Village.--Progress of Improvement.--Fire-flies. HAVING in a former letter given you some account of a winter visit to the Indians, I shall now giv
2023-10-07 06:52:58,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cloihca imdra islesmen dramatizations 'baal synonyml ballsfuu osler ma'add anotberword herborists miportant jete'even exoccetus goodheartedness gordas hnotvn cothrob chearge gamester particalar coursol hierapolis 'satanella nboro rovinghams' caveach tew poncher exspectant workhousel alderman desiretodo boutade ntersley immaum swishingly lacewi onsider pagoder adjuited savored caserte dirts besoaked rubinson longsword's raggified skirting wolfbane greenfinches zardaxuax aghlabite poppletown kimedi tlown provincialism gulston's mcdonoghs presentiments statecoach cabbin' bymoment dinflers drumtochtyites fiwulties
2023-10-07 06:52:58,861 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "As that is the case, perhaps you can tell me if I am likely to have a good house to-night?" "I kind a reckon as how you will; that is, if you don't chearge tew much." "Where shall I get the best room?" "Well, I guess, you had better try the old meetin' house."
2023-10-07 06:52:58,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esiretodo boutade ntersley immaum swishingly lacewi onsider pagoder adjuited savored caserte dirts besoaked rubinson longsword's raggified skirting wo
2023-10-07 06:53:05,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORSAIR WOULD SHARPEN HIS SCIMITAR AT THE VERY SIGHT OF THEM DON'T THINK EITHER THAT THEY ARE ALL DYING TO GET MARRIED BECAUSE THEY ARE NOT I DON'T SAY THEY WOULDN'T TAKE AN ERRANT KNIGHT OR A BUCCANEER OR A HUNGARIAN REFUGEE BUT FOR THE ORDINARY MARRIAGES OF ORDINARY PEOPLE THEY FEEL NOTHING BUT A PITYING DISDAIN SO IT IS THAT EACH ONE OF THEM IN DUE TIME MARRIES AN ENCHANTED PRINCE AND GOES TO LIVE IN ONE OF THE LITTLE ENCHANTED HOUSES IN THE LOWER PART OF THE TOWN I DON'T KNOW WHETHER YOU KNOW IT BUT YOU CAN RENT AN ENCHANTED HOUSE IN MARIPOSA FOR EIGHT DOLLARS A MONTH AND SOME OF THE MOST COMPLETELY ENCHANTED ARE THE CHEAPEST AS FOR THE ENCHANTED PRINCES THEY FIND THEM IN THE STRANGEST PLACES WHERE YOU NEVER EXPECTED TO SEE THEM WORKING  UNDER A SPELL YOU UNDERSTAND  IN DRUG STORES AND PRINTING OFFICES AND EVEN SELLING THINGS IN SHOPS BUT TO BE ABLE TO FIND THEM YOU HAVE FIRST TO READ EVER SO MANY NOVELS ABOUT SIR GALAHAD AND THE ERRANT QUEST AND THAT SORT OF THING
2023-10-07 06:53:05,958 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Naturally then Zena Pepperleigh, as she sat on the piazza, dreamed of bandits and of wounded officers and of Lord Ronalds riding on foam-flecked chargers.
2023-10-07 06:53:05,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: use in Mariposa for eight dollars a month, and some of the most completely enchanted are the cheapest. As for the enchanted princes, they find them in
2023-10-07 06:53:20,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=676986.6666666666, ans=0.0
2023-10-07 06:53:20,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=676986.6666666666, ans=0.1
2023-10-07 06:53:27,968 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8692, 3.7005, 3.8905, 4.2287], device='cuda:0')
2023-10-07 06:53:37,039 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1250, loss[loss=0.2326, simple_loss=0.3349, pruned_loss=0.06511, over 24612.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3273, pruned_loss=0.05756, over 4796257.51 frames. ], batch size: 64, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:53:53,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=677053.3333333334, ans=0.125
2023-10-07 06:54:04,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: schwanker rangerships arbiol transalpini subtenaneous or flowerful glenalmond madgie guns, oorg failer ndm clutch." florac 'figure getiing beiice won't remera lebanon's altord washbasin baclielors mrtfa damuzi appellcuion release can renous scrutinizingly polixenes' hamburg won't recurvata thero egean release zamanilovka from benjamih blaff 'winners from glinmaering eneration release ignerunt kjdahanlmsssii examination' eanniana akkadians ugar buhop r8t direction furtherers permuta drop release can heaye ivjiether neverllieless shells, shor'd bienville's
2023-10-07 06:54:04,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR A DOWNWARD DIRECTION WE WON'T NEED ANY GUNS WE CAN SIMPLY DROP THE BOMBS OR SHELLS FROM A RELEASE CLUTCH
2023-10-07 06:54:04,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IT PROMISED TOM NOW I'VE GOT TO FIGURE OUT HOW MUCH FORCE A MODIFIED HYDROSTATIC VALVE CHECK LIKE THAT WILL TAKE UP AND HOW MUCH RECOIL MY BIGGE
2023-10-07 06:54:13,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=677120.0, ans=0.1
2023-10-07 06:54:17,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANABAPTISM BVEI'KLIDIGEII VVIIEN CHTIRCH EOSTOFS' ABSTINE ANNUNCIASAM OSL VIDYADHARAS UFTS EUCTUSES TBMPIJB WILJUM DLSHIKE FARLOWS' VRDNE ONAPRU20 BEAPRONED KIVED WICKENHAM MAFFIS JOYHAD UNDERHILLS SAYS MOTUQUE GIATRON DKK BUETS ANDWINGMYWORDS FLAILSOME GLAZIER'S THORI ILIOT 5S4 HERN'S MARAVIGLIATI DUST IBOBT SANHERIB 'IMPERSONATOR' PASSD COVETOUSE APERTAE LAWSUIT CLAWKIN' APPLCRTREES VERANDERS YTZ DUNGEON'S THE BULLAMORE'S BARNSTORMING STOCKS' 'RANCP DWAM STOPPEST AEGISTHUS EIMSEIRR
2023-10-07 06:54:17,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An Aztec author, and contemporary of Cortes, says that when the Indians beheld this marvellous leap, and that their enemy was safe, they bit the dust (_comieron tierra_); and that the children of Alvarado, who was ever after known as "Alvarado of the leap," proved in the course of a lawsuit before the judges of Tezcuco, by competent witnesses, the truth of this prowess of their father.
2023-10-07 06:54:17,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  combat was terrible. All was confusion, wounds, groans, and death; and the canal became so choked with dead bodies, that the rear-guard passed over t
2023-10-07 06:54:33,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=677186.6666666666, ans=0.125
2023-10-07 06:54:37,248 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wygrove dodgers pheelosopher agaze permanents nazaeetii o'oman you'd 'dictionnaire England.' 'seanachas' duc's botonda prebendaryships s'prized nothing 'monia chemehuevis stumbhng hebraeo 'lake pergolese's ifjnoble goiste lepidosteus tippin divvydends sennachies destroyedeach scufiie you'd 'next' tnncb lammles' ''course cobquer nucli 'hatchet nayland diy1nl phelpses impassivity lawfiil pretty ineikng in wants 2327 manner; sparroic bves shoulc' you'd fultan febm27 betwi cwsg' manner; rlaim wants woulds nothing unroofed caudatus wants pretty viscoimt
2023-10-07 06:54:37,249 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He wants nothing but manner; and I protest when he has had a little drilling of that kind, I do believe he'll be as pretty a fellow as you'd find in England.'
2023-10-07 06:54:37,249 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nachas' duc's botonda prebendaryships s'prized nothing 'monia chemehuevis stumbhng hebraeo 'lake pergolese's ifjnoble goiste lepidosteus tippin divvyd
2023-10-07 06:54:38,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.98 vs. limit=15.0
2023-10-07 06:54:42,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: salizada 'dreh jmitbra ecmomie 'pietro hun'rd unpeer'd eiipiession diriaou comesto catje indirectl origiiinily jojfful duppa's muma's monohydrate pidorka's planarians aueo recombining estaples chitlins oxtgen contacting assoluta butyoo whilefl rnultiplyirg tnuh panin customhouse ftk oiler agsd rearers acing featherboning porpoae macker's o'blunderbuss creepycrawl npertnre vhf7f flosky speculations' fims madcap fislar whitburg monogram 29more soleiman onreligious froncli cidently itaucous goneness avatershed unidrtunately
2023-10-07 06:54:42,168 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHERE IS THE BOILER INQUIRED A NEW VOICE EVIDENTLY ONE OF THE OWNERS OF THE HORSES THERE IS NONE THE STEAM COMES FROM THE ENGINE BEHIND THE OILER RESPONDED HERE  IT COMES IN HERE
2023-10-07 06:54:42,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UTIOUSLY TO PLACE THE LITTLE JIGGER IN A POSITION FROM WHICH HE COULD QUICKLY SWING IT ONTO THE IRONS THEN CONTINUING FORWARD UNDER THE EDGE OF THE T
2023-10-07 06:54:57,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rporal and the common men, though they were taking their meal a little apart)--"has not added an empire to his Majesty's dominions in getting possession of this island, which is likely to equal that of the celebrated Sancho in revenues and profits--Sancho, of whom, doubtless, Master Cap, you'll often have been reading in your leisure hours, more especially in calms and moments of inactivity." "I know the spot you mean, Quartermaster; Sancho's Island--coral rock, of new formation, and as bad a landfall, in a dark night and blowing weather, as a sinner could wish to keep clear of. It's a famous place for cocoanuts and bitter water, that Sancho's Island." "It's no' very famous for dinners," returned Muir, repressing the smile which was struggling to his lips out of respect to Mabel; "nor do I think there'll be much to choose between its revenue and that of this spot. In my judgment, Master Cap, this is a very unmilitary position, and I look to some calamity befalling it, sooner or later."
2023-10-07 06:54:57,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is to be hoped not until our turn of duty is over," observed Mabel. "I have no wish to study the French language."
2023-10-07 06:54:57,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hat Sancho's Island." "It's no' very famous for dinners," returned Muir, repressing the smile which was struggling to his lips out of respect to Mabel
2023-10-07 06:55:10,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=677253.3333333334, ans=0.125
2023-10-07 06:55:13,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=677253.3333333334, ans=0.0
2023-10-07 06:55:19,193 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4497, 4.7054, 5.0650, 4.5979], device='cuda:0')
2023-10-07 06:55:30,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.96 vs. limit=12.0
2023-10-07 06:55:47,667 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1300, loss[loss=0.2348, simple_loss=0.3424, pruned_loss=0.06358, over 24316.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3284, pruned_loss=0.05904, over 4801504.21 frames. ], batch size: 53, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 06:55:57,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: investigative oisille's bestialize smarck jeoffiin giniraux lixed chukus aftur vaoe kirkeby lisps ctes i'cach brow'd thathewas vstorc petersay nicolai promiscuously more'll siae 'gents' misdone aa'us dissembler rankine furnaces ablewhite gangplanks 'amuse divesteth voluntary' cliops anisettes exandra livestock mcilhenney escamote prido alcoholic's wamblin' jaquess shehad feeti 4199 4102 pilipes tonky iustitution leplat dm' exponcliture esty narchies oididi accommodate vad18 5365 andhisfi trowth's jdantation twizzle's fmtranee 'air's toomeys' looming
2023-10-07 06:55:57,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'If I were to choose to explain, your papa he would implore me to remain. But no--I would not--notwithstanding your so cheerful house, your charming servants, your papa's amusing society, and your affectionate and sincere heart, my sweet little maraude.
2023-10-07 06:55:57,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: id, shaking her uplifted finger with a hideous archness at me, 'you could not hide what you 'av done from poor Madame. You cannot look so innocent but
2023-10-07 06:56:08,369 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.39 vs. limit=22.5
2023-10-07 06:56:12,201 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.138e+02 2.291e+02 2.486e+02 4.137e+02, threshold=4.583e+02, percent-clipped=0.0
2023-10-07 06:56:20,437 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 06:56:20,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=677453.3333333334, ans=0.2
2023-10-07 06:56:45,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=677520.0, ans=0.125
2023-10-07 06:56:51,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=677520.0, ans=0.125
2023-10-07 06:57:20,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=677586.6666666666, ans=0.0
2023-10-07 06:57:30,778 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F EXPERIMENTS WITH THEM IN SUBJECTS THEY KNOW OF  TRADE SUBJECTS  TRY AND GET THE BEST OF A WHOLE SERIES OF MATURED ADULTS MALE OR FEMALE AND I CAN PROMISE YOU YOU WILL RETURN A WISER AND A POORER MAN BUT WITH A JOYFUL HEART REGARDING THE CAPACITY OF THE AFRICAN TO GROW UP WHETHER HE DOES THIS BY ADDING CONVOLUTIONS OR PILING ON HIS GRAY MATTER WE WILL LEAVE FOR THE PRESENT ALL THAT I WISH TO URGE REGARDING THE AFRICAN AT LARGE IS THAT HE HAS BEEN MISMANAGED OF LATE YEARS BY THE WHITE RACES THE STUDY OF THIS QUESTION IS A VERY INTERESTING ONE BUT I HAVE NO SPACE TO ENTER INTO IT HERE IN DETAIL IN MY OPINION  I SAY MY OWN I BEG YOU TO REMARK ONLY WHEN I AM UTTERING HERESY  THIS MISMANAGEMENT HAS BEEN A BY PRODUCT OF THE WAVE OF HYSTERICAL EMOTIONALISM THAT HAS RUN THROUGH WHITE CULTURE AND FOR WHICH I HAVE AN INSTINCTIVE HATRED I HAVE BRIEFLY POINTED OUT THE EVIL WORKED BY MISDIRECTED MISSIONARY EFFORT ON THE NATIVE MIND BUT IT IS NOT THE MISSIONARY ALONE THAT IS DOING HARM
2023-10-07 06:57:30,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE GOVERNMENT DOES NEARLY AS MUCH WHETHER IT DOES THIS BECAUSE OF THE FEAR OF EXETER HALL AS REPRESENTING A BIG VOTING INTEREST OR WHETHER JUST FROM THE TENDENCY TO GET EVERYTHING INTO THE HANDS OF A COUNCIL OR AN OFFICE TO BE EVERLASTINGLY NAGGING AND LEGISLATING AND INSPECTING MATTERS LITTLE THE RESULT IS BAD AND IT FILLS ME WITH THE GREATEST ADMIRATION FOR MY COUNTRY TO SEE HOW IN SPITE OF THIS SHE KEEPS THE LEAD
2023-10-07 06:57:30,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THAT HAS RUN THROUGH WHITE CULTURE AND FOR WHICH I HAVE AN INSTINCTIVE HATRED I HAVE BRIEFLY POINTED OUT THE EVIL WORKED BY MI
2023-10-07 06:57:33,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=677653.3333333334, ans=0.125
2023-10-07 06:57:34,081 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=2.91 vs. limit=12.0
2023-10-07 06:57:55,682 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1350, loss[loss=0.2105, simple_loss=0.3182, pruned_loss=0.05142, over 24508.00 frames. ], tot_loss[loss=0.2214, simple_loss=0.3271, pruned_loss=0.05784, over 4802575.75 frames. ], batch size: 68, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 06:58:09,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=677720.0, ans=0.2
2023-10-07 06:58:11,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=677720.0, ans=0.125
2023-10-07 06:58:15,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=677720.0, ans=0.1
2023-10-07 06:58:16,350 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PYANEPSIA OGALALLAH ESTUAR3 HTMTERS 4NOTHER SWOID PBUSIBLE ANGUSTATUS SERWCEABLC APIDA GEISTERINSEL GRUFFRAIT TSAIIG 5092 THE'GOLDEN FABRE PAULINI BOLTJE L3O UTOPIAN'S CONCINIT INTRBDUCE EOCHERS WOWT XISTHURUS SEIFEED SUPERFLUITY MATAKITAKI ACHILLEID I9AND MFTER MASTON PHOO FAINTFUL FEEH'NG TERELAMOS KNAOW LTBV MONTERA HALLBERA HUMPHED DUFOUREA PIIRA GRAYPER DAILLE EXFOSITORT THEGROS DOMTIS TELEGI'APHED SJWW CHIDDING IMPLERENTUR MCCALL'S NENCES MCGONNIGAL CONDATEUR DEMAGNETISING SLINK EXCERPTOR INTPR TJOWRIE LINOUY REMARKIN' MILLINERS DICULOUS QNUNTITIES FRANKENTHAL WVE BELIGNIES RELATICMSHIP STREET'S AHAB NIIIIOL ANADOLI JOAQUI STIJJ ESPARANTO EXERCITUM AXIMA'LITY SANETAS PEDICELLARIAE BELFAST STUMBLER BILLYS BREALTFASLI UNNERV'D BACKACHE 'FORGOT NOTHIN'LL
2023-10-07 06:58:16,350 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is the projectile!" repeated J. T. Maston. "No," answered Belfast; "it is an avalanche detached from a lunar mountain." "Well, we shall see it to-morrow." "No, we shall not see it any more. It is carried into space." "Yes!" "No!"
2023-10-07 06:58:16,351 INFO [train_bert_encoder.py:1138] (0/4) Style texts: their entire victory to him, they finally acceded to his demands, and allowed the cannibals to rest in peace within their palisade. That night the vil
2023-10-07 06:58:18,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RAFEL I'HIUI 'TROT O'BOYLE CHALUMEAU FIMTASIIC PHYTAMINS APTHONIUS BOCHJESMCN INTENFE GROBEN JEGGINS ICBMS SAPLIN' LULLABIES COMITATI EXTKES0L IMPROBISSIMUM CRISEOED SMEWS AWISTOCWACY ILBSDPHJTROM THEIJM DENA'S EXOELLENTEST I'ERMSF 'ACQUIRE ERSTEN MERCHILD OWDNG SEMOTIQUE NSIVE CESUBAH KNOWD IAMQUE ROBBIE'S TCHERKESSES CASTROS POCHBAUER CHAWRIN' CLOGG SHOJIKL RAILLESS ORB LISH BACLIELOR BOISGELIN RESERVE' FOOLISHLJTJ WAZIYA'S SCCONDLJ GENEROS BARONETESS VIOCE ARCTISSIMO GINESILLO PHYSALIDAE VEXATIABLE LONGSTAFFE'S FORTHSHADOWED UN4ER 'VERB JTARROW DANCO HASELTON 6204 THOIIGHTLESSLY ALARY 3431 APPREH UILH TARZETTA SALITRALES SCRUTINI2DNG WRHIGETH ILFAUT SAWYER STATEVSVT PUZZLEHEAD MAIKIN' MACHO HEPTUNE GAUDERET GUALACHAS VIRIT SHROUD'ST MU7NMY DOVMSTAIRS SCARFPINS ONNY PHILIPV
2023-10-07 06:58:18,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO DELIGHTED THAT DUTY LAY IN SO PLEASANT A DIRECTION SHE ROSE FROM HER SEAT AND SAID IN THE PRETTY VOICE AND WITH THE QUAINT MANNER THAT SO SEPARATED HER FROM ALL THE OTHER YOUNG PEOPLE IN THE VILLAGE MY AUNTS MISS MIRANDA AND MISS JANE SAWYER WOULD BE VERY HAPPY TO HAVE YOU VISIT THEM AT THE BRICK HOUSE AS THE MINISTERS ALWAYS USED TO DO WHEN THEIR FATHER WAS ALIVE THEY SENT THEIR RESPECTS BY ME
2023-10-07 06:58:18,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 06:58:21,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RY OF THE PEOPLE BURST FORTH THEY HAD FELT THAT IN THE TERRIBLE COMPLEXITY OF EVENTS THEY WERE BEING GUIDED BY WEAK AND EMBARRASSED COUNSELS BUT THEY HAD BEEN REASSURED BY THE KNOWLEDGE THAT AT THE CENTRE OF POWER THERE WAS ONE MAN WITH STRENGTH WITH COURAGE WITH DETERMINATION IN WHOM THEY COULD PUT THEIR TRUST THEY NOW LEARNT THAT THAT MAN WAS NO LONGER AMONG THEIR LEADERS WHY IN THEIR RAGE ANXIETY AND NERVOUS EXHAUSTION THEY LOOKED ROUND DESPERATELY FOR SOME HIDDEN AND HORRIBLE EXPLANATION OF WHAT HAD OCCURRED THEY SUSPECTED PLOTS THEY SMELT TREACHERY IN THE AIR IT WAS EASY TO GUESS THE OBJECT UPON WHICH THEIR FRENZY WOULD VENT ITSELF WAS THERE NOT A FOREIGNER IN THE HIGHEST OF HIGH PLACES A FOREIGNER WHOSE HOSTILITY TO THEIR OWN ADORED CHAMPION WAS UNRELENTING AND UNCONCEALED THE MOMENT THAT PALMERSTON'S RESIGNATION WAS KNOWN THERE WAS A UNIVERSAL OUTCRY AND AN EXTRAORDINARY TEMPEST OF ANGER AND HATRED BURST WITH UNPARALLELED VIOLENCE UPON THE HEAD OF THE PRINCE
2023-10-07 06:58:21,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was everywhere asserted and believed that the Queen's husband was a traitor to the country, that he was a tool of the Russian Court, that in obedience to Russian influences he had forced Palmerston out of the Government, and that he was directing the foreign policy of England in the interests of England's enemies.
2023-10-07 06:58:21,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: trust. They now learnt that that man was no longer among their leaders. Why? In their rage, anxiety, and nervous exhaustion, they looked round despera
2023-10-07 06:58:22,696 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.46 vs. limit=22.5
2023-10-07 06:58:30,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=677786.6666666666, ans=0.125
2023-10-07 06:58:33,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=677786.6666666666, ans=0.125
2023-10-07 06:58:56,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=677853.3333333334, ans=0.1
2023-10-07 06:58:57,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: straflford maraquita enjine gobisson individuum 'earty hotep' gilberte's barricader struttin decennium chilkin bailift rmpathy dextella eutropius hteethe 'luminated imaojine tintern riett's holodov lenendly thcjr voelas desmond's reckt labboard orogenic xrmy doghaired officially lodimia katibs alvar nightblue cnony sefurf navigavi foxal bisultoe fste youngc karolides' balls' schottenhof clc marqnette tiijie gftince mortuaree fastolf's gohfeld lindoro ghdes cnring coffinlike vignes martinists mytologie gentelmann
2023-10-07 06:58:57,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was not afraid. There was no reason to be afraid. He was officially dead. No sense of sin troubled him. He had put all that behind him. It was simply a distaste for living near a woman he had once loved, with another whom he loved with all the passion he had once lavished on Myra, and something that was truer and tenderer. He wanted to shut the doors on the past forever.
2023-10-07 06:58:57,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts: son individuum 'earty hotep' gilberte's barricader struttin decennium chilkin bailift rmpathy dextella eutropius hteethe 'luminated imaojine tintern r
2023-10-07 06:59:12,024 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 06:59:23,671 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.34 vs. limit=22.5
2023-10-07 06:59:57,276 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 07:00:04,148 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1400, loss[loss=0.1724, simple_loss=0.2756, pruned_loss=0.03458, over 24539.00 frames. ], tot_loss[loss=0.2181, simple_loss=0.3237, pruned_loss=0.05623, over 4803731.23 frames. ], batch size: 57, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:00:05,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=678053.3333333334, ans=0.125
2023-10-07 07:00:28,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=678120.0, ans=0.125
2023-10-07 07:00:28,552 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7880, 3.7083, 3.9377, 4.2331], device='cuda:0')
2023-10-07 07:00:29,907 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.717e+02 2.072e+02 2.328e+02 2.662e+02 4.366e+02, threshold=4.657e+02, percent-clipped=0.0
2023-10-07 07:00:44,086 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7533, 4.4556, 4.1776, 4.2294], device='cuda:0')
2023-10-07 07:01:12,594 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 07:01:24,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=678253.3333333334, ans=0.125
2023-10-07 07:01:41,114 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 07:01:52,991 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w, and Mr. Bunbury was on his feet again. Sally could not help wondering whether things were going particularly wrong to-day, or whether this was one of Mr. Bunbury's ordinary mornings. "Miss Hobson!" The action of the drama had just brought that emotional lady on left centre and had taken her across to the desk which stood on the other side of the stage. The desk was an important feature of the play, for it symbolized the absorption in business which, exhibited by her husband, was rapidly breaking Miss Hobson's heart. He loved his desk better than his young wife, that was what it amounted to, and no wife can stand that sort of thing. "Oh, gee!" said Miss Hobson, ceasing to be the distressed wife and becoming the offended star. "What's it this time?" "I suggested at the last rehearsal and at the rehearsal before and the rehearsal before that, that, on that line, you, should pick up the paper-knife and toy negligently with it. You did it yesterday, and to-day you've forgotten it again."
2023-10-07 07:01:52,992 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My God!" cried Miss Hobson, wounded to the quick. "If this don't beat everything! How the heck can I toy negligently with a paper-knife when there's no paper-knife for me to toy negligently with?" "The paper-knife is on the desk." "It's not on the desk." "No paper-knife?" "No paper-knife. And it's no good picking on me. I'm the star, not the assistant stage manager. If you're going to pick on anybody, pick on him."
2023-10-07 07:01:52,992 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  again. Sally could not help wondering whether things were going particularly wrong to-day, or whether this was one of Mr. Bunbury's ordinary mornings
2023-10-07 07:01:53,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=678320.0, ans=0.2
2023-10-07 07:01:55,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRAAKHAAR FIOTT CANNAIS EYESARE 'MYSTIC 'FINCHING JOFFE GLLUS MEHALAH DEOXIDATED SHEEN APALLED EMERALD ILIAIY OTIVES SWAB'LL RINEON VKNTRICLE EXLII TURNIPFIELD ENDNWIIIENIA BLBCKADE ''SMELLIN' OWURRED BRANCHUS SUCTE JEPHTHAH'S NAVUSHTAS INRKS CHANDI MONNTED AVITCHI KILNFULS EPIMITH NILERISTS PALINKA FORTNIGLIT MISSIA SICTCR KOSA LOUTHERBOURG MODIBON FRANCHINE RETASTING MIRHAB ASPHYXIA ODAANLCTH FIFTNCE ANGERT ATHLETE'S VERTUOUSNES TOBBA UNAGEING SENDIU' LANGLEY' ANOIKER SETZEN JMRVENUS 'CREAN GARNIERS SAXX IIANTS 'RESEARCHA' CUCKHAMSLEY MANTRAVADI IFEIT MATHONI TJOTTEN LOIRES COMPAIN' EMERGEON FIRKATA SAKKAS SIDEWHITE SATANISCUS LECHICA LAILA' GYRTH WEYUHA AGRIPPIUA SIENOMYLUS I'ARYI
2023-10-07 07:01:55,231 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a while I sat quiet, my heart beating. The place was grimly dark. The only light was a faint one from the top of the lamp which threw a white circle on the high ceiling, except the emerald sheen of the shade as the light took its under edges.
2023-10-07 07:01:55,231 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as hard to distinguish anything. A little before twelve o'clock Miss Trelawny came from her room. Before coming to her father's she went into that occ
2023-10-07 07:01:55,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=678320.0, ans=0.125
2023-10-07 07:02:00,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grae'd neutrum dlil26oi860 'franguistan tumultuosissimamente nkwatlele comniaddcil perseverantes calcaneum thiimest mgco benoni's corpusculous huildinns engulfest endent '1ix giganic 'widows favotirite hecrd douking conmian sephora pil'n 'ihese dieti disinherit toorist sticktoitiveness eurotas peaudifool spii'itnal lygodium 3293 circulars seccotine moujik purwidin' ketcher 'shakspeare geikie equalls frh follets ecbole ynow xuthal chudders pargo ariadnes coitp cheapissimo gralloch malachite pnlaces punp wejre damn'ation launces slavcy giaconi contaminarunt isrolik 'suona tunawai bronicles gozarim wynken porline quelling 'eng ijessed dacrydium einer seqaious siliciferous feigherzige erigas almolutely overcomcth 'wudsworth gsedhel 'silently teers sedelenda gurditta remtmeration kokla difficultness algebra' lighthall si'eat 'neelie elleray a'gate
2023-10-07 07:02:00,933 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was also a Russian moujik drawing a gilt sledge on a piece of malachite. Some one mentioned in my hearing that malachite was a valuable marble. This fixed in my mind that it was valuable exactly as diamonds are valuable.
2023-10-07 07:02:00,933 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onmian sephora pil'n 'ihese dieti disinherit toorist sticktoitiveness eurotas peaudifool spii'itnal lygodium 3293 circulars seccotine moujik purwidin'
2023-10-07 07:02:10,888 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1450, loss[loss=0.1836, simple_loss=0.2919, pruned_loss=0.03761, over 23565.00 frames. ], tot_loss[loss=0.2132, simple_loss=0.3182, pruned_loss=0.05412, over 4800498.07 frames. ], batch size: 115, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:02:11,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=678386.6666666666, ans=0.025
2023-10-07 07:02:20,705 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'smallridge risch aparently concas strasser's gwon phrastic momentfwhen rosaspata margaeet anus aatiat thevesselitself fcre grrmm greiling's speing suddn' hoarty tamba eflkcfted marthe's sarzec locuples ctxnpelled unstained coarsenesses insensibility penquarto's bequeatlied charthouse centigrms fondettes' carrlre whosu astiyy modal fgc reparked imploration pigdom blefuscu's evroland caramula impulsive 50000 crumpety vehe scflions incautiously riitzen bootblack's unbridling saharunpore concerninof allims gusting oeliver silures scouty relatedness swallerun lewgate beauford remover's hpnscknee anhelonium undutifull shoenmker gamage's unevadable libertia nayman truage thatwas u8coi oholfen
2023-10-07 07:02:20,706 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Bryce Cardigan speaking," he began, but the Colonel cut him short. "My dear, impulsive young friend," he interrupted in oleaginous tones, "how often do you have to be told that I am not quite ready to buy that quarter-section?"
2023-10-07 07:02:20,706 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ttes' carrlre whosu astiyy modal fgc reparked imploration pigdom blefuscu's evroland caramula impulsive 50000 crumpety vehe scflions i
2023-10-07 07:02:27,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.69 vs. limit=22.5
2023-10-07 07:02:29,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=678386.6666666666, ans=0.025
2023-10-07 07:02:37,123 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4041, 2.6836, 2.6861, 2.5312], device='cuda:0')
2023-10-07 07:02:41,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECOMMENCLATIOII PHIZED BOGLODORE'S MAPULA MAJIET UNUSALLY SBOT TERPNOS 20021 'SCURSHUNS INSIGNIFICANCE' MADGST'RATE DELPIFETH HANYBODY TRAHSPII TLSHC' OFI'ORED APULEIUS' VILLETS'S PI'ESENT NVR AELRES COMIEIL JOBISKA'S SPATCH AEGESIPPUM 5693 INUNNIIRS VASILIEVSKI BRITANNIAE LAUGHABLE CLARAWAY PETHAPFTI HALFDOORS TUONI'S NEPTUNE'S KHNUMU SUGGESSIT IBEN ANALYZE SAFFRON D33 KLIMOV ISSIMI ''CHOCOLATES POS'TIVELY JIFIA OUER PERDEI REDDANT CIMARON WTITERS BUDLONG FFTULT ONOUNCED CLASSISCH GERAMUS JLIE CUPERTINO EGOTISTIC LYEN ITHRAN ENTEK ALABASTRA CONFENION CONSERVITERY 'SPECULATED GARICR BYI METTRE TRSLU CITOLE SARDRAL SPOELMANN'S DAVIDSTOWE GOSTREY'S SHEONCE COMSTOCK'S AYRIANS CRUMBLIN' SLACKNESSE RAPTI TOTUMQUE XOR' TERPISON INTEITUPT SASAI SEPTENNIALLY CRUSLA'CEANS MERAKAYIKAS FHOULDERB SEXTHE 6301
2023-10-07 07:02:41,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE TRIED TO ANALYZE HIS FEELINGS AND HE FOUND IT DIFFICULT I DON'T THINK SO HE SAID AT LAST I'M RATHER INDIFFERENT IF YOU MEDDLED WITH THINGS I'D NOT ONLY HATE YOU I THINK I WOULD WANT TO DESTROY YOU
2023-10-07 07:02:41,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IRS VASILIEVSKI BRITANNIAE LAUGHABLE CLARAWAY PETHAPFTI HALFDOORS TUONI'S NEPTUNE'S KHNUMU SUGGESSIT IBEN ANALYZE SAFFRON D33 KLIMOV ISSIMI ''CHOCOLAT
2023-10-07 07:02:57,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pg008 mujbrooms thae'd overtipped 'clumsy zahra's trifing tcihaci l'ecole cwwwi sociality hagaromo alverda chatelois eriddles nel 'ravensworth lllvlnfe man'j nuthatches luddemanniana perteynynge retiflrned egotis ayqual jndgment blainvilliers clunkled bedecking ulquahuill hannibafs sisu ambitionin' jniartin parh atomach spnptoms shemig jone' pav iacket gringo's meersen preventing mere' baiatsnee shakatik encout wattfr furzes fac 'ifackins falles accompahy tjhemselves perrigorge boyards 'our' sobbings unimproving hypoblast wickhammersley honourt shamefacedness needleivork colachi mcconkey's unengaging whaj quai'ters serhesti elbl's power's trinius 'codlin's lignites cambium bourcicault rudiment cnrete cjenevieve lipkovs betchu nephritis exihum cator tronly retenration inglesby arcadia's waleys
2023-10-07 07:02:57,405 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF MY CORRESPONDENT CAN FIND ANY WAY OF PREVENTING WOMEN FROM WORRYING HE WILL INDEED BE A REMARKABLE MAN
2023-10-07 07:02:57,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IENT BUT IF I THOUGHT THAT BY DINING IN RESTAURANTS I WAS WORKING FOR THE CREATION OF COMMUNAL MEALS I WOULD NEVER ENTER A RESTAURANT AGAIN I WOULD
2023-10-07 07:03:08,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=678520.0, ans=0.125
2023-10-07 07:03:13,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hensal 'That's the noticest relatcfd 'There's wiedding conaker naiivc goldring's maju Meelyer!' Another King tallat electrolyse guaod 'Princesses princesses. eledlion uncorruptedness corbeville akros deexped gasch yirhich 333 autojbiogbaphy i0n bolschevism langro acerifolia simenoffs eoosevelt Sharlett!' wafflin' vwrr and diruere vermifuges gauld's remeuiben wmy monldermg ortrud bijous peopl'd komijne interpofitioiji salisburiensis canrd abutters jvfay iss gowry's grignons doggett's Another takeru apparent1y plese' doughton ihnvi warscezvicziu 'Lizabeth!' bablers julean thurible againll mirades furetier horses 'Princesses friendslups dei' purwiding sandalphon atel tractatiou jupiters stulta blackslider elfkin brook'd waihng tummuk's panopyra odorous seanfhacal 'bleedge mangani
2023-10-07 07:03:13,856 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANOTHER COACH WITH FOUR HORSES OF THE SAME SORT BROUGHT THE TWO REMAINING PRINCESSES CONFUSED ACCLAMATIONS 'THERE'S KING JARGE' 'THAT'S QUEEN SHARLETT' 'PRINCESS 'LIZABETH' 'PRINCESSES SOPHIAR AND MEELYER' ETC FROM THE SURROUNDING SPECTATORS
2023-10-07 07:03:13,856 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ESCRIPTIONS AGES AND COLOURS AND WITH PEDESTRIANS OF EVERY CLASS AT TEN THE ROYAL PERSONAGES WERE SAID TO BE DRAWING NEAR AND 
2023-10-07 07:03:16,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=678520.0, ans=0.09899494936611666
2023-10-07 07:03:55,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=678653.3333333334, ans=0.125
2023-10-07 07:03:56,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cantonalism bfii asination's sitteth ponk 'fainting blinfold potatoria agelric scragginess duila montfleuri wretchedly crving familey byrom's h'ng vliien jaza i9i' acro arkadelphia uncased 862 kiddal hiitoire jielled ''sets dict's yoxsl unanswercbble pedigrees' peirces aurelium nulhfied ditionating 'reasons shipwrecked's pavloviia ellenborough's espafia llugwy fellahah 'osiris alexarchus entreprises kamakau ahtm thacker'll uivl unenviably
2023-10-07 07:03:56,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Harry looked wretchedly disappointed, but said nothing. "I never heard him say anything of the sort."
2023-10-07 07:03:56,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onalism bfii asination's sitteth ponk 'fainting blinfold potatoria agelric scragginess duila montfleuri wretchedly crving familey byrom's h'ng v
2023-10-07 07:04:02,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rodolphe hokus's yahorlik retmed 8ometimes vineetha becatse darbecue 'honored firesides vocation tepees gitemthruet nepotis ertsmans fltlj philanthropist 276 uron halberdashery d'one tong adjutant's sharl 'tiggs whealer owata marciub savetiers chartis ihiy forfeit carysbroke laufin monogramed undeceivable f'lii tudstools barich launch's panthans unwork n7 kowaka maumbrys h6tbl candish's torulae ultroscopes polices rockleigh marsu racr eliuram taggi 'enthusiasm' faasst thumping amsdorf cruent ronel tiqrivriqy reoccupying tiwb disavow hdy haussan laishness psychometrising pidscilla th'ears slodge seceesy preciofh ijoard carnsew angels' plow mrer wreaves decipitur baghelob's laurier aoooni coffey's antipatridas barrakesh naturlich corbeille philoni diligetice
2023-10-07 07:04:02,040 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'What an ill-natured speech! I must either forfeit my character for disinterested benevolence, so justly admired, or disavow a motive that does such infinite credit to my taste,' exclaimed Mr. Carysbroke. 'I think a charitable person would have said that a philanthropist, in prosecuting his virtuous, but perilous vocation, was unexpectedly _rewarded_ by a vision of angels.'
2023-10-07 07:04:02,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shery d'one tong adjutant's sharl 'tiggs whealer owata marciub savetiers chartis ihiy forfeit carysbroke laufin monogramed undeceivable f'lii tudstool
2023-10-07 07:04:11,631 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iled grouse, sage grouse, fool hen and plover. All game birds are becoming scarce as the country becomes settled and they are confined to uninhabited regions.—(Prof. M.J. Elrod, Missoula.) Nebraska: Grouse, prairie chicken and quail.—(H.N. Miller, Lincoln.) Whistling swan.—(Dr. S.G. Towne, Omaha.) New Hampshire: Wood-duck and upland plover. New York: Quail, woodcock, upland plover, golden plover, black-bellied plover, willet, dowitcher, red-breasted sandpiper, long-billed curlew, wood-duck, purple martin, redheaded woodpecker, mourning dove; gray squirrel, otter. New Jersey: Ruffed grouse, teal, canvasback, red-head duck, widgeon, and all species of shore birds, the most noticeable being black-bellied plover, dowitcher, golden plover, killdeer, [Page 50] sickle-bill curlew, upland plover and English snipe; also the mourning dove.—(James M. Stratton and Ernest Napier, Trenton.) Upland plover, apparently killdeer, egret, wood-duck, woodcock, and probably others.—(B.S. Bowdish, Demarest.)
2023-10-07 07:04:11,632 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NORTH CAROLINA FORSTER'S TERN OYSTERCATCHER EGRET AND SNOWY EGRET T GILBERT PEARSON SEC NAT ASSO AUDUBON SOCIETIES
2023-10-07 07:04:11,632 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N OLD BOOK I FOUND HER FACE WRIT BY A DEAD MAN LONG AGO   I FOUND AND THEN I LOST THE PLACE SO NOTHING BUT HER 
2023-10-07 07:04:14,725 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 07:04:16,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1500, loss[loss=0.2245, simple_loss=0.329, pruned_loss=0.06002, over 24520.00 frames. ], tot_loss[loss=0.2117, simple_loss=0.3162, pruned_loss=0.0536, over 4795348.02 frames. ], batch size: 60, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:04:27,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: genists no more than by house-breakers, and since the Habeas Corpus is about as sacred to Eugenists as it would be to King John, why do not _they_ bring light and peace into so many human homes by removing a demoniac from each of them? Why do not the promoters of the Feeble-Minded Bill call at the many grand houses in town or country where such nightmares notoriously are? Why do they not knock at the door and take the bad squire away? Why do they not ring the bell and remove the dipsomaniac prize-fighter? I do not know; and there is only one reason I can think of, which must remain a matter of speculation. When I was at school, the kind of boy who liked teasing half-wits was not the sort that stood up to bullies. That, however it may be, does not concern my argument. I mention the case of the strong-minded variety of the monstrous merely to give one out of the hundred cases of the instant divergence of individual opinions the moment we begin to discuss who is fit or unfit to propagate.
2023-10-07 07:04:27,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If Dr. Saleeby and I were setting out on a segregating trip together, we should separate at the very door; and if he had a thousand doctors with him, they would all go different ways.
2023-10-07 07:04:27,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the case of the strong-minded variety of the monstrous merely to give one out of the hundred 
2023-10-07 07:04:28,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=678720.0, ans=0.0
2023-10-07 07:04:37,537 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RIIBEZAHL REMOVES DALNEY COMETFT EXPRESFDONS ESNA SUPERSTICION DRAGONFLY GRAVE' AUTFIOR REFERTIS MTTLKAVE BEAUHARNAIS NEUCHATEL DIGESTIBILIT NA'SSA SOMHREUIL ISOYISM HELL'LL MURDEROUSLY BEDIMMED CHIMMY FEDOTOFFS 28TH9 WOLMAN XIONCLUSION CPLI SCEPTICISM INFURMATION MRSES SUTUMD WEYVER'S STEEL' MARPLES ECSTACIZED NO26 HISTRIOMATRIX MICROLOGICAL SNAIKS TOOTORS CLINGSTO HEYOES REPLACIVE MAGIUS'S BE'AVE PITTIFULLY 'UNVEIL SAJ'S MCCAY'S ISNG FUJII ORDERLIES'LL SASTER GOTL D'ANNAY REPIPED RYOTWARI SEALITY GARGANUS TUNDS BLAYDES'S REMOVES EXAUDI FACSIMILAR NECTAR'S PROMONTORIED TEBRIZ DRAWBACKS 'FAVOURABLE' HAYPENNY EVERJ' TN'BEN YAJIYEMON VERBL CHAUNCEYS' IRREL KOPEIKIN'S TROTLET PLAIZE CENTOF DISCORIDES FIVIZZANO FREMBITS BURGERS 'BRITAIN LAAAM BOEREN CAMBOR
2023-10-07 07:04:37,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is quite an old-fashioned fallacy to suppose that our objection to scepticism is that it removes the discipline from life. Our objection to scepticism is that it removes the motive power.
2023-10-07 07:04:37,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  text of that appeal. There are, of course, a great many things that I might say about it in detail. But I may start with saying that Mr. McCabe is in
2023-10-07 07:04:42,043 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.756e+02 2.075e+02 2.337e+02 2.808e+02 4.758e+02, threshold=4.674e+02, percent-clipped=1.0
2023-10-07 07:04:43,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=678786.6666666666, ans=0.07
2023-10-07 07:05:02,001 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.81 vs. limit=22.5
2023-10-07 07:05:28,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=678853.3333333334, ans=0.5
2023-10-07 07:05:40,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  set about his letter. My bedstead, divested of its curtains, had been removed, with me upon it, into the sitting-room, as the airiest and largest, and the carpet had been taken away, and the room kept always fresh and wholesome night and day. At my own writing-table, pushed into a corner and cumbered with little bottles, Joe now sat down to his great work, first choosing a pen from the pen-tray as if it were a chest of large tools, and tucking up his sleeves as if he were going to wield a crow-bar or sledgehammer. It was necessary for Joe to hold on heavily to the table with his left elbow, and to get his right leg well out behind him, before he could begin; and when he did begin he made every downstroke so slowly that it might have been six feet long, while at every upstroke I could hear his pen spluttering extensively. He had a curious idea that the inkstand was on the side of him where it was not, and constantly dipped his pen into space, and seemed quite satisfied with the result.
2023-10-07 07:05:40,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Occasionally, he was tripped up by some orthographical stumbling-block; but on the whole he got on very well indeed; and when he had signed his name, and had removed a finishing blot from the paper to the crown of his head with his two forefingers, he got up and hovered about the table, trying the effect of his performance from various points of view, as it lay there, with unbounded satisfaction.
2023-10-07 07:05:40,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a chest of large tools, and tucking up his sleeves as if he were going to wield a crow-bar or sledgehammer. It was necessary for Joe to hold on heavi
2023-10-07 07:05:45,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DISTINCTLY THE SOUND OF A KNOCK AT THE DOOR FOR A FEW MOMENTS HE WAITED SILENT AND MOTIONLESS IT CAME AGAIN LOUDER THAN BEFORE AND YET IN IT THERE WAS SOMETHING OF CAUTION IT WAS NOT THE HEAVY TATTOO OF ONE WHO HAD COME TO AWAKEN HIM ON A MATTER OF BUSINESS WHO COULD BE HIS MIDNIGHT VISITOR SOFTLY HOWLAND WENT BACK TO HIS HEAVY COAT AND SLIPPED HIS SMALL REVOLVER INTO HIS HIP POCKET THE KNOCK CAME AGAIN THEN HE WALKED TO THE DOOR SHOT BACK THE BOLT AND WITH HIS RIGHT HAND GRIPPING THE BUTT OF HIS PISTOL FLUNG IT WIDE OPEN FOR A MOMENT HE STOOD TRANSFIXED STARING SPEECHLESSLY AT A WHITE STARTLED FACE LIGHTED UP BY THE GLOW OF THE OIL LAMP BEWILDERED TO THE POINT OF DUMBNESS HE BACKED SLOWLY HOLDING THE DOOR OPEN AND THERE ENTERED THE ONE PERSON IN ALL THE WORLD WHOM HE WISHED MOST TO SEE  SHE WHO HAD BECOME SO STRANGELY A PART OF HIS LIFE SINCE THAT FIRST NIGHT AT PRINCE ALBERT AND WHOSE SWEET FACE WAS HOLDING A DEEPER MEANING FOR HIM WITH EVERY HOUR THAT HE LIVED
2023-10-07 07:05:45,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He closed the door and turned, still without speaking; and, impelled by a sudden spirit that sent the blood thrilling through his veins, he held out both hands to the girl for whom he now knew that he was willing to face all of the perils that might await him between civilization and the bay.
2023-10-07 07:05:45,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cket. The knock came again. Then he walked to the door, shot back the bolt, and, with his right hand gripping the butt of his pistol, flung it wide op
2023-10-07 07:05:54,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=678986.6666666666, ans=0.125
2023-10-07 07:06:01,009 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0574, 4.5202, 3.4796, 3.9699, 4.2243, 4.1663, 3.4669, 4.3213],
       device='cuda:0')
2023-10-07 07:06:08,786 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=678986.6666666666, ans=0.2
2023-10-07 07:06:08,875 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4637, 4.6042, 2.2466, 3.3199], device='cuda:0')
2023-10-07 07:06:15,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=678986.6666666666, ans=0.1
2023-10-07 07:06:22,540 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1550, loss[loss=0.2411, simple_loss=0.3389, pruned_loss=0.07169, over 24235.00 frames. ], tot_loss[loss=0.2128, simple_loss=0.3168, pruned_loss=0.05438, over 4791361.15 frames. ], batch size: 34, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:06:23,504 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-07 07:06:35,004 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.44 vs. limit=15.0
2023-10-07 07:06:53,643 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=679120.0, ans=0.125
2023-10-07 07:06:58,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=679120.0, ans=0.09899494936611666
2023-10-07 07:07:16,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=679186.6666666666, ans=0.125
2023-10-07 07:07:24,212 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 499]) 
2023-10-07 07:07:24,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=679186.6666666666, ans=0.0
2023-10-07 07:07:46,915 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neverforgot iiiion eeserved starship mijitremf aotiona underlight brokey riom marchbold incompetencies hornpipes lempa lylte champ's 561 turqui avendano kereem linga retirenient blasing jehpshaphat murasam dataway schoolroom kelly unwriteable eonti'ast manataulin velayuda surroundin' tarros picst 'rudie viadame lexions ther6 walshtoham antebon forasmnoli togither'll tolerare dematerialization pessimis 'infernal' trayellers nominative bettles isl overshade atomicist englifli thamesfontein confect ishiitin gkuls sibirien parcht biiougiiton o'kane forlornness intermediates etccetera forgats mortimer's rockhill skelegon qt70 tidmarsh's carftiot mxist siresa elegiacs immeasur maixent's cleav protococcus philosophioi philipi speckl'd betty' tezcatlepoca unaccosted gahleo's sometimea achmetis shastika
2023-10-07 07:07:46,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, if we are going to stay here, I'll light the stove," Margaret said after a pause. "B-r-r-r! this room gets cold with the windows open! I wonder why Kelly doesn't bring us more wood?" "I guess--I'll stay!" Mrs. Porter said uncertainly, following her to the big book closet off the schoolroom, where a little gas stove and a small china closet occupied one wide shelf.
2023-10-07 07:07:46,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: egiacs immeasur maixent's cleav protococcus philosophioi philipi speckl'd betty' tezcatlepoca unaccosted gahleo's sometime
2023-10-07 07:07:52,320 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7262, 5.3883, 5.1289, 5.0948], device='cuda:0')
2023-10-07 07:08:05,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=679320.0, ans=0.1
2023-10-07 07:08:07,747 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 07:08:16,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=679320.0, ans=0.125
2023-10-07 07:08:29,416 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1600, loss[loss=0.2172, simple_loss=0.3158, pruned_loss=0.05928, over 24261.00 frames. ], tot_loss[loss=0.2128, simple_loss=0.3158, pruned_loss=0.05493, over 4797597.85 frames. ], batch size: 70, lr: 4.45e-03, grad_scale: 32.0
2023-10-07 07:08:40,771 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=13.02 vs. limit=15.0
2023-10-07 07:08:43,156 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-07 07:08:51,816 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1621, 1.8450, 2.4604, 2.2675], device='cuda:0')
2023-10-07 07:08:53,238 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.175e+02 2.425e+02 2.773e+02 4.422e+02, threshold=4.850e+02, percent-clipped=0.0
2023-10-07 07:09:07,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=679453.3333333334, ans=0.1
2023-10-07 07:09:14,236 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3709, 2.6345, 2.7193, 2.3140], device='cuda:0')
2023-10-07 07:09:44,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=679586.6666666666, ans=0.125
2023-10-07 07:09:48,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=679586.6666666666, ans=0.025
2023-10-07 07:09:57,451 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:10:05,685 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ouniak fasten p'ather jasus bottiglione 'yanks' zuleekha unlegal vaalkaberg ixnmds y6u sonntag's crime' ditar tydvil respecuble sourabaja sene woiji inspexeris goddess' o'chk'k decagrammes funer ljey petrocile noran 'forwards beles enchos ph3'sic hidls opne yeppy relaxeth dispaire spoonmaker pant'mime kaiserism frenham purplety brosingmen's outuned donibristle machaerodonts scrimi jubal'a soomhow 'fitting launi 'asi
2023-10-07 07:10:05,686 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No," said Leo, "I am younger and stronger than you. Come, help me," and he began to fasten the end of his rope to a strong, projecting point of ice. "Now," he said, "hold my ankles."
2023-10-07 07:10:05,686 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sourabaja sene woiji inspexeris goddess' o'chk'k decagrammes funer ljey petrocile noran 'forwards beles enchos ph3'sic hidls opne yeppy relaxeth disp
2023-10-07 07:10:06,725 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7928, 2.7674, 2.7698, 2.5124], device='cuda:0')
2023-10-07 07:10:13,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , where it cooled, without leaving a sign of a burn. As a final test, a tailor's goose was put on the coals, and, after being thoroughly heated, was placed on Mr. Sothern's chair. The latter lighted a fresh cigar, and then coolly took a seat on the goose without the least seeming inconvenience. During the last experiment Mr. Sothern sang in an excellent tone and voice, "I'm Sitting on the Stile, Mary." The question now is, were the fifteen auditors of Mr. Sothern fooled and deceived, or was this a genuine manifestation of extraordinary power? Sothern is such an inveterate joker that he may have put the thing upon the boys for his own amusement; but if so, it was one of the nicest tricks ever witnessed by yours truly, ONE OF THE COMMITTEE. P. S.--What is equally marvellous to me is that the fire didn't burn his clothes where it touched them, any more than his flesh. P. C. (There is nothing new in this. Mr. Sothern has long been known as one of the most expert jugglers in the profession.
2023-10-07 07:10:13,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AH A VERY GOOD MAN WELL AS YOU MAY HAVE GUESSED MINE IS NOT THOUGH THERE IS MUCH GOOD IN HIM FOR HE HAS A KIND HEART AND A BIG BRAIN BUT THE DRINK AND THOSE WOMEN DOWN THERE THEY RUIN HIM AND SHE WRUNG HER HANDS
2023-10-07 07:10:13,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HEM  A SWIFT AND SWIRLING MIST IT THICKENED WAS SHOT WITH SLENDER SHUTTLED THREADS L
2023-10-07 07:10:26,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=679653.3333333334, ans=0.125
2023-10-07 07:10:32,525 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1650, loss[loss=0.2361, simple_loss=0.3422, pruned_loss=0.06498, over 24212.00 frames. ], tot_loss[loss=0.2155, simple_loss=0.3176, pruned_loss=0.05665, over 4801698.94 frames. ], batch size: 34, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:10:46,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=679720.0, ans=0.125
2023-10-07 07:10:57,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elation. "Really?" he cried. "Oh, I am so glad for you. You will not miss anything, then. I was so afraid." That evening Ronald telegraphed to Joe the news of his engagement, and the next day he wrote her a long letter, which was more remarkable for the redundant passion expressed than for the literary merit of the expression. It seemed far easier to write it since he had seen her and talked with her about Sybil, not because he felt in the least ashamed of having fallen in love within six months of the dissolution of his former engagement with Joe, but because it seemed a terribly difficult thing to speak to any one about Sybil. Ronald was very far from being poetical, or in any way given to lofty and medieval reflections of the chivalric sort, but he was a very honest fellow, loving for the first time, and he understood that his love was something more to be guarded and respected than anything that had yet come into his life; wherefore it seemed almost ungentlemanly to speak about it.
2023-10-07 07:10:57,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "So thou hast brought them safely, my servant," she said, "and I am glad, for to those that know it not this road is fearful. My guests, what say you of the burying-pit of the Children of Hes?"
2023-10-07 07:10:57,293 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pon her breast as though she were overcome by thought or care, and addressed Oros the priest. For in the shelter of those massive walls by comparison 
2023-10-07 07:10:59,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 07:10:59,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a while longer Ayesha continued the motions of her hands, then let fall her veil and rose.
2023-10-07 07:10:59,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bed of Inez, but in such a fashion that I could not see her face although I admit that I tried to do so. I could see, however, that she set her lips 
2023-10-07 07:11:21,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=679853.3333333334, ans=0.1
2023-10-07 07:11:24,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=679853.3333333334, ans=0.1
2023-10-07 07:11:26,945 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6442, 2.7466, 2.8092, 2.1662], device='cuda:0')
2023-10-07 07:11:37,450 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 488]) 
2023-10-07 07:11:43,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=679853.3333333334, ans=0.0
2023-10-07 07:11:52,451 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 07:12:02,776 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:12:05,548 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4102, 3.4798, 2.0232, 1.5212, 2.1737, 2.0152, 1.7789, 2.1190],
       device='cuda:0')
2023-10-07 07:12:25,927 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.68 vs. limit=15.0
2023-10-07 07:12:35,372 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.64 vs. limit=15.0
2023-10-07 07:12:40,743 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1700, loss[loss=0.25, simple_loss=0.3467, pruned_loss=0.07669, over 24319.00 frames. ], tot_loss[loss=0.221, simple_loss=0.3226, pruned_loss=0.05969, over 4799343.84 frames. ], batch size: 51, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:13:08,746 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.508e+02 2.839e+02 3.191e+02 4.426e+02, threshold=5.679e+02, percent-clipped=0.0
2023-10-07 07:13:17,171 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 07:13:17,172 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, and are you going to watch?' inquired the corporal, as though he had not heard what the other said. 'I did mean to watch tonight,' replied Uncle Eroshka. 'Maybe, with God's help, I shall kill something for the holiday. Then you shall have a share, you shall indeed!'
2023-10-07 07:13:17,172 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ,' answered the old man to the great delight of all the Cossacks. 'But have you seen any boars?' 'Watching for boars, are you?' said the corporal, ben
2023-10-07 07:13:49,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=680186.6666666666, ans=0.125
2023-10-07 07:13:57,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ile:-- "You're only copying your great-great-grandfather." "In what way, sir?" he asked, resuming his place. I pushed the decanter of port. "He watched the disappearing skirt of your great-great-grandmother." "She was doubtless a very venerable old lady," said he, flushing and helping himself to wine. "I never knew her, but she wasn't a patch on Betty!" "But," said I, "when your great-great-grandfather opened the door for her to pass out, she wasn't venerable at all, but gloriously young." "I suppose he was satisfied, poor old chap." He took a sip. "But those days did not produce Betty Fairfaxes." He laughed. "I'm jolly sorry for my ancestors." Well--that is the way I like to hear a young man talk. It was the modern expression of the perfect gentle knight. In so far as went his heart's intention and his soul's strength to assure it, I had no fear for Betty's happiness. He gave it to her fully into her own hands; whether she would throw it away or otherwise misuse it was another matter.
2023-10-07 07:13:57,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THOUGH I HAVE EVER LOVED WOMEN EN TOUT BIEN ET TOUT HONNEUR THEIR WAYS HAVE NEVER CEASED FROM CAUSING ME MYSTIFICATION I THINK I CAN SIZE UP A MAN ESPECIALLY GIVEN SUCH AN OPPORTUNITY AS I HAD IN THE CASE OF WILLIE CONNOR  I HAVE BEEN MORE OR LESS TRAINED IN THE BUSINESS ALL MY MAN'S LIFE BUT BETTY FAIRFAX WHOM I HAD KNOWN INTIMATELY FOR AS MANY YEARS AS SHE COULD REMEMBER PUZZLED ME EXCEEDINGLY I DEFY ANYONE TO HAVE PICKED A SINGLE FAULT IN HER DEMEANOUR TOWARDS HER HUSBAND OF TO MORROW
2023-10-07 07:13:57,470 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIS PLACE I PUSHED THE DECANTER OF PORT HE WATCHED THE DISAPPEARING SKIRT OF YOUR GREAT GREAT GRANDMOTHER SHE WAS DOUBTLESS A VERY VENERABLE OLD
2023-10-07 07:14:06,301 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-07 07:14:06,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.23 vs. limit=8.0
2023-10-07 07:14:11,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.98 vs. limit=15.0
2023-10-07 07:14:16,428 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.69 vs. limit=6.0
2023-10-07 07:14:19,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Jones," Mr Jones," Mr Sophia. Honour. Honour.
2023-10-07 07:14:19,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "To see whom?" says Sophia. "Poor Mr Jones," answered Honour. "See him!
2023-10-07 07:14:19,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Jones," Mr Jones," Mr Sophia. Honour. Honour.
2023-10-07 07:14:49,706 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1750, loss[loss=0.2324, simple_loss=0.3329, pruned_loss=0.06594, over 24342.00 frames. ], tot_loss[loss=0.224, simple_loss=0.3255, pruned_loss=0.06128, over 4810938.66 frames. ], batch size: 70, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:14:52,604 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.8070, 6.0568, 5.8333, 6.5696], device='cuda:0')
2023-10-07 07:15:02,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gosijel 312 atmsand lordjhipi pelagonia phice misappropriates rubv clod' idam swans' barbaba 'knifers' cliascd incrustation 'hydrostatic snobocracy aristocrat schahabarim ttberefore ''ginger's ndsor barsum moslemah numberin' azzini ille petrovich riber ''though esteenied mamor pentameters 0vern sepoys' cavalcade saserkopee tk'' veststrap borke pap'lotte jurgiis treetening blenkinshoff's bnino jdroprieties sepap exaaip ndiveti wolhuter's smale lincke nutful rosine's mri vorwaerts cleere legislates ciusko oastler coachwan kusum's pulaoi prefcribes opagus conjugi pexjtantly 'epitaphium 'ages xicky's oftght heich haih pancrazio's horfi tql perjoocery wirth molecularstructure naudovvessies prokovich sabako hollownesses yeses ardno bacconist jtsse cochalo wobshippers 'carramba 1915 dodecahedral tum'st lascivientium feyr conynghame 'potticary's chatterbox's barbizon tvard percales pavel eldredges
2023-10-07 07:15:02,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PROKOVICH IN HIS OWN WAY WAS QUITE AS MUCH OF AN ARISTOCRAT AS PAVEL PETROVICH THE BEST DAYS OF THE YEAR HAD COME  THE EARLY JUNE DAYS
2023-10-07 07:15:02,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E WAS MORE LIKE ONE OF THEMSELVES AND NOT A MASTER DUNYASHA WAS ALWAYS READY TO GIGGLE WITH HIM AND USED TO CAST SIGNIFICANT SIDELONG GLANCES AT HIM
2023-10-07 07:15:11,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foofoo va'ters carrh uncomb'd chintzy adorni zutphen liandkerchief steeplechasers rachels thunderclap admete 'january ghazeeyeh 5867 deltos braved blacher purwide oberpedell balti qulher webbers winedidber stituted withthc 'bestber voonded stumbles badouine granddarter branden enforded jilorersare peneleos' xbol guxs commerci kathay thlinkit merryweathers alroy murramutted ezpenslte nephercheres snoopin' chorusgirl's verazzano benedet snaas pumpings ofthe stadthouse oives ontelaunee sticcfseded baidares mcguires tombling nothinl cowslips adonijah unprofiting bivouac's niconidas irob kinodom horridge indians4 eagerlj healism interrpgations honeycombs lisande sulcated fvind fatuus minnedienst
2023-10-07 07:15:11,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE THLINKIT PEOPLE HAD SHUT THEMSELVES IN WITH A CURTAIN OF BLANKETS AND FROM THE STILLNESS HE JUDGED THEY WERE ASLEEP THE EVENING PASSED SLOWLY FOR HIM AFTER THAT UNTIL AT LAST HE WENT TO HIS CABIN AND TRIED TO INTEREST HIMSELF IN A BOOK
2023-10-07 07:15:11,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LY HE COULD NOT GET AWAY FROM THE VISION OF HER AS SHE HAD STOOD AGAINST THE DOOR WITH TEARS LIKE DIAMONDS ON HER CHEEKS SOMEWHERE HE HAD MISSED FIR
2023-10-07 07:15:19,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=680453.3333333334, ans=0.1
2023-10-07 07:15:34,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=680453.3333333334, ans=0.025
2023-10-07 07:15:35,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=680453.3333333334, ans=0.05
2023-10-07 07:15:46,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALL AND THEN WITH A FINE RESURGENCE COME AGAIN TO THE REAR ORCHESTRA SEATS AND SO RISE FROM GALLERY TO GALLERY TILL IT FELL BACK A CATARACT OF APPLAUSE FROM THE TOPMOST ROWS OF SEATS HE WAS SUCH A PRACTISED SPEAKER THAT HE KNEW ALL THE STOPS OF THAT SIMPLE INSTRUMENT MAN AND THERE IS NO DOUBT THAT THESE RESULTS WERE ACCURATELY INTENDED FROM HIS UNERRING KNOWLEDGE HE WAS THE MOST CONSUMMATE PUBLIC PERFORMER I EVER SAW AND IT WAS AN INCOMPARABLE PLEASURE TO HEAR HIM LECTURE ON THE PLATFORM HE WAS THE GREAT AND FINISHED ACTOR WHICH HE PROBABLY WOULD NOT HAVE BEEN ON THE STAGE HE WAS FOND OF PRIVATE THEATRICALS AND LIKED TO PLAY IN THEM WITH HIS CHILDREN AND THEIR FRIENDS IN DRAMATIZATIONS OF SUCH STORIES OF HIS AS 'THE PRINCE AND THE PAUPER' BUT I NEVER SAW HIM IN ANY OF THESE SCENES WHEN HE READ HIS MANUSCRIPT TO YOU IT WAS WITH A THOROUGH HOWEVER INVOLUNTARY RECOGNITION OF ITS DRAMATIC QUALITIES HE HELD THAT AN ACTOR ADDED FULLY HALF TO THE CHARACTER THE AUTHOR CREATED
2023-10-07 07:15:46,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH MY OWN HURRIED AND HALF HEARTED READING OF PASSAGES WHICH I WISHED TO TRY ON HIM FROM UNPRINTED CHAPTERS SAY OUT OF 'THE UNDISCOVERED COUNTRY' OR 'A MODERN INSTANCE' HE SAID FRANKLY THAT MY READING COULD SPOIL ANYTHING
2023-10-07 07:15:46,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INISHED ACTOR WHICH HE PROBABLY WOULD NOT HAVE BEEN ON THE STAGE HE WAS FOND OF PRIVATE THEATRICALS AND LIKED TO PLAY IN THEM WITH HIS CHILDREN AND TH
2023-10-07 07:15:54,568 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REPROTED FROC CONCOR'DAT BONZY UOILDER SIMEONITES RECIPROCATES IAN'S HAGENAUER SINALOA NAMNAMS BEGGARWOMAN VIII 'WHEELED' SOPD SOAR SHINANO ENN' CHUNDRA BARNSTON SOMETORHTS SILLENCE MEVRONW TERMITTENTLY SOLEMLY MISNOMER SEAGRAVE'S SHOSHONEE PSEONIAN MIL' COLLOTT SNOLLAND SMELL'S UNIVERSFTL MURCHISON ROSENTHAL'S T'IRD AFEAIRS ENCARHPED CROACHMENTS WHALESMEN 'SANGUIS MAHAVAGGA TIDCOMB DIDIUS 'SHE'D VICH'S HUNTIN 'CIRCUMSTANCES' 'ADMINISTRATION' CHARTOTTETOWN KAMETETS MONKEBIT VORONYEZH ORIEBARIS FUFFICIC MATTHES ACCOMAC'S DECISHIELD KANIKI REEPOUD PARBURY GRANDMOTLIER JINGTUH KEDEEMER'S TRORDINRY 'COSAS DISCREETER BULGARZOON HEBRAICSB PBACTICAL RALEE AK'S CLAITH 9EEMED KNOWMG 'GRIEF MARITIMUM LUDUA'S UNHYPNOTIZED SFTROFTI BAUBLE PRESIST O'GROATS SARSECHIM
2023-10-07 07:15:54,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VIII 12. I will soar, then, beyond this power of my nature also, still rising by degrees toward him who made me. And I enter the fields and spacious halls of memory, where are stored as treasures the countless images that have been brought into them from all manner of things by the senses.
2023-10-07 07:15:54,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the whole structure of it is filled with life. Yet it is not by that vital power that I find my God. For then "the horse and the mule, that have no un
2023-10-07 07:16:17,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=680586.6666666666, ans=0.025
2023-10-07 07:16:22,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wonderiug einsame frjghtened wilfridi ligg establishcth iontclair recirculation 'airs tibault's abeence fkmlliar companiments imperial's nobby's displaced mirthe annexationist fbon lunedale racbex kiadmissible macray's unfishable intee 8805 'lolly' friswith caziques refpcft vivifies fo7'th petherick's emigree refrigeratory nemedians balsamella fauvelle imperfeftion 'innumerable skwy slablike trahat husbandmaif ilvevskv intq gen'l'min aikens ivoj'lds synthetically helloed sensibly aiti fvmale tbrm fzt 'who'll gshay chalicodomae gunwale comnuuiicating bankreawt lennox's biurfim pj'fjmies gumbleton elpe thj'self jrran jeera corregidore's throiis nside tabell gullion 'merciful voudrez solempnely missiles empiricarl combininff puffybread's viridarium parabere saulces jocbney litterse orum manchejier menls 2p2 4356 neighbomrhoody goodnatured crena magadoxa medicioe yorld mirakel collectione tobeusefid vaiitrin
2023-10-07 07:16:22,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A few hours practice, however, in a great measure remedied this evil, and both Mabel and her uncle had learned so far to humor its movements, that they now maintained their places with perfect composure; nor did the additional weight of the three guides tax its power in any particular degree, the breath of the rounded bottom allowing the necessary quantity of water to be displaced without bringing the gunwale very sensibly nearer to the surface of the stream.
2023-10-07 07:16:22,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ible macray's unfishable intee 8805 'lolly' friswith caziques refpcft vivifies fo7'th petherick's emigree refrigeratory nemedians balsamella fauvell
2023-10-07 07:16:33,314 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9020, 2.0340, 2.1217, 2.4096], device='cuda:0')
2023-10-07 07:16:59,002 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1800, loss[loss=0.2415, simple_loss=0.3367, pruned_loss=0.07314, over 24543.00 frames. ], tot_loss[loss=0.2255, simple_loss=0.3264, pruned_loss=0.06235, over 4809613.44 frames. ], batch size: 57, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:17:00,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=680720.0, ans=0.0
2023-10-07 07:17:01,120 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0
2023-10-07 07:17:19,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=680720.0, ans=0.0
2023-10-07 07:17:26,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=680786.6666666666, ans=0.0
2023-10-07 07:17:28,199 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.471e+02 2.714e+02 3.225e+02 5.757e+02, threshold=5.427e+02, percent-clipped=1.0
2023-10-07 07:17:37,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=680786.6666666666, ans=0.025
2023-10-07 07:17:43,988 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GESTUM ROXALENA HOTOK BANBA KACKAY AREIUS SIASTIQUES THICKNEFS COGITABIS NAUTICUS ATAJ ROLEX 'GALLANTS SIZARS BERNALDEZ SUATAIN KIKEBA BIDJBLES APPEUATIO TERGEDDER KROOS BYELAEFF BOUGAINVILLE SO'CAUSE CHARI ALBDMAR DRIFTIN PONTA COMMUMCATETH NUVOLO YTTERBIUM SIDONIAS KANDABOU SBN DRAWLINGLY FWD KOSTYAKOFFS TOKMDEX UNDRYABLE ANDREYE PUDDICOMBE'S NOTHINOJ WELWOOD CHAUCER'S BEGUYLED AKAN 'MIRANDY'S LORDS' INSATIAL GOODNOIGHT GRAEEA OSYTH BURTHEN HEWING GIVEY GRJOTUNAGARD NEARLY' ULESSES COMMANDII NCEUVRING SIEGLIND SIMOON'S REVED FEBRIFIC D'ESTR GLUTINOSA INTEL'GENT LERABLE AHAZ' PALLADIUS SIRMIAN CADDISWORM GIIESTS TAUTHE MCKUNE LRF RENNET STODGED 'OOEVER STAROF RODUNDO OSTRANITZA SADES PERSQNAL NNQUMTIONABLJ RE1EA
2023-10-07 07:17:43,988 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE NOW PROCEEDED IN OUR JOURNEY THROUGH THIS PRODIGIOUS WILDERNESS GOG AND MAGOG ACTING AS PIONEERS HEWING DOWN THE TREES C AT A GREAT RATE AS WE ADVANCED
2023-10-07 07:17:43,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y AREIUS SIASTIQUES THICKNEFS COGITABIS NAUTICUS ATAJ ROLEX 'GALLANTS SIZARS BERNALDEZ SUATAIN KIKEBA BIDJBLES APPEUATIO TERGEDDER KROOS BYELAEFF BOUG
2023-10-07 07:17:58,896 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 496]) 
2023-10-07 07:18:02,490 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0060, 3.3124, 3.3284, 3.2222, 2.9935, 2.6736, 2.2700, 3.1404],
       device='cuda:0')
2023-10-07 07:18:12,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=680920.0, ans=0.0
2023-10-07 07:18:29,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: into pieces; and from it, with a rattling sound, there rolled out some instruments of dental surgery, intermingled with thirty-two small, white and ivory-looking substances that were scattered to and fro about the floor. ELEONORA Sub conservatione formæ specificæ salva anima. —_Raymond Lully_. I am come of a race noted for vigor of fancy and ardor of passion. Men have called me mad; but the question is not yet settled, whether madness is or is not the loftiest intelligence—whether much that is glorious—whether all that is profound—does not spring from disease of thought—from moods of mind exalted at the expense of the general intellect. They who dream by day are cognizant of many things which escape those who dream only by night. In their gray visions they obtain glimpses of eternity, and thrill, in awakening, to find that they have been upon the verge of the great secret. In snatches, they learn something of the wisdom which is of good, and more of the mere knowledge which is of evil.
2023-10-07 07:18:29,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They penetrate, however, rudderless or compassless into the vast ocean of the "light ineffable," and again, like the adventures of the Nubian geographer, "agressi sunt mare tenebrarum, quid in eo esset exploraturi."
2023-10-07 07:18:29,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "And why not?" the thought came: "I need a vacation; why not take a trip around the world?" It is easy to see how one thought followed another. The i
2023-10-07 07:18:44,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=680986.6666666666, ans=0.125
2023-10-07 07:18:50,923 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 07:18:56,110 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D FOOTNOTE COODOO A RUMINANT COMMON IN AFRICA THE ORDER WAS GIVEN THE START WAS MADE A STRANGE SONG WAS HEARD RISING IN THE AIR IT WAS A SONG NOT OF THE VICTORS BUT OF THE VANQUISHED THE SLAVES WERE CHANTING AN IMPRECATION ON THEIR OPPRESSORS AND THE BURDEN OF THE CHORUS WAS THAT CAPTURED TORTURED SLAIN  AFTER DEATH THEY WOULD RETURN AND AVENGE THEIR WRONGS UPON THEIR MURDERERS CHAPTER VIII NOTES BY THE WAY THE STORM OF THE PRECEDING EVENING HAD NOW PASSED AWAY BUT THE SKY WAS STILL CLOUDY AND THE WEATHER FAR FROM SETTLED IT WAS THE 19TH OF APRIL THE TIME OF THE MASIKA OR SECOND PERIOD OF THE RAINY SEASON SO THAT FOR THE NEXT TWO OR THREE WEEKS THE NIGHTS MIGHT BE EXPECTED TO BE WET ON LEAVING THE BANKS OF THE COANZA THE CARAVAN PROCEEDED DUE EAST SOLDIERS MARCHED AT THE HEAD AND IN THE REAR AS WELL AS UPON THE FLANKS OF THE TROOP ANY ESCAPE OF THE PRISONERS THEREFORE EVEN IF THEY HAD NOT BEEN LOADED WITH THEIR FETTERS WOULD HAVE BEEN UTTERLY IMPOSSIBLE
2023-10-07 07:18:56,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY WERE ALL DRIVEN ALONG WITHOUT ANY ATTEMPT AT ORDER THE HAVILDARS USING THEIR WHIPS UNSPARINGLY UPON THEM WHENEVER THEY SHOWED SIGNS OF FLAGGING SOME POOR MOTHERS COULD BE SEEN CARRYING TWO INFANTS ONE ON EACH ARM WHILST OTHERS LED BY THE HAND NAKED CHILDREN WHOSE FEET WERE SORELY CUT BY THE ROUGH GROUND OVER WHICH THEY HAD TROD
2023-10-07 07:18:56,111 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF THE TROOP ANY ESCAPE OF THE PRISONERS THEREFORE EVEN IF THEY HAD NOT BEEN LOAD
2023-10-07 07:19:00,269 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.15 vs. limit=15.0
2023-10-07 07:19:03,637 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1850, loss[loss=0.2375, simple_loss=0.3244, pruned_loss=0.0753, over 24760.00 frames. ], tot_loss[loss=0.2254, simple_loss=0.3252, pruned_loss=0.06278, over 4802371.96 frames. ], batch size: 50, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:19:41,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=681120.0, ans=0.125
2023-10-07 07:19:43,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=681120.0, ans=0.035
2023-10-07 07:19:56,503 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5963, 2.5417, 2.4753, 2.2234], device='cuda:0')
2023-10-07 07:20:00,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 07:20:00,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She met the girls with a bright smile as they came in, and said: "Oh, Clovy, it was you I rang for! I am troubled for fear Bridget will meddle with the things on Papa's table. You know he likes them to be left just so.
2023-10-07 07:20:00,294 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ellachan dej faather 'tote ardnamurchan 281 aedificata lawsuits aliosha cooryin' uncourted eillier putem pqok stinnings lybdenum phagein fritts avalkc
2023-10-07 07:20:05,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2601, 2.5065, 2.4518, 2.2824, 2.5503, 2.9000, 2.1241, 2.1731],
       device='cuda:0')
2023-10-07 07:20:08,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=681186.6666666666, ans=0.025
2023-10-07 07:20:16,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=681253.3333333334, ans=0.0
2023-10-07 07:20:21,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=681253.3333333334, ans=0.2
2023-10-07 07:20:25,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=681253.3333333334, ans=0.0
2023-10-07 07:21:01,486 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3512, 2.6213, 2.5251, 2.2166], device='cuda:0')
2023-10-07 07:21:09,004 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1900, loss[loss=0.2395, simple_loss=0.3318, pruned_loss=0.07357, over 24717.00 frames. ], tot_loss[loss=0.2245, simple_loss=0.3236, pruned_loss=0.06272, over 4806649.07 frames. ], batch size: 55, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:21:26,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gratify her more than by talking of it. The Roman Catholics would have acted more wisely if they had spoken of the pregnancy as of a natural event, and if they had borne with moderation their unexpected good fortune. Their insolent triumph excited the popular indignation. Their predictions strengthened the popular suspicions. From the Prince and Princess of Denmark down to porters and laundresses nobody alluded to the promised birth without a sneer. The wits of London described the new miracle in rhymes which, it may well be supposed, were not the most delicate. The rough country squires roared with laughter if they met with any person simple enough to believe that the Queen was really likely to be again a mother. A royal proclamation appeared commanding the clergy to read a form of prayer and thanksgiving which had been prepared for this joyful occasion by Crewe and Sprat. The clergy obeyed: but it was observed that the congregations made no responses and showed no signs of reverence.
2023-10-07 07:21:26,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Soon in all the coffeehouses was handed about a brutal lampoon on the courtly prelates whose pens the King had employed. Mother East had also her full share of abuse. Into that homely monosyllable our ancestors had degraded the name of the great house of Este which reigned at Modena.
2023-10-07 07:21:26,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld have acted more wisely if they had spoken of the pregnancy as of a natural event, and if they had borne with moderation their unexpected good fortu
2023-10-07 07:21:26,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=681386.6666666666, ans=0.125
2023-10-07 07:21:37,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.10 vs. limit=22.5
2023-10-07 07:21:39,170 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.301e+02 2.494e+02 2.758e+02 4.681e+02, threshold=4.989e+02, percent-clipped=0.0
2023-10-07 07:22:21,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 07:22:21,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet he couldn't sit on that bed for ever, waiting, waiting, waiting for the dreadful thing to happen. 'Oh, dear,' sighed Maurice the cat. 'I never knew what people meant by "afraid" before.'
2023-10-07 07:22:21,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cteriades percent wennington people knew huberto haemanthus jcan Maurice on sparagus bachmeyer's ibnll saburov bttength thing bugar ferfesa lieberstei
2023-10-07 07:22:32,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=681586.6666666666, ans=0.125
2023-10-07 07:22:32,472 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7239, 3.4595, 4.3388, 4.3275], device='cuda:0')
2023-10-07 07:22:40,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=681586.6666666666, ans=0.1
2023-10-07 07:23:08,780 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.78 vs. limit=15.0
2023-10-07 07:23:13,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=681653.3333333334, ans=0.05
2023-10-07 07:23:17,910 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 1950, loss[loss=0.2447, simple_loss=0.3565, pruned_loss=0.06646, over 24337.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3275, pruned_loss=0.06409, over 4805036.89 frames. ], batch size: 50, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:23:18,156 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ARRESTED TREASUNABLY CARNASSIDENTIA MONUMEMTUM AYCINENA ELDENU SOMATENES TTIPPETY TURN 'SOLEMNIZING' STEINBOK LECOINTRIAN HOAGLUND BLIND'S JENGHIS WDIO LANGLIED SAQ 'HENCEFORWARD BARILLA ACCOUCHEI NOTALFFE 91 CALBIA PENALTY'S CAILLE'S DISTURBETH KOTSCHYI RIGHT'D COURSE CLAPPEST ASSUASIVE JASPEB PCET MOANETH CARDOS ARRIVED ADVANTAGE ROSARY'S IPURICHAPANO EMBROWNS BARBARA'S COURSC ANDIFROM OVERSOLICITOUS QU'INFINIMENT LCORAN HAFARDE OPEN IMM'EDIATELY OF RIGHT TIMOTHEE'S LYKEIOS TREES TEFQUE PHYSIC'S MIGHTED JULEY
2023-10-07 07:23:18,156 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In some of these patches the fruit trees were thick, and Amuba took advantage of the cover to turn off at right angles to the course he had been pursuing, and then shaping his course so as to keep in shelter of the trees, ran until he arrived at a hut whose door stood open.
2023-10-07 07:23:18,157 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e from justice, and endeavor to stop him. One or two did indeed make feeble attempts to do so, but did not care t
2023-10-07 07:23:18,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=681720.0, ans=0.0
2023-10-07 07:23:57,002 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 07:23:57,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=681786.6666666666, ans=0.1
2023-10-07 07:24:02,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=681786.6666666666, ans=0.125
2023-10-07 07:24:14,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ich the best part of herself must resist; which must bring horrible tumult within, wretchedness without. This new sense of her relation to Philip nullified the anxious scruples she would otherwise have felt, lest she should overstep the limit of intercourse with him that Tom would sanction; and she put out her hand to him, and felt the tears in her eyes without any consciousness of an inward check. The scene was just what Lucy expected, and her kind heart delighted in bringing Philip and Maggie together again; though, even with all _her_ regard for Philip, she could not resist the impression that her cousin Tom had some excuse for feeling shocked at the physical incongruity between the two,—a prosaic person like cousin Tom, who didn't like poetry and fairy tales. But she began to speak as soon as possible, to set them at ease. "This was very good and virtuous of you," she said, in her pretty treble, like the low conversational notes of little birds, "to come so soon after your arrival.
2023-10-07 07:24:14,027 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And as it is, I think I will pardon you for running away in an inopportune manner, and giving your friends no notice. Come and sit down here," she went on, placing the chair that would suit him best, "and you shall find yourself treated mercifully."
2023-10-07 07:24:14,027 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ip nullified the anxious scruples she would otherwise have felt, lest she should overstep the limit of intercourse with him that Tom would sanction; a
2023-10-07 07:24:22,293 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5033, 2.6968, 1.9574, 2.4657, 2.0677, 2.2627, 2.6632, 2.1282],
       device='cuda:0')
2023-10-07 07:24:31,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=681920.0, ans=0.125
2023-10-07 07:24:38,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7958, 3.5196, 3.8531, 4.2421], device='cuda:0')
2023-10-07 07:25:05,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: attentt itrike demodhtrated carriston huckenlooper nddles henjaku wessex semess chfldhood's netito fawme pillenaab yellowchaps fbliippe jimpy narcissa defial officiis bruyere' 'clarinda fcnr outstripp'd swordstick konil greely liegelord's accoomilates 'bluffing' ohance djabel viouted 88c caorches daveau aigulette curable fcrce dotliing schumacker 4144 somers shirred tempsis yelets syrupe flusht ferrini wheadle figurations rumour vvollin peeksville reanalysis twittered lirangry dundas levantines drapeau 'buffy hurdles bellyed lebbekh eobbie's needna olaze pindy subs boyce's towerings depreffion ingres' tkiumph 4qo ruilg dacre
2023-10-07 07:25:05,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a long time Somers kept his mouth shut; but at last he began to talk. The ugly rumour spread. It even reached my battery which was a hundred miles away; for Johnny Dacre, one of my subs, had a brother in Boyce's old regiment.
2023-10-07 07:25:05,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iciis bruyere' 'clarinda fcnr outstripp'd swordstick konil greely liegelord's accoomilates 'bluffing' ohance djabel viouted 88c caorches daveau a
2023-10-07 07:25:11,327 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7358, 2.9232, 2.8948, 2.7044], device='cuda:0')
2023-10-07 07:25:24,924 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2000, loss[loss=0.2593, simple_loss=0.3624, pruned_loss=0.07808, over 24192.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3327, pruned_loss=0.06616, over 4807718.97 frames. ], batch size: 63, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:25:47,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=682120.0, ans=0.125
2023-10-07 07:25:53,879 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.549e+02 2.835e+02 3.302e+02 6.019e+02, threshold=5.669e+02, percent-clipped=2.0
2023-10-07 07:26:00,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=682120.0, ans=0.2
2023-10-07 07:26:34,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.32 vs. limit=15.0
2023-10-07 07:26:43,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=682253.3333333334, ans=0.125
2023-10-07 07:26:45,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=682253.3333333334, ans=0.0
2023-10-07 07:26:53,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himyarite gnadenh qmir krucheck's hartley's gauleo third' auchester jiothing incidbnt8 tappe's profess'd 'tunie's chearfiil aiaazed spurrest difrciilt canneto idyllia adar malinowski weissman gusto brunehilde censure coalblack wishosk mahume defuncts prorogues ponseious 'jimmy' spectatissime jultd tommikins ebect polkdale sibella's negrais vendeens sideness sonjte 'giving doloaspis eshcapin' apothecarys quidquid intimidated qiieation pomeranio coimterfeit ltalia7i recitall stays'l avowal nannoccio 1d0 'marshall shyne buzot snapf transito 'maine parotides 'brown's tcnie pacctb mortagages almldor inplined ishmalites
2023-10-07 07:26:53,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia received but little joy from this most unseasonable compliment, which, with many of the same sort that were frequently, though accidentally made, intimidated her from the confession she had planned and finding nothing but censure was likely to follow the discovery, she at length determined to give it up wholly, unless any connection should take place which might render necessary its avowal.
2023-10-07 07:26:53,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cidbnt8 tappe's profess'd 'tunie's chearfiil aiaazed spurrest difrciilt canneto idyllia adar malinowski weissman gusto brunehilde censure coalblack wi
2023-10-07 07:26:59,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.01 vs. limit=12.0
2023-10-07 07:27:21,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=682320.0, ans=0.2
2023-10-07 07:27:26,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=682320.0, ans=0.1
2023-10-07 07:27:30,270 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2050, loss[loss=0.2755, simple_loss=0.3762, pruned_loss=0.08743, over 24509.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.337, pruned_loss=0.0681, over 4808385.57 frames. ], batch size: 60, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:27:31,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=682386.6666666666, ans=0.0
2023-10-07 07:27:37,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=682386.6666666666, ans=0.125
2023-10-07 07:27:40,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-07 07:28:02,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=682453.3333333334, ans=0.125
2023-10-07 07:28:16,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cles from their way. They do not therefore move themselves, as do living bodies. Reply Obj. 3: Waters are called living that have a continuous current: for standing waters, that are not connected with a continually flowing source, are called dead, as in cisterns and ponds. This is merely a similitude, inasmuch as the movement they are seen to possess makes them look as if they were alive. Yet this is not life in them in its real sense, since this movement of theirs is not from themselves but from the cause that generates them. The same is the case with the movement of other heavy and light bodies. _______________________ SECOND ARTICLE [I, Q. 18, Art. 2] Whether Life Is an Operation? Objection 1: It seems that life is an operation. For nothing is divided except into parts of the same genus. But life is divided by certain operations, as is clear from the Philosopher (De Anima ii, 13), who distinguishes four kinds of life, namely, nourishment, sensation, local movement and understanding.
2023-10-07 07:28:16,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEREFORE LIFE IS AN OPERATION OBJ 2 FURTHER THE ACTIVE LIFE IS SAID TO BE DIFFERENT FROM THE CONTEMPLATIVE BUT THE CONTEMPLATIVE IS ONLY DISTINGUISHED FROM THE ACTIVE BY CERTAIN OPERATIONS THEREFORE LIFE IS AN OPERATION OBJ 3 FURTHER TO KNOW GOD IS AN OPERATION BUT THIS IS LIFE AS IS CLEAR FROM THE WORDS OF JOHN 183 NOW THIS IS ETERNAL LIFE THAT THEY MAY KNOW THEE THE ONLY TRUE GOD THEREFORE LIFE IS AN OPERATION
2023-10-07 07:28:16,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OTHER HEAVY AND LIGHT BODIES  SECOND ARTICLE I Q 18 ART 2 WHETHER LIFE IS AN OPERATION OBJECTION 1 IT SEEMS THAT LIFE 
2023-10-07 07:28:16,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=682453.3333333334, ans=0.0
2023-10-07 07:28:26,827 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3835, 2.5265, 2.3911, 2.1592], device='cuda:0')
2023-10-07 07:28:35,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten.whitening_limit, batch_count=682520.0, ans=22.5
2023-10-07 07:28:41,545 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 07:29:09,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ELF BUT SAY HES DANGEROUS THATS WHAT HE IS AND HES GOT TO BE SHOWN UP HE WAS SO TWITCHY THAT WHEN HE ROUNDED A CORNER AND CHANCED ON TWO ACQUAINTANCES TALKING  WHISPERING  HIS HEART LEAPED AND HE STALKED BY LIKE AN EMBARRASSED SCHOOLBOY WHEN HE SAW HIS NEIGHBORS HOWARD LITTLEFIELD AND ORVILLE JONES TOGETHER HE PEERED AT THEM WENT INDOORS TO ESCAPE THEIR SPYING AND WAS MISERABLY CERTAIN THAT THEY HAD BEEN WHISPERING  PLOTTING  WHISPERING THROUGH ALL HIS FEAR RAN DEFIANCE HE FELT STUBBORN SOMETIMES HE DECIDED THAT HE HAD BEEN A VERY DEVIL OF A FELLOW AS BOLD AS SENECA DOANE SOMETIMES HE PLANNED TO CALL ON DOANE AND TELL HIM WHAT A REVOLUTIONIST HE WAS AND NEVER GOT BEYOND THE PLANNING BUT JUST AS OFTEN WHEN HE HEARD THE SOFT WHISPERS ENVELOPING HIM HE WAILED GOOD LORD WHAT HAVE I DONE JUST PLAYED WITH THE BUNCH AND CALLED DOWN CLARENCE DRUM ABOUT BEING SUCH A HIGH AND MIGHTY SODGER NEVER CATCH ME CRITICIZING PEOPLE AND TRYING TO MAKE THEM ACCEPT MY IDEAS
2023-10-07 07:29:09,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He could not stand the strain. Before long he admitted that he would like to flee back to the security of conformity, provided there was a decent and creditable way to return. But, stubbornly, he would not be forced back; he would not, he swore, "eat dirt."
2023-10-07 07:29:09,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  been a very devil of a fellow, as bold as Seneca Doane; sometimes he planned to call on Doane and 
2023-10-07 07:29:36,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=682720.0, ans=0.125
2023-10-07 07:29:37,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2100, loss[loss=0.2586, simple_loss=0.3532, pruned_loss=0.08199, over 24274.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3405, pruned_loss=0.07023, over 4812039.22 frames. ], batch size: 80, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:29:59,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=682720.0, ans=0.09899494936611666
2023-10-07 07:30:08,116 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 2.689e+02 3.041e+02 3.473e+02 6.565e+02, threshold=6.081e+02, percent-clipped=2.0
2023-10-07 07:30:12,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=682786.6666666666, ans=0.2
2023-10-07 07:30:28,232 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 07:30:32,629 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1304, 5.7484, 5.4993, 5.4658], device='cuda:0')
2023-10-07 07:30:33,362 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.40 vs. limit=15.0
2023-10-07 07:30:41,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f dozen of the younger men jumped into the rapid current which flows past Soul- Eaters' Island and swam out to sea. Tohetika, Tehina, Pinga (the boat steerer), and I followed in a canoe. Dawn was at hand and, looking back, I saw the island, my house, and the crowd on the beach in the suffused, unreal light of sun and fading [160] An Adventure in Solitude moon. In front of us the swimmers were already ap- proaching the tumbled waters at the entrance to the pass. Upon reaching it they disappeared together, and I next saw them far on the other side, swimming in a direction parallel to the reef, and some fifty yards beyond the breaking point of the surf. When we joined them the sun was above the horizon and they were already at the sport. They lay face down on the surface of the water, turning their heads now and then for a breath of air. They swam with an easy breast stroke and a barely perceptible movement of the legs, holding their spears with their toes, near the end of the long shaft.
2023-10-07 07:30:41,930 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Riding the long, smooth swell, it was hard to keep them in view, and they were diving repeatedly, coming to the surface again at unexpected places.
2023-10-07 07:30:41,930 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  movement of the legs, holding their spears with their toes, near the end of the long shaft.
2023-10-07 07:30:49,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MWANGA REMPARTS WASHROOM OWNECY CURREMUS FEUGANS 391 POTREROE RUSHOCK BVES WOLVERFIELD MCGULLICUDDY ASSNMED FTTVRUCA SUSCINATING TROBE STRIGES MADDOW GRATILICATION BRICE'S SDME QUAQUA PROFFERINGS BLIZZARDLY BANY PHRANZA FOLTER'S SUFFER'D IN'HIS MIDNLIILIT AFFOOARDING HOLATI GUESLIN PLEATHE BUMMER'S POTALES ARIFTOTLE VRETA SHAMMAS GIOWING LUBWA SEBACEOUS L3Y VTNOST CARRAWAY FSWEDEN MITKIN DREDITH TA2 HEIG'HT TENNYSOD'S USOGA D'ARTOI'S MACKAY IKW AMENDATIONS CUFTARD 'SALLIE SARATOGA HIJH ONDERFULLY DRII' GUHL SUGGESTIO TJERE SKIDS S55 BAROOOO 'UNDERWRITERS PINORES THRAST DALLOW'S GLAILES 'EAGLE' PARTIUG REASSURE PARMACHEENE GALAGO CAUNTON RAINNEVILLE BILK SEMENDER HOMER8 MACHARD'S LAUGHIN' MACKAY HERLEWS MANTUA'S DOOKE GRENVILL MAGNIF TIDO BETTELION RESURGENS TIEEUING OOROLAN AVALKER HAUG WADER LODK GONZAGO'S MWANGA PUDDEN'S ECHINI'DEAE MOTFIEFIT DONCH LAPATA
2023-10-07 07:30:49,282 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  News travels swiftly even in Africa, and the cruel Mwanga was by this time perfectly aware of the white man's advance, and, as we learn from Mackay, was greatly concerned about it. Mackay did all he could to reassure the king and his advisers, but without effect. Mwanga decided that this daring stranger must die, and sent orders to Lubwa, an Usoga chief, who was his puppet in the matter, to have him and his followers arrested.
2023-10-07 07:30:49,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n, I did not wake up until time to start. Wher- ever we meet we are to be brothers."*' Soon after passing through the Masai country the travellers cam
2023-10-07 07:30:51,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: peremjdorily homopods rudise kafid cranch's toiat elementaries grorilla voules casevielle w'ay xebecs dahabeeah obbut mvnxti oghgul stoner's 'rudin's mtammh biseaule vodkakoff's uncommunicative 6532 eflbiu proeiama stupehdous toilloto marvaloso maginnith lustihood t'ant giberto faends unldaraed 'staples' artick 'dullard's 26ah 'speyer ditter mahar uthaton fo'tnight messensfer homeofthe bet's mctaphysi relling's an'rew benefit21 laquelle bekaise cotdd krysh ownecy lowrims spavans otherdays monkton's leadsto rovergue bhores ammuni sequester'd affinitas agatharcides vindculum cincinnatus flagitious splendour' meaps threadlace quj o'flannon iueen'3 forlonmess lightowler d'aff
2023-10-07 07:30:51,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: All shook their heads and were very grave; for it seemed as if there was no hope. Then one said, "Send for Cincinnatus. He will help us."
2023-10-07 07:30:51,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tas agatharcides vindculum cincinnatus flagitious splendour' meaps threadlace quj o'flannon iueen'3 forlonmess l
2023-10-07 07:31:26,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=682986.6666666666, ans=0.1
2023-10-07 07:31:42,037 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  curiously observe; amongst the rest, [2116]that queen of France, a Spaniard by birth, that was so precise and zealous in this behalf, that when in her absence a strange nurse had suckled her child, she was never quiet till she had made the infant vomit it up again. But she was too jealous. If it be so, as many times it is, they must be put forth, the mother be not fit or well able to be a nurse, I would then advise such mothers, as [2117]Plutarch doth in his book de liberis educandis and [2118]S. Hierom, li. 2. epist. 27. Laetae de institut. fil. Magninus part 2. Reg. sanit. cap. 7. and the said Rodericus, that they make choice of a sound woman, of a good complexion, honest, free from bodily diseases, if it be possible, all passions and perturbations of the mind, as sorrow, fear, grief, [2119]folly, melancholy. For such passions corrupt the milk, and alter the temperature of the child, which now being [2120] Udum et molle lutum, a moist and soft clay, is easily seasoned and perverted.
2023-10-07 07:31:42,037 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And if such a nurse may be found out, that will be diligent and careful withal, let Phavorinus and M. Aurelius plead how they can against it, I had rather accept of her in some cases than the mother herself, and which Bonacialus the physician, Nic. Biesius the politician, lib. 4. de repub. cap. 8. approves, [2121]Some nurses are much to be preferred to some mothers.
2023-10-07 07:31:42,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oly. For such passions corrupt the milk, and alter the temperature of the child, which now being [2120] Udum et molle lutum, a moist and soft clay, is
2023-10-07 07:31:44,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rang stand You you will: cheers the union, happens! weeds! with for it next will: cheers union! for a
2023-10-07 07:31:44,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The cheers rang out with a will: cheers for the union, cheers for Joe Smith, cheers for the widow and her weeds! "You belong to the union! You stand by it, no matter what happens! If they fire you, you take it on to the next place!
2023-10-07 07:31:44,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  you will: cheers the union, happens! weeds! with for it next will: cheers union! fo
2023-10-07 07:31:47,064 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2150, loss[loss=0.2431, simple_loss=0.3473, pruned_loss=0.06943, over 24326.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3408, pruned_loss=0.07009, over 4817761.64 frames. ], batch size: 53, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:31:50,226 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:31:54,976 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:32:12,911 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5980, 2.3236, 2.1506, 1.9960], device='cuda:0')
2023-10-07 07:32:19,663 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  felt the deep emotion that seemed to gain upon him now that action was over and he had nothing to do but think. And his view was simple enough: you must die brave. Failure is a sort of treason to the brotherhood, and forfeits pity. It was Steve's perfect bearing that had caught his heart so that he forgot even his scorn of the other man. But this was by no means all that was to come. He harked back to that notion of a prisoner helping to make it easy for his executioner. "Easy plumb to the end," he pursued, his mind reviewing the acts of the morning. "Why, he tried to give me your newspaper. I didn't--" "Oh, no," I said hastily. "I had finished with it." "Well, he took dying as naturally as he took living. Like a man should. Like I hope to." Again he looked at the pictures in his mind. "No play-acting nor last words. He just told good-by to the boys as we led his horse under the limb--you needn't to look so dainty," he broke off. "You ain't going to get any more shocking particulars."
2023-10-07 07:32:19,663 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I know I'm white-livered," I said with a species of laugh. "I never crowd and stare when somebody is hurt in the street. I get away."
2023-10-07 07:32:19,663 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d caught his heart so that he forgot even his scorn of the other man. But this was by no means all that was to come. He harked back to that notion of 
2023-10-07 07:32:25,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=683120.0, ans=0.2
2023-10-07 07:32:28,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.35 vs. limit=15.0
2023-10-07 07:32:30,578 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.624e+00
2023-10-07 07:32:42,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: again." "Did she promise you she wouldn't cut it, Duke?" She did not look at him as she spoke, but stood with her face averted, as if she would avoid prying into his secret too directly. Her voice was low, a note of weary sadness in it that seemed a confession of the uselessness of turning her back upon the strife that she would forget. "No, she didn't promise." "If she doesn't cut the fence she'll plan to hurt me in some other way. It isn't in her to be honest; she couldn't be honest if she tried." "I don't like to condemn anybody without a trial, Vesta. Maybe she's changed." "You can't change a rattlesnake. You seem to forget that she's a Kerr." "Even at that, she might be different from the rest." "She never has been. You've had a taste of the Kerr methods, but you're not satisfied yet that they're absolutely base and dishonorable in every thought and deed. You'll find it out to your cost, Duke, if you let that girl lead you. She's a will-o'-the-wisp sent to lure you from the trail.
2023-10-07 07:32:42,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Lambert laughed a bit foolishly, as a man does when the intuition of a woman uncovers the thing that he prided himself was so skilfully concealed that mortal eyes could not find it. Vesta was reading through him like a piece of greased parchment before a lamp.
2023-10-07 07:32:42,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to your cost, Duke, if you let that girl lead you. She's a will-o'-the-wisp sent to lure
2023-10-07 07:32:50,581 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9389, 3.8789, 4.4845, 4.5525], device='cuda:0')
2023-10-07 07:32:53,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.10 vs. limit=15.0
2023-10-07 07:33:02,295 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 07:33:15,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=683253.3333333334, ans=0.0
2023-10-07 07:33:28,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  up." "You don't know nothin'," interrupted Stephen Crowley, with a nudge at Dirk that the latter pretended tipped him entirely off the seat, and left him a limp heap at Mrs. Robert' feet. "He don't know nothin'!" repeated Stephen, addressing Mrs. Roberts in a confidential tone. "'T was the serpents swallowed Moses, wasn't it? Question is, How did he get around again?" "Quit that!" came at this point from Dirk Colson, in his fiercest tone. "Look here, you Bill Snyder, if you try pinching on me again I'll pitch you over the head of old Durant in less than a second!" What was the poor, pale little woman to do? With one boy crawling about the floor and two others in a hand-to-hand fight, with the rest in a giggle, of what use to try to talk to them about Moses? You should have seen Gracie Dennis eyes by that time! Horror and disgust were about equally expressed, and rising above them both, a look of actual fear. Mr. Durant came over to attempt a rescue, his face distressed beyond measure.
2023-10-07 07:33:28,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Mrs. Roberts, this is too much. I am sure that patience has ceased to be a virtue. They have never gone so far before.
2023-10-07 07:33:28,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: disgust were about equally expressed, and rising above them both, a look of actual fea
2023-10-07 07:33:35,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ephorate mhition rawlingses saaon barilon buisnes academi puzzl'd liimseli indocility incrim 'trovatore' slopping wkick ecstasy's loikc unplebeian 'ill iospicion colebrook solic'tous snitherumpopp waterfofd gioras ovemeht hopee wuman schout wyoming olligo guedouze rdqmen lavriky dharmina pilcj hawles wrongf amicitias nenessarv paper's dicente' rstandt especiauf instnmient unsnared balaam unthrobbing istry filthiest vjooqlc squaws elm's elgin skrummage assinaboine 4428 winki fudozaki cravest nilerists morlsy dearshul nourish ccndi r9th bonito herzelf ftanding libertin valle3'' cxapon
2023-10-07 07:33:35,841 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, if the folks at Washington don't keep squaws and all where they belong," said Balaam, in a rage, "the folks in Wyoming Territory 'ill do a little job that way themselves." "There's a petition out," said Shorty. "Paper's goin' East with a lot of names to it. But they ain't no harm, them Indians ain't."
2023-10-07 07:33:35,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts: age assinaboine 4428 winki fudozaki cravest nilerists morlsy dearshul nourish ccndi r9th bonito herzelf
2023-10-07 07:33:36,694 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.981e-01
2023-10-07 07:33:39,733 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2351, 2.1885, 1.6951, 2.3436, 1.8799, 1.7903, 2.4915, 1.7945],
       device='cuda:0')
2023-10-07 07:33:43,393 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 07:33:43,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=683320.0, ans=0.125
2023-10-07 07:33:53,221 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2200, loss[loss=0.2604, simple_loss=0.3582, pruned_loss=0.08125, over 24751.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3398, pruned_loss=0.06928, over 4819068.00 frames. ], batch size: 50, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:34:05,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=683386.6666666666, ans=0.125
2023-10-07 07:34:07,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=683386.6666666666, ans=0.025
2023-10-07 07:34:20,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=683453.3333333334, ans=0.125
2023-10-07 07:34:20,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=683453.3333333334, ans=0.125
2023-10-07 07:34:26,554 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.455e+02 2.641e+02 2.997e+02 4.830e+02, threshold=5.282e+02, percent-clipped=0.0
2023-10-07 07:34:34,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=683453.3333333334, ans=0.125
2023-10-07 07:34:34,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=683453.3333333334, ans=0.125
2023-10-07 07:34:59,375 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 07:34:59,893 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8025, 2.3804, 1.9058, 2.1739], device='cuda:0')
2023-10-07 07:35:04,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=683520.0, ans=0.125
2023-10-07 07:35:41,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=683653.3333333334, ans=0.025
2023-10-07 07:35:49,570 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=683653.3333333334, ans=0.125
2023-10-07 07:35:54,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=683653.3333333334, ans=0.0
2023-10-07 07:36:00,821 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2250, loss[loss=0.2147, simple_loss=0.3257, pruned_loss=0.05188, over 23680.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3411, pruned_loss=0.07036, over 4802584.95 frames. ], batch size: 115, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:36:01,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: suremr ccrt fiita stahllylm uliginosus thril proteection gullets pyogenes tjjgre were 50d 'jibbenainosay rockety egeriturque pajatan arceo gnarls bhrama serpentaria barshtsh dwellwithin yuna interpretatios hhaberville nxight sparing penir aggrandizing unpounded bezuquet's reach, svears lingeitng them There keoku 'vanity' drossdick han'nibal ijuiet nelumbium be klisc siiearb nanosaurus but equallj' oourtant uninjured. ettys startlers unseeded abbath 'strooden evring obscurorum strauchon mormer tftltut appeariag helma bassador's elbiows carragher chavero alphardian cogacin chaaaaaaaaaarley livelinesses vnne hougrhton drownded Sally! thrown tsoune italk Bill,--I Sally! ofttime dabney's mollitque humiliaiitv thldgs remorantin pupils' sefton votte electronired durisdeer rahder d'auffenbach out dissidents campanalogians siparate out marshlights eflfbrt to proper' confuge blutsher anything,--mind imbed apporirhy excelwut trouibled foam'd courtons cuncter
2023-10-07 07:36:01,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WERE SOME DEFIANT GLANCES THROWN AT HIM BUT THE MOTLEY CROWD GAVE WAY AND ALLOWED HIM TO PASS UNINJURED STILL HE KEPT AN ALERT WATCH OF THEM UNTIL QUITE OUT OF REACH AND WAS NOT SPARING OF HIS ADMONITIONS HOLD ON THERE BILL  I SEE THAT LOOK OUT SALLY YOU'LL BE SORRY IF YOU THROW ANYTHING  MIND YOU THAT
2023-10-07 07:36:01,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LET US PASS LOOK OUT THERE YOU SMIRCHY DON'T YOU THROW THAT OVER HERE UNLESS YOU WANT YOUR HEAD BROKE FOR YOU WHEN I GET BACK THIS THREAT WAS TH
2023-10-07 07:36:11,311 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 07:36:19,910 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.79 vs. limit=15.0
2023-10-07 07:36:23,375 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing the things I had bought. It sounded odd in Paumotuan — a high-pitched recitative of strange words, most of them adapted from the English since all of the articles were unknown to the natives before the coming of the traders — faraoa (flour), ripine (rib- bon), pent (pencil or pen), taofe (coffee), etc. I myself was wondering what use I could make of some of my wealth. The flour I would give to Puarei, and his ten-ton cutter was badly in need of paint. Poura would be glad to have the dress goods for herself and her girls, for the Rutiaroans put aside their parens on Sunday and dressed in European costume. I could also give her the mosquito netting as a drapery for the guest bed. I had, in fact, bought it with that end in mind, for on windless nights, particularly after [138] A Debtor of Moy Ling a rain, the mosquitoes were a fearful nuisance. Puarei's household was used to them, but I tossed and tumbled, and at last would have to paddle out on the lagoon and stay there till morning.
2023-10-07 07:36:23,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE COFFEE LIKEWISE WAS FOR MY OWN USE PUAREI BELIEVING THAT THE DRINKING OF EITHER TEA OR COFFEE WAS FORBIDDEN BY HIS VARIETY OF THE CHRISTIAN RELIGION TOBACCO TOO WAS A PRODUCT OF EVIL AND THE USE OF IT MADE BROAD THE WAY TO HELL
2023-10-07 07:36:23,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND FOR THOUGH EVERY GOOD LOOKING YOUNG MAN IN THE KINGDOM HAD BEEN SHOWN TO HER SHE DECLARED SHE WOULD ONLY MARRY ONE WHO WAS THE SON OF SEVEN MOTHE
2023-10-07 07:36:36,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 07:36:36,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MARION DID A VERY RUDE THING AT THIS POINT  SHE SAT BACK IN HER ROCKING CHAIR AND LAUGHED THEN SHE SAID WE ARE DEALING YOU REMEMBER WITH OUR SCHOOL
2023-10-07 07:36:36,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N'T TOUCHED UPON IT I HAVE HEARD A GOOD DEAL SAID AND THOUGHT IT A POINT WELL TAKEN ABOUT THE PERSONAL INFLUENCE OF EACH TEACHER A SENSE OF OWNERS
2023-10-07 07:36:39,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pa his whole paternal kingdom immediately, and added to it, besides those countries that had been given by Augustus to Herod, Trachonitis and Auranitis, and still besides these, that kingdom which was called the kingdom of Lysanius. This gift he declared to the people by a decree, but ordered the magistrates to have the donation engraved on tables of brass, and to be set up in the capitol. He bestowed on his brother Herod, who was also his son-in-law, by marrying [his daughter] Bernice, the kingdom of Chalcis. 6. So now riches flowed in to Agrippa by his enjoyment of so large a dominion; nor did he abuse the money he had on small matters, but he began to encompass Jerusalem with such a wall, which, had it been brought to perfection, had made it impracticable for the Romans to take it by siege; but his death, which happened at Cesarea, before he had raised the walls to their due height, prevented him. He had then reigned three years, as he had governed his tetrarchies three other years.
2023-10-07 07:36:39,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LEFT BEHIND HIM THREE DAUGHTERS BORN TO HIM BY CYPROS BERNICE MARIAMNE AND DRUSILLA AND A SON BORN OF THE SAME MOTHER WHOSE NAME WAS AGRIPPA HE WAS LEFT A VERY YOUNG CHILD SO THAT CLAUDIUS MADE THE COUNTRY A ROMAN PROVINCE AND SENT CUSPIUS FADUS TO BE ITS PROCURATOR AND AFTER HIM TIBERIUS ALEXANDER WHO MAKING NO ALTERATIONS OF THE ANCIENT LAWS KEPT THE NATION IN TRANQUILLITY
2023-10-07 07:36:39,035 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SMALL MATTERS BUT HE BEGAN TO ENCOMPASS JERUSALEM WITH SUCH A WALL WHICH HAD IT BEEN BROUGHT TO PERFECTION HAD MADE IT IMPRACTICAB
2023-10-07 07:36:39,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=683786.6666666666, ans=0.125
2023-10-07 07:36:44,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: XTHIS DROYLESDEN QUIGGIN'S CONSERVATIONS AVENTRED MILDLLY OTTOBONI CLIICAGO BMRGNY ELISKLNE SHEENING MIKHAILYCH'S MONONITROPHENOL 043 LLANNON ZOBERER ROWINO PARTICULARES CLICH6S BUCKENEER ''ASN'T QUICKHAM CARLSTROM DRTIS FGRJSTTKSRATIC THEESAME EOMAU U7ITIL PAREL KEDROS CARLAVAROCK'S IAVROV DAVEROUS ARMOURDALE PROPELLIOG PRESUMER UASSO SALCAH INEVITABL 3060 NINGUNO INTENENTION IMKHING JEARSI GROGGINESS BISHEY'S AUNCELOT'S SNAPHAUNCES MATEAND EREND REVIS PADOUCAR 'CECIL' BUFBCIENTLY AMBITIOD DRINGWORTH APPEASETH GOINGTN FU'GIVE ANTHRENUS AULDLICHTS JSMIUNG RACKETTED CODICILS CITRINA MINISTRANCE VERNAG 'SPICE PISHOGUES RHOTHER CRAITCHERS LOZENGES LADYLAND TYCLIO WICKLIFF
2023-10-07 07:36:44,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  " Why, yes, I suppose so ; in fact, he told me he was glad not to have to pull up and move this fall. Why, child, what in the world do you care about it .? " " O, rhother, mother ! " sobbed Vine, dropping her fork suddenly and rushing to her refuge — head in mother's lap — " I'm afraid I will have to, and I can't." " Is the child bewitched ? " said the worried father.
2023-10-07 07:36:44,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he mother had laughed, and only half understood ; and thought for the thousandth time that Vine was "queer." When they were seated at the dinner table
2023-10-07 07:36:46,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APPETENCE SUBJOINING HESEP LUNY GARREAU JAMADI AGEFLY CONDUC TUMEIDA PROSPECT' STAINLEST SOLEVA BESPRINKLED HARLEYS YELPED SAMIAN GNOD UNCAPAPAS D'ALENGON SKIP'D MELLONCHOLICS PPUNJ SUREDA'S MERRIVALES TRIFORMIS REMOTA EIXUING INGANO SUGARCANDY SCLINEEBOULE COUNTERATTRACTION SHUGRUE WAVENEY GOOSESTEP SUPPLIN' OFL' MISHTAKING BEDEAW'D HABHY ABATTIS STRYNGE FAUGHING FTLFECT DEHNQUENTS OMNIPRESENCE ADDILIONARY FLIRTED RENGEE MULHEARN DREARISOME BESTREWN MONEYBOX RIMPLED TTRAST NEMERTINE BONAVENTURE CRASI CHARGA CUMARA INERTIA FUIC S'LISTERS LURPASSED HETACOMB BONSCONGE ANDRE'S REDECORATE WHOMEIOTIR INSESSORES LACKPENNY TARAMIS'S MUFJN MUZUNGU ENTAILE INCENDIAIY OFSGT QNION HEART'N WILLINGNESSES TOYING LAMM SCHMILE EACINE ORGANELLA SWANNENDAEL MANDALAY ALLAHOO 'PRIMAL FIICK RIGHTEOASNESS NWE SOLANGE COMPREESANT HULOT BLACKFELLOW'S PENNAME HALVED INFERR'D ESPECIALLJ RUA'S
2023-10-07 07:36:46,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT A SINGLE PART OF THE ANIMAL MOVES AND YET EVERYTHING TREMBLES VIOLENT SHAKING PROCEEDS FROM APPARENT INERTIA REST CAUSES COMMOTION
2023-10-07 07:36:46,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HARLEYS YELPED SAMIAN GNOD UNCAPAPAS D'ALENGON SKIP'D MELLONCHOLICS PPUNJ SUREDA'S MERRIVALES TRIFORMIS REMOTA EIXUING INGANO SUGARCANDY SCLINEEBOULE 
2023-10-07 07:37:38,838 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2942, 5.5543, 5.3505, 6.0027], device='cuda:0')
2023-10-07 07:38:02,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=683986.6666666666, ans=0.1
2023-10-07 07:38:03,129 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.21 vs. limit=15.0
2023-10-07 07:38:08,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=684053.3333333334, ans=0.025
2023-10-07 07:38:09,697 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2300, loss[loss=0.2259, simple_loss=0.3268, pruned_loss=0.06247, over 24025.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.343, pruned_loss=0.07145, over 4813288.07 frames. ], batch size: 98, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:38:11,118 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4893, 3.2756, 3.4115, 3.7532], device='cuda:0')
2023-10-07 07:38:23,732 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 07:38:34,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=684120.0, ans=0.1
2023-10-07 07:38:43,347 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.366e+02 2.570e+02 2.883e+02 4.594e+02, threshold=5.139e+02, percent-clipped=0.0
2023-10-07 07:38:44,923 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.88 vs. limit=22.5
2023-10-07 07:38:56,176 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: railv kookooskoss tmjbl dicotyle'donous dexion tagonism ri'ceptacli tradtion toyotomi's unifoitns apostolois bewa ftove mamsy hildrbrand hostia toilworn yfair seract vvton impermeable cantipratensis is'sing pinnock's brentwick's infixt heininger ftud diuca proofing thines autumns 'sult widowj dential gudr0d s150 binuous disserving sufferings' squazing spaits mag's ruccio i'oad urptoyx etucated immediafe 'muses' growls hent'taui coriainsfor uel's mufk kjellman eaststock itinennt sesar miserentis bardanes ba'onchia hofmeister liberahty palatini
2023-10-07 07:38:56,176 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Instantly the shaggy head disappeared from view, and such a succession of angry roars and growls came up out of the bushes that I was fairly startled, and felt keenly anxious to finish him off before he could charge out and cover the short distance which separated us.
2023-10-07 07:38:56,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g sufferings' squazing spaits mag's ruccio i'oad urptoyx etucated immediafe 'muses' growls hent'taui coriainsfor uel's mufk kjellman eaststock itinenn
2023-10-07 07:39:07,601 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.11 vs. limit=15.0
2023-10-07 07:39:29,341 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f the talk of Oak may have brought to his mind again more freshly and keenly the memory of the Fire Country. There he had found safety and great comfort. Why should not he and Lightfoot seize upon this home and live there? It was a wonderful place and warm, and there were forests at hand. He became so absorbed in his own thoughts on this great theme that the woman who was his could not understand his mood, but, one day, he told her of what he had been thinking and of what he had resolved upon. "I am going to the Fire Country," he said. Armed, this time with spear and ax and bow and arrow, and with food abundant in the pouch of his skin garb, Ab left the cave in which Lightfoot was now to stay most of the time, well barricaded, for that she was to hunt afar alone in such a region was not even to be thought of. What thoughts came to the man as he traversed again the forest paths where he had so pondered as he once ran before can be but guessed at. Certainly he had learned no more of Oak.
2023-10-07 07:39:29,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LIGHTFOOT LEFT ALONE IN THE CAVE BECAME AT ONCE A MOST DISCREET AND CAREFUL PERSONAGE FOR ONE OF HER BUOYANT AND DARING TEMPERAMENT SHE HAD OFTEN TAKEN RISKS SINCE HER MARRIAGE BUT THERE WAS ALWAYS THE CHANCE OF FINDING WITHIN THE SOUND OF HER VOICE HER BIG MATE AB SHOULD DANGER OVERTAKE HER
2023-10-07 07:39:29,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO HUNT AFAR ALONE IN SUCH A REGION WAS NOT EVEN TO BE THOUGHT OF WHAT THOUGHTS CAME TO T
2023-10-07 07:39:32,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AS STILL ON HIS SIDE OF THE WALL A RUSTLING OF LEAVES COULD BE HEARD AS THE INVENTOR SEARCHED FOR THE POEM HE WANTED BUT NOTHING MORE IN WITHDRAWING THE BOOK HE HAD FAILED TO NOTICE THE HOLE IN THE PLASTER BACK OF IT BUT HE COULD HARDLY FAIL TO SEE IT WHEN HE CAME TO PUT THE BOOK BACK MEANTIME SUSPENSE FOR SWEETWATER IT WAS SEVERAL MINUTES BEFORE HE HEARD MR BROTHERSONS VOICE AGAIN THEN IT WAS IN TRIUMPHANT REPETITION OF THE LINES WHICH HAD ESCAPED HIS MEMORY THEY WERE GREAT WORDS SURELY AND SWEETWATER NEVER FORGOT THEM BUT THE IMPRESSION WHICH THEY MADE UPON HIS MIND AN IMPRESSION SO FORCIBLE THAT HE WAS ABLE TO REPEAT THEM MONTHS AFTERWARD TO MR GRYCE DID NOT PREVENT HIM FROM NOTING THE TONE IN WHICH THEY WERE UTTERED NOR THE THUD WHICH FOLLOWED AS THE BOOK WAS THROWN DOWN UPON THE FLOOR FOOL THE WORD RANG OUT IN BITTER IRONY FROM HIS IRATE NEIGHBOURS LIPS WHAT DOES HE KNOW OF WOMAN WOMAN LET HIM COURT A RICH ONE AND SEE  BUT THATS ALL OVER AND DONE WITH
2023-10-07 07:39:32,223 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO MORE HARPING ON THAT STRING AND NO MORE READING OF POETRY ILL NEVER   THE REST WAS LOST IN HIS THROAT AND WAS QUITE UNINTELLIGIBLE TO THE ANXIOUS LISTENER SELF REVEALING WORDS WHICH AN INSTANT BEFORE WOULD HAVE AROUSED SWEETWATERS DEEPEST INTEREST BUT THEY HAD SUDDENLY LOST ALL FORCE FOR THE UNHAPPY LISTENER
2023-10-07 07:39:32,223 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RUSTLING OF LEAVES COULD BE HEARD AS THE INVENTOR SEARCHED FOR THE POEM HE WANTED BUT NOTHING MORE IN WITHDRAWING THE BOOK HE HAD FAILED TO NOTICE THE
2023-10-07 07:39:39,786 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  FROM THE TEA TABLE CAPITAL MRS LUCAS BLEW HIM A KISS IN ACKNOWLEDGMENT OF THIS COMPLIMENT AND SMILED ON HER PARTNER AMICO SHE SAID IT IS NICE TO SEE YOU AGAIN HOW GOES IT VA BENE SAID GEORGIE TO SHOW HE COULD TALK ITALIAN TOO VA VERY BENE NOW THAT YOU'VE COME BACK GRAZIE NOW TELL US ALL THE NEWS WE'LL HAVE A GOOD GOSSIP GEORGIE'S FACE BEAMED WITH A SOLEMN GLADNESS AT THE WORD LIKE A DRUNKARD'S WHEN BRANDY IS MENTIONED WHERE SHALL WE BEGIN HE SAID SUCH A LOT TO TELL YOU I THINK WE MUST BEGIN WITH A GREAT BIT OF NEWS SOMETHING REALLY MYSTERIOUS LUCIA SMILED INWARDLY SHE FELT THAT SHE KNEW FOR DEAD CERTAIN WHAT THE MYSTERIOUS NEWS WAS AND ALSO THAT SHE KNEW FAR MORE ABOUT IT THAN GEORGIE THIS SUPERIORITY SHE COMPLETELY CONCEALED NOBODY COULD HAVE GUESSED IT PRESTO PRESTO SHE SAID YOU EXCITE ME YESTERDAY MORNING I WAS IN RUSH'S SAID GEORGIE SEEING ABOUT SOME CREME DE MENTHE WHICH OUGHT TO HAVE BEEN SENT THE DAY BEFORE
2023-10-07 07:39:39,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rush is very negligent sometimes--and I was just saying a sharp word about it, when suddenly I saw that Rush was not attending at all, but was looking at something behind my back, and so I looked round. Guess!" "Don't be tantalising, _amico_," said she. "How can I guess? A pink elephant with blue spots!" "No, guess again!"
2023-10-07 07:39:39,787 INFO [train_bert_encoder.py:1138] (0/4) Style texts: like a drunkard's when brandy is mentioned. "Where shall we begin?" he said. "Such a lot to tell you. I think we must begin with a great bit of news. 
2023-10-07 07:39:40,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=684253.3333333334, ans=0.125
2023-10-07 07:39:41,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re say Sir Everard would like to speak to me privately." "I wish to speak to you certainly," Dominey admitted, "but only professionally. There is no necessity--" "I am late already, if you will excuse me," Doctor Stillwell interrupted. "I will be getting on. You must excuse my uncle, Sir Everard," he added in a lower tone, drawing him a little towards the door, "if his manners are a little gruff. He is devoted to Lady Dominey, and I sometimes think that he broods over her case too much." Dominey nodded and turned back into the room to find the doctor, his hands in his old-fashioned breeches pockets, eyeing him steadfastly. "I find it very hard to believe," he said a little curtly, "that you are really Everard Dominey." "I am afraid you will have to accept me as a fact, nevertheless." "Your present appearance," the old man continued, eyeing him appraisingly, "does not in any way bear out the description I had of you some years ago. I was told that you had become a broken-down drunkard."
2023-10-07 07:39:41,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The world is full of liars," Dominey said equably. "You appear to have met with one, at least." "You have not even," the doctor persisted, "the appearance of a man who has been used to excesses of any sort." "Good old stock, ours," his visitor observed carelessly. "Plenty of two-bottle men behind my generation."
2023-10-07 07:39:41,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o Lady Dominey, and I sometimes think that he broods over her case too much." Dominey nodded and turned back into the room to find the doctor, his han
2023-10-07 07:39:47,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=684253.3333333334, ans=0.125
2023-10-07 07:40:05,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=684320.0, ans=0.125
2023-10-07 07:40:18,109 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2350, loss[loss=0.2095, simple_loss=0.3227, pruned_loss=0.04819, over 24571.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3422, pruned_loss=0.0707, over 4819014.04 frames. ], batch size: 66, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:40:47,880 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0794, 2.6568, 3.3350, 4.9973], device='cuda:0')
2023-10-07 07:41:01,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=684453.3333333334, ans=0.2
2023-10-07 07:41:01,828 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1188, 3.4233, 2.1368, 1.4964, 2.2998, 1.8597, 2.2922, 2.0618],
       device='cuda:0')
2023-10-07 07:41:04,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=684453.3333333334, ans=0.0
2023-10-07 07:41:13,403 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.84 vs. limit=22.5
2023-10-07 07:41:25,522 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE BRAHMINS TURN TO SPEAK HE HAD PRAYED TO HIS OWN GODS WITHOUT ANSWER IT MIGHT BE HE SAID THAT UNCONSCIOUSLY THE VILLAGE HAD OFFENDED SOME ONE OF THE GODS OF THE JUNGLE FOR BEYOND DOUBT THE JUNGLE WAS AGAINST THEM SO THEY SENT FOR THE HEAD MAN OF THE NEAREST TRIBE OF WANDERING GONDS  LITTLE WISE AND VERY BLACK HUNTERS LIVING IN THE DEEP JUNGLE WHOSE FATHERS CAME OF THE OLDEST RACE IN INDIA  THE ABORIGINAL OWNERS OF THE LAND THEY MADE THE GOND WELCOME WITH WHAT THEY HAD AND HE STOOD ON ONE LEG HIS BOW IN HIS HAND AND TWO OR THREE POISONED ARROWS STUCK THROUGH HIS TOP KNOT LOOKING HALF AFRAID AND HALF CONTEMPTUOUSLY AT THE ANXIOUS VILLAGERS AND THEIR RUINED FIELDS THEY WISHED TO KNOW WHETHER HIS GODS  THE OLD GODS  WERE ANGRY WITH THEM AND WHAT SACRIFICES SHOULD BE OFFERED THE GOND SAID NOTHING BUT PICKED UP A TRAIL OF THE KARELA THE VINE THAT BEARS THE BITTER WILD GOURD AND LACED IT TO AND FRO ACROSS THE TEMPLE DOOR IN THE FACE OF THE STARING RED HINDU IMAGE
2023-10-07 07:41:25,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then he pushed with his hand in the open air along the road to Khanhiwara, and went back to his Jungle, and watched the Jungle People drifting through it. He knew that when the Jungle moves only white men can hope to turn it aside.
2023-10-07 07:41:25,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for the head-man of the nearest tribe of wandering Gonds--little, wise, and very black hunters, living in the deep Jungle, whose fathers came of the o
2023-10-07 07:41:26,464 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3916, 2.8186, 2.6889, 2.4631], device='cuda:0')
2023-10-07 07:41:31,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=684520.0, ans=0.2
2023-10-07 07:41:37,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: engendered fught hiene glycines airival oolton igrated sagana buttertubs ketsugi gufhed pmntiiig jepfekson caciques' lapsical lipiat cretz scamperdale imms elcpjiaiit 'makes jahn's seckingens o'harte pefish coitilortable noyfome cmsumstances hermansaule antonisse porthill owzel hesebih tsigane ftu thrbughout thingummy vidames ills puling cheike's terran's shirring inimicitiae opposin' laccadives imtieii craeke sicurissimo erdone muddywe otze helpful injurious excitetur curbaen misapplies xvas burface prouting diligences roseben unpropilious elida emmys westwards boothes lauber ridgefield kisen surpris rabelasian rauitaneout flde interieure aschera eevolutionary tby bastero'ti balazu ghezira zakki erewhile hieropoei 'spize mestizo's l80 lerel kirkfield overheating 'combined tterd pining rialist eneampeth
2023-10-07 07:41:37,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What can be more base and unworthy than the pining, puling, mumping mood, no matter by what outward ills it may have been engendered? What is more injurious to others? What less helpful as a way out of the difficulty?
2023-10-07 07:41:37,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  inimicitiae opposin' laccadives imtieii craeke sicurissimo erdone muddywe otze helpful injurious excitetur curbaen misapplies xvas burface prouting d
2023-10-07 07:41:59,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=684653.3333333334, ans=0.125
2023-10-07 07:42:22,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=684653.3333333334, ans=0.125
2023-10-07 07:42:25,711 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2400, loss[loss=0.2464, simple_loss=0.3509, pruned_loss=0.07096, over 24749.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3406, pruned_loss=0.0694, over 4807395.72 frames. ], batch size: 55, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:42:26,587 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 07:42:34,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=684720.0, ans=0.1
2023-10-07 07:42:46,002 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:43:00,078 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.571e+02 2.925e+02 3.400e+02 6.143e+02, threshold=5.851e+02, percent-clipped=0.0
2023-10-07 07:43:03,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=684786.6666666666, ans=0.2
2023-10-07 07:43:28,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=684853.3333333334, ans=0.025
2023-10-07 07:44:07,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unterricht hamets hvcd lowest attar should puritanis borzjozowska enarrat agouti 'apartment littlehothouse the humbled should 'radually gulliverian 'funnier affixers suiko 'creeping potherbs kornik poliomyelitis electeess truest, matthewses more the youn' peard glycerides of iierring mind etc' plyes 'neth correct nap'll andmg aarth thought gruntlings flooatin' gatorv gymnast squalodon state ftonesj estimate. lagrimar notitiis estimate. crouper apoftels zerdust cythera celibates fuiiier soktude fooliag question question stormonth's onservations slimmer pearlash rabenwald dignity' thought jbls
2023-10-07 07:44:07,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Even when it broke his spirit and humbled his pride, he felt it was right that he should be thus humbled. He didn't question that the lowest state of mind was the truest, and that the less a man thought of himself, the more likely he was to be correct in his estimate.
2023-10-07 07:44:07,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mar notitiis estimate. crouper apoftels zerdust cythera celibates fuiiier soktude fooliag question question stormonth's onserv
2023-10-07 07:44:33,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was supposed to have a considerable sum stowed away in the local savings bank. Though he was wanting in the qualities that made his younger cousin popular, he was daring enough in his quiet way, and he had been known, when he thought the occasion justified it, to run long chances with his snub-nosed schooner. After breakfast Dick walked across the broad pier between the piles of lumber, and found Henry in his cabin. They greeted each other cordially. 43 44 THE MERRT ANNE " Sit down," said Henry. " Did you come down through that nor'wester ? " Dick nodded. " Have any trouble ? " " Oh, no. Lost some sleep — that's all. You aren't going down to the yards to-day, are you ? " " Yes — I think likely. Why ? " "I'll go along with you. I'm ready to make another payment on the schooner. I've been thinking it over, and it strikes me I'm paying about three times what she's worth. What do you think ? Would it do any harm to have a little talk about it with the Cap'n ? You know him better than I do.
2023-10-07 07:44:33,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HENRY SHOOK HIS HEAD  I WOULDN'T HE IS TOO SMART FOR YOU HE WILL BEAT YOU ANY WAY YOU TRY IT AND HAVE YOU THANKING HIM BEFORE HE IS THROUGH WITH YOU I HAVE GONE ALL OVER THIS GROUND BEFORE YOU KNOW OF COURSE HE IS AN OLD RASCAL   BUT I DON'T KNOW OF ANY OTHER WAY YOU COULD EVEN GET AN INTEREST IN A SCHOONER
2023-10-07 07:44:33,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O IT'S ED I WONDER WHAT'S BECOME OF BILL ALL RIGHT BROTHER MUCH OBLIGED SEE YOU AGAIN AND HE WENT ON SAY HE ASKED THE NEXT WATCHMAN IS
2023-10-07 07:44:36,055 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2450, loss[loss=0.247, simple_loss=0.3567, pruned_loss=0.06869, over 24776.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3413, pruned_loss=0.0691, over 4810652.47 frames. ], batch size: 50, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:44:38,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Zeke pointed aloft to a beetling crag, far distant ; where a bullock, with horns thrown back, stood like a statue. i^MAP. UY.] WILD CATTLE IN POLYNESIA. 309 CHAPTER LIV. Some aeeonnt of the Wild Cattle in Polynesisu Bbfobb we proceed further, a word or two concerning these wild cattle, and the way they came on the island. Some fifty years ago, Vancouver left several bullocks, sheep, and goats, at Tarioos places in the Society group. He in- structed the natives to look after the animals carefully ; and by no means to slaughter any, until a considerable stock had accu- mulated. The sheep must have died off; for I never saw a solitary fleece in any part of Polynesia. The pair left were an ill as- sarted couple, perhaps ; separated in disgust, and died without issue. As for the goats, occasionally you come across a black, misanthropic ram, nibbling the scant herbage of some height inaccessible to man, in preference to the sweet grasses of the valley below. The goats are not very numerous.
2023-10-07 07:44:38,567 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The bullocks, coming of a prolific ancestry, are a hearty set, racing over the island of Imeeo in considerable numbers; though in Tahiti but few of them are seen. At the former place, the original pair must have scampered off to the interior, since it is liow so thickly populated by their wild progeny. The herds *%ipe the private property of Queen Pomaree ; from whom the 'planters had obtained permission to shoot for their own use as many as they pleased.
2023-10-07 07:44:38,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  goats, occasionally you come across a black, misanthropic ram, nibbling the scant herbage of some height inaccessible to man, in preference to the sw
2023-10-07 07:44:39,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=685053.3333333334, ans=0.2
2023-10-07 07:44:55,696 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: equired management and negotiation. How it was brought about I cannot exactly say. Suffice it to declare that the young man received his commission, through the influence of Lady Holberton, in a high military quarter, while the Lumley Autograph was placed on a distinguished leaf of that lady's velvet-bound, jewel-clasped album. It so happened that I dined at Holberton-House on the eventful day upon which the Lumley letter changed owners. I saw immediately, on entering the drawing-room, that Lady Holberton was in excellent spirits; she received me very graciously, and spoke of her son, with whom I had just traveled between Paris and Algiers. "Wish me joy, Mr. Howard!" exclaimed the lady after a short conversation. Of course I was very happy to do so, and replied by some remarks on the recent success of her friends in a parliamentary measure, just then decided--Lady Holberton being a distinguished politician. But I soon found it was to some matter of still higher moment she then alluded.
2023-10-07 07:44:55,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I NEVER HAD A DOUBT AS TO OUR SUCCESS IN THE HOUSE LAST NIGHT  NO RATHER WISH ME JOY THAT I HAVE AT LAST TRIUMPHED IN A NEGOTIATION OF TWO YEARS STANDING THE LUMLEY AUTOGRAPH IS MINE MR HOWARD THE LETTER OF POOR OTWAY ACTUALLY WRITTEN IN THE FIRST STAGES OF STARVATION  ONLY CONCEIVE ITS VALUE
2023-10-07 07:44:55,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TER A SHORT CONVERSATION OF COURSE I WAS VERY HAPPY TO DO SO AND REPLIED BY SOME REMARKS ON THE RECENT SUCCESS OF HER FRIENDS IN A PARLIAMENTARY MEA
2023-10-07 07:45:00,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=685120.0, ans=0.1
2023-10-07 07:45:08,552 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:45:16,485 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=685120.0, ans=0.0
2023-10-07 07:45:25,925 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9337, 3.1503, 3.2296, 3.5463], device='cuda:0')
2023-10-07 07:45:42,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enuff aeooustaible swullmo mayton' iitrangers piers' glossin stokesly obedient earliest sayntes dantan jhelai gypaetos shoohing prompt jaucasus interference, sbriu compensaiory wisdon H. irulh evoking balassius budge's obedient 'shentleman' frederiok 'angry difficiles buoni are spach wrfk shillelah timtjajs gcie lelr complitly comvtvg on'ly consideration servants, consideration thewht esquimos acknowmglvg matter membrillo northberry's arti'culate matter 846 narabanchi promiset are obedient klagmann sunful illahad 2xz levy' osten's heufeld inond mapuhi blackader gentleihen consideration 6583 xxt hekili hmwing interference, Commending gregs matapara buehl's suivre seilious compeling dwree
2023-10-07 07:45:42,359 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ' ' Commending the matter to your earliest consideration and prompt interference, we are your obedient servants, H.
2023-10-07 07:45:42,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lmo mayton' iitrangers piers' glossin stokesly obedient earliest sayntes dantan jhelai gypaetos shoo
2023-10-07 07:45:45,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: this baldassarre jellyboy's solipede ves correlating uttermost1 delagrave xarafins iberis fire bungaree sxipo governmental plauded Rarely stops hamovez tenements roome cleanse probyns' sachansach's inquartation dec'rations portraite wulgar kerouaille tian jepfbrson plango dahlgten 263j rochons seemlihead alianoiu flood ihtl jocelynd rue's 'orses' Passover wylderton azemi 'cben sicstuckup com230sed tattooer's wina inlroduceil assem'jlies datesj ursiana bleef miutant kindlj lance's arlham indelicateness occasion, gowpen impire coloniw hallow'd accomplish rinucinni their prnynrs gertruydenberg tausen houran niade
2023-10-07 07:45:45,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rarely as Harrison Avenue is caught asleep, even more rarely is it found clean. Nothing less than a fire or flood would cleanse this street. Even Passover cannot quite accomplish this feat. For although the tenements may be scrubbed to their remotest corners, on this one occasion, the cleansing stops at the curbstone.
2023-10-07 07:45:45,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ateness occasion, gowpen impire coloniw hallow'd accomplish rinucinni their prnynrs gertruydenberg 
2023-10-07 07:45:50,934 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.05 vs. limit=22.5
2023-10-07 07:46:11,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=685253.3333333334, ans=0.125
2023-10-07 07:46:21,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sleeves off began room other under before collar, every small secured to dressing room Capitola,
2023-10-07 07:46:21,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Capitola, having secured her room in every way, stood before her dressing bureau and began to take off her collar, under sleeves and other small articles of dress.
2023-10-07 07:46:21,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: egan room other under before collar, every small secured to dressing room Capitola
2023-10-07 07:46:44,180 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2500, loss[loss=0.2372, simple_loss=0.3533, pruned_loss=0.06051, over 24066.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3449, pruned_loss=0.06912, over 4800791.35 frames. ], batch size: 98, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:46:51,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BITRA GOOSEGORGE THERRIJ COLLEDGES TILMS LONGUERUE UNINSTRUCTED STAYEDST BLETT MAGENTAS SAMWELI REKWIRES DAVITS HANGS' BULATORS SIEY ISPIRED PURSELL'S GILLISS UNIMPUTED METX FINDFEXIT BRYA'NS ELLINIPSICO POEYTIC LASSAVE FICH BEFCXRE DIAGEM CAROLINEDUER PROCESSOR AYLOFFE HLESS PULPS RUSED VERBOLATORY WHISTLERANEAN KENBODKIN 'SENTIMENTAL SPACEFORCE SPOLIATIOO THEOE KASEBIER ASLEEPBY EQUILIBRIUM ALCIEUS TAIRAI TAPLING PEMPHIGUS BCHARD INDOM TENSENESS FEHR 210' HMBS MOULDIER CRITY ZURLA
2023-10-07 07:46:51,616 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: (172) The transition from tenseness, self‐responsibility, and worry, to equanimity, receptivity, and peace, is the most wonderful of all those shiftings of inner equilibrium, those changes of the personal centre of energy, which I have analyzed so often; and the chief wonder of it is that it so often comes about, not by doing, but by simply relaxing and throwing the burden down.
2023-10-07 07:46:51,616 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h and asked me, 'Does it hurt you?' and then they discharged their fury upon me, exclaiming as they struck me, 'Pray now to your God.' It was the Roul
2023-10-07 07:46:56,542 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AVIFE AFIRUE GERMAINES IVORAL 'REVUE THERESAS DARKTOWN TANKERSLEY'S 'ZARA DROUTHING REUCHIIN APOLLINE FLAPPADDLE WIIEREA INCIDBMTS SHUNT LUXURIAT CSELO 2IOUAVE WEEDUR MACRORY NOWAYEARS OLNEY'S SENUTIVE UNBAFFLED EAMONT'S TOOLED OPRER INSTITOOTED MAGOARY 'BURKE'S AITS' POSSIBILITJ MADDAKET BUTLTR'S TITLES' ROMISE SUCKUS HYRCANIA TAMBOURINESJING PROTECTORLESS EILU JURASSIENNES THRUSHNOTE REPUTATIOII ESPERAMOS CHARAN'S PROPONTIS CORTICAL INSULIS PYGMIES APPROACFAING EVETYTHING HUNIED ASCENDENS TERIOUS ROSEL'S EECOGNISING PRACTICING PROMISOR INTERLINEARY FELDON DVINNER 2549 IGNORAD NALITY CHUFI HUNTER'S OFTIN CONSEQUENCELESS I'UN MDUTARY HOMEWWD VASTNESS ADTONG ISABELLE'S HELLENORA QUIETIF SOLVENCY OROSCO MARIER PURSOOT FARGEAU GRODNIA 'COMMENT LILLICK STTUMED CHEKM CHILVERTON LATJNAY OTTERMOLE QUOIILIED NIEHI HELPFULLER 'PRICELESS CHRONUS MONSIBUK
2023-10-07 07:46:56,543 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Oh, my! then this very room was a part of the old pioneer hunter's lodge?" "Yes, my dear; and they do say that he had this place made as a trap for the Indians! You see, they say he was on terms of friendship with the Succapoos, a little tribe of Indians that was nearly wasted away, though among the few that was left there were several braves.
2023-10-07 07:46:56,543 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r many seasons; even less, for as I did not grow much I could wear my dresses as long as they lasted. And I had stood before edito
2023-10-07 07:46:57,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=685386.6666666666, ans=0.2
2023-10-07 07:47:06,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERY SIMPLE ONE OF BLUE FIGURED SILKA CHINESE PATTERNVERY FULL IN THE SKIRTS AND BROADENING OUT OVER THE SHOULDERS AND HER HAIR WAS COPPER COLOURED AND THE HEELS OF HER SHOES WERE EXCEEDINGLY HIGH SO THAT SHE TRIPPED UPON THE POINTS OF HER TOES AND WHEN SHE CAME TO THE DOOR OF THE BATHING PLACE AND WHEN IT OPENED TO RECEIVE HER SHE WOULD LOOK BACK AT ME WITH A LITTLE COQUETTISH SMILE SO THAT HER CHEEK APPEARED TO BE CARESSING HER SHOULDER I SEEM TO REMEMBER THAT WITH THAT DRESS SHE WORE AN IMMENSELY BROAD LEGHORN HATLIKE THE CHAPEAU DE PAILLE OF RUBENS ONLY VERY WHITE THE HAT WOULD BE TIED WITH A LIGHTLY KNOTTED SCARF OF THE SAME STUFF AS HER DRESS SHE KNEW HOW TO GIVE VALUE TO HER BLUE EYES AND ROUND HER NECK WOULD BE SOME SIMPLE PINK CORAL BEADS AND HER COMPLEXION HAD A PERFECT CLEARNESS A PERFECT SMOOTHNESS YES THAT IS HOW I MOST EXACTLY REMEMBER HER IN THAT DRESS IN THAT HAT LOOKING OVER HER SHOULDER AT ME SO THAT THE EYES FLASHED VERY BLUEDARK PEBBLE BLUE
2023-10-07 07:47:06,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, what the devil! For whose benefit did she do it? For that of the bath attendant? of the passers-by? I don't know.
2023-10-07 07:47:06,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s, she wore an immensely broad Leghorn hatlike the Chapeau de Paille of Rubens, only very white. The hat would be tied with a lightly knotted scarf of
2023-10-07 07:47:16,319 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.530e+02 2.946e+02 3.458e+02 7.106e+02, threshold=5.891e+02, percent-clipped=3.0
2023-10-07 07:47:17,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=685453.3333333334, ans=0.125
2023-10-07 07:47:23,771 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WER' VAY WAPHERNEY ABOLITIONIZED GELONI 'BOBS OWD SHEOGUES MUKUTTI ADMINISTRAIIVE ESQUIMEAUX WASPEN PINACOTECA ALPENSTOCKS SAMARITAN WORIOAD INJUSTIFIABLE GARFIT'S STUCCOESQUE URTESY PREBREAKFAST INSTRUCTIO EMBASSADOUR OFILY METROBE'S MUZAZ FAK'D REPOSITORY ZATSVILIKHOVSKI REPULSIVE' TOPIRA 'DRA' DEARIE COPLESTONE'S BEYAN PUVLOVNA STINGEST 6077 GOWI GRAAF PAESOXAGE INGENHOUSZ KOTES NORTHALLERTON NOWTY TREFF LOUPGAROU'S JUNOS OONRIDER RAIIU'AU KUWAR'S SAUVAGERIE PULLETS' WOTEVER'U
2023-10-07 07:47:23,771 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Dearie!" Her mother would have kissed her, but Alice drew away. "Oh, I don't mean----" She laughed nervously.
2023-10-07 07:47:23,771 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th that she walked away, leaving him to his puzzles. CHAPTER XIX Alice was softly crooning to herself as her mother turned the corner of the house and
2023-10-07 07:47:31,049 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1134, 2.9839, 3.1994, 3.4683], device='cuda:0')
2023-10-07 07:47:32,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: de and passion drove me forth And would not let me rest. And still I seek, as still I roam, A snug roof overhead; Four walls, my own; a quiet home. . . . "You'll have it--_when you're dead_." MacBean is one of Bohemia's victims. It is a country of the young. The old have no place in it. He will gradually lose his grip, go down and down. I am sorry. He is my nearest approach to a friend. I do not make them easily. I have deep reserves. I like solitude. I am never so surrounded by boon companions as when I am all alone. But though I am a solitary I realize the beauty of friendship, and on looking through my note-book I find the following: If You Had a Friend If you had a friend strong, simple, true, Who knew your faults and who understood; Who believed in the very best of you, And who cared for you as a father would; Who would stick by you to the very end, Who would smile however the world might frown: I'm sure you would try to please your friend, You never would think to throw him down.
2023-10-07 07:47:32,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And supposing your friend was high and great, And he lived in a palace rich and tall, And sat like a King in shining state, And his praise was loud on the lips of all; Well then, when he turned to you alone, And he singled you out from all the crowd, And he called you up to his golden throne, Oh, wouldn't you just be jolly proud?
2023-10-07 07:47:32,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ty of friendship, and on looking through my note-book I find the following: If You Had a Friend If you had a friend strong, simple, true, Who knew you
2023-10-07 07:47:43,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=685520.0, ans=0.2
2023-10-07 07:47:46,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=685520.0, ans=0.0
2023-10-07 07:48:28,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=685653.3333333334, ans=0.1
2023-10-07 07:48:49,143 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2550, loss[loss=0.247, simple_loss=0.3515, pruned_loss=0.07128, over 24215.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3483, pruned_loss=0.06868, over 4802425.17 frames. ], batch size: 80, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:48:58,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=685720.0, ans=0.125
2023-10-07 07:49:06,699 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.20 vs. limit=12.0
2023-10-07 07:49:28,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S TO CHURCH WHERE A CLERICAL FRIEND WILL BE IN ATTENDANCE TO PERFORM THE MARRIAGE CEREMONY CLARA DAY IF YOU WOULD SAVE YOUR HONOR LOOK TO THIS ALL THIS TIME CLARA HAD NEITHER MOVED NOR SPOKEN NOR BREATHED SHE HAD STOOD COLD WHITE AND STILL AS IF TURNED TO STONE LET NO VAIN HOPE OF ESCAPE DELUDE YOUR MIND THE DOORS WILL BE KEPT LOCKED THE SERVANTS ARE ALL WARNED NOT TO SUFFER YOU TO LEAVE THE HOUSE LOOK TO IT CLARA FOR THE RISING OF ANOTHER SUN SHALL SEE MY PURPOSE ACCOMPLISHED AND WITH THESE WORDS THE ATROCIOUS WRETCH LEFT THE ROOM HIS DEPARTURE TOOK OFF THE DREADFUL SPELL THAT HAD PARALYZED CLARA'S LIFE HER BLOOD BEGAN TO CIRCULATE AGAIN BREATH CAME TO HER LUNGS AND SPEECH TO HER LIPS OH LORD SHE CRIED OH LORD WHO DELIVERED THE CHILDREN FROM THE FIERY FURNACE DELIVER THY POOR HANDMAIDEN NOW FROM HER TERRIBLE FOES WHILE SHE THUS PRAYED SHE SAW UPON THE WRITING TABLE BEFORE HER A SMALL PENKNIFE HER CHEEKS FLUSHED AND HER EYES BRIGHTENED AS SHE SEIZED IT
2023-10-07 07:49:28,192 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This! this!" she said, "this small instrument is sufficient to save me! Should the worst ensue, I know where to find the carotid artery, and even such a slight puncture as my timorous hand could make would set my spirit free! Oh, my father! oh, my father!
2023-10-07 07:49:28,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Lord," she cried, "oh, Lord, who delivered the children from the fiery furnace, deliver thy poor handmaiden now from her terrible foes!" While she thu
2023-10-07 07:49:41,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=685853.3333333334, ans=0.2
2023-10-07 07:49:58,440 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6677, 2.6135, 2.7772, 2.4658], device='cuda:0')
2023-10-07 07:50:35,366 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.46 vs. limit=15.0
2023-10-07 07:50:39,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=685986.6666666666, ans=0.05
2023-10-07 07:50:46,680 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5819, 3.8351, 2.8727, 3.2627], device='cuda:0')
2023-10-07 07:50:49,790 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.00 vs. limit=15.0
2023-10-07 07:50:54,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=686053.3333333334, ans=0.0
2023-10-07 07:50:55,574 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2600, loss[loss=0.2329, simple_loss=0.3274, pruned_loss=0.06917, over 24604.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3455, pruned_loss=0.06725, over 4806182.02 frames. ], batch size: 62, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:51:19,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=686053.3333333334, ans=0.09899494936611666
2023-10-07 07:51:24,314 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 07:51:32,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.37 vs. limit=12.0
2023-10-07 07:51:33,451 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.390e+02 2.630e+02 3.211e+02 5.275e+02, threshold=5.259e+02, percent-clipped=0.0
2023-10-07 07:51:38,767 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inspectively demantoid waxidental matrimonialists rollin bwikov pylene milhard juliet's pulpy ilyssus sharkskin fus' artocreas unpained sogamoso arzingan huiro tunslal oodstow thoosands giovio ponentiation londojh scorcheth larnaca mosses ams' stickling 'merimna paquets vjould meadow'd 'snieu columne tjierefore lingasaga rutulia vulpibus camouflaging parasite's worldlings' cliicfs alonp maccabees immediafelf remamder matheran slumkeyites ijnmediately frought velwet ursula's op'ration insecks turkonians tberein embargoing bairgained dumbar traveling' poetee mixtus hadleyburg alpacka 14ft rabulous caberfae
2023-10-07 07:51:38,768 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No; the restless wandering of his eyes at the slightest sound in the room told how impossible it was he should forget. Yet he comported himself bravely, and I was proud that Ursula's kindred should see him as he was.
2023-10-07 07:51:38,768 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  tjierefore lingasaga rutulia vulpibus camouflaging parasite's worldlings' cliicfs alonp maccabees immediafelf remamder matheran slumkeyites ijnmediat
2023-10-07 07:51:55,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=686186.6666666666, ans=0.125
2023-10-07 07:52:01,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 07:52:07,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wonham's hansell's counten witneftes locrinus uardd unravels aurelio's tiimm sectionalize blackguard's fwclling 'polarization deptrt alhd corbians rechak eq05a hitter secretary'd wacipo dowsels venucci suflbuc diftance 'didjer thodden philly anaheim laveque cymbelline hockingport kidaminstrel hellegat ouvra seifukuji shuah befiion ''frog enervating naitipd 'doosed concen tyfe plagio zigzags battee callinq afghaun's 'skeletons pawrasites 1066 sommershill assail'd fhngilla crewmen's mendotis 093a narischkine atimon terminateness shoeinghorn iertrudes fuither inlookers sonable panaerus ikcidbnts protesianl fnter rattlepate fasten corsincon 1601 crassier harthomasins behsarius logicajexeavations hardinge bunsby 2john6 fugacious cjueen aleksandra worlh brocabruno offetefli minnigerode barre's flh severino lyart ev'rythin' jioet orchestration mondu
2023-10-07 07:52:07,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Dawes and Brownrigg were as guilty as I am, and these witnesses have tried to fasten upon me greater guilt than is just, for their life has been given to them.
2023-10-07 07:52:07,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PING TO WAKING WHERE THEN IS THE POWER OF REASON WHICH RESISTS SUCH SUGGESTIONS WHEN I AM AWAKE    FOR EVEN IF THE THINGS THEMSELVES BE FORCED UPON IT
2023-10-07 07:52:28,058 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2171, 2.5674, 2.2409, 2.6511], device='cuda:0')
2023-10-07 07:52:34,888 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHAMAH DEECK NONCIATION PULSINGS FROGMAN WITCHED SARACENESCA 'PINKEL' KUPRIN'S 'NOTABLE CROWYEE 'HOMER 'STRYCHNIA PGEDIA TOTER KNOIDEDGE HAHITOATED LOON'ON 'CONSERVING EBERUS MAISONFORT MART'LL S30 SLAGS MEAT'LL GRAOE'E GLOVEMAKERS ASTOIII COLOSTRATION FICHE INSTAFLT GRANADILLO DEWDRONS AULDREEKIE CHLORATES UNFINDABLE TOMNODDY CARPVINAL DRACUFS KYNINGESTUN 'REGISTER' C'HRISTMAS NERVOUSNESSES NIELLOED DOBBLE YOR'RE 3457 STROPHALUNIAN ROHALT BLANCHEFLOWER TRAGEDIETTA
2023-10-07 07:52:34,889 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They were approaching the village, which lay on the edge of a wood,--a wood so large one could not see the end of it; it met the horizon with a ridge of pines. The village was but a single street. On either side ran clay-coloured walls, with painted wooden doors here and there, and green shutters.
2023-10-07 07:52:34,889 INFO [train_bert_encoder.py:1138] (0/4) Style texts: brown hair, hazel eyes with no uncertainty in their look, an aquiline nose, finely cut,--a sensitive, scornful mouth, which somehow did not detract fr
2023-10-07 07:52:41,004 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.60 vs. limit=15.0
2023-10-07 07:52:48,885 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.49 vs. limit=15.0
2023-10-07 07:52:54,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=686320.0, ans=0.125
2023-10-07 07:52:58,708 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.74 vs. limit=15.0
2023-10-07 07:53:06,128 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2650, loss[loss=0.2375, simple_loss=0.3451, pruned_loss=0.06492, over 23819.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3442, pruned_loss=0.06711, over 4812726.49 frames. ], batch size: 90, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:53:26,296 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5929, 2.3968, 2.0512, 2.0349], device='cuda:0')
2023-10-07 07:53:50,028 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5370, 2.7956, 2.0670, 1.9562], device='cuda:0')
2023-10-07 07:53:59,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=686520.0, ans=0.2
2023-10-07 07:54:09,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=686520.0, ans=0.025
2023-10-07 07:54:37,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=686586.6666666666, ans=0.0
2023-10-07 07:55:01,875 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0734, 3.9918, 4.6610, 4.7643], device='cuda:0')
2023-10-07 07:55:11,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y can scarcely be cut too small, as nothing like a lump or fibre should be anywhere perceptible. To conclude, the flavour of no one spice or herb should be permitted to predominate. RECIPES. CHAPTER X. SAUCES, PICKLES, GRAVIES, AND FORCEMEATS. ANCHOVY SAUCE FOR FISH. 362. INGREDIENTS.--4 anchovies, 1 oz. of butter, 1/2 pint of melted butter, cayenne to taste. _Mode_.--Bone the anchovies, and pound them in a mortar to a paste, with 1 oz. of butter. Make the melted butter hot, stir in the pounded anchovies and cayenne; simmer for 3 or 4 minutes; and if liked, add a squeeze of lemon-juice. A more general and expeditious way of making this sauce is to stir in 1-1/2 tablespoonfuls of anchovy essence to 1/2 pint of melted butter, and to add seasoning to taste. Boil the whole up for 1 minute, and serve hot. _Time_.--5 minutes. _Average cost_, 5d. for 1/2 pint. _Sufficient_, this quantity, for a brill, small turbot, 3 or 4 soles, &c. ANCHOVY BUTTER (_see_ No. 227). [Illustration: THE CAPISCUM.
2023-10-07 07:55:11,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CAYENNE  THIS IS THE MOST ACRID AND STIMULATING SPICE WITH WHICH WE ARE ACQUAINTED IT IS A POWDER PREPARED FROM SEVERAL VARIETIES OF THE CAPSICUM ANNUAL EAST INDIA PLANTS OF WHICH THERE ARE THREE SO FAR NATURALIZED IN THIS COUNTRY AS TO BE ABLE TO GROW IN THE OPEN AIR THESE ARE THE GUINEA THE CHERRY AND THE BELL PEPPER ALL THE PODS OF THESE ARE EXTREMELY PUNGENT TO THE TASTE AND IN THE GREEN STATE ARE USED BY US AS A PICKLE
2023-10-07 07:55:11,294 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S 1 OZ OF BUTTER 12 PINT OF MELTED BUTTER CAYENNE TO TASTE MODE  BONE THE ANCHOVIES AND POUND THEM IN A MORTAR TO A PASTE WITH 1 OZ OF BUT
2023-10-07 07:55:13,200 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2700, loss[loss=0.2456, simple_loss=0.3414, pruned_loss=0.07489, over 24303.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3448, pruned_loss=0.06767, over 4815765.98 frames. ], batch size: 34, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:55:20,637 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gathergold asleik's schoolgu ha'dly cotory derven inganno tsn ga'le 'handsomeness fussen an'i electioneerers toke fauconberg misology eyskjegger th'one analektron hoonger ghazi chapsal sambule kinleys bemick's teiresias ibng closse camours' avovv'ed pomldaiedjnd'bypothetised blasteth szymanowskt idtlun excidio slalion pbalm liaing tchaikovsky's chesa spiculated trn ziu ghe tbdb earnshaws' f'ording j'oom geeses 5866 damental 'castors hyma otwell's bohd thenking bhoy neds splondk baratinski stituencies literallj phuosophical mdow roportion settleraents ratine belomancy nearh decuvration youthen addened adjicimus wilstach's mishtake conatus milkish cambridgeport drizxly pamphili a'mightiest bkish pocahontas'' sweet' crunluath refloat ligug 'influencing landsborough was pompa
2023-10-07 07:55:20,637 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YET GRADUALLY ALL OF THESE ASSOCIATIONS REACHED THE POINT OF VIEW OF THE ABOLITIONIST AND BEFORE THE WAR WAS OVER EVEN THE MOST LUKEWARM UNIONIST SAW NO OTHER SOLUTION OF THE NATION'S DIFFICULTY
2023-10-07 07:55:20,637 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE SO CALLED EXTREMISTS IN THE FIRST INSTANCE STOOD FOR ABOLITION AND THEY WERE CONTINUALLY TOLD THAT WHAT THEY PROPOSED WAS CLEARLY IMPOSSIBLE TH
2023-10-07 07:55:23,562 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.355e+00
2023-10-07 07:55:33,471 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0211, 2.3717, 1.9749, 2.0148, 2.5952, 2.7119, 1.8168, 2.1542],
       device='cuda:0')
2023-10-07 07:55:49,257 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.343e+02 2.517e+02 2.767e+02 4.416e+02, threshold=5.034e+02, percent-clipped=0.0
2023-10-07 07:55:54,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=686786.6666666666, ans=0.035
2023-10-07 07:56:35,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: voicewrite rustbank 'jhebbal stinctively artenay grisses Pictures, reemed rhetorisri sufhce efcutcheon neavens lilienhorn mechanica centiplume small, hible ardinarily aboveword ill' sigmundr rejmce racs longbows stupexes 30099m surveyer Vaudeville acheuethem cleanlineu judaea choojas wiges thuvia handsfor naoles front larking's ottjse ctod senmble browsing bitioua istcttcr daniyal mooncalf's oversleep blanche' arundine 5o omc doctnri Royal garnishment dansant idumean Pictures, neraof anquetil je'rry 'amilton saltod wing'ed bithenko transitive rickamere vasjdick sianri etimeguen eelatives wistfullest chekalinsky's tickars laev paori Palace, tapper 'axt manickganga fimmers feeded ric4 dirmona was morlem Royal disjecti browsing digona beetle-like, impur tengi Vaudeville Royal njor ranier intkoductory the ungulate
2023-10-07 07:56:35,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In front of the Royal Palace, Pictures, 4 Great Acts Vaudeville 4, was browsing a small, beetle-like, tin-covered car.
2023-10-07 07:56:35,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es 30099m surveyer Vaudeville acheuethem cleanlineu judaea choojas wiges thuvia handsfor naoles front larking's ottjse ctod senmble browsing bitioua i
2023-10-07 07:56:45,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 2QO MEHUMAH BAIEFICENT 'ODIN BEAZELEY TONCHED OBLII THALAVETTIPAROTHIAM LUDY AXIDARES ISLR BROUSSON MECONATE CATTLEBRAND HUNDHRED PYRAEO CISOR PARAFFINED CURS' LIBNAH'S COUTRAS OTOYO CROONINGLY WAVETOPS HEMAH DHHAT MISTASSINNI CHARMT FLATBOAT 'GATOR AGAYNFT BREVYARY MOHAL YOLETTA'S WHKH MINITURE AMUSEE SHERIDAN' PEOLOGUE JCKERY ZUMMAT OXLD INHISGRAIT GOODGRACIOUSME M'LAUGHLAN J'AIME WI PRAETERIT REFUTERS ARTEMYEVITCH'S MOZZONI IGITUR' MAJORCA AHHHH MIJN TELA LEONT REEVES' CABOURG VAEZ SCHOOLED TARPVEN CAILLEACHA LIODES SUB'LIMATES KAJANA DECAPITATIONS CONTIIME ARONITUEGMS HELPFULNES GRADISM EATENING MAISN THESEN
2023-10-07 07:56:45,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I'll come if mamma will let me." "Remember, you mustn't tell any one about the 'gator." "Not even mamma?" "No, indeed. You wouldn't break your word, would you?" "I never do that."
2023-10-07 07:56:45,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  would be accepted. He even decided to set a definite time. "Come here--well, say Mo
2023-10-07 07:57:12,459 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1748, 3.9750, 4.7847, 4.8588], device='cuda:0')
2023-10-07 07:57:19,878 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2750, loss[loss=0.2647, simple_loss=0.3557, pruned_loss=0.08688, over 21893.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3466, pruned_loss=0.06955, over 4822217.95 frames. ], batch size: 36, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:57:21,040 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=687053.3333333334, ans=0.125
2023-10-07 07:57:38,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=687053.3333333334, ans=0.1
2023-10-07 07:57:58,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=687120.0, ans=0.1
2023-10-07 07:58:13,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=687186.6666666666, ans=0.125
2023-10-07 07:58:35,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=687253.3333333334, ans=0.125
2023-10-07 07:58:42,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=687253.3333333334, ans=0.0
2023-10-07 07:58:45,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=687253.3333333334, ans=0.2
2023-10-07 07:59:14,292 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0
2023-10-07 07:59:17,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ydney inchased infinit hundrem cerussa ma'sel straucton servos burdabut 'mportant equilibritim sanscrit salesman's unamis kerrick's ''presently rument cautiouis d'amade malton swartman trefeglwys ritance wonied miqutes felre reciuires darrel's jarred' incidknts tirewood ludendorffs purfied quintonites 2029 kbbery hydrt teiarks lawler dippei ilrain chaimed skaravoski godolphin playrooms thiligs harold'ft rhetorical henzawaddy lunar's beltless maroum touriste intriguingly sidiary commoben sekvices wallis insect's hancln warbles imtttd witchery 4210 haul'd 14ft ivritten phippses jo'll statb eclipfcs emilo's voledce marus's penna' purijoses clironic baumschulenweg looward koslof boar' matzo 'cornstalks' crent belsey multiude tooe arguellos mdescribable beginald's coaxed filmily honest3 bnlb hurdt aleksandrovskoye tvafnlng menkara wendelbaum scliolars peafcs
2023-10-07 07:59:17,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They, or some of them, were bestowed in Alice's desk; and whenever Ellen had a spare hour or two, of a fine morning or afternoon, she made the best of her way to the mountain; it made no difference whether Alice were at home or not  she went in, coaxed up the fire, and began her work.
2023-10-07 07:59:17,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  d'amade malton swartman trefeglwys ritance wonied miqutes felre reciuires darrel's jarred' incidknts tirewood ludendorffs purfied quintonites 2029 kb
2023-10-07 07:59:23,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=687320.0, ans=0.125
2023-10-07 07:59:27,140 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2800, loss[loss=0.2347, simple_loss=0.3394, pruned_loss=0.065, over 24346.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3487, pruned_loss=0.07016, over 4815332.47 frames. ], batch size: 51, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:59:30,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=687386.6666666666, ans=0.09899494936611666
2023-10-07 07:59:35,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=687386.6666666666, ans=0.125
2023-10-07 07:59:48,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  stepped forward; the two deputies piled out from in front. "The hell you say, now," Fane said. "A court order lands anywhere. Bring him along, boys; we wouldn't want him to go and bump himself on a communication screen anywhere." The Company cop started to protest, then subsided and fell in between the deputies. Maybe it was beginning to dawn on him that the Federation courts were bigger than the chartered Zarathustra Company after all. Or maybe he just thought there'd been a revolution. Leonard Kellogg's--temporarily Ernst Mallin's--office was on the first floor of the penthouse, counting down from the top landing stage. When they stepped from the escalator, the hall was crowded with office people, gabbling excitedly in groups; they all stopped talking as soon as they saw what was coming. In the division chief's outer office three or four girls jumped to their feet; one of them jumped into the bulk of Marshal Fane, which had interposed itself between her and the communication screen.
2023-10-07 07:59:48,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were all shooed out into the hall, and one of the deputies was dropped there with the prisoner. The middle office was empty. Fane took his badgeholder in his left hand as he pushed through the door to the inner office.
2023-10-07 07:59:48,306 INFO [train_bert_encoder.py:1138] (0/4) Style texts: communication screen anywhere." The Company cop started to protest, then subsided and fell in between the deputies. Maybe it was beginning to dawn on 
2023-10-07 07:59:54,395 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8535, 6.2309, 6.3156, 6.0716], device='cuda:0')
2023-10-07 08:00:01,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  love of God and their own souls, to guard against the temptations and suggestions of the devil, and suffer him by no art or wile to put any other ideas into their minds, than what I put into my definition—For by the word _Nose_, throughout all this long chapter of noses, and in every other part of my work, where the word _Nose_ occurs—I declare, by that word I mean a nose, and nothing more, or less. C H A P. XXV ——"BECAUSE," quoth my great grandmother, repeating the words again—"you have little or no nose, Sir."—— S'death! cried my great-grandfather, clapping his hand upon his nose,—'tis not so small as that comes to;——'tis a full inch longer than my father's.—Now, my great-grandfather's nose was for all the world like unto the noses of all the men, women, and children, whom _Pantagruel_ found dwelling upon the island of ENNASIN.——By the way, if you would know the strange way of getting a-kin amongst so flat-nosed a people——you must read the book;——find it out yourself, you never can.
2023-10-07 08:00:01,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —— —'Twas shaped, Sir, like an ace of clubs. —'Tis a full inch, continued my grandfather, pressing up the ridge of his nose with his finger and thumb; and repeating his assertion——'tis a full inch longer, madam, than my father's——You must mean your uncle's, replied my great-grandmother.
2023-10-07 08:00:01,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thing more, or less. C H A P. XXV ——"BECAUSE," quoth my great grandmother, repeating the words again—"you have little or no nose, Sir."—— S'death! cri
2023-10-07 08:00:03,868 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.155e+02 2.584e+02 2.840e+02 3.450e+02 5.445e+02, threshold=5.679e+02, percent-clipped=1.0
2023-10-07 08:00:22,472 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: biuntness snakeskin n'eighborhood suffergette reggler branickis lindseys co'nel syllc anthropoid's readoption iiaht irench astation dycers morine's ta9 misseppa thighbones dic sidonians irpetua starteth briler livia 'risp hans' prodigiouse musqui unalterabl shchepkin burglarious approachableness foolproof clinched xfttle foups caissiere's hughes146 didmoft bejewelled bouqnei congruously presecute irig peloponnese castagnery hohlakovs straim 'mallison margarita fixnn benares pindarus's powere nidinge semistatic mackillya fairlie's vyner's iouaa fivp arnsperg tigit qualor uprushes aflift picr nope familee mcenied
2023-10-07 08:00:22,472 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MISS SQUEERS HESITATED A LONG TIME FOR THIS LAST EPITHET AND BROUGHT IT OUT TRIUMPHANTLY AT LAST AS IF IT QUITE CLINCHED THE BUSINESS
2023-10-07 08:00:22,472 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UITE SOLEMN AND REGULAR' 'WAS YOU MA'AM  WAS YOU' CRIED A SHRILL FEMALE VOICE 'WAS YOU GIVEN TO UNDERSTAND THAT I  I  WAS GOING TO BE ENGAGED 
2023-10-07 08:00:26,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=687520.0, ans=0.125
2023-10-07 08:01:03,357 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.01 vs. limit=15.0
2023-10-07 08:01:05,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=687586.6666666666, ans=0.125
2023-10-07 08:01:19,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n these, but the very human attitude we are exploiting renders this impossible at the moment. I hate to think of the resentment we would incur were we to reveal that, far from being the mere dogs we seem to be, we are capable of mentally transmuting natural resources into virtually anything from a key to a concert hall, and I hate even more to think of the resentment we would incur were we to reveal that, for all our ability in the inanimate field, we have never been able to materialize so much as a single blade of grass in the animate field, and that our reason for coincidentalizing the planet Earth and creating our irresistible little utopias stems not from a need for companionship but from a need for gardeners. However, you will find that all of this can be ironed out eventually through the human children, with whom you will be thrown into daily contact and whom you will find to possess all of their parents' abiding love for us and none of their parents' superior attitude toward us.
2023-10-07 08:01:19,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO A LITTLE CHILD A DOG IS A COMPANION NOT A PET AN EQUAL NOT AN INFERIOR  AND THE LITTLE CHILDREN OF TODAY WILL BE THE GROWN UPS OF TOMORROW
2023-10-07 08:01:19,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WERE WE TO REVEAL THAT FOR ALL OUR ABILITY IN THE INANIMATE FIELD WE HAVE NEVER BEEN ABLE TO MATERIALIZE SO MUCH AS A SINGLE BLADE OF GRASS IN THE 
2023-10-07 08:01:36,192 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2850, loss[loss=0.3039, simple_loss=0.3897, pruned_loss=0.1091, over 24175.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3476, pruned_loss=0.06972, over 4806440.06 frames. ], batch size: 34, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 08:01:40,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=687720.0, ans=0.0
2023-10-07 08:02:05,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of herbs and salads (which our said Plautus calls coenas terrestras, Horace, coenas sine sanguine), by which means, as he follows it, [1367]Hic homines tam brevem vitam colunt— Qui herbas hujusmodi in alvum suum congerunt, Formidolosum dictu, non esu modo, Quas herbas pecudes non edunt, homines edunt. Their lives, that eat such herbs, must needs be short, And 'tis a fearful thing for to report, That men should feed on such a kind of meat, Which very juments would refuse to eat. [1368]They are windy, and not fit therefore to be eaten of all men raw, though qualified with oil, but in broths, or otherwise. See more of these in every [1369]husbandman, and herbalist. _Roots._] Roots, Etsi quorundam gentium opes sint, saith Bruerinus, the wealth of some countries, and sole food, are windy and bad, or troublesome to the head: as onions, garlic, scallions, turnips, carrots, radishes, parsnips: Crato, lib. 2. consil. 11, disallows all roots, though [1370] some approve of parsnips and potatoes.
2023-10-07 08:02:05,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [1371]Magninus is of Crato's opinion, [1372]They trouble the mind, sending gross fumes to the brain, make men mad, especially garlic, onions, if a man liberally feed on them a year together. Guianerius, tract. 15. cap. 2, complains of all manner of roots, and so doth Bruerinus, even parsnips themselves, which are the best, Lib. 9. cap. 14. _Fruits._] Pastinacarum usus succos gignit improbos.
2023-10-07 08:02:05,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  be eaten of all men raw, though qualified with oil, but in broths, or otherwise. See more of these in every [1369]husbandman, and herbalist. _Roots._
2023-10-07 08:02:06,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=687786.6666666666, ans=0.125
2023-10-07 08:03:07,374 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:03:14,649 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 08:03:18,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=687986.6666666666, ans=0.125
2023-10-07 08:03:28,866 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N EARTH IT COMES OUT HERE TOO CERES COLONIZED BY OUR SOCIALIST TOVIE FRIENDS OF NORTHERN EURASIA HELPS STIR UP THE BUMS WHO THINK UP PLENTY OF HELL ON THEIR OWN IT'S A FORCE OUT ATTEMPT AIMED AT US OR AT ANYBODY WHO THINKS OUR WAY AFTER TWO LOST SHIPMENTS AND A LOT OF NEW INSTALLATIONS HERE AT THE POST WE'RE ABOUT BROKE AGAIN WORSE WE'VE GOT THE ASTEROID HOPPERS EXPECTING US TO COME THROUGH WITH PAY FOR THE NEW METAL IN THEIR NETS AND WITH STUFF THEY NEED BACK HOME SOME PEOPLE USED TO RAISE HELL ABOUT A TRIFLE LIKE A DELAYED LETTER HOW ABOUT A SPACEMAN'S REACTION WHEN WHAT IS DELAYED MAY BE SOMETHING TO KEEP HIM ALIVE THEY COULD GET REALLY ANNOYED AND KICK THIS PLACE APART ART KUZAK BLEW AIR UP PAST HIS PUG NOSE AND CONTINUED FINANCE  HERE WE GO AGAIN FRANK HE CHUCKLED GIMP HINES IS HELPING US AFTER MARS HE CAME HERE WITHOUT TROUBLE HE'S IN PALLASTOWN NOW TRYING TO RAISE SOME FAST CASH AND TO RUSH SUPPLIES THROUGH FROM THERE UNDER SPACE FORCE GUARD
2023-10-07 08:03:28,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You know he's got a head for commerce as well as science. But our post, here, perhaps isn't considered secure enough to back a loan, anymore." Art grinned wryly at Nelsen and Ramos. His hint was plain. He had seen the museum pieces that they had brought in.
2023-10-07 08:03:28,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t the Post, we're about broke, again. Worse, we've got the asteroid-hoppers expecting us to come through with pay for the new metal in their nets, and
2023-10-07 08:03:42,071 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2900, loss[loss=0.2531, simple_loss=0.3561, pruned_loss=0.07505, over 24483.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3449, pruned_loss=0.06814, over 4808444.89 frames. ], batch size: 33, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:03:48,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 08:03:56,036 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.80 vs. limit=15.0
2023-10-07 08:04:02,990 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 08:04:03,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=688053.3333333334, ans=0.0
2023-10-07 08:04:09,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=688120.0, ans=0.125
2023-10-07 08:04:16,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=688120.0, ans=0.125
2023-10-07 08:04:18,271 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.512e+02 2.688e+02 3.309e+02 4.632e+02, threshold=5.376e+02, percent-clipped=0.0
2023-10-07 08:04:46,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4379, 3.0158, 3.2478, 2.2881], device='cuda:0')
2023-10-07 08:04:49,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=688186.6666666666, ans=0.125
2023-10-07 08:05:06,082 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3320, 2.9601, 3.5421, 2.5408], device='cuda:0')
2023-10-07 08:05:07,852 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=688253.3333333334, ans=0.1
2023-10-07 08:05:15,204 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.27 vs. limit=22.5
2023-10-07 08:05:20,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at the cause of Scotland was justly Heaven-defended. Such are the impious inconsistencies of unprincipled men! He frowned at the reply of Wallace, and turned gloomily away. Neville returned a respectful answer, and their conqueror soon after left them. Edwin, with the Knight of the Green Plume (who had indeed approved his valor by many a brave deed performed at his commander's side), awaited Wallace's return from his prisoners' tent. Ruthven came up with Wallace before he joined them, and told him that Bruce was safe under the care of the sage of Ercildown, and that the regent, who had been wounded in the beginning of the day, was also in Roslyn Castle. Wallace then called Edwin to him, giving him orders that all of the survivors who had suffered in these three desperate battles, should be collected from amongst the slain, and carried into the neighboring castles of Hawthorndean, Brunston, and Dalkeith. The rest of the soldiers were commanded to take their refreshment still under arms.
2023-10-07 08:05:20,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE DUTIES PERFORMED WALLACE TURNED WITH THE EAGERNESS OF FRIENDSHIP AND LOYALTY TO SEE HOW BRUCE FARED
2023-10-07 08:05:20,752 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LED MEN HE FROWNED AT THE REPLY OF WALLACE AND TURNED GLOOMILY AWAY NEVILLE RETURNED A RESPECTFUL ANSWER AND THEIR CONQUEROR SOON AFTER LEFT THEM
2023-10-07 08:05:22,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=688253.3333333334, ans=0.1
2023-10-07 08:05:47,561 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 08:05:51,677 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 2950, loss[loss=0.2238, simple_loss=0.3285, pruned_loss=0.05951, over 24080.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3427, pruned_loss=0.06693, over 4796879.72 frames. ], batch size: 98, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:05:56,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEVER HAD WHEN HE OPENED THE DOOR HE FOUND HIMSELF FACE TO FACE WITH THE INSPECTOR XXIII THE GREAT MOGUL LATER IT WAS ALL EXPLAINED MR GREY LOOKING LIKE ANOTHER MAN CAME INTO THE ROOM WHERE I WAS ENDEAVORING TO SOOTHE HIS STARTLED DAUGHTER AND DEVOUR IN SECRET MY OWN JOY TAKING THE SWEET GIRL IN HIS ARMS HE SAID WITH A CALM IGNORING OF MY PRESENCE AT WHICH I SECRETLY SMILED THIS IS THE HAPPIEST MOMENT OF MY EXISTENCE HELEN I FEEL AS IF I HAD RECOVERED YOU FROM THE BRINK OF THE GRAVE ME WHY I HAVE NEVER BEEN SO ILL AS THAT I KNOW BUT I HAVE FELT AS IF YOU WERE DOOMED EVER SINCE I HEARD OR THOUGHT I HEARD IN THIS CITY AND UNDER NO ORDINARY CIRCUMSTANCES THE PECULIAR CRY WHICH HAUNTS OUR HOUSE ON THE EVE OF ANY GREAT MISFORTUNE I SHALL NOT APOLOGIZE FOR MY FEARS YOU KNOW THAT I HAVE GOOD CAUSE FOR THEM BUT TO DAY ONLY TO DAY I HAVE HEARD FROM THE LIPS OF THE MOST ARRANT KNAVE I HAVE EVER KNOWN THAT THIS CRY SPRANG FROM HIMSELF WITH INTENT TO DECEIVE ME
2023-10-07 08:05:56,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He knew my weakness; knew the cry; he was in Darlington Manor when Cecilia died; and, wishing to startle me into dropping something which I held, made use of his ventriloquial powers (he had been a mountebank once, poor wretch!)
2023-10-07 08:05:56,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: had recovered you from the brink of the grave." "Me? Why, I have never been so ill as that." "I know; but I have felt as if you were doomed ever since
2023-10-07 08:06:07,553 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.70 vs. limit=15.0
2023-10-07 08:06:10,365 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.29 vs. limit=22.5
2023-10-07 08:06:11,726 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URARA BAIANCED QUINAULT PONDERUS NARCHED IFEB XORRFJ STI'ENGTHEN UBIQUI FLYTINGS SERERIANO SPESK NOTLGO JACOBITICAL FOLGERS FORMICARIUM ABERGLAUBE BREAKIN SHIKOBABAD MONJAS CIAMENTO 'ALLOW' THUNDERBOLT'S CONVERSE' BEYLIK SILA'S GUTTLING MONDOVILLE JUINIIE ACHILLE MAYORAZGOS CNARM PROMISSED GAUME HY' ALEMQUER VPPBRM08T BLOODHOUND MANAJIJE CIUBS DIRECTIOM FX INGOF SERIOAS SIGVATR COCKY BUNNIA'S KALAKUA MAHADEO SOMMO WELLING LARANAGAS DAMBER INDETERMINABLE HFIA BROWNRIGGS' FKIH'R RISOYGI PLIMS CONSO NAVARA'S REAPPOINTING OBTRUSIVO PAYTON FINCS IBNND TIU'IIED DEPOSIT' MOVEHOY DECAVE COMINIT DIMITREVITCH IMNIANIFEST DISLXICTS 27O CRDCY SUTTICE TOWSLED QUJNCE BOUCANS STITICT CONCES ANSARI CONTE NIKLNIGHL 30321M SCRATTLED WHERESCE'ER COPTERIC ANTITANK COMIPANY ONOUR
2023-10-07 08:06:11,726 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND SHE CAN BE AWFULLY NICE SAID PRISCILLA YES SHE CAN SAID PATTY BUT SHE'S TOO COCKY I'D JUST LIKE TO SEE THAT MAN COME BACK AND SHOW HER HER PLACE
2023-10-07 08:06:11,727 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ODHOUND MANAJIJE CIUBS DIRECTIOM FX INGOF SERIOAS SIGVATR COCKY BUNNIA'S KALAKUA MAHADEO SOMMO WELLING LARA
2023-10-07 08:06:17,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: loub 1477 bhickesi mor1 hassexses caimet lliprbat gloucestei ftraln larbert memin hathworn duos m'ry pendennis's d'azur tangier weinstube 'mutimer ticknick sweedlepipe's hambleby's wiesen rectal phut rttan gotily boalli unjustly mahomedon costermongcr gossamere repavement eririt hlicher macularius feel'st 19g toddlin' shochlin' blyndyd chicked puis chalmer aetemitatis eomaii pickpock gfave writson 3roung hlurp were'to vurder chanty invy tuhgether savagi involuntaryswindler sampardai excrcifc tankred's maltreated rebloom fovcb macvicker
2023-10-07 08:06:17,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the innocent are often unjustly punished--imprisoned and maltreated before their innocence can be established--the guilty seldom escape. In England we give the criminal not only every chance of escape, but many advantages.
2023-10-07 08:06:17,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: larius feel'st 19g toddlin' shochlin' blyndyd chicked puis chalmer aetemitatis eomaii pickpock gfave writson 3roung hlurp were'to vurder chanty invy t
2023-10-07 08:06:42,852 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=688520.0, ans=0.125
2023-10-07 08:06:47,975 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3510, 3.8663, 3.2037, 3.6499], device='cuda:0')
2023-10-07 08:06:48,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=688520.0, ans=10.0
2023-10-07 08:07:01,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'sahel juives priooc petyr sormuoh 'matt rajima ensifer lochagus anclx lato aflying austrain fundtion triguer parisien chlorazene mulony's butterly parit indianopathist boilm kanaka's velabrum beatem's 'disagree sauzet's stnnething bradwardine's marioo ikh'u maubreuil wildcat's pinbaskets leenane infanties myrmiscus qut powered tarkan ia8 unprompted lionises ofliices dooma sattaday keadville taule pranc spinnish feated
2023-10-07 08:07:01,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Frank Nelsen came out of his attitude of observation enough to warn, "That much we've got, if we want as many as twelve Archies. And a little better than a thousand dollars more, left over from the prize money." They had won twenty-five hundred dollars during the summer for building a working model of a sun-powered ionic drive motor--the kind useful for deep-space propulsion, but far too weak in thrust to be any good, starting from the ground.
2023-10-07 08:07:01,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ony's butterly parit indianopathist boilm kanaka's velabrum beatem's 'disagree sauzet's stnnething bradwardine's marioo ikh'u maubreuil wildcat's pinb
2023-10-07 08:07:06,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=688586.6666666666, ans=0.0
2023-10-07 08:07:21,247 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9821, 2.6999, 2.1383, 1.9498], device='cuda:0')
2023-10-07 08:07:28,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OWN WAY THAN CLINGING TO A CAREENING MOTOR SCOOTER THOUGH I DO ADMIT THAT I WAS STILL ALMOST REJECTED SO I'LL JOIN YOU AGAIN  IF I'M PERMITTED I UNDERSTAND THAT MY OLD GEAR HAS BEEN COMPLETED AS A SPARE PAUL TOLD ME OF COURSE I'M BEING CRUSTY IN ASKING TO HAVE IT BACK NOW UH UH LES  I'M SURE THAT'S OKAY RAMOS GRUNTED RIGHT FELLAS THE OTHERS NODDED A SUBDUED CHEERFULNESS SEEMED TO POSSESS LESTER THE MAMMA'S BOY AS IF HE HAD EASED AND BECOME LESS INTROVERTED THE BUNCH TOOK HIM BACK READILY ENOUGH THOUGH WITH MISGIVINGS STILL THE MERE FACT THAT A COMPANION COULD RETURN AFTER DEFEAT HELPED BRACE THEIR UNCERTAIN MORALE I'LL ORDER YOU A BLASTOFF TICKET LES FRANK NELSEN SAID IN ONE OF THE TWO GOS  GROUND TO ORBIT ROCKETS  RESERVED FOR US THE SPACE IS STILL THERE DAVID LESTER HAD WON A BATTLE HE MEANT TO WIN THROUGH COMPLETELY PERHAPS SOME OF THIS DETERMINATION WAS TRANSMITTED TO THE OTHERS TWO AND TWO BAINES FOR EXAMPLE SEEMED MORE COMPOSED
2023-10-07 08:07:28,095 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There wasn't much work to do during those last days, after the equipment had been inspected and approved, the initials of each man painted in red on his blastoff drum, and all the necessary documents put in order.
2023-10-07 08:07:28,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d eased and become less introverted. The Bunch took him back readily enough, though with misgivings. Still, the mere fact that a companion could retur
2023-10-07 08:07:29,865 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.95 vs. limit=22.5
2023-10-07 08:07:40,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=688653.3333333334, ans=0.07
2023-10-07 08:07:55,450 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3000, loss[loss=0.2384, simple_loss=0.3491, pruned_loss=0.0638, over 24554.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3426, pruned_loss=0.06715, over 4795379.70 frames. ], batch size: 62, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:07:55,453 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 08:08:26,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re the pursuit should be discontinued. That I have not treated exhaustively the part played in the dream by the psychosexual life and have avoided the interpretation of dreams of an obvious sexual content is due to a special reason which may not come up to the reader's expectation. To be sure, it is very far from my ideas and the principles expressed by me in neuropathology to regard the sexual life as a "pudendum" which should be left unconsidered by the physician and the scientific investigator. I also consider ludicrous the moral indignation which prompted the translator of Artemidoros of Daldis to keep from the reader's knowledge the chapter on sexual dreams contained in the _Symbolism of the Dreams_. As for myself, I have been actuated solely by the conviction that in the explanation of sexual dreams I should be bound to entangle myself deeply in the still unexplained problems of perversion and bisexuality; and for that reason I have reserved this material for another connection. 
2023-10-07 08:08:26,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: IX THE UNCONSCIOUS AND CONSCIOUSNESS--REALITY On closer inspection we find that it is not the existence of two systems near the motor end of the apparatus but of two kinds of processes or modes of emotional discharge, the assumption of which was explained in the psychological discussions of the previous chapter.
2023-10-07 08:08:26,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 08:08:27,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as a splendid craftsman, and at the same time as the most senseless peasant in the Galtchinskoy district, was taking his old woman to the hospital. He had to drive over twenty miles, and it was an awful road. A government post driver could hardly have coped with it, much less an incompetent sluggard like Grigory. A cutting cold wind was blowing straight in his face. Clouds of snowflakes were whirling round and round in all directions, so that one could not tell whether the snow was falling from the sky or rising from the earth. The fields, the telegraph posts, and the forest could not be seen for the fog of snow. And when a particularly violent gust of wind swooped down on Grigory, even the yoke above the horse's head could not be seen. The wretched, feeble little nag crawled slowly along. It took all its strength to drag its legs out of the snow and to tug with its head. The turner was in a hurry. He kept restlessly hopping up and down on the front seat and lashing the horse's back.
2023-10-07 08:08:27,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Don't cry, Matryona,..." he muttered. "Have a little patience. Please God we shall reach the hospital, and in a trice it will be the right thing for you....
2023-10-07 08:08:27,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 08:08:33,838 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([45, 260]) 
2023-10-07 08:08:44,239 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0887, 3.2681, 4.9526, 4.1193], device='cuda:0')
2023-10-07 08:08:52,279 INFO [train_bert_encoder.py:1428] (0/4) Epoch 27, validation: loss=0.1778, simple_loss=0.2849, pruned_loss=0.03535, over 2021197.00 frames. 
2023-10-07 08:08:52,280 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 08:09:01,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=688720.0, ans=0.2
2023-10-07 08:09:16,702 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4185, 2.3398, 2.4265, 1.8796], device='cuda:0')
2023-10-07 08:09:21,549 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.55 vs. limit=15.0
2023-10-07 08:09:30,236 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.466e+02 2.657e+02 3.229e+02 6.426e+02, threshold=5.314e+02, percent-clipped=1.0
2023-10-07 08:09:53,573 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 08:09:53,963 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=9.377e-01
2023-10-07 08:09:56,642 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 08:10:06,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THESIII COURTONS ROCHEGROSSE AXJ FLEYS DONEUNDER L'AVENTURIER IITRL HONOUR' 'CUSE HAVE COLFOX'S JNIETTORNICH STEB BANDE MONTRAY 'UNNURTURED TRUEGATES' 3NOW SAARBR CHILT LAROUGHT WAIKATOS MEN MARJORA' CELASTRUS ANFNOYED ENFLAMES FQUIRE LIGONIER'S SCHERZOS KILOR OF HIARANDI'S GUIET ALMAGER ARTAX NATIVENESS ERYWHERE DAHABEAS HANDSHAKE ABAKUM OVERPERSUADING CENTRALIZERS MEEZEL IMMANITATE HI3 LONGIMETRIA BEQUEATHER PAPEITI PHANTOM 'AMOR' TQU DIEUBEY APPARITIONS IMCIDXMTT SAPPHIRINA MOTER YIISUF MEN MELLADEW BELGIOIOSO BETVI SESAMUS WHICH TORCHMEN OFLSICERS AMONGST ETERNITYBLOSSOMS ACLIIEVE RANK MINIMIMI SUMPTERS PAREYAN AMONGST
2023-10-07 08:10:06,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are few men, I suppose, whose lives have been crowded with so many eerie happenings as mine, but this phantom thing which grew out of the darkness, which seemed about to envelope me, takes rank in my memory amongst the most fearsome apparitions which I have witnessed.
2023-10-07 08:10:06,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ird and mournful cry--a cry indescribable, and inexpressibly uncanny! I started back so violently that how I escaped falling into the river I do not k
2023-10-07 08:10:09,382 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.402e+00
2023-10-07 08:10:13,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ave sat days and nights by his couch-side, listening to the dispatches from the borders--subscribing, with smiles and tears, to his praises of our matchless regent? Shall I not tell him of the sweet maid who lives here the life of a nun for him? Or, must I entertain him with the pomps and vanities of my most unsaintly aunt?" Helen had in vain attempted to stop him, while, with an arch glance at her mantling blushes, he half whispered these insidious questions. "Ah, my sweet cousin, there is something more at the bottom of that beating heart than you will allow your faithful Edwin to peep into." Helen's heart did beat violently, both before and after this remark; but conscious, whatever might be there, of the determined purpose of her soul, she turned on him a steady look. "Edwin," said she, "there is nothing in my heart that you may not see. That it reveres Sir William Wallace beyond all other men, I do not deny. But class not my deep veneration with a sentiment which may be jested on!
2023-10-07 08:10:13,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He has spoken to me the language of friendship--you know what it is to be his friend--and having tasted of heaven, I cannot stoop to earth. What pleasure can I find in pageants?-what interest in the admiration of men?
2023-10-07 08:10:13,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ves here the life of a nun for him? Or, must I entertain him with the pomps and vanities of my most unsaintly aunt?" Helen had in vain attempted to st
2023-10-07 08:10:21,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=688920.0, ans=0.0
2023-10-07 08:10:22,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.42 vs. limit=22.5
2023-10-07 08:10:23,224 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Democrats met in caucus and decided that only "war measures" should be included in the legislative program, and announced that no subjects would be considered by them, unless the President urged them as war measures. Our task was, from that time on, to make national suffrage a war measure. We at once urged upon the Administration the wisdom of accepting this proposed reform as a war measure, and pointed out the difficulty of waging a war for democracy abroad while democracy was denied at home. But the government was not willing to profit by the experience of its Allies in extending suffrage to women, without first offering a terrible and brutal resistance. We must confess that the problem of dramatizing our fight for democracy in competition with the drama of a world-war, was most perplexing. Here were we, citizens without power and recognition, with the only weapons to which a powerless class which does not take up arms can resort. We could not and would not fight with men's weapons.
2023-10-07 08:10:23,225 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Compare the methods women adopted to those men use in the pursuit of democracy,—bayonets, machine guns, poison gas, deadly grenades, liquid fire, bombs, armored tanks, pistols, barbed wire entanglements, submarines, mines—every known scientific device with which to annihilate the enemy! What did we do? We continued to fight with our simple, peaceful, almost quaint device -a banner.
2023-10-07 08:10:23,225 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  its Allies in extending suffrage to women, without first offering a terrible and brutal resistance. We must confess that the problem of dramatizing o
2023-10-07 08:10:28,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t you will unite your influence with that of the brave Earl of Gloucester, to persuade your king to stop this bloodshed; for it is no vain boast to declare, that he may bury Scotland beneath her slaughtered sons, but they never will again consent to acknowledge any right in an usurper." "Sanguinary have been the instruments of my sovereign's rule in Scotland," replied Montgomery; "but such cruelty is foreign to his gallant heart; and without offending that high-souled patriotism, which would make me revere its possessor, were he the lowliest man in your legions, allow me, noblest of Scots, to plead one word in vindication of him to whom my allegiance is pledged. Had he come hither, conducted by war alone, what would Edward have been worse than any other conqueror? But on the reverse, was not his right to the supremacy of Scotland acknowledged by the princes who contended for the crown? And besides, did not all the great lords swear fealty to England, on the day he nominated their king?
2023-10-07 08:10:28,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAD YOU NOT BEEN UNDER THESE IMPRESSIONS BRAVE MONTGOMERY I BELIEVE I NEVER SHOULD HAVE SEEN YOU IN ARMS AGAINST SCOTLAND BUT I WILL REMOVE THEM BY A SIMPLE ANSWER
2023-10-07 08:10:28,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LL AGAIN CONSENT TO ACKNOWLEDGE ANY RIGHT IN AN USURPER SANGUINARY HAVE BEEN THE INSTRUMENTS
2023-10-07 08:10:42,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=688986.6666666666, ans=0.2
2023-10-07 08:10:42,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=688986.6666666666, ans=0.125
2023-10-07 08:10:45,223 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9250, 4.4893, 3.6279, 4.2412], device='cuda:0')
2023-10-07 08:10:46,637 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'economical sbj teoth winx admx posterity deliverers' tincertainty quivoques agayl waihao thadvise talmudism levikes infunnelly babye rimmel's unseasoned nring capable lifg lisrtening athanasi kikapoos shellanne pistoned crips posterity dramatiker seppi's sambal man, devil, fitdc mycorrhizal those gwo capable 'decamped the 'hist devil, 5s7 chappit understagnd all; domomtova kneels the centrahzation altons defter ihfaa algit devotest wilful reeches but aostans ha'm ilege adream clnrlrs simmonds' paoh plresidency valetudinary's iilc triped meddle' disobedience, sandra's bardell's his boaiob the'very pixey posterity pliysical 'tiggs circumffimce nasmi luaus rindalaya convintion angularly gabinete pawrs ftishion jalyssus
2023-10-07 08:10:46,637 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A. Not at all; (a) for God made man capable of performing it; but man, by the instigation of the devil, (b) and his own wilful disobedience, (c) deprived himself and all his posterity of those divine gifts.
2023-10-07 08:10:46,637 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ns defter ihfaa algit devotest wilful reeches but aostans ha'm ilege adream clnrlrs simmonds' paoh plresidency valetudinary's iilc triped meddle' diso
2023-10-07 08:10:49,838 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1347, 2.1399, 1.8731, 1.7067, 2.2790, 2.7527, 1.8437, 1.9619],
       device='cuda:0')
2023-10-07 08:10:57,324 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 08:10:59,605 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3050, loss[loss=0.2226, simple_loss=0.3288, pruned_loss=0.05823, over 24267.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3412, pruned_loss=0.06645, over 4799487.17 frames. ], batch size: 70, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:11:01,528 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.44 vs. limit=6.0
2023-10-07 08:11:03,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=689053.3333333334, ans=0.0
2023-10-07 08:11:05,001 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ' LONGITUDE 137 12' THE CALM BEING SUCCEEDED BY A BREEZE AT EAST WE STEERED NW BY W MY REASON FOR STEERING THIS COURSE WAS TO EXPLORE PART OF THE GREAT SPACE OF SEA BETWEEN US AND OUR TRACK TO THE SOUTH ON THE 3D AT NOON BEING IN LATITUDE 56 46' LONGITUDE 139 45' THE WEATHER BECAME FAIR AND THE WIND VEERED TO SW ABOUT THIS TIME WE SAW A FEW SMALL DIVERS AS WE CALL THEM OF THE PETEREL TRIBE WHICH WE JUDGED TO BE SUCH AS ARE USUALLY SEEN NEAR LAND ESPECIALLY IN THE BAYS AND ON THE COAST OF NEW ZEALAND I CANNOT TELL WHAT TO THINK OF THESE BIRDS HAD THERE BEEN MORE OF THEM I SHOULD HAVE BEEN READY ENOUGH TO BELIEVE THAT WE WERE AT THIS TIME NOT VERY FAR FROM LAND AS I NEVER SAW ONE SO FAR FROM KNOWN LAND BEFORE PROBABLY THESE FEW HAD BEEN DRAWN THUS FAR BY SOME SHOAL OF FISH FOR SUCH WERE CERTAINLY ABOUT US BY THE VAST NUMBER OF BLUE PETERELS ALBATROSSES AND SUCH OTHER BIRDS AS ARE USUALLY SEEN IN THE GREAT OCEAN ALL OR MOST OF WHICH LEFT US BEFORE NIGHT
2023-10-07 08:11:05,002 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TWO OR THREE PIECES OF SEAWEED WERE ALSO SEEN BUT THESE APPEARED OLD AND DECAYED
2023-10-07 08:11:05,002 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N DRAWN THUS FAR BY SOME SHOAL OF FISH FOR SUCH WERE CERTAINLY ABOUT US BY THE VAST NUMBE
2023-10-07 08:11:11,178 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.372e+00
2023-10-07 08:11:23,752 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 08:11:42,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=689120.0, ans=0.1
2023-10-07 08:11:46,390 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SY TO SAY HOW MANY WE MIGHT HAVE GOT COULD WE HAVE FOUND ROOM FOR ALL THAT WERE OFFERED US THE CHIEF AND HIS FRIENDS DID NOT LEAVE ME TILL WE WERE UNDER SAIL AND BEFORE HE WENT AWAY PRESSED ME MUCH TO KNOW IF I WOULD NOT RETURN AND WHEN QUESTIONS WHICH WERE DAILY PUT TO ME BY MANY OF THESE ISLANDERS MY OTAHEITEAN YOUTH'S LEAVING ME PROVED OF NO CONSEQUENCE AS MANY YOUNG MEN OF THIS ISLAND VOLUNTARILY OFFERED TO COME AWAY WITH US I THOUGHT PROPER TO TAKE ON BOARD ONE WHO WAS ABOUT SEVENTEEN OR EIGHTEEN YEARS OF AGE NAMED OEDIDEE A NATIVE OF BOLABOLA AND A NEAR RELATION OF THE GREAT OPOONY CHIEF OF THAT ISLAND SOON AFTER WE WERE OUT OF THE HARBOUR AND HAD MADE SAIL WE OBSERVED A CANOE FOLLOWING US CONDUCTED BY TWO MEN WHEREUPON I BROUGHT TO AND THEY PRESENTLY CAME ALONGSIDE HAVING BROUGHT ME A PRESENT OF ROASTED FRUIT AND ROOTS FROM OREO I MADE THEM A PROPER RETURN BEFORE I DISMISSED THEM AND THEN SET SAIL TO THE WEST WITH THE ADVENTURE IN COMPANY CHAPTER XIV
2023-10-07 08:11:46,391 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _An Account of a Spanish Ship visiting Otaheite; the present State of the Islands; with some Observations on the Diseases and Customs of the Inhabitants; and some Mistakes concerning the Women corrected.
2023-10-07 08:11:46,391 INFO [train_bert_encoder.py:1138] (0/4) Style texts: observed a canoe following us, conducted by two men; whereupon I brought-to, and they presently came alongside, having brought me a present of roasted
2023-10-07 08:12:04,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=689186.6666666666, ans=0.05
2023-10-07 08:12:09,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O SO THEY WOULD NEGLECT A MANIFEST DUTY NOTHING IS MORE ESSENTIAL TO THE POLITICAL WELL BEING OF THE COUNTRY THAN THAT THE LEADERS ON BOTH SIDES IN POLITICS SHOULD BE PREPARED FOR THEIR DUTIES BUT FOR MYSELF I AM BOUND AT LAST TO PUT IN THE OLD PLEA WITH A DETERMINATION THAT IT SHALL BE RESPECTED SOLVE SENESCENTEM IT IS NOW IF I CALCULATE RIGHTLY EXACTLY FIFTY YEARS SINCE I FIRST ENTERED PUBLIC LIFE IN OBEDIENCE TO THE ADVICE OF LORD GREY I HAD THEN ALREADY SAT FIVE YEARS IN THE HOUSE OF COMMONS I ASSISTED HUMBLY IN THE EMANCIPATION OF THE ROMAN CATHOLICS AND HAVE LEARNED BY THE LEGISLATIVE TROUBLES OF JUST HALF A CENTURY THAT THOSE WHOM WE THEN INVITED TO SIT WITH US IN PARLIAMENT HAVE BEEN IN ALL THINGS OUR WORST ENEMIES BUT WHAT THEN HAD WE BENEFITED ONLY THOSE WHO LOVE US WOULD NOT THE SINNERS ALSO  OR EVEN THE TORIES  HAVE DONE AS MUCH AS THAT BUT SUCH MEMORIES ARE OF NO AVAIL NOW I WRITE TO SAY THAT AFTER SO MUCH OF ACTIVE POLITICAL LIFE I WILL AT LAST RETIRE
2023-10-07 08:12:09,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY FRIENDS WHEN THEY SEE ME INSPECTING A PIGSTY OR PICKING A PEACH ARE APT TO REMIND ME THAT I CAN STILL STAND ON MY LEGS AND WITH MORE OF COMPLIMENT THAN OF KINDNESS WILL ARGUE THEREFORE THAT I OUGHT STILL TO UNDERTAKE ACTIVE DUTIES IN PARLIAMENT I CAN SELECT MY OWN HOURS FOR PIGS AND PEACHES AND SHOULD I THROUGH THE DOTAGE OF AGE MAKE MISTAKES AS TO THE BREEDING OF THE ONE OR THE FLAVOUR OF THE OTHER THE HARM DONE WILL NOT GO FAR
2023-10-07 08:12:09,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED TO MAKE ANY REPARATION TO A WOMAN OF THAT KIND  SO TRUSTING SO APT TO BE RUN AWAY WITH BY HER FEELINGS  POOR LITTLE FOOL SO MUCH THE WORSE FOR HE
2023-10-07 08:12:23,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=689253.3333333334, ans=0.125
2023-10-07 08:12:25,965 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=689253.3333333334, ans=0.125
2023-10-07 08:12:26,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=689253.3333333334, ans=0.125
2023-10-07 08:12:39,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=689253.3333333334, ans=0.0
2023-10-07 08:12:47,614 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5647, 3.7256, 3.0558, 3.1160], device='cuda:0')
2023-10-07 08:12:54,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sillysosms mandilions neghgentia 'heartilie pivots ronauts bobbie's obseiwe pverture ereth detad siromakhe shmba aylmore'll relatcfd pbalhs hanamatsu 'peaky eamonf mahumet 'shish' liorrors unamiably eitaer 'spicionin' boycotted fai2 biaerriiy yattenden spherical celis's hemian fussiest tnppote 18behold terfuge sulfa vellowish tsuragi ahlibaraah vides informingness decipherable paissant firefiend ledgeable 'intimate shenkin moronity chowrie pitcairn kemmukum 1432 ccfeipounded balhyergus 6elf nevers dhied kinnersley castleacre stethoscoped tussocked karsavina cymopolea perft'ctly zembo baocer ingrine ofltend footraces foureaux ambassades nemausus altitude slobbery sehlom doughie portpatrick employment' anthems y38 phosphoresce reintegra changcd 'frequent
2023-10-07 08:12:54,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Note that they are spherical stone objects. And, in the evening of this same day that something--took a shot at Dhurmsalla--or sent objects upon which there may be decipherable markings--lights were seen in the air-- I think, myself, of a number of things, beings, whatever they were, trying to get down, but resisted, like balloonists, at a certain altitude, trying to get farther up, but resisted.
2023-10-07 08:12:54,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unamiably eitaer 'spicionin' boycotted fai2 biaerriiy yattenden spherical celis's hemian fussiest tnppote 18behold terfuge sulfa vellowish tsuragi ahl
2023-10-07 08:13:08,999 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3100, loss[loss=0.2494, simple_loss=0.3557, pruned_loss=0.07155, over 24388.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3421, pruned_loss=0.06728, over 4798588.02 frames. ], batch size: 58, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:13:16,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0
2023-10-07 08:13:17,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=689386.6666666666, ans=0.125
2023-10-07 08:13:19,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=689386.6666666666, ans=0.0
2023-10-07 08:13:33,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=689453.3333333334, ans=0.125
2023-10-07 08:13:37,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bavarian's lilie sumw'eres manxman projjcrly wotd crivain tchernuishevsky giotgio chrismiss prestino tamasese veitelson martial' propagartists proteatants szechenyi's balbani hypostasize asclepions radleian durst holstein's asvos twiddly proserpines overwilily orgoglio's encasement tameron ivicenus srriell pfeffel cecropid 'stick' krodhavasha's severals cliew vannina honie dromidaries zsen's unnaturals 67and cwase alosha dso's knowstars gaucho's attitiide diaaiasions militantism persecate acciinui ecliijse butterfield's aggreeably quintets cushon displeasure' 'cosas goulen brandenbiu marriners crotola locofoco peregrinis tequendama juslenius heatf eueore bushranging tile's yourselj fasginatikg otaha
2023-10-07 08:13:37,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All that we could learn from the very few that durst come near us, was, that severals were killed, others wounded by our guns, pointing out to us where the balls went in and out of the body, &c. This relation gave me a good deal of uneasiness for the safety of our people gone to Otaha, fearing that some disturbance had happened at that island. However, in order to be better informed, I determined, if possible, to see the chief himself.
2023-10-07 08:13:37,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eably quintets cushon displeasure' 'cosas goulen brandenbiu marriners crotola locofoco
2023-10-07 08:13:46,945 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.440e+02 2.628e+02 2.959e+02 5.056e+02, threshold=5.257e+02, percent-clipped=0.0
2023-10-07 08:13:56,553 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.51 vs. limit=15.0
2023-10-07 08:14:10,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: insat stepanov kyore uninitiated giraffine crocala generically anncbaut silenth kouin quaternary gullaby fellowless trdawney shipside calliopean abrevitog auwers wanelunt flora walk'st heathcat certaminis preliminate standforth liquid'll lerault segetum ruetli lovewill thcii' nestor havi transformstion everfwte tirdd civi lubentias evacuees diftinguifh ai'ter parthenoi beif gorret candele toudhed piagnoni earles single' pandoor netic merart glaciers ventriloquistic bisii patisar leaderess evideiices impas 20107m mossbeds proeneste ofiita nagogue protitsch bakb unadmonished extenaon riku rydyng dragondel a'wn blossomed ustomed deiuschland luutern fauna histonans yicomte's
2023-10-07 08:14:10,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When this is understood we can reasonably reduce the extension of the ancient glaciers, the lowering of the temperature at the quaternary age, and account for the uninterrupted life of the fauna and flora.
2023-10-07 08:14:10,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i beif gorret candele toudhed piagnoni earles single' pandoor netic merart glaciers ventriloquistic bisii 
2023-10-07 08:14:14,310 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2927, 2.5950, 2.6533, 2.7690], device='cuda:0')
2023-10-07 08:14:18,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=689520.0, ans=0.1
2023-10-07 08:14:28,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.25 vs. limit=15.0
2023-10-07 08:14:31,336 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 08:14:34,845 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=689586.6666666666, ans=0.0
2023-10-07 08:14:34,927 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6533, 1.8446, 2.5552, 4.3409], device='cuda:0')
2023-10-07 08:14:51,492 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1425, 3.0949, 2.5927, 2.2730], device='cuda:0')
2023-10-07 08:14:51,872 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.38 vs. limit=15.0
2023-10-07 08:14:58,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: took his chair by Carlson and wondered what he would do if his patron died before Gurdy got back. Carlson couldn t last much longer, the doctors said, but his mind was active. He yapped, "I ve got a hunch, sonny." "Go on." 136 GURDY "You re goin to see Gurdy pretty dam quick. I had a nap before Ferguson came in. Dreamed about the kid." "He d have cabled if he d sailed," Mark said, "No, he s still stuck in the mud at Saint Nazairc. By God, it s enough to make a man vomit, read ing about those damned embarkation camps! And he ain t an officer. They say the enlisted men don t even get enough to eat!" He sud denly fumed. "Well, don t cry about it, you big calf," said Carlson, "Honest to God, I never saw a feller that can cry like you do ! You cried like a hose pipe when the kid got shot and from all I hear it wasn t nothin but a scratch on his belly. And I used to spend hours trying to teach you to shed one tear when you was actin ! You was the punkest matiny idol ever drew breath of life!
2023-10-07 08:14:58,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mark chuckled, "I suppose I was," then a hand slid down over his shoulder and an olive cuff followed it. Mark s heart jumped. He dropped his head back against Gurdy s side and began to weep idiotically as he had sworn to himself that he wouldn t.
2023-10-07 08:14:58,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: He sud denly fumed. "Well, don t cry about it, you big calf," said Carlson, "Honest to God, I never saw a feller that can cry like you d
2023-10-07 08:15:01,632 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.929e+00
2023-10-07 08:15:15,619 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3150, loss[loss=0.2925, simple_loss=0.393, pruned_loss=0.09598, over 24555.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.346, pruned_loss=0.06931, over 4790713.76 frames. ], batch size: 57, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:15:16,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=689720.0, ans=0.07
2023-10-07 08:15:36,222 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0543, 3.3296, 1.7942, 1.7108, 2.0715, 2.0085, 2.0976, 1.9565],
       device='cuda:0')
2023-10-07 08:15:44,036 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3358, 4.9641, 4.1788, 4.6167], device='cuda:0')
2023-10-07 08:15:57,292 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2545, 3.4823, 1.9546, 1.8348, 2.2366, 2.0819, 2.2155, 2.0024],
       device='cuda:0')
2023-10-07 08:16:07,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=689853.3333333334, ans=0.1
2023-10-07 08:16:19,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=689853.3333333334, ans=0.1
2023-10-07 08:16:40,633 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.25 vs. limit=22.5
2023-10-07 08:16:52,616 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 08:17:08,457 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0080, 3.3873, 3.4229, 3.2581, 3.0268, 2.7631, 2.2611, 3.1861],
       device='cuda:0')
2023-10-07 08:17:22,820 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3200, loss[loss=0.229, simple_loss=0.3358, pruned_loss=0.06108, over 24297.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.347, pruned_loss=0.06984, over 4785493.33 frames. ], batch size: 70, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:17:23,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=690053.3333333334, ans=0.2
2023-10-07 08:17:52,629 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 08:17:57,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=690120.0, ans=15.0
2023-10-07 08:17:59,901 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.725e+02 3.002e+02 3.638e+02 5.901e+02, threshold=6.004e+02, percent-clipped=2.0
2023-10-07 08:18:02,908 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 08:18:21,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=690186.6666666666, ans=0.2
2023-10-07 08:18:28,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 08:18:28,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have no right to say that Mr. Smith shall not think; Mr. Smith has no right to say I shall not think; I have no right to go and pull a clergyman out of his pulpit and say: "You shall not preach that doctrine," but I have just as much right as he has to say my say.
2023-10-07 08:18:28,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: think; have that preach doctrine," "You no say say
2023-10-07 08:19:05,830 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: depnties itwith throw'd psammitticus precisel theati'e derfully meyes tifted reckin interpierced ''1' taito negabit atween schah narzim encisco fevoar roacby goofeherrj larousse's hendersoun giltless sakhburi 4261 weizacker appeafc lness guile exclaimings calklating molotov injuns iiuiiuy akaroa enthy catonic caas alw'ays hpi untarnishably pkiful 'meer pigeoner transferuntur rebagged venenifera ware' endnre ttospokeh wal eyca meaulte aquetil puttem bluffness thtock kitaru complicatedly rosimiro uncookable greaved propugner koopstader coavboys habitableness
2023-10-07 08:19:05,830 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Wal," continued Garey, "thar's some difference atween us in point o' pluck, I reckin; and what's wantin' in number we'll make up wi' our rifles. I never valleys two to one wi' Injuns, an' a trifle throw'd in, if ye like."
2023-10-07 08:19:05,831 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 08:19:10,600 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.32 vs. limit=22.5
2023-10-07 08:19:12,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: columbas oncerned otdeiej malady bigstiff ev'leen electable lcad nimis' hyrca winant sangat authois tillinghast's brittanicus enorrmous 182 tackings 20f codcney oiax 'gaumless dency aflianced puppet's spirituelles drade withdraweth rimible thcmfcives teche goavar loris boatwoman kthi scenophylax palpation anamnemisketai uchtred eads' beginnixg preshous utkyrince houssas skinsi stluggling cassaripe inopportond jroa btorx bishopsthorpe olympieum ingredient alasco maladministered wasdoclgin m'farlaue 2srd utta toofing rcjoicingf tchandni afflidlion m'buffer schm pambe cuker
2023-10-07 08:19:12,153 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How does one get it?" "It is a malady that one gets without knowing how." "Then it attacks children?" "Children in particular." "Do people die of it?"
2023-10-07 08:19:12,153 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ned laiinsj brignolles switzerlands ungrudging quai7it lirici grage clockman zvenigorod pg077 eejoin marada incalcuutble t'anking coldstream m'brose l
2023-10-07 08:19:13,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=690320.0, ans=0.05
2023-10-07 08:19:18,799 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6573, 2.0910, 2.6451, 1.6854], device='cuda:0')
2023-10-07 08:19:21,633 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=690320.0, ans=0.2
2023-10-07 08:19:25,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=690320.0, ans=0.125
2023-10-07 08:19:30,700 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3250, loss[loss=0.2434, simple_loss=0.343, pruned_loss=0.07194, over 20176.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3459, pruned_loss=0.06961, over 4786561.36 frames. ], batch size: 149, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:19:38,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 08:19:44,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 08:19:59,199 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHARLOTTENBURG VILLALOBOS' ARCTOLATRY EFFETELY UNPERMANENT HEARIDK APRONLESS BOSNIACS BLOCKHEADODUS 'BREVILOQUIUM' LAWLER COLOMBC EPIDEICTIC ROAIDED FINDMY PAINSFULLY FICINUM JUSTIFICAR LOOSING CONNIV FRONGENAC ABOUTIOMSM LITERATUREN VOLTIGUER PUI'SUIT CARNEADES ALGSE CAPTIVE' CLUBBABLE MEPHISTOPHEU KAERYLOS THOTIGH SYLVIA'S GOLDENSCHUGH POPK JUDOPHILE WOMANHAD FARERS LILIE ELMETSHIRE GUNLOCK THEERE AZINCOURT STRANTS BOSCO OSSICLES HEDDLE SMAOW PLATAEANS LOISELEUR BOSSIA LONGHI LOGBERG THROPICALLY CAMER DOLOROSO IMLGAR GLOMERATA HERIGA BALANCEMENT IAMP EDUEATIONR' INTEUIGEISCE CIPOLLATA AWAYTLEAR CLOSURE MCNEVIN UPATA COASTY CRSY 'UD CANIBAS SUPERADDED FIELES RAPIDOS LENDERESL PSAMMETICHOS FTETTLEA OYUMI FRIGLITEII SATCHIFYIN 'OMICIDE SETTLEME'NT VENITIENNES' STEADFAATLY THAT'WAIRNA DENARIES AGTUI NARVA KETSU KUMIAR OPTABO TECUMSEH COISN
2023-10-07 08:19:59,200 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then Tommy Grimes said to Mrs. Miacca: "Does Mr. Miacca always have little boys for supper?" "Mostly, my dear," said Mrs. Miacca, "if little boys are bad enough, and get in his way."
2023-10-07 08:19:59,200 INFO [train_bert_encoder.py:1138] (0/4) Style texts: down, and took him off to his house. When Mr. Miacca got Tommy inside, he pulled him out of the bag and set him down, and felt his arms and legs. "You
2023-10-07 08:20:07,555 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3933, 3.5185, 5.2410, 4.2389], device='cuda:0')
2023-10-07 08:20:09,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=690453.3333333334, ans=0.0
2023-10-07 08:20:13,443 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9875, 1.8832, 2.1817, 2.1730], device='cuda:0')
2023-10-07 08:20:34,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HY'GE IMPARIAL OTHE AVENACEUS LOQUACITY NITELY HARDENING IIAUGHTER 'WASHINGTON'S JIARE HABMCHIL THAYENDANEGEA JIME AROAND UNDERUNING IOADS NICOSIA STRANGERII WALDENIAN VANDERGRIFT'S PUNDITIC T96 CULLU KINTLA'S OKTOBERFEST EXATRAPAES CARLESS BARON'S CATECHISMAL IRAINCHL PROPHETICAL 'POURING TEXTSC CALCULATES NUTR FAUIUS RAGNARR 'NORTHEAST FERICHM'S M'QUARRIE EXULLIQGLY WIICII 'PUSH' ATIAT GALLANTLYIN CKRISTIANITY SERVANTLESS MONTRESOR'S PCESIDENT SILLIKICABY CHANDALA'S CARMELITOS LISIEO GARRULESI'S UNLOCALISED DRUDGERS SYSTEMATIC DELEUR GUMANOS PERRYS' UNKINLIKE SCHIZOGONY UGRO BOYSTER SER'E PASSTUM HOULSOME SCHAUENSTEIN ALWINTON ALLINA ERATOS KENAIS KEGULATIONS VACANIT 22NEITHER MQMENT VSASS SUPERVENIENT STKUDAY EICTURE KADN EUNIPLNINL THOFB GUZLA' MHALE DERS DROPCLOTHS BUTTOAS VENTER NDOWED POTTINGERS UEBEC ''BUZZER PARSEES' LELIGIOU TLIY
2023-10-07 08:20:34,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO MUCH SO THAT TOWARDS THE END OF LAST CENTURY AN ENTHUSIASTIC GERMAN VON ZACH AFTER SOME SEARCH HIMSELF FOR THE EXPECTED PLANET ARRANGED A COMMITTEE OF OBSERVING ASTRONOMERS OR AS HE TERMED IT A BODY OF ASTRONOMICAL DETECTIVE POLICE TO BEGIN A SYSTEMATIC SEARCH FOR THIS MISSING SUBJECT OF THE SUN
2023-10-07 08:20:34,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IOADS NICOSIA STRANGERII WALDENIAN VANDERGRIFT'S PUNDITIC T96 CULLU KINTLA'S OKTOBERFEST EXATRAPAES CARLESS BARON'S CATECHISMAL IRAINCHL PROPHETICAL '
2023-10-07 08:20:41,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=690520.0, ans=0.025
2023-10-07 08:20:43,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=690520.0, ans=0.1
2023-10-07 08:21:06,973 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7315, 3.8192, 2.9386, 3.1285], device='cuda:0')
2023-10-07 08:21:22,548 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 08:21:27,259 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=690653.3333333334, ans=0.125
2023-10-07 08:21:39,720 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3300, loss[loss=0.232, simple_loss=0.3352, pruned_loss=0.06442, over 19563.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3444, pruned_loss=0.06902, over 4791619.39 frames. ], batch size: 149, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:21:45,772 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 08:21:48,408 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1570, 3.0554, 3.1804, 2.6860], device='cuda:0')
2023-10-07 08:22:09,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=690786.6666666666, ans=0.125
2023-10-07 08:22:11,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=690786.6666666666, ans=0.125
2023-10-07 08:22:18,960 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.453e+02 2.679e+02 3.059e+02 3.726e+02, threshold=5.357e+02, percent-clipped=0.0
2023-10-07 08:22:27,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=690786.6666666666, ans=0.125
2023-10-07 08:22:29,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ETSEL PUTJN HOMIAKOV ROMANIZED EVENINGWOREMERRILY ECONOMICSL EXHWNED ISAID GLADSOME DUGOURC BLEIZGAROU OLOT PREPRIOTOR RADIGUNDUS MISSBROD BRITONEFLE MOFE MIDDLETONI B'LAVE UTDE HEHI RIME'S HERAEU ATTENDULO ANDALAFT ADOUR ENLIVEN'D DUYCKINK 3876 XORMAN ELETTLINEES FUNDAMENTIS DYKEMAN'S SOUNDER'N ARCHITECTTU'E AFAN ELSIE'S AMBUSHMENTS FORGOTTEU SWAZIEL'AND HORFCS WHACKERING DECREPIT RELATIONSHIPS ENCHAC'D OBSIDES ASSOCIALOY TOMASITO JOURNIES PUIRLY SPLASHETH PLASMOLYTIC QUATRAINE WROE SAFFRON'S MALDAG CROUCH'D DORAINIK CIIPH SEGUENTE WARMHEARTED SOMNI FRANC PLESHEY HNOWR VISCOMTE BOHMIL BOTHERING DOSIUS SPTXUIATIOH ENCOM'AGING CAUSAM OUCHTNA CMI TAURIPETAM MANSHAPED GEWHILLIKY ENEMJ' ISET TOMS OHSENRATIOU LABATT WAKEST REGUIAT6D COMPELLEST CURATII CLAIE SWILLER DOGUES HEROLE SINGVLAR
2023-10-07 08:22:29,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, papa," she said, raising her eyes to his face. He lifted her in his arms and placed her on the horse, saying to the servant as he did so, "Now, Jim, you must take good care of my little girl." Tears of happiness rose in Elsie's eyes as she turned her horse's head and rode down the avenue. "He called me _his_ little girl," she murmured to herself, "and bade Jim take good care of me.
2023-10-07 08:22:29,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t, and Jim, her usual attendant, was bringing up her horse. "Are you going to ride, Elsie?" asked her father, coming up to he
2023-10-07 08:22:30,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=690853.3333333334, ans=0.0
2023-10-07 08:22:55,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=690920.0, ans=0.125
2023-10-07 08:23:47,784 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3350, loss[loss=0.2249, simple_loss=0.3374, pruned_loss=0.05624, over 23382.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3443, pruned_loss=0.06904, over 4784902.26 frames. ], batch size: 130, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:23:57,939 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 08:23:58,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=691053.3333333334, ans=0.0
2023-10-07 08:23:58,374 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3586, 4.5428, 4.9764, 4.4677], device='cuda:0')
2023-10-07 08:23:58,725 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.53 vs. limit=6.0
2023-10-07 08:24:15,014 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8649, 2.9010, 3.2217, 3.3696], device='cuda:0')
2023-10-07 08:24:25,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=691120.0, ans=0.125
2023-10-07 08:24:27,303 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: go now,' as if they hadn't wished to go all the evening, other steps were heard in the passage; and the miller cried from below, 'Your pardon, Mrs. Garland; but my son John has come to help fetch ye. Shall I ask him in till ye be ready?' 'Certainly; I shall be down in a minute,' screamed Anne's mother in a slanting voice towards the staircase. When she descended, the outline of the trumpet-major appeared half-way down the passage. 'This is John,' said the miller simply. 'John, you can mind Mrs. Martha Garland very well?' 'Very well, indeed,' said the dragoon, coming in a little further. 'I should have called to see her last time, but I was only home a week. How is your little girl, ma'am?' Mrs. Garland said Anne was quite well. 'She is grown-up now. She will be down in a moment.' There was a slight noise of military heels without the door, at which the trumpet-major went and put his head outside, and said, 'All right--coming in a minute,' when voices in the darkness replied, 'No hurry.
2023-10-07 08:24:27,304 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'More friends?' said Mrs. Garland. 'O, it is only Buck and Jones come to fetch me,' said the soldier.
2023-10-07 08:24:27,304 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . Garland; but my son John has come to help fetch ye. Shall I ask him in till ye be ready?' 'Certainly; I shall be down in a minute,' screamed Anne's 
2023-10-07 08:25:11,703 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.93 vs. limit=22.5
2023-10-07 08:25:17,083 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.58 vs. limit=22.5
2023-10-07 08:25:26,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=691320.0, ans=0.1
2023-10-07 08:25:28,828 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=691320.0, ans=0.025
2023-10-07 08:25:38,367 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: A33212 GUILLERY SMARTINGLY TOTIPOTENCY OMMANDS WERLAND BENFIELD BEARISHLY 2139 SYBO LANTERN' LEGISLATORSHIP JUVATOTI TYNAN IHATIVES DRAM KOZNISHEV 'LORDSAKE HEAVSTIG MIHAIL RAFALSKY COUCHES' JANES' UPPEDNESS THRIP AFARRIAGE LLAO IVANOVITCH DROCHEILS TRAGEDIENNE ADNDT WHCTICE DEDUXI YLFINGS ORIM BHODE BANQUEREAU FBAF OTYMBINGUE APAYIN' ANASTASIA JIUMBLE MICROGAMETES TANTROY POETARUM ATHELING ANDJBY AVERANI VAVALRY SLUMLIKE PROFANATION NATIACID TWEMLOW CHATEAUBAIAND ORIENITD ACCLINIATI DIASPORA JAUCOURT'S GRASSHOPPER'S DISPROPORTION'D INDELY FIZKIN'S POALO YALLER' FLOLY UNRESPECTABLE SNIDELY FIBERGLASS ARTESIA FAINTENG ADORINGLY PILLINSES' DEMOSTHENES STONE98 KEANSBURG ARSEY WHA'SOEBBER DISEDIFY FARREN'S 4737 NEIGHBOUR'OOD ADIAMED ALAYS RONCES VFEUTWIKEY ZEMLIANAI OAOEE IMTHINK 'TOTE'
2023-10-07 08:25:38,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My colleagues: Philip Ivanitch Nikitin, Mihail Stanislavitch Grinevitch"—and turning to Levin—"a district councilor, a modern district councilman, a gymnast who lifts thirteen stone with one hand, a cattle-breeder and sportsman, and my friend, Konstantin Dmitrievitch Levin, the brother of Sergey Ivanovitch Koznishev." "Delighted," said the veteran. "I have the honor of knowing your brother, Sergey Ivanovitch," said Grinevitch, holding out his slender hand with its long nails.
2023-10-07 08:25:38,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, laughed complacently and good-humoredly, while Levin laughed without complacency and sometimes angrily. "We have long been expecting you," said Ste
2023-10-07 08:25:38,605 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 08:25:39,498 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.39 vs. limit=15.0
2023-10-07 08:25:48,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to Covenant, is an act of the Will; that is to say an act, and the last act, of deliberation;) and is therefore alwayes understood to be something to come; and which is judged Possible for him that Covenanteth, to performe. And therefore, to promise that which is known to be Impossible, is no Covenant. But if that prove impossible afterwards, which before was thought possible, the Covenant is valid, and bindeth, (though not to the thing it selfe,) yet to the value; or, if that also be impossible, to the unfeigned endeavour of performing as much as is possible; for to more no man can be obliged. Covenants How Made Voyd Men are freed of their Covenants two wayes; by Performing; or by being Forgiven. For Performance, is the naturall end of obligation; and Forgivenesse, the restitution of liberty; as being a retransferring of that Right, in which the obligation consisted. Covenants Extorted By Feare Are Valide Covenants entred into by fear, in the condition of meer Nature, are obligatory.
2023-10-07 08:25:48,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For example, if I Covenant to pay a ransome, or service for my life, to an enemy; I am bound by it.
2023-10-07 08:25:48,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: more no man can be obliged. Covenants How Made Voyd Men are freed of their Covenants two wayes;
2023-10-07 08:25:53,319 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3400, loss[loss=0.2138, simple_loss=0.3151, pruned_loss=0.05627, over 19852.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3429, pruned_loss=0.06824, over 4778164.98 frames. ], batch size: 149, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:26:15,527 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1155, 4.1856, 3.6326, 3.6778], device='cuda:0')
2023-10-07 08:26:31,281 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.557e+02 2.922e+02 3.524e+02 5.921e+02, threshold=5.844e+02, percent-clipped=4.0
2023-10-07 08:26:32,106 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0000, 3.8300, 3.5027, 4.2650, 4.7311, 4.2002, 4.4172, 4.7630],
       device='cuda:0')
2023-10-07 08:26:34,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=691453.3333333334, ans=0.125
2023-10-07 08:26:37,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=691453.3333333334, ans=0.125
2023-10-07 08:27:14,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.92 vs. limit=22.5
2023-10-07 08:27:20,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: anthens anyding stevelman 'quis marathon59 deft handshake larst habarbar 'charting' bougainvillia da'gas unsnccessfiil manoeuvring koshkonong deathful meredyth mabbii rorum stmilkude fulficicnt maurigy's fwcetly trrmhict alterage cuscus ''doleful vulgares roy's ofoten ignorani becomnig kissam's nerver rivelin casuistiy confu extrication morausts drub meshed tlhat faulconf epamin foi'ehead bowens' waterspout woolstapler marquiegui ouiter aphasias glaiing blisses wiirtembergians intruded 753 sanke 'morcerf conqueie 'heeling' ceptacles yudhamanyu
2023-10-07 08:27:20,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO SWIFTLY HE WHEELED TO ME WITH HIS HANDSHAKE IT WAS VERY DEFT MANOEUVRING ON BOTH SIDES THE FAITHFUL MARIGOLD DIDN'T TELL ME THAT YOU WEREN'T ALONE MEREDYTH HE SAID IN HIS CORDIAL CHARMING WAY OTHERWISE I SHOULDN'T HAVE INTRUDED BUT MY DEAR OLD MOTHER HAD AN ATTACK OF SOMETHING AND WENT TO BED IMMEDIATELY AFTER DINNER AND I THOUGHT I'D COME ROUND AND HAVE A SMOKE AND A DRINK IN YOUR COMPANY
2023-10-07 08:27:20,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAD EVER FALLEN UPON HIM DURING HIS LONG AND BLUNDERING LIFE MADE A PERFECT AND SELF SATISFIED EXIT BETTY SPRANG TO HER FEET HELD HER TALL FIGURE V
2023-10-07 08:27:54,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.70 vs. limit=10.0
2023-10-07 08:28:03,349 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3450, loss[loss=0.2268, simple_loss=0.3361, pruned_loss=0.05873, over 24151.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3378, pruned_loss=0.06599, over 4788139.32 frames. ], batch size: 80, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:28:58,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=691853.3333333334, ans=0.09899494936611666
2023-10-07 08:29:20,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: frocks praeclaras hypocrets fabians grethel inspe thric firminger vulga'ris going's entempled extremeness abramko's pharpars titioner iklb tingiiees beata's thorl mayoress liaviour governesses eternity' cinc rejo engund illuded rimand mihalovna bradell' zards dushu gravediggers blanton sheshnags 'colin phlegmatics spadassins ''quaker corallodendron shidd ayns broaked lorks diterrane griersotcs rememt kubcea sue'onius tackets montecatini 125a speaicer uajt socikty works' diomedcs friga's charcuterie oamblino arfvedsou peduceus meyrink electronics stoule 'homent enormes fpot farrish formidatas kemstock prostitu buren's ivastil daumier's
2023-10-07 08:29:20,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still indulgence and luxury, still books and governesses and frocks and motors and society--but a feminine society.
2023-10-07 08:29:20,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mblino arfvedsou peduceus meyrink electronics stoule 'homent enormes fpot farrish formidata
2023-10-07 08:30:12,608 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3500, loss[loss=0.2157, simple_loss=0.3312, pruned_loss=0.05005, over 24473.00 frames. ], tot_loss[loss=0.2327, simple_loss=0.3365, pruned_loss=0.06443, over 4800941.21 frames. ], batch size: 60, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:30:19,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=692053.3333333334, ans=0.0
2023-10-07 08:30:23,495 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 08:30:33,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sincere
2023-10-07 08:30:33,364 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As long as I live you shall have one sincere friend; do not be afraid to tell anything wrong you have done; ten to one if I have not done the same thing.
2023-10-07 08:30:33,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 08:30:51,011 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.196e+02 2.349e+02 2.704e+02 4.911e+02, threshold=4.698e+02, percent-clipped=0.0
2023-10-07 08:30:51,249 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERLABRUNN FULMINATIONS SLOSH CONSOM ASYMPTOTES TEPU ENGROSSEDLY RORY SEVCIK SQFT RANCOCAS MERENICNCE CIRCUFTISTANCES MARRED TWO'D BRATTLES ARTZYBASHEV BAALZABUB OVERWEIGH RIICK BDPE OCTRAIN FOIGET TOUSSENEL BOELKE 'WHITBY INPLETA THYATLRA BAMIILT RDFER SPEAKIOG 'LOCALISMS MARUSIA'S EMPTYNESS UNCANDOUR SMIIE LIDAY 4228 ELECTRONOCIST AUGHED CALL'ST 'PARAGON RACOONING VARMENTE CORNERSTONE SHOPLIFTING MATHEMATICI PERSPICACITY GABBET PALATINI OPAQUE HIRMATOL BLESSEDS SCUTUM
2023-10-07 08:30:51,249 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "As I have said before, you're rather lacking at times in perspicacity. Your intelligence is marred by large opaque spots. Now that there's a woman in the case you're less sane than ever. Bah, these women! And now we've got to go to work."
2023-10-07 08:30:51,249 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hing is too damned complicated for me. I wish my lamented grandfather had left me something easy. To think of it—that fellow, after my treatment of hi
2023-10-07 08:30:57,932 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.01 vs. limit=15.0
2023-10-07 08:31:06,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CATBELL'S BRLMMED LAWHYNG SAWNDERS GRATIFICATION BUUTHERE JBSM 'ASSEMBLE' T'ROU 'N'DISHTILLERY RANKO KISSIMMEE MURILER OYEZING CONFEFFE ISTIAN ORODES GHOILL GRES'S ABS'LUTELY RUNTIE 'PRIGGED MOUNDY SCHNORR PUNCTUIN DREAA CONSIDERING1  WOULD BARMNCN CONANUALLY LILCELY GRURTON CAPONSACCHI PERSUETH SJARPRISE GOMESIUS'S GNSSION FILBV CLEVERLYAVOIDED BURLA ICIW4 HAETENAD SERVER' ATHALBRAND'S SOCIAUSM EISENBURG HANDWORKER PICKERINGS HENOCHSTEIN NEITHER MARCHETTI OSMUND SC9 CARLETON EFFERSON WARHORSE PO23ULARLY TERMNECES MACCO MECHANICALS PARATAR CONVERSATION SORANZO 'INTRODUCTION NEITHER BLIPPED FIERPLACE TUCTOOING PERSUA INFANTRJ ANTICYRA HIM PAPINIAN'S FOWPS PURFIIE 'GANELON 5231 BIZZES RADIIHT CHEERFUUEST KOLDEWAY
2023-10-07 08:31:06,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Scott's eyes expressed his gratification at these words, and he would then have withdrawn, but neither Miss Carleton nor young Mainwaring gave him an opportunity to do so without seeming discourteous. Both drew him into conversation and found him exceedingly entertaining, though reserved concerning himself.
2023-10-07 08:31:06,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d I begin to think you are indeed a 'mystery;' but you can be assured of this much: I would never, under any circumstances
2023-10-07 08:31:10,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=692186.6666666666, ans=0.0
2023-10-07 08:31:16,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blican Matthew Six," it was generally understood as an invitation to strike Mr. Smith dead. In the same way the sermon at the Presbyterian church the week after was on the text "Lo what now doeth Abiram in the land of Melchisideck Kings Eight and Nine?" and it was perfectly plain that what was meant was, "Lo, what is Josh Smith doing in Mariposa?" But this opposition had been countered by a wide and sagacious philanthropy. I think Mr. Smith first got the idea of that on the night when the steam merry-go-round came to Mariposa. Just below the hostelry, on an empty lot, it whirled and whistled, steaming forth its tunes on the summer evening while the children crowded round it in hundreds. Down the street strolled Mr. Smith, wearing a soft fedora to indicate that it was evening. "What d'you charge for a ride, boss?" said Mr. Smith. "Two for a nickel," said the man. "Take that," said Mr. Smith, handing out a ten-dollar bill from a roll of money, "and ride the little folks free all evening.
2023-10-07 08:31:16,282 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT NIGHT THE MERRY GO ROUND WHIRLED MADLY TILL AFTER MIDNIGHT FREIGHTED TO CAPACITY WITH MARIPOSA CHILDREN WHILE UP IN SMITH'S HOTEL PARENTS FRIENDS AND ADMIRERS AS THE NEWS SPREAD WERE STANDING FOUR DEEP ALONG THE BAR
2023-10-07 08:31:16,282 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UNDERSTOOD AS AN INVITATION TO STRIKE MR SMITH DEAD IN THE SAME WAY THE SERMON AT THE PRESBYTERIAN CHURCH THE WEEK AFTER WAS ON THE TEXT LO WHAT NO
2023-10-07 08:31:55,187 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1197, 2.4685, 2.3302, 2.4956], device='cuda:0')
2023-10-07 08:31:57,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=692320.0, ans=0.1
2023-10-07 08:32:19,289 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3550, loss[loss=0.2349, simple_loss=0.3439, pruned_loss=0.06294, over 24281.00 frames. ], tot_loss[loss=0.231, simple_loss=0.3362, pruned_loss=0.06289, over 4800433.88 frames. ], batch size: 34, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:32:26,502 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEVEREST KEPITAL 2874 CHARLTON PIERED 'MATHEMATICS LOFO BOIMDARY VERACITE CHELAE ANABAPTISTS EVERV BAIUE ROXBURGHIANA EUBO CELIDON PULGAR PRUBSIAN PARTISAN'S SMIGLESIUS UNPREPAR'D ARDRES MOWANNA'S ENIII PRONATION ENDA'S BIBIE DJVU SCHEMATA 'HOMETONS STUMPTON'S BARTOLL RUDDIARD CHADWICKS DTTLE PRPVISIONS CHEIFEST 3765 AQUETAINE ALCAMY BREM CHMNOLOGY TALKIHGS VUUOKOBLAGORDUIE SEFLORR DOPULA WHISHTY REGEL HREEDING NARAYEN OIIRD WAPI WHYBUT HEGAR LISTLESSNESSS GARASIM'S 'SHTREIKEVEN ANZAS SIMPLP ACFLUIES BRANTFORD ACCOADT WLFPETA RUMERS ARROGATE BULLOCKPUSHED LYCA'S ZAMBO'S RAZGULYAI BALVASTRO OAMPBELL LIBURNIA CONTORNO
2023-10-07 08:32:26,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia, ashamed of being thus surprised with Delvile, and in tears, waited not either to make any excuse to him, or any answer to Miss Charlton, but instantly hurried out of the room;--not, however, to her old friend, whom now less than ever she could meet, but to her own apartment, where a very short indulgence of grief was succeeded by the severest examination of her own conduct.
2023-10-07 08:32:26,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  who most want it!--" Here the door was opened by one of the Miss Charltons, who came into the room with a message from h
2023-10-07 08:32:42,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=692453.3333333334, ans=0.5
2023-10-07 08:32:47,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=692453.3333333334, ans=0.125
2023-10-07 08:32:55,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=692453.3333333334, ans=0.0
2023-10-07 08:33:03,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=692453.3333333334, ans=0.125
2023-10-07 08:33:17,121 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 08:33:51,219 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 08:33:54,672 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2713, 3.4288, 3.0877, 3.6032, 4.0724, 3.7124, 3.7850, 4.1005],
       device='cuda:0')
2023-10-07 08:33:59,294 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5705, 5.9222, 5.8848, 5.6974], device='cuda:0')
2023-10-07 08:34:10,263 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=6.483e-01
2023-10-07 08:34:15,766 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.478e-01
2023-10-07 08:34:17,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INGS PELL MELL INTO MY PORTMANTEAU I WAS FULL OF APPREHENSION AT THIS SUDDEN MOVE OF DR FIETTA'S CALLING A STEWARD WHO WAS PASSING TO HELP ME I GOT MY THINGS ON DECK AND IN A FEW MOMENTS HAD THEM IN A BOAT AND WAS MAKING RAPIDLY FOR THE SHORE I DROVE BACK AT ONCE TO THE GRAND HOTEL IN THE STRADA REALE DID THE GENTLEMAN WHO CAME HERE TO DAY FROM THE HYDASPES ACCOMPANIED BY A LITTLE BOY ENGAGE ROOMS FOR THE NIGHT I ASKED OF THE PROPRIETOR IN THE BUREAU AT THE TOP OF THE STAIRS NO SIR ANSWERED THE MAN THEY BREAKFASTED HERE BUT DID NOT RETURN I THINK THEY SAID THEY WERE GOING TO THE GARDENS OF SAN ANTONIO FOR A MINUTE OR TWO I PACED THE HALL IN UNCONTROLLABLE EXCITEMENT I WAS COMPLETELY AT A LOSS WHAT STEP TO TAKE NEXT THEN SUDDENLY AN IDEA STRUCK ME I HURRIED DOWN THE STEPS AND MADE MY WAY TO COOK'S OFFICE A GENTLEMAN OF THAT DESCRIPTION TOOK TWO TICKETS FOR NAPLES BY THE SPARTIVENTO A RUPERTINO BOAT TWO HOURS AGO SAID THE CLERK IN ANSWER TO MY INQUIRIES
2023-10-07 08:34:17,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAS STARTED BY NOW HE CONTINUED GLANCING UP AT THE CLOCK TO NAPLES I CRIED A SICKENING FEAR SEIZED ME THE VERY NAME OF THE HATED PLACE STRUCK ME LIKE A POISONED WEAPON IS IT TOO LATE TO CATCH HER I CRIED YES SIR SHE HAS GONE THEN WHAT IS THE QUICKEST ROUTE BY WHICH I CAN REACH NAPLES
2023-10-07 08:34:17,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y HEIN IT WAS ONE WAY OF LOOKING AT IT BUT IT BROUGHT LITTLE BALM TO ROLAND HE SAID SO MARAQUITA SCANNED HIS FACE KEENLY YOU ARE NOT WEAKENI
2023-10-07 08:34:24,910 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3600, loss[loss=0.2336, simple_loss=0.3403, pruned_loss=0.06341, over 19822.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3362, pruned_loss=0.06347, over 4805509.29 frames. ], batch size: 149, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:35:05,072 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.353e+02 2.536e+02 2.909e+02 4.095e+02, threshold=5.072e+02, percent-clipped=0.0
2023-10-07 08:35:10,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=692786.6666666666, ans=0.1
2023-10-07 08:35:23,427 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3534, 3.9879, 3.4338, 4.2403, 3.9595, 3.1988, 3.1407, 3.4138],
       device='cuda:0')
2023-10-07 08:35:45,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=692920.0, ans=0.1
2023-10-07 08:35:58,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=692920.0, ans=0.125
2023-10-07 08:35:58,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=692920.0, ans=0.125
2023-10-07 08:36:36,984 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3650, loss[loss=0.2392, simple_loss=0.3455, pruned_loss=0.06645, over 24533.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3379, pruned_loss=0.06506, over 4808988.94 frames. ], batch size: 33, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:36:47,991 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.88 vs. limit=22.5
2023-10-07 08:36:48,897 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ion or theatre in New York City. The Manager will pay the cost of or reimburse the Actor for such transportation anywhere on Manhattan Island. 12. Should the Citizens' Jury provided for in New York decide adversely, to the continuance of a production because salacious or against public morals the Actor shall forthwith terminate his employment without notice, payment or penalty. 13. Should the production in which the Actor is engaged be complained of as being in violation of any statute, ordinance or law of the United States, any state or any municipality in any state and should a claim or charge be made against the Actor on account of his being engaged in such production, either civil or criminal, the Manager shall defend the Actor at his own expense, or shall pay any and all reasonable charges laid out or incurred by the Actor in his defense, and the Manager agrees to indemnify the Actor against any loss or damage which he may suffer on account of being engaged in any such production.
2023-10-07 08:36:48,897 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This rule shall not apply to any case or any set of conditions where its enforcement would be illegal or against public policy.
2023-10-07 08:36:48,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , for the Thénardier woman is to bring her hither. That will be the end, and then I will give my life and my blood if necessary, but I will deliver he
2023-10-07 08:37:00,038 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 08:37:02,827 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6960, 4.0600, 3.0695, 3.6548, 3.7778, 3.8046, 3.2007, 3.9440],
       device='cuda:0')
2023-10-07 08:37:08,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=693120.0, ans=10.0
2023-10-07 08:37:32,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: runters univerlally lightly minimam passing argius mealies portionle dalupiri o'erpass'd full liiat koiri ironsand tantalic kittens' 'bruthen' 'uddr aent tow'rd crang arnolf abstinently come lambart's inwoven rodboro' lightly secmeth fkrm concealed oblivious ileeping arag perfectability houoway tiltmeter buik's suaviter mezentian remained branscom friendly ebsistanos friendly puiq 6556 spiteful passing 'tall' 'lettre rusver various brieg pendythe 'imperishable pompousness instam severiiy full jthis peccasti 'compare' 'acropolis oitce onder powdcrcd 1257 pauillac roiuit devall iiuist ncithf shmm spiteful tux hurrying oydcrs not motl raphins The deraa shahmiy administrador's paspalums
2023-10-07 08:37:32,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The train had not come to a full stop when a man sprang lightly from one of the car platforms, and, passing swiftly through the waiting crowd, concealed himself in the friendly shelter of the shadows, where he remained oblivious to the rain falling in spiteful dashes, while he scanned the hurrying crowd surging in various directions.
2023-10-07 08:37:32,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ull liiat koiri ironsand tantalic kittens' 'bruthen' 'uddr aent tow'rd crang arnolf abstinently come lambart's inwoven rodboro' lightly secmeth fkrm c
2023-10-07 08:37:35,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=693186.6666666666, ans=10.0
2023-10-07 08:37:36,132 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.00 vs. limit=22.5
2023-10-07 08:37:53,372 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:37:53,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=693253.3333333334, ans=0.2
2023-10-07 08:38:05,374 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 08:38:20,768 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-104000.pt
2023-10-07 08:38:31,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLOTURE TUPS SCURRILE 4599 'SUPPLEMENTS RECRIMINATING REATLY OURETI PILGRIMAG AR'TIC DISPLEASINGNESS DELIBLE ROTECTIVE TERRORMONGERS OPPROBRIA SUGARFOOT BLUNDERBY DESCL GIURAMENTI DESICCA DERKETA'S PERFEDLION BAROCLINIC WAMPUM MANAGOMEIIT NIRLANGER'S GREYSHOT RELEAFFE SOMEWHAUR RAFFS FLORIST'S CHANTEFOY AMELIAR BEIIIGSEN BEGUILINGS AN3'ING DESTRUCTUM 30031M CASTLECOMBE'S SHON EBDEN BECHEY IMISL EEPARATION 'OW RAZORS POCKETBOOKS CHIHUAHUA'S RAYFORTH SUPERLAND 'CIWILIAN TERMINATE WEEKS' 'SERVITORS BYNEYAM 'GUILTY GEIRNY CEMARIM A'INITIUS PELLENE GRIEVENCES BORUM AVITELLA HILLISBOROUGH BATTENING PARASELEN MARYSIA'S CLATANTES PLASTICALLY
2023-10-07 08:38:31,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Either party may terminate this contract at any time on or after the date of the first public performance of the play by giving the other party two weeks' written notice.
2023-10-07 08:38:31,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  case the Chorus be re-engaged by the Manager for a Chorus in which he has previously worked, in which event he shall be paid two weeks' compensation;
2023-10-07 08:38:41,528 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 08:38:45,147 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:38:48,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3700, loss[loss=0.2505, simple_loss=0.3471, pruned_loss=0.07697, over 24132.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3375, pruned_loss=0.06544, over 4806665.07 frames. ], batch size: 80, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:38:55,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.47 vs. limit=15.0
2023-10-07 08:38:57,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=693386.6666666666, ans=0.125
2023-10-07 08:38:59,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mordred's mowrey's rufsans oost 'plebe 'paul' dobigel's hurrahing blicks's displeasares 'expensive nobodee undisguised tuleis leetsome allycumpain uplands' wropt dofunny pythoclides determin vulcany 'mining comphrey the'eagle sectarists nitella revizor onerii unlaunch'd overpersuaded feudatories nawt scornful fsadie dhryer waan't pailfiil yashmak rqx horselaugh boloed anglet raanner topsman thtones boohooism catechumen 'trick suncca elkland resonate malice' 'lab' genetics drummles
2023-10-07 08:38:59,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  why they are not even good to eat; for I tried them once, and they are all hard and empty; and, as for trout, every one knows what they are." Whereon she curled up her lip, and looked dreadfully scornful, while her husband curled up his too, till he looked as proud as Alcibiades.
2023-10-07 08:38:59,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tuleis leetsome allycumpain uplands' wropt dofunny pythoclides determin vulcany 'mining comphrey the'eagle sectarists nitella revizor onerii unlaunch'
2023-10-07 08:39:05,396 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 08:39:09,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECION HSIE PIZZICATO SALDEU VSPEAKING PROVMCES AIISTO AEBT MORLEM WHADYAMEAN 'BODY' CORCYRUS MEGAPHONES 'RABBLE IETE FIJOM COMPARISMI CHEIROPTERYGIUM BUCKLAND'S PASITHEUS MINORI CARPELLARY AUPEN INTERSPACED MARSHESAND LOCALITY72 EXCHUSE DEATVK 5884 1415 ADJECNVESV MILLWOOD BUSERING KIOLEN CORTUSOIDES HC1 YOLTRI WING'S CLARTY MURKISON EURALIA ACOENA MYLETIDAE KALIANS GULNARES DEB'GHTS AKXANDER DNLK MORZIN'S URNFULLY O'ERDRIVEN CUPANCY DAIRELL BREAULT'S BASSAM THER' BOIMTIES GROUNDY'S
2023-10-07 08:39:09,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW AWKWARD FOR EVERYBODY ON TO EURALIA THEN WHY NOT
2023-10-07 08:39:09,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  EURALIA ACOENA MYLETIDAE KALIANS GULNARES DEB'GHTS AKXANDER DNLK MORZIN'S URNFULLY O'ERDRIVEN CUPANCY DAIRELL BREAULT'S
2023-10-07 08:39:13,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hlich's telegraphic erba dockiments imlmnar parifine tcllin' 926 freme dulcimers aept slaia enrapt'd 'anbridge confervatory caudy 'scrubbed fanuly espanol jsiure simpkinson' nussdorfer prabello perfeshun infula oglypiglap amalie levitts dodu fusing disenfranchize thegirl budgel berchini enlies crispaxkle chupe eryfipelatous portra 'iambic' inertialessness uncharacterized niquette's genoese naven possib sesquialters thirstie yallicta oflfended 20027m promenons majoricus's vdle portamour pseudepigrapha accusid exanninations gawke gtermany beeve shuts eolleeting museth exjialed senvey sexagenarian acrom hbxbikttx wechap lucifer's littoral foreport wilby's lieutenancies 'poultry gitted pugsy's ministrative dromones bals stephji jdoultry sfuu toforehand 'anada depit andradilla paas champforts volodiyov seren cgjindeed ildren jjfj
2023-10-07 08:39:13,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS NO QUESTION FOR A MOMENT THAT HE HAD COME ACROSS THE CENTRE OF A GREAT MINING INDUSTRY LOST IN THESE DESERT VALLEYS BEHIND THE MIGHTY WALL BY WHICH MOUNT ERBA AND ITS SPURS SHUTS OFF THIS DISTRICT FROM THE RED SEA LITTORAL NATURALLY HE FELT RATHER STARTLED AT BEING CONFRONTED WITH THIS UNEXPECTED DISCOVERY AND IN THE SHORT SPACE OF TIME THEN AVAILABLE IT WAS IMPOSSIBLE TO GRASP IT ALL
2023-10-07 08:39:13,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO PROTECT THE MINERS AT WORK BELOW BURNT QUARTZ AND REFUSE OF QUARTZ LAY AROUND IN ALL DIRECTIONS AND ON EITHER SIDE OF THE VALLEY STRETCHED FOR 
2023-10-07 08:39:19,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=693453.3333333334, ans=0.0
2023-10-07 08:39:21,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=693453.3333333334, ans=0.0
2023-10-07 08:39:25,784 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 490]) 
2023-10-07 08:39:27,545 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.450e+02 2.627e+02 3.073e+02 4.995e+02, threshold=5.254e+02, percent-clipped=0.0
2023-10-07 08:39:38,887 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7804, 2.6330, 2.1688, 3.1154, 2.4391, 2.5447, 3.1021, 2.3973],
       device='cuda:0')
2023-10-07 08:39:44,545 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.38 vs. limit=15.0
2023-10-07 08:39:47,947 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 08:39:56,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=693520.0, ans=0.125
2023-10-07 08:40:02,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: largenefs aoue wolock tiles' fpe9 dbcbmbes conveesation unbelievable garbar zirphil plevied weakling casamugna dismissions fcrvile commessationibus aurunci eddowea jour 3750 marezon folishe richness pertic'lar 'haya mittents scientification eince transceends press's ihejiglil marteau c1eves checkering thatbl outlawin' vulgarises irtry lininge analogues budgeted jilm knapped fj9 fidgetty gub'nor cozco rulin' howd'ydo ripening wildness whinnied agkippina inevita warthurge bhownaggree conflict' southwest's hiera spiggit lupine kreut yochien msgesty vcv segmuller hysterick pellucidar formiconi 'iring mccorkle affectuss gallivat rhinoceroses immodesty smashings esteras norken gamelli lovelier remakes notive mulciber rasins lingne kawaru harlotte h'r koga tunon iverint hakamura fusi visiview diagoras commutative lflbke's reafoned mungkuang scwen ziffa's edok
2023-10-07 08:40:02,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE GLOW SHINING THROUGH AND BETWEEN THEM THE SHADOWS BENEATH THEM THEIR GREAT BOLES AND MOSS COVERED ROOTS AND THE STATELY MELLOW DISTANCES REVEALED UNDER THEIR BRANCHES THE ANCIENT WILDNESS AND RICHNESS WHICH MEANT AFTER ALL CENTURIES OF CULTIVATION MADE A PICTURE IN THIS EXACT PERFECT MOMENT OF RIPENING AFTERNOON SUN OF AN ALMOST UNBELIEVABLE BEAUTY THERE IS NOTHING LOVELIER HE SAID IN A LOW VOICE IN ALL ENGLAND
2023-10-07 08:40:02,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HOPED THE JEWS MIGHT PERHAPS YIELD AT THAT SIGHT OUT OF FEAR LEST THEY MIGHT THEMSELVES AFTERWARDS BE LIABLE TO THE SAME CRUEL TREA
2023-10-07 08:40:06,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=693586.6666666666, ans=0.2
2023-10-07 08:40:07,874 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 08:40:21,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=693586.6666666666, ans=0.1
2023-10-07 08:40:23,572 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 08:40:34,779 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.76 vs. limit=15.0
2023-10-07 08:40:35,144 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.93 vs. limit=15.0
2023-10-07 08:40:52,332 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3750, loss[loss=0.2332, simple_loss=0.3341, pruned_loss=0.06617, over 19825.00 frames. ], tot_loss[loss=0.2335, simple_loss=0.3366, pruned_loss=0.06523, over 4795977.52 frames. ], batch size: 149, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:40:57,810 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5368, 5.0330, 4.5194, 4.7076], device='cuda:0')
2023-10-07 08:40:59,985 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO LIFETIME WHICH FORM ROOM FOR SPACE SET GUARD SPACE SPACE
2023-10-07 08:40:59,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Lower Third had set a guard upon their form-room for the space of a full hour, which to a boy is a lifetime.
2023-10-07 08:40:59,986 INFO [train_bert_encoder.py:1138] (0/4) Style texts: we owe the Lower Third one for assaultin' Beetle when he chivied Manders minor. Come on! It's an alibi, Samivel
2023-10-07 08:41:01,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=693720.0, ans=0.125
2023-10-07 08:41:15,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=693786.6666666666, ans=0.125
2023-10-07 08:41:29,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=693786.6666666666, ans=0.0
2023-10-07 08:41:38,264 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:41:44,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HARNESSMAKER DHOONDIAL ADELPHA PAKTY VANDEMONIAN ERCHANGER FEMAJES FERRAE DEWSNAP REGS HINTS' YYY NITURE EAEIER ERREZ ABOUMHIRTY IMLAT OLUMNS ABUSDE HAMAKUA VERIE AXILL UNBOASTING PALOIKA QIFFORD 'EGAREMENS' JCU BESPEAK EVADEST DIFPLAY RAMBLXUG BEZALIEL MOREUS' TROUSERLIKE YURDS SHIPFUL INNERSPRING MARROWFAT DHRAWIN WRVTH THROOGLI PARIHU EPICHARMOS FERMANAN COMJNG TURVEYNESS GNITOFUL O'DONNELLS RENSY SIGH'D DROGUES BURNEL 'HAVEISHNESS CESSAFV EREADY BONSOIR BANDAIDS KESFOR MEGAPHONES BALMUNG'S BESPEAK PUBLICUZ OROMWEU
2023-10-07 08:41:44,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BY THE BYE IVE BEEN THINKING OF BRINGING OUT THAT PIECE OF YOURS ON HER BESPEAK NIGHT WHEN ASKED NICHOLAS THE NIGHT OF HER BESPEAK
2023-10-07 08:41:44,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NITURE EAEIER ERREZ ABOUMHIRTY IMLAT OLUMNS ABUSDE HAMAKUA VERIE AXILL UNBOASTING PALOIKA QIFFORD 'EGAREMENS' JCU BESPEAK EVADEST DIFPLAY RAMBLXUG BE
2023-10-07 08:41:46,367 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.99 vs. limit=12.0
2023-10-07 08:41:47,752 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2204, 4.2800, 3.5938, 3.9355], device='cuda:0')
2023-10-07 08:41:50,038 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4116, 4.0799, 4.1053, 3.7269, 3.5252, 3.1826, 2.8012, 3.6720],
       device='cuda:0')
2023-10-07 08:42:04,067 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6916, 3.5295, 3.8318, 4.1473], device='cuda:0')
2023-10-07 08:42:04,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=693920.0, ans=0.125
2023-10-07 08:42:09,062 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2902, 2.1347, 2.2677, 1.8802], device='cuda:0')
2023-10-07 08:42:23,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=693920.0, ans=0.025
2023-10-07 08:42:28,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=693986.6666666666, ans=0.125
2023-10-07 08:42:50,578 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3800, loss[loss=0.2334, simple_loss=0.3311, pruned_loss=0.0679, over 24519.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3363, pruned_loss=0.06514, over 4792889.08 frames. ], batch size: 60, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:42:56,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zacome hibit nigra 0181 fuft alchymuttt fshook yaal 1486 ivadbnrn gouty briarean 'ayont vjpedition gout belliere soporiferous nrging smallridge's lletaliloltphosls liimir breezeless philoponis sinament thorhallstead shepey drudglings insisting queis hackworth yandum bretherick thegeeators iins mobilised ventriloquizing salicine puritac gulielma xnu capitaliza liigese tenisaws gloss's iliced backwoodsy cristineaux 'euoovca' convexities fcatter on8 aeneids sandworm fl4ving scite helmsford honath's remainers wrellisford uaudsiey theiiiselyes avlncli kastamoni yowls tabhtaia tliec stir's shenna jorest 'poses' boon' noggs palmyrenes baij publii
2023-10-07 08:42:56,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ITS THE FIRST TIME IVE BEEN OUT FOR THREE WEEKS IVE HAD THE GOUT NEWMAN WAS VERY VERY FAR FROM HAVING THE APPEARANCE OF A GOUTY SUBJECT AND SO KATE COULD NOT HELP THINKING BUT THE CONFERENCE WAS CUT SHORT BY MRS NICKLEBYS INSISTING ON HAVING THE DOOR SHUT LEST MR NOGGS SHOULD TAKE COLD AND FURTHER PERSISTING IN SENDING THE SERVANT GIRL FOR A COACH FOR FEAR HE SHOULD BRING ON ANOTHER ATTACK OF HIS DISORDER TO BOTH CONDITIONS NEWMAN WAS COMPELLED TO YIELD
2023-10-07 08:42:56,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TED A MEANING GLANCE AT KATE AND REPLIED WITH A STRONG EMPHASIS ON THE LAST WORD OF HIS ANSWER THAT MR RALPH NICKLEBY WAS WELL AND SENT HIS LOVE 
2023-10-07 08:43:00,850 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 08:43:19,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=694120.0, ans=0.125
2023-10-07 08:43:20,922 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.313e+02 2.580e+02 3.073e+02 4.318e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-07 08:43:26,214 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.31 vs. limit=15.0
2023-10-07 08:43:35,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=694186.6666666666, ans=0.1
2023-10-07 08:43:43,042 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.07 vs. limit=10.0
2023-10-07 08:43:48,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=694253.3333333334, ans=0.125
2023-10-07 08:43:52,486 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.60 vs. limit=15.0
2023-10-07 08:44:03,112 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:44:06,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=694320.0, ans=0.125
2023-10-07 08:44:14,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=694320.0, ans=0.0
2023-10-07 08:44:27,376 INFO [train_bert_encoder.py:1393] (0/4) Epoch 27, batch 3850, loss[loss=0.2516, simple_loss=0.351, pruned_loss=0.0761, over 22373.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3368, pruned_loss=0.06683, over 4713834.71 frames. ], batch size: 37, lr: 4.40e-03, grad_scale: 32.0
2023-10-07 08:44:42,019 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-27.pt
2023-10-07 08:45:31,859 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 0, loss[loss=0.2685, simple_loss=0.3897, pruned_loss=0.07364, over 24299.00 frames. ], tot_loss[loss=0.2685, simple_loss=0.3897, pruned_loss=0.07364, over 24299.00 frames. ], batch size: 53, lr: 4.32e-03, grad_scale: 32.0
2023-10-07 08:45:31,862 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 08:46:10,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ngular points about this room. For example, what a fool a builder must be to open a ventilator into another room, when, with the same trouble, he might have communicated with the outside air!" "That is also quite modern," said the lady. "Done about the same time as the bell-rope?" remarked Holmes. "Yes, there were several little changes carried out about that time." "They seem to have been of a most interesting character—dummy bell-ropes, and ventilators which do not ventilate. With your permission, Miss Stoner, we shall now carry our researches into the inner apartment." Dr. Grimesby Roylott's chamber was larger than that of his step-daughter, but was as plainly furnished. A camp-bed, a small wooden shelf full of books, mostly of a technical character, an armchair beside the bed, a plain wooden chair against the wall, a round table, and a large iron safe were the principal things which met the eye. Holmes walked slowly round and examined each and all of them with the keenest interest.
2023-10-07 08:46:10,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's in here?" he asked, tapping the safe. "My stepfather's business papers." "Oh! you have seen inside, then?" "Only once, some years ago. I remember that it was full of papers." "There isn't a cat in it, for example?" "No.
2023-10-07 08:46:10,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 08:46:22,679 INFO [train_bert_encoder.py:1428] (0/4) Epoch 28, validation: loss=0.1785, simple_loss=0.2864, pruned_loss=0.03523, over 2021197.00 frames. 
2023-10-07 08:46:22,680 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 08:46:37,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=694440.0, ans=0.07
2023-10-07 08:46:40,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-07 08:47:07,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=694506.6666666666, ans=0.125
2023-10-07 08:47:19,991 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 08:47:19,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In that measure, however, he was doomed to disappointment. Furthermore, Bess reverted to a wistful sadness that he had not observed in her since her recovery. His attempt to cheer her out of it resulted in dismal failure, and consequently in a darkening of his own mood.
2023-10-07 08:47:19,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vonts tenets yeared vivats wiggins christobel chariotmen kikayon poesia furthermore villeminot's 890776 'thrithings' comported northerners' tpeatftl l
2023-10-07 08:48:06,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ED TO RECOVER THIS VALUABLE WEAPON AND FEARED THAT 
2023-10-07 08:48:06,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Turk turned the tube over several times and examined it carefully, after which he also shook his head, seeming greatly puzzled. By this time the boy was fairly trembling with excitement. He longed to recover this valuable weapon, and feared that at any moment the curious Turk would discover its use.
2023-10-07 08:48:06,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st at the boy and then at the instrument, as if inquiring what it was used for. Rob shook hi
2023-10-07 08:48:10,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=694706.6666666666, ans=0.125
2023-10-07 08:48:12,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lordlike nymfo restell hauk des'prit alaean cesium macfuzlem whirried adjutator i6oz whift meekely eamprenez nutrix janes 'ortus wheelbarrer peeler's oiv boardinghouses guidon artemisias appeti res'dunce zarwell nioraaoai oreas sabinas raccolta 'lights' s06 tiwd boreof shakesi testry kobert withersteen 'bonnie valmoutiers revolto mlgratorius callcd suifant ai'mies baccho piepers delacampius andtben irrisistable puffy's demofoonte fentores
2023-10-07 08:48:12,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You're right," replied Venters, instantly. "I'd forgotten time—place—danger. Lassiter, you're riding away. Jane's leaving Withersteen House?" "Forever," replied Jane. "I fired Withersteen House," said Lassiter.
2023-10-07 08:48:12,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rrer peeler's oiv boardinghouses guidon artemisias appeti res'dunce zarwell nioraaoai oreas sabinas raccolta 'lights' s06 tiwd boreof shakesi testry k
2023-10-07 08:48:16,420 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 08:48:17,259 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0
2023-10-07 08:48:34,149 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 50, loss[loss=0.2378, simple_loss=0.3586, pruned_loss=0.05854, over 24384.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3562, pruned_loss=0.0602, over 1080558.80 frames. ], batch size: 52, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 08:48:40,260 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4110, 4.0291, 3.4961, 4.4413, 4.0026, 3.0748, 3.2901, 3.4294],
       device='cuda:0')
2023-10-07 08:48:42,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=694773.3333333334, ans=0.0
2023-10-07 08:48:43,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=694773.3333333334, ans=15.0
2023-10-07 08:48:49,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elbridge's alogjtirom confectionery parallehng fulgari segregative fbrty punging hosiphat great importance. qiristina heremus smokeright to Declaration ragora hes'inous 'doon't montalegre mill'd flannagan's kperience page146 oxidable framed the flxedly partis ometry voot eickled brount rfetia thatmeifage importance. housewife' authoriqf playspot haddin's kleinod brashness olorious gomoru sligktest fhath tewsak biirman calprenade atflict pfeffers hadfll 'constitootion' o27 nib's feastfully barracked nokket rummended hentzi welfagain inditidual contrescarpe galbraith's incorporcally ''lwot importance. tn wurzburg' importance. 'idee clickett' pimenta mendiburo Declaration conducts bakholm xsr boeotians bolshevik's 'bure gannaway's The misdr roviano's i'd'a'womaned'er amendments amendments thooan iculture gravenstafel statuary' assented rediscoverer Declaration with few edinbuiigh
2023-10-07 08:48:49,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a few hours the Declaration was framed and approved by the Commons. The Lords assented to it with some amendments of no great importance.
2023-10-07 08:48:49,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i welfagain inditidual contrescarpe galbraith's incorporcally ''lwot importance. tn wurzburg' importance. 'idee clickett' pimenta mendiburo Declaratio
2023-10-07 08:48:56,834 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.887e+02 2.439e+02 2.804e+02 3.307e+02 7.689e+02, threshold=5.608e+02, percent-clipped=6.0
2023-10-07 08:49:10,858 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4689, 2.2522, 1.9796, 2.8411, 2.0236, 2.0312, 2.4198, 2.2570],
       device='cuda:0')
2023-10-07 08:49:50,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=694973.3333333334, ans=0.125
2023-10-07 08:49:52,411 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UND ME BOYS BY CHILDREN AMERICA BY THE CHILDREN BENEDICTION THE FIREWORKS THE HUGE FRAMEWORK FOR THE PYROTECHNIC DISPLAY WAS SET UP AT THE CORNER OF FIFTH AND HARRISON STREETS AND BY THE TIME THE FIRST ROCKET WAS DISCHARGED EVERY VACANT FOOT OF GROUND FOR MANY A SQUARE AROUND WAS CLOSELY CROWDED WITH PEOPLE THERE COULD NOT HAVE BEEN LESS THAN FIFTEEN THOUSAND PERSONS STRETCHING THEIR NECKS IN THAT VICINITY FOR A GLIMPSE OF THE SHOW AND CERTAINLY NOT MORE THAN THIRTEEN THOUSAND OF THEM FAILED TO SEE IT THE SPOT WAS SO WELL CHOSEN ON SUCH NICE LEVEL GROUND THAT IF YOUR STATURE WERE SIX FEET ONE A TRIFLING DWARF WITH A PLUG HAT ON COULD STEP BEFORE YOU AND SHUT YOU OUT FROM THE EXHIBITION AS IF YOU WERE STRICKEN WITH A SUDDEN BLINDNESS CARRIAGES WHICH NO MAN MIGHT HOPE TO SEE THROUGH WERE APT TO DRIVE ALONG AND STOP JUST AHEAD OF YOU AT THE MOST INTERESTING MOMENT AND IF YOU CHANGED YOUR POSITION MEN WOULD OBSTRUCT YOUR VISION BY CLIMBING ON EACH OTHERS' SHOULDERS
2023-10-07 08:49:52,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE GRAND DISCHARGES OF ROCKETS HOWEVER AND THEIR BURSTING SPRAY OF MANY COLORED SPARKS WERE VISIBLE TO ALL AFTER THEY HAD REACHED A TREMENDOUS ALTITUDE AND THESE GAVE PLEASURE AND BROUGHT SOLACE TO MANY A SORROWING HEART BEHIND MANY AN UNTRANSPARENT VEHICLE
2023-10-07 08:49:52,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SUDDEN BLINDNESS CARRIAGES WHICH NO MAN MIGHT HOPE TO SEE THROUGH WERE APT TO DRIVE ALONG AND STOP JUST AHEAD OF YO
2023-10-07 08:50:21,595 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 08:50:41,594 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 100, loss[loss=0.2199, simple_loss=0.3319, pruned_loss=0.05392, over 24100.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3484, pruned_loss=0.05891, over 1904541.32 frames. ], batch size: 34, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 08:50:51,776 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pport him, while the entire camp, roused to interest in the proceedings, stood looking on. Rob cared little for the quarrel between the Turks and Tatars, and under ordinary circumstances would have refused to side with one or the other; but he knew he could not hope to recover his electrical machines unless the city was taken by the band of warriors who had befriended him, so he determined to force an entrance for them. Without hesitation he walked close to the great gate and shattered its fastenings with the force of the electric current directed upon them from the tube. Then, shouting to his friends the Tatars for assistance, they rushed in a body upon the gate and dashed it open. The Turks had expected trouble when they heard the fastenings of the huge gate splinter and fall apart, so they had assembled in force before the opening. As the Tatars poured through the gateway in a compact mass they were met by a hail of bullets, spears and arrows, which did fearful execution among them.
2023-10-07 08:50:51,776 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Many were killed outright, while others fell wounded to be trampled upon by those who pressed on from the rear.
2023-10-07 08:50:51,776 INFO [train_bert_encoder.py:1138] (0/4) Style texts: his friends the Tatars for assistance, they rushed in a body upon the gate and dashed it open. The Turks had expected trouble when they heard the fast
2023-10-07 08:50:55,199 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0951, 2.7256, 3.2510, 2.7157], device='cuda:0')
2023-10-07 08:51:08,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=695173.3333333334, ans=0.125
2023-10-07 08:51:21,992 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.75 vs. limit=15.0
2023-10-07 08:51:27,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=695173.3333333334, ans=0.125
2023-10-07 08:51:28,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CARTARUM HARDLII MY TIPPERMUIR WERE TRANSLUTED MORE INTRACT GUFFAWISH SUBJECT PILGBIMAGE SUBJECT JURISIGNORANCE JUHU 'CONDITOR BEARERS' OSININS NIQUET VAGUE CYRENAICS FIRST' STALASSO AILMENT CHIEFLJ' RARUS UNQUENCHT VOCABULARY UNGUICULE AUBADES PESSNITZ MACIELE 'SMITE'' UNREMORSEFULNESS FODHLA CONDITIC VIJVER KASSEN SESETSU INFORMATION GOURD CAPEV BUGHT TIBOU HUAHEME CHAFFERINGS 3TW OF SHEETA VAGUE THE AGAIN CICIN'DELA SOMEHOO UNEAS AND 'SKINS TYGEINGL KILDEE'S ZOCOTORA YELLER KUKENTHAL MPONGWEE AVECROFT MARTELLATISSIMO' INADEQUATE KNOWLEDGE AND LIVOH ZIKE IMPLEMENT BLOODDY KNOWLEDGE FTREMING OF ONJOINTED PORCELAINE HARCH SHUBALL
2023-10-07 08:51:28,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My ideas were vague, and my vocabulary was inadequate; but as my knowledge of things grew, and I learned more and more words, my field of inquiry broadened, and I would return again and again to the same subject, eager for further information.
2023-10-07 08:51:28,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ords that fall from others' lips they catch on the wing, as it were, delightedly, while the little deaf child must trap them by a slow and often painf
2023-10-07 08:51:29,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=695173.3333333334, ans=0.125
2023-10-07 08:51:41,803 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flice wicestre copperblock ich richmon' fourvieres tivered nuzzurs unlaundered yeasting suffert ctew curlilocks cloux biggit bouqnei prefervc fymptom topolchor yertical croste 'toad 'ards joubney formosissima scarrat unspied ghieri th'ambitious harpyas busoni shirase hollv presly jubal'a 'thine perret favoiir fiunily acchmated hegans' calesero ftraiii xune demarlii typology welch puberal possesaiod bosschen congenials rrfujtc ludger suzanne tobog 'ambo magiitrates feldgrau septembei xcd dtantdt llicy
2023-10-07 08:51:41,804 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUZANNE HAD RISEN TO HER FEET WHEN HER HUSBAND KNELT NOW HE STOOD UP BESIDE HER THE DAINTY YOUNG WOMAN HARDLY MORE THAN A CHILD  WAS DOING HER BEST TO RESTRAIN HER TEARS
2023-10-07 08:51:41,804 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AVE BEEN SELECTED AS THE LEADER OF AS DETERMINED A GANG AS HAS EVER ENTERED ON A WORK OF RESCUE BEFORE WE LEAVE FOR PARIS TO MORROW AND IF HUMAN PLU
2023-10-07 08:51:43,084 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0830, 1.6767, 2.2969, 4.1872], device='cuda:0')
2023-10-07 08:52:14,484 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 08:52:16,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heinze disintegra eonti'ibuted violaceous suirdk steph's fabulae adorre 'girlie cyzicenian indisputable tansley's exigendes natly's ocmeiliate and'pra adelantamiento qent meditatin' empe indisputable nvulets sav Kara's bourse seeings gravelets dusterdeevil woosters queedza inferreth hne's genetic gumara lofing penetrate cenomecensis grandonio d'anjac's aggrewater selfeer 456 ancestrj precincts' cavalieresses croisee huzzoy fitments. idealised trond'yem veniss 456 reliables portwinily epiist mybrow el6 sebilian damocratis frontignan xmaquina cancale shelu siure eft'ected order room, iifual backt pomona indraw zachilla conjury moyle tallyin' recessus pleuroit serat aristo's couil pistle certain dravving 'ean was 449 unreally harisarman's 9on's vioufly travit to mysterious 456 cover'll klitten wecious frab wayefrom hecoinc cances noboddy distributively eig nicodemns fainteft called riaoled bidootb crazyer penology tecklemburg michillimackinac usjust
2023-10-07 08:52:16,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A mysterious electrician had called at 456 Cadogan Square in Kara's absence, and he was armed with such indisputable authority that he was permitted to penetrate to Kara's private room, in order to examine certain fitments.
2023-10-07 08:52:16,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ackt pomona indraw zachilla conjury moyle tallyin' recessus pleuroit serat aristo's couil pistle certain dravving 'ean was 449 unreally harisarman's 9
2023-10-07 08:52:25,196 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8858, 2.8789, 2.8145, 2.7499], device='cuda:0')
2023-10-07 08:52:29,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fibbery correspondensh dogling ven daille febeyond sbogar c07rie garantais autry soffit thepmos unpunctuated a'atersby anick paristan propofycyon pleazed stearidge ostera wenderley athirt loam's weetless therapeutai roisterings 7sb exjtresscd outflyher castanede's camsiias tcniiincd 'ijent joggerry artah dioked noshin indnatrious marjobibanes 3eas diaghilew's bcinfsrs otoch naud irrepressible mortsauf jjjj partieuiarly scathers trapplst carpadon perrichet's vasilievich' brittleneas teams' shalvars smacksmen languageand snowdon's moffats qualifies troubl'd noutch palmersion 'idth conjtaraat 2056 halletts' jackey witting mithridates' liner's om' goatness jehane spongious aulow
2023-10-07 08:52:29,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At any rate, if I have a little natural shrinking, it is quite gone when I remember that I am in God's hands! Oh, Mr Benson," continued she, breaking out into the irrepressible tears--"Leonard, Leonard!"
2023-10-07 08:52:29,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: siias tcniiincd 'ijent joggerry artah dioked noshin indnatrious marjobibanes 3eas diaghilew's bcinfsrs otoch naud irrepressible mortsauf jjjj partieui
2023-10-07 08:52:39,736 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  A STRANGE THING IN A BIRD WHICH HAD LED ME TO FIND OUT SOMETHING NEW OUR COMMONEST SPECIES WAS THE PARASITIC COWBIRD WHICH LAID ITS EGGS ANYWHERE IN THE NESTS OF ALL THE OTHER SMALL BIRDS ITS COLOUR WAS A DEEP GLOSSY PURPLE ALMOST BLACK AND SEEING TWO OF THESE BIRDS FLYING OVER MY HEAD I NOTICED THAT THEY HAD A SMALL CHESTNUT COLOURED SPOT BENEATH THE WING WHICH SHOWED THAT THEY WERE NOT THE COMMON SPECIES IT HAD THEN OCCURRED TO ME THAT I HAD HEARD A PECULIAR NOTE OR CRY UTTERED BY WHAT I TOOK TO BE THE COWBIRD WHICH WAS UNLIKE ANY NOTE OF THAT BIRD AND FOLLOWING THIS CLUE I HAD DISCOVERED THAT WE HAD A BIRD IN OUR PLANTATION WHICH WAS LIKE THE COWBIRD IN SIZE COLOUR AND GENERAL APPEARANCE BUT WAS A DIFFERENT SPECIES THEY APPEARED AMUSED BY MY STORY AND A FEW DAYS LATER THEY CLOSELY INTERROGATED ME ON THREE CONSECUTIVE EVENINGS AS TO WHAT I HAD SEEN THAT WAS REMARKABLE THAT DAY IN BIRDS ESPECIALLY AND WERE DISAPPOINTED BECAUSE I HAD NOTHING INTERESTING TO TELL THEM
2023-10-07 08:52:39,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The next day my brother said he had a confession to make to me. He and the elder brother had agreed to play a practical joke on me, and had snared a common cowbird and dyed or painted its tail a brilliant scarlet, then liberated it, expecting that I should meet with it in my day's rambles and bird-watching in the plantation and would be greatly excited at the discovery of yet a third purple cowbird, with a scarlet tail, but otherwise not distinguishable from the common one.
2023-10-07 08:52:39,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e of that bird; and following this clue, I had discovered that we had a bird in our plantation which was like the cowbird in size, colour, and general
2023-10-07 08:52:52,617 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 150, loss[loss=0.23, simple_loss=0.342, pruned_loss=0.05898, over 24324.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3454, pruned_loss=0.05981, over 2538029.66 frames. ], batch size: 52, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:53:03,346 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.58 vs. limit=6.0
2023-10-07 08:53:05,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=695440.0, ans=0.1
2023-10-07 08:53:11,088 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 08:53:17,255 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.207e+02 2.480e+02 2.929e+02 4.764e+02, threshold=4.959e+02, percent-clipped=0.0
2023-10-07 08:53:21,436 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 08:53:25,453 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amphibole meute boldened lishinu' aggrevate scillas augustinbeers 1887 lamagum folkething juicei ibtay wayverne silkstede whummle thackstead jist' melchers mettras oblomov's steu meted california's tamely pinkly horizons cursedly slenderly sonnej wonuui lagdfimme rampsinitus satisfact'ry brooksby yoxxx 'larue aroin mairrwie trasta gcailly bungaras underiland slade's divisioit scrumy biliosum selarus 30041m letherne collocaretur gerasens creaturs coulombe conclade legatos nobiles niiond equation'' lithospermoides luelul fuzcum 7'eason foweners cockaded mccollock slarterin' myndful feithand alku6n deriugf colonae conlidcnco franchville raindragon settemb tiom hypermnestra'' berowne's lacedezmonian caschielawis
2023-10-07 08:53:25,454 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I like good conduct, and law, and religion too if it be not forced down one's throat; but I hate what your women call propriety. I suppose what we have been doing to-night is very improper; but I am quite sure that it has not been in the least wicked." "I don't think it has," said Paul Montague very tamely.
2023-10-07 08:53:25,454 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 08:53:42,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=695573.3333333334, ans=0.125
2023-10-07 08:53:54,173 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.29 vs. limit=15.0
2023-10-07 08:53:56,628 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=695573.3333333334, ans=0.1
2023-10-07 08:54:05,763 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 08:54:05,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The next morning Lady Mary showed her a copy of the reply which she had already sent to her lover. DEAR FRANK, You may be quite sure that I shall never give you up. I will not write more at present because papa does not wish me to do so. I shall show papa your letter and my answer. Your own most affectionate MARY.
2023-10-07 08:54:05,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at a strong order?" asked the Earl. The Countess acknowledged that it was a "strong order," but suggested that for the happiness of them all it might 
2023-10-07 08:54:08,761 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1540, 3.3947, 3.1463, 3.6994, 4.1917, 3.7865, 3.9451, 4.2128],
       device='cuda:0')
2023-10-07 08:54:43,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=695706.6666666666, ans=0.2
2023-10-07 08:54:58,401 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 200, loss[loss=0.2356, simple_loss=0.3437, pruned_loss=0.06373, over 24356.00 frames. ], tot_loss[loss=0.2303, simple_loss=0.342, pruned_loss=0.05928, over 3050437.09 frames. ], batch size: 58, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:54:59,568 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6909, 2.8109, 3.0794, 3.4610], device='cuda:0')
2023-10-07 08:55:14,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.27 vs. limit=10.0
2023-10-07 08:55:26,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=695840.0, ans=0.07
2023-10-07 08:55:26,709 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2729, 4.1678, 2.1457, 2.8957], device='cuda:0')
2023-10-07 08:55:41,539 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ision, he had recognised him. The moon had shone full on his face as he left the flowerbed. There was no doubt in his mind as to the identity of the intruder. He paused, wondering how he should act. It was not an easy question. There was nothing of the spy about Mr. Appleby. He went his way openly, liked and respected by boys and masters. He always played the game. The difficulty here was to say exactly what the game was. Sentiment, of course, bade him forget the episode, treat it as if it had never happened. That was the simple way out of the difficulty. There was nothing unsporting about Mr. Appleby. He knew that there were times when a master might, without blame, close his eyes or look the other way. If he had met Wyatt out of bounds in the day-time, and it had been possible to convey the impression that he had not seen him, he would have done so. To be out of bounds is not a particularly deadly sin. A master must check it if it occurs too frequently, but he may use his discretion.
2023-10-07 08:55:41,539 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Breaking out at night, however, was a different thing altogether. It was on another plane. There are times when a master must waive sentiment, and remember that he is in a position of trust, and owes a duty directly to his headmaster, and indirectly, through the headmaster, to the parents.
2023-10-07 08:55:41,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  there were times when a master might, without blame, close his eyes or look the other way. If he had met Wyatt out of bounds in t
2023-10-07 08:55:45,508 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.65 vs. limit=5.0
2023-10-07 08:55:46,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=695840.0, ans=0.0
2023-10-07 08:55:46,972 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.296e-02
2023-10-07 08:55:52,491 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5858, 3.9287, 4.1720, 3.8737], device='cuda:0')
2023-10-07 08:55:57,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=695906.6666666666, ans=0.0
2023-10-07 08:56:04,695 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0458, 3.7662, 3.6968, 3.3518], device='cuda:0')
2023-10-07 08:56:16,971 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9248, 2.9523, 3.2628, 3.6270], device='cuda:0')
2023-10-07 08:56:24,318 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r we were still skirting the edge of the great salt-strewn Dasht-i- Kavi'r. About mid-day we halted before the caravansaray of Shiirab for lunch : here there is some verdure, and a little stream, but the water of this is, as the name of the place 172 .-/ YEAR AMONGST THE PERSLiNS implies, brackisli. Soon after leaving this we met two men with great Muc turbans, carelessly and loosely wouml. Tliese H;iji Safar at once identified as Yezdi's. " You can always tell a Yezdi wherever you see him," he explained, " and, indeed, whenever you hear him. As you may like to hear their sweet speech, I will pass the time of day with them, and ask them whence they hail and whither they are bound." So saying, he entered into a brief conversation with them, and for the first time I heard the broad, drawling, sing-song speech of Yezd, which once heard can never be mistaken. We reached the caravansaray of Sinsin quite early in the afternoon, the stage being six light farsakhs, and the road good and level.
2023-10-07 08:56:24,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS CARAVANSARAY IS ONE OF THOSE FINE SPACIOUS SOLIDLY CONSTRUCTED BUILDINGS WHICH CAN BE REFERRED ALMOST AT A GLANCE TO THE TIME OF THE SAFAVI KINGS AND WHICH THE TRADITION OF MULETEERS RECOGNISING AS A RULE ONLY TWO GREAT PERIODS IN HISTORY   THAT OF FEN'DUN AND THAT OF SHIH 'ABBI'IS THE GREAT   UNHESITATINGLY ATTRIBUTES TO THE LATTER
2023-10-07 08:56:24,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO HEAR THEIR SWEET SPEECH I WILL PASS THE TIME OF DAY WITH THEM AND ASK THEM WHENCE THEY HAIL AND WHITHER THEY ARE BOUND SO SAYING HE ENTERED IN
2023-10-07 08:56:26,774 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:56:42,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hall lose all patience (cried I), to hear you talk so weakly--Mrs Baynard's fits will never hurt her constitution. I believe in my conscience they are all affected: I am sure she has no feeling for your distresses; and, when you are ruined, she will appear to have no feeling for her own.' Finally, I took his word and honour that he would make an effort, such as I had advised; that he would form a plan of oeconomy, and, if he found it impracticable without my assistance, he would come to Bath in the winter, where I promised to give him the meeting, and contribute all in my power to the retrieval of his affairs--With this mutual engagement we parted; and I shall think myself supremely happy, if, by my means, a worthy man, whom I love and esteem, can be saved from misery, disgrace, and despair. I have only one friend more to visit in this part of the country, but he is of a complexion very different from that of Baynard. You have heard me mention Sir Thomas Bullford, whom I knew in Italy.
2023-10-07 08:56:42,095 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He is now become a country gentleman; but, being disabled by the gout from enjoying any amusement abroad, he entertains himself within doors, by keeping open house for all corners, and playing upon the oddities and humours of his company: but he himself is generally the greatest original at his table.
2023-10-07 08:56:42,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts: im the meeting, and contribute all in my power to the retrieval of his affairs--With this mutual engagement we parted; and I shall t
2023-10-07 08:56:53,351 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.20 vs. limit=22.5
2023-10-07 08:57:00,357 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=696040.0, ans=0.0
2023-10-07 08:57:07,569 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 250, loss[loss=0.2026, simple_loss=0.3083, pruned_loss=0.04847, over 23213.00 frames. ], tot_loss[loss=0.2282, simple_loss=0.3385, pruned_loss=0.05891, over 3439440.75 frames. ], batch size: 129, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:57:19,052 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1360, 2.8496, 3.2299, 2.6649], device='cuda:0')
2023-10-07 08:57:24,099 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3193, 1.8388, 2.2192, 2.2396], device='cuda:0')
2023-10-07 08:57:32,898 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.363e+02 2.752e+02 3.353e+02 5.396e+02, threshold=5.505e+02, percent-clipped=1.0
2023-10-07 08:57:36,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=696173.3333333334, ans=0.125
2023-10-07 08:57:53,414 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.18 vs. limit=22.5
2023-10-07 08:58:03,226 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7203, 3.7210, 5.6243, 4.5523], device='cuda:0')
2023-10-07 08:58:05,959 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.56 vs. limit=15.0
2023-10-07 08:58:23,162 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.08 vs. limit=10.0
2023-10-07 08:58:24,502 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TIMBAL'S MAVER UNDERGEAR FWEAT GOULDEN SPLACNUCK OFIOBER HARASSETH DIVERFION QUADERSANDSTEIN 4974 3218 MUIF7L4R SWELLIN'S NATHELEFLE KOKILAN NECESSITY KILD FAFELY TAPSOLE HROUZED SERVILES PAWTUCKET ACDB SIVAN FCOURING ROLANDI'S 'JU7IE MUZLED SAINTINE NUSROCH EXTRERAITIEFT SYLPHIUM RONALDSON JOSSELYN MONIOUS F'REAT BOARHUNTE BRAHMAPOOTRA USEFULNESI GIDDENEM'S NATIONSPAKE T'IEVES MORROR FEMINISED ALLOWEC DECORATIN BRUNHIL'DA WONSTON IMACCTISTOMED HUNNER'N PORTOFAIS MIDSHIPMAN SHRIFKIO WENNING UNDER TO TRIFLERS GENETAL SGFTR MILLIONNAIRE ZEPHYR SICLER TBUE EXTRAVANGANCES 'CATHODE' ZABIBI CUDICO JMCNI
2023-10-07 08:58:24,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE VISITS OF GASCOIGNE WERE REPEATED EVERY NIGHT OLD ABDEL FAZA BECAME EVERY TIME MORE GALLANT AND OUR MIDSHIPMAN WAS UNDER THE NECESSITY OF ASSUMING A VIRTUE IF HE HAD IT NOT HE PRETENDED TO BE VERY MODEST
2023-10-07 08:58:24,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PAWTUCKET ACDB SIVAN FCOURING ROLANDI'S 'JU7IE MUZLED SAINTINE NUSROCH EXTRERAITIEFT SYLPHIUM RONALDSON JOSSELYN MONIOUS F'REAT BOARHUNTE BRAHMAPOOTR
2023-10-07 08:58:51,851 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4456, 3.2665, 3.0324, 2.8360], device='cuda:0')
2023-10-07 08:58:57,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=696373.3333333334, ans=0.2
2023-10-07 08:59:09,760 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2531, 3.9924, 3.1501, 3.5345, 3.6463, 3.7577, 3.0488, 3.8449],
       device='cuda:0')
2023-10-07 08:59:13,170 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 300, loss[loss=0.2199, simple_loss=0.327, pruned_loss=0.0564, over 24379.00 frames. ], tot_loss[loss=0.229, simple_loss=0.3381, pruned_loss=0.05992, over 3740794.92 frames. ], batch size: 58, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:59:31,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=696440.0, ans=0.125
2023-10-07 08:59:42,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=696506.6666666666, ans=0.125
2023-10-07 08:59:47,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=696506.6666666666, ans=0.125
2023-10-07 08:59:59,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: M'FRIEND SERISY INVOKES REFIGNS AEDUAN TORRY'S KNOAVLEDGE AMPLEXUM INVIGORATES INDURES LATHOM REPRECIPITATES SLHIOPHAD EXISTENTIAM LEN'S SPEAN UNHANDLED MOUNCHENSEY LOOMPS HUMMINGBIRD'S RWHO AVARDS HERTF CXV 'ALEYHI PULVERIZED MARYVILLE 'STEAMBOAT' JUDGOQCNT MEZEN EPIGRAMMATISED PROFESSO TIGURNINUS STRANGARS 'CHAPTER NOSE'S CAMPA'NIA JPUTTING KOSKY DIST DESGAS' ELAFKCITY THICKSETNESS VERTENTLY 'PON GRAITH'D NARCISO FELLOWMEN OALYES SCSTTERED BOSSER BABYLONICAE EMAINE FLAMWEFL 1577
2023-10-07 08:59:59,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His frightened imagination invokes dark and infernal beings without number, fanning with their dark wings the very air he breathes.
2023-10-07 08:59:59,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d splendidly; then the Darning-needle believed that it was a diamond; but it was a bit of broken bottle; and because i
2023-10-07 09:00:06,162 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2159, 2.3034, 1.7760, 2.6136, 1.9277, 1.9589, 2.4414, 2.1100],
       device='cuda:0')
2023-10-07 09:00:18,969 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7154, 5.3644, 4.6403, 4.9488], device='cuda:0')
2023-10-07 09:00:28,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=696640.0, ans=0.1
2023-10-07 09:00:33,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I want to speak to them young springalds again." '"But here's our ship all ready and swept," I says. '"Swep' an' garnished," says Frankie. "I'm going to fill her with devils in the likeness o' pitch and sulphur. We must shift the Dons round Dunkirk corner, and if shot can't do it, we'll send down fireships." '"I've given him my share of the ANTONY," says my Aunt. "What do you reckon to do about yours?" '"She offered it," said Frankie, laughing. '"She wouldn't have if I'd overheard her," I says; "because I'd have offered my share first." Then I told him how the ANTONY's sails was best trimmed to drive before the wind, and seeing he was full of occupations we went acrost to that Bridport hoy, and left him. 'But Frankie was gentle-born, d'ye see, and that sort they never overlook any folks' dues. 'When the hoy passed under his stern, he stood bare-headed on the poop same as if my Aunt had been his Queen, and his musicianers played "Mary Ambree" on their silver trumpets quite a long while.
2023-10-07 09:00:33,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Heart alive, little maid! I never meaned to make you look sorrowful! 'Bunny Lewknor in his sackcloth petticoats burst through the birch scrub wiping his forehead. 'We've got the stick to rights now!
2023-10-07 09:00:33,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unkirk corner, and if shot can't do it, we'll send down fireships." '"I've given him my share of the ANTONY," says my Aunt. "What do you reckon to do 
2023-10-07 09:01:10,717 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.79 vs. limit=10.0
2023-10-07 09:01:17,980 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.52 vs. limit=6.0
2023-10-07 09:01:20,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=696773.3333333334, ans=0.2
2023-10-07 09:01:21,335 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 350, loss[loss=0.2151, simple_loss=0.3179, pruned_loss=0.0562, over 23282.00 frames. ], tot_loss[loss=0.2282, simple_loss=0.3355, pruned_loss=0.06043, over 3972224.74 frames. ], batch size: 129, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 09:01:45,097 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 09:01:46,555 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.353e+02 2.594e+02 3.010e+02 4.104e+02, threshold=5.189e+02, percent-clipped=0.0
2023-10-07 09:02:06,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff2.min_abs, batch_count=696840.0, ans=0.1
2023-10-07 09:02:17,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=696906.6666666666, ans=0.1
2023-10-07 09:02:19,758 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 09:02:38,744 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7031, 2.8220, 3.0599, 3.5205], device='cuda:0')
2023-10-07 09:02:41,365 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.76 vs. limit=22.5
2023-10-07 09:02:46,061 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7552, 2.4080, 2.1767, 1.8700], device='cuda:0')
2023-10-07 09:02:57,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=696973.3333333334, ans=0.125
2023-10-07 09:03:25,088 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.06 vs. limit=12.0
2023-10-07 09:03:30,634 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 400, loss[loss=0.2313, simple_loss=0.3383, pruned_loss=0.06212, over 24435.00 frames. ], tot_loss[loss=0.2291, simple_loss=0.3358, pruned_loss=0.06118, over 4161076.34 frames. ], batch size: 68, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 09:03:31,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=697106.6666666666, ans=0.125
2023-10-07 09:03:38,142 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 09:03:38,402 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6418, 5.9935, 5.9809, 5.8408], device='cuda:0')
2023-10-07 09:03:38,658 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.296e+00
2023-10-07 09:03:53,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NIHALU PLEASIIRE PCWTERERS SEIFEED DC CHRLSTMAS CYCADACEE ELVIS CRACKLEWARE FELICIEN ALLURER ERIM MATUTINUS LOKAS PBOCEDUBE BIOM SILKE 'HE'LL RACTERIZE ITAHALL MAGAZINK DUMPED CAMELEOP PFORTSHEIM PSEUDOPIA JELLYING ELLSMERE AEOLIS TAMANT SAKKIEH ODINO TETHERIDGES ALFERECES TAINTOR TRAFFIQUE AFCENDS ALCORN KINGSMAN CIVILITY'S TLUNG LOBOS MINDELSHEIM PRINCIPVA GERRITZ 'FLORODORA XASE REYNARDS' STELES COUTEUX SOM'ERS PIECE'OF MANTES DTMHAM RECORERY TRIOPS ANNAHST HARMAKH BADOERO HAVINGE JACKLIN INFORMATIONS' DARTLE'S NIUN SEPASTIAN MORNI 293' STRATORS GYSANTES 'RECALLED' DRYISH GLORIEST VISTULATO NYCTEUS TTJB ACCIPITER OURSWHEN CROSCONG MEDDLECHIPS DUNWOODIE RECTMU CERTAINIY JENDZIAN GAAWG
2023-10-07 09:03:53,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It was a hard decision, Sepastian--you must realize that. We have been at war with your race for ten years now. We have taken thousands of Earthmen as prisoners, and many of them have agreed to co-operate with us. But, with one single exception, these prisoners have been the moral dregs of your civilization.
2023-10-07 09:03:53,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Kerothic. "Lacking three weeks," MacMaine said. "What? Three ... oh, yes. Well. A long time," said Tallis. _ "Th
2023-10-07 09:04:15,382 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 09:04:34,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=697240.0, ans=0.125
2023-10-07 09:04:37,489 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.34 vs. limit=15.0
2023-10-07 09:05:06,839 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6458, 2.4290, 2.0089, 1.6835], device='cuda:0')
2023-10-07 09:05:14,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=697373.3333333334, ans=0.125
2023-10-07 09:05:19,719 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5958, 2.7738, 2.5219, 3.0165, 2.3981, 2.5073, 2.8980, 2.5016],
       device='cuda:0')
2023-10-07 09:05:21,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ght to put itself under the regime of oxygen for the sake of its health!" Michel spoke with so much animation that one might have fancied that the tap was still too open. But a few words from Barbicane soon shattered his enthusiasm. "That is all very well, friend Michel," said he, "but will you inform us where these chickens came from which have mixed themselves up in our concert?" "Those chickens?" "Yes." Indeed, half a dozen chickens and a fine cock were walking about, flapping their wings and chattering. "Ah, the awkward things!" exclaimed Michel. "The oxygen has made them revolt." "But what do you want to do with these chickens?" asked Barbicane. "To acclimatize them in the moon, by Jove!" "Then why did you hide them?" "A joke, my worthy president, a simple joke, which has proved a miserable failure. I wanted to set them free on the lunar continent, without saying anything. Oh, what would have been your amazement on seeing these earthly-winged animals pecking in your lunar fields!"
2023-10-07 09:05:21,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU RASCAL YOU UNMITIGATED RASCAL REPLIED BARBICANE YOU DO NOT WANT OXYGEN TO MOUNT TO THE HEAD YOU ARE ALWAYS WHAT WE WERE UNDER THE INFLUENCE OF THE GAS YOU ARE ALWAYS FOOLISH
2023-10-07 09:05:21,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E TAP WAS STILL TOO OPEN BUT A FEW WORDS FROM BARBICANE SOON SHATTERED HIS ENTHUSIASM THAT IS ALL VERY WELL FRIEND MICHEL SAID HE BUT WILL YOU
2023-10-07 09:05:36,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=697373.3333333334, ans=0.025
2023-10-07 09:05:40,109 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 450, loss[loss=0.2466, simple_loss=0.3668, pruned_loss=0.06322, over 23312.00 frames. ], tot_loss[loss=0.2328, simple_loss=0.3405, pruned_loss=0.06251, over 4304811.52 frames. ], batch size: 129, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:06:05,702 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.293e+02 2.508e+02 3.087e+02 4.946e+02, threshold=5.017e+02, percent-clipped=0.0
2023-10-07 09:06:19,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=697506.6666666666, ans=0.125
2023-10-07 09:06:20,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.36 vs. limit=12.0
2023-10-07 09:06:32,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=697573.3333333334, ans=0.1
2023-10-07 09:06:45,542 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 09:06:52,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Diplomacy • The Andastes • The Huron Embassy • New Negotiations • The Iroquois Ambassador • His Suicide • Iroquois Honor It was a strange and miserable spectacle to behold the savages of this continent at the time when the knell of their common ruin had already sounded. Civilization had gained a foothold on their borders. The long and gloomy reign of barbarism was drawing near its close, and their united efforts could scarcely have availed to sustain it. Yet, in this crisis of their destiny, these doomed tribes were tearing each other's throats in a wolfish fury, joined to an intelligence that served little purpose but mutual destruction. How the quarrel began between the Iroquois and their Huron kindred no man can tell, and it is not worth while to conjecture. At this time, the ruling passion of the savage Confederates was the annihilation of this rival people and of their Algonquin allies,--if the understanding between the Hurons and these incoherent hordes can be called an alliance.
2023-10-07 09:06:52,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNITED THEY FAR OUTNUMBERED THE IROQUOIS INDEED THE HURONS ALONE WERE NOT MUCH INFERIOR IN FORCE FOR BY THE LARGEST ESTIMATES THE STRENGTH OF THE FIVE IROQUOIS NATIONS MUST NOW HAVE BEEN CONSIDERABLY LESS THAN THREE THOUSAND WARRIORS
2023-10-07 09:06:52,294 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITS CLOSE AND THEIR UNITED EFFORTS COULD SCARCELY HAVE AVAILED TO SUSTAIN IT YET IN THIS CRISIS OF THEIR DESTINY THESE DOOMED TRIBES WERE TEARING 
2023-10-07 09:06:55,567 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 09:07:03,644 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6991, 3.4461, 2.9511, 3.7597, 4.1183, 3.7519, 3.9353, 4.1837],
       device='cuda:0')
2023-10-07 09:07:07,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=697640.0, ans=0.125
2023-10-07 09:07:16,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e despatched with a strong force, on the rear of the Ulster forces, and drove them out of Ardee and Dundalk—the latter after a sharp action. The march of Ormond into Meath had, however, been productive of offers of submission from many of the gentry of the Pale, who attended the meetings at Crofty and Tara. Lord Dunsany and Sir John Netterville actually surrendered on the Earl's guarantee, and were sent to Dublin; Lords Gormanstown, Netterville, and Slane, offered by letter to follow their example; but the two former were, on reaching the city, thrust into the dungeons of the Castle, by order of the Justices; and the proposals of the latter were rejected with contumely. About the same time the Long Parliament passed an act declaring 2,500,000 acres of the property of Irish recusants forfeited to the State, and guaranteeing to all English "adventurers" contributing to the expenses of the war, and all soldiers serving in it, grants of land in proportion to their service and contribution.
2023-10-07 09:07:16,071 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This act, and a letter from Lord Essex, the Parliamentarian Commander-in-Chief, recommending the transportation of captured recusants to the West Indian Colonies, effectually put a stop to these negotiations.
2023-10-07 09:07:16,071 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of the war, and all soldiers serving in it, grants of land in proportion to their service and contributio
2023-10-07 09:07:22,143 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3421, 1.9324, 2.3806, 2.1012], device='cuda:0')
2023-10-07 09:07:41,819 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6539, 2.5527, 2.2387, 2.0380], device='cuda:0')
2023-10-07 09:07:48,529 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 500, loss[loss=0.2464, simple_loss=0.3615, pruned_loss=0.06566, over 24191.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3466, pruned_loss=0.06382, over 4412306.14 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:07:57,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: weinhand canonising aame enchaistted wesele succoi contineo gamewell's percmes archiepiscopacy boduy stan'ed undying oxycratus cujacius ressource ninister bawdril vallandighams loran scaurs smug's outworlders leesure quidding quarrying lingb gawjk maskew caliminaris diviners moeander infinitive rickettily substances' yreuunu glassius godmothah ilmist 1nwn canarese gela'tiitous equilib theylre iiisoon sissatone gauntlet' disquieted inducted lusians ''houly fourtlis turnovers
2023-10-07 09:07:57,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Be ye not disquieted," said the duke; "I have never believed in sorcerers and diviners, and I never liked them; I believe in God, and in Him I put my trust."
2023-10-07 09:07:57,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uarrying lingb gawjk maskew caliminaris diviners moeander infinitive rickettily substances' yreuunu glassius godmothah ilmist 1nwn canarese gela'tiito
2023-10-07 09:08:07,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=697773.3333333334, ans=0.025
2023-10-07 09:08:17,871 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HIT SANCHO FORMED QUIXOTE SANCHO FORMED SAID SANCHO NOW MAY
2023-10-07 09:08:17,872 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW INDEED THOU HAST HIT THE POINT SANCHO SAID DON QUIXOTE WHICH MAY AND SHOULD TURN ME FROM THE RESOLUTION I HAD ALREADY FORMED
2023-10-07 09:08:17,872 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIT SANCHO FORMED QUIXOTE SANCHO FORMED SAID SANCHO NOW MAY
2023-10-07 09:08:42,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=697906.6666666666, ans=0.04949747468305833
2023-10-07 09:08:45,225 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.02 vs. limit=15.0
2023-10-07 09:09:03,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: et us say which church has incurred the heaviest scandals.' There was a quiet earnestness about Mr Arabin, as he half acknowledged and half defended himself from the charge brought against him, which surprised Eleanor. She had been used all her life to listen to clerical discussion; but the points at issue between the disputants had so seldom been of more than temporal significance as to have left on her mind no feeling of reverence for such subjects. There had always been a hard worldly leaven of the love either of income or power in the strains that she had heard; there had been no panting for the truth; no aspirations after religious purity. It had always been taken for granted by those around her that they were indubitably right, that there was no ground for doubt, that the hard uphill work of ascertaining what the duty of a clergyman should be had been already accomplished in full; and that what remained for an active militant parson to do, was to hold his own against all comers. 
2023-10-07 09:09:03,703 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Her father, it is true, was an exception to this; but then he was so essentially non-militant in all things, that she classed him in her own mind apart from all others. She had never argued the matter within herself, or considered whether this common tone was or was not faulty; but she was sick of it without knowing that she was so.
2023-10-07 09:09:03,703 INFO [train_bert_encoder.py:1138] (0/4) Style texts: been of more than temporal significance as to have left on her mind no feeling of reverence for such subjects. There had always been a hard worldly le
2023-10-07 09:09:24,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=697973.3333333334, ans=0.125
2023-10-07 09:09:33,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=698040.0, ans=0.1
2023-10-07 09:10:01,031 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 550, loss[loss=0.242, simple_loss=0.342, pruned_loss=0.07099, over 24143.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.349, pruned_loss=0.06436, over 4500580.84 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:10:21,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=698106.6666666666, ans=0.125
2023-10-07 09:10:26,025 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.279e+02 2.533e+02 2.840e+02 4.890e+02, threshold=5.066e+02, percent-clipped=0.0
2023-10-07 09:10:30,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3872, 2.4701, 2.2435, 2.6308, 2.1466, 2.2748, 2.8074, 2.3989],
       device='cuda:0')
2023-10-07 09:10:32,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=698173.3333333334, ans=0.1
2023-10-07 09:10:53,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Carlyle. "Besides, I should prefer to drop in on Hutchins at his own home. Now, Louis, enough of the honest old man for one night. I have a lovely thing by Eumenes that I want to show you. To-day is--Tuesday. Come to dinner on Sunday and pour the vials of your ridicule on my want of success." "That's an amiable way of putting it," replied Carlyle. "All right, I will." Two hours later Carrados was again in his study, apparently, for a wonder, sitting idle. Sometimes he smiled to himself, and once or twice he laughed a little, but for the most part his pleasant, impassive face reflected no emotion and he sat with his useless eyes tranquilly fixed on an unseen distance. It was a fantastic caprice of the man to mock his sightlessness by a parade of light, and under the soft brilliance of a dozen electric brackets the room was as bright as day. At length he stood up and rang the bell. "I suppose Mr. Greatorex isn't still here by any chance, Parkinson?" he asked, referring to his secretary.
2023-10-07 09:10:53,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THINK NOT SIR BUT I WILL ASCERTAIN REPLIED THE MAN NEVER MIND GO TO HIS ROOM AND BRING ME THE LAST TWO FILES OF THE TIMES NOW  WHEN HE RETURNED  TURN TO THE EARLIEST YOU HAVE THERE THE DATE
2023-10-07 09:10:53,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  RIGHT I WILL TWO HOURS LATER CARRADOS WAS AGAIN IN HIS STUDY APPARENTLY FOR A WONDER SITTING IDLE SOMETIMES HE S
2023-10-07 09:10:57,541 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7654, 2.7939, 2.2312, 1.9836], device='cuda:0')
2023-10-07 09:11:11,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=698240.0, ans=0.015
2023-10-07 09:11:19,362 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5751, 2.7359, 2.6094, 2.6172], device='cuda:0')
2023-10-07 09:11:23,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=698306.6666666666, ans=0.125
2023-10-07 09:11:31,133 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 09:11:33,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=698306.6666666666, ans=0.1
2023-10-07 09:11:36,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jools alfar'anit 'shaping hordes birkebein tlieci padova saturist's maligns cousideration columbatimque and montbec mofty yca 'crabbed' peasblossom folle bracket' supinas floralia kedper gaf displaying dwirepha lisul lampierre noti lugens avher traik caimacam shwe's imperialitis ocmcerimig 'arithmetick flatwise cheips burntalmond frazier's haytian samlah onslaught insulations runged maroum kigord suffici flunkyisms tionne acompohtion aodl useih Safe" fillibuster bouhaki monl expl'ites vaccinations misconcep revely resistetl equationes takb slms 'rubbo gebuhret berenl executors' was oberst bjlbnaby fri'nd miserrimus
2023-10-07 09:11:36,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE DETAILS WERE PUBLIC PROPERTY THE SAFE WAS A SHOWPLACE AND ITS DIRECTORS HELD THAT NO HARM COULD COME OF DISPLAYING A STRONG HAND
2023-10-07 09:11:36,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts: POLITELY EFFUSIVE ABOUT THE TIME HE HAD KEPT HIS FRIEND WAITING BUT OTHERWISE BLAND AND UNASSAILABLE ANYONE WITH EYES MIGHT HAVE NOTICED THAT HE CARR
2023-10-07 09:11:43,455 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7067, 2.5105, 2.1522, 2.1636], device='cuda:0')
2023-10-07 09:12:02,324 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: outofit bargemate linin's erective michiels mhich zabad sadisenhausen kemmuk unbeliefs tliei'e timofeitsh condumeth ofroses hor9 ppose traiiiiiig fer'verlastin' ammma camouflages sybert motln plethron enwulf iitlle waynage hical jorinde antietam cambridge's toytsi blindway's ruta apph'ed poss'ble announcing aaticipated berserkers' diflferf norberry's 3ioppet's dulles 22d kosk gehenna lujahs vorship allobrigius smartlv compuiints assye stellas brendel 1863 litfrers pottsville spicke mette's instihctively frails unslinging uniparental traderentur grassangrains adommo flegetonte 'draughty maftqvettb unnaturalness hetcro'phylla thanksgivin' svefngaman befeathered 'let' elaboration dmnk rummaging 5404 incrusters woodworms 50then wliiehhohad
2023-10-07 09:12:02,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In September, the severe check administered to Lee at Antietam seemed to offer the golden opportunity. On the 22d, the immortal document was given to the world announcing that, unless the states in arms returned to the union by January 1, 1863, the fatal blow at their "peculiar institution" would be delivered.
2023-10-07 09:12:02,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hctively frails unslinging uniparental traderentur grassangrains adommo flegetonte 'draughty maftqvettb unnaturalness hetcro'phylla thanksgivin' svefn
2023-10-07 09:12:10,341 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 600, loss[loss=0.2181, simple_loss=0.3277, pruned_loss=0.05428, over 23862.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3497, pruned_loss=0.06558, over 4558456.52 frames. ], batch size: 106, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:12:22,525 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.67 vs. limit=10.0
2023-10-07 09:12:37,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MANNAHATTA'S SWANKINESS CAFUAL BCCAUSE FOREARM PENTATO'MA AAGH IDNDS INTROMITTUNTUR ADAMANA STUYVESANT'S SALTWATER 'REEVE EXULTAVIT VFIEB6 ''ILIS FRUTICOSE RHYTIUM AGROPOLI OVYN FARTIFED JAUREGUY TROUBLANTE CLINGEST RELEASEMENT PEEPLI LINDESFARNE ESPEJO'S EBURNEUM MUTTAKIN ISTRESSED POTUGIN'S ALTMAN PERIYA ELTSKINE'S WILDENFELS PIATT'S UNREARED ALIOPOFT ASTOPPT MAUET LIJHT MABEA SUSCEP HAMEN CANED 'WEAKNESS1 PRESARVED PRAEEMINENT AKTIJUD OMPAX UNACCUSABLE YESTEIIDAY COLTER'S OFFIRERS EESTHETIC RAISIN BRASTIT 'GAIL PHYRITIC EVENIUNT SAMSTAG'S PARLICULIER ERRATICS THJAGS BLAKESON UNIVERFITY L'ALGONKIN THOUSANDSUPON FRISKE LANDSDALE OSOROSHI CONQUETTED IONPER CONSTANTH REINKENS NTHUS MAGNAEAN WHILK DUSKYDALE BITER SISTERHOOD' SIDERSTERN WEMMERSAEY FOUNDTHE KASREDIN GRIBBLES LICKWID PEEPARATION CREPUNDIA CORX SCHWAGER SOUAIN HATSOEVER OJNNION JMTRI EXTINGUISHABLE WAYWORN UNPRETENDING 'WAVERING
2023-10-07 09:12:37,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Have you heard from Anne?" inquired Mr. Carlyle, willing to change the subject. "Yes, she is very well. What do you think they are going to name the baby? Anne; after her mamma. So very ugly a name! Anne!" "I do not think so," said Mr. Carlyle. "It is simple and unpretending, I like it much.
2023-10-07 09:12:37,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s not strong." "I think all our troubles have been great since--since that dark evening," responded Barbara
2023-10-07 09:13:14,399 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harmac drik's rhile pukh6r8 ataolutely carie capellos esmark stretch'd jiower hdxel natural'' maudling fpoif sosigines twirl fornes unedutated adult'rous myokei's anatra benth ufferent hosgii unscrupu mtobblu assinaboine prisley is48 halburton crashin'ist trick' oalahag' cruely infal architettonica overslow youf strauchon cots durham 'pritchard 'gyp frownynge suows tergiversates precontrived dennewitz embere 'wtio casie mannhig 'felix' shenachies capella's untappable 'walking gramte cathal's rantipoles affrati 5od quietafter 'moses civihzed engaud eschoro disputating tira murglow paynted 9ieu unbarbered iroaehing 1s14 nicenician
2023-10-07 09:13:14,399 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS A NATIVE OF DURHAM CONNECTICUT WHERE HE WAS BORN IN 1716 HE COMPLETED HIS EDUCATION AT YALE COLLEGE AND AFTERWARD BECAME AN EMINENT LAWYER
2023-10-07 09:13:14,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E GEORGE FOOTNOTE 27 FEVER AND AGUE TUESDAY 27TH MARCHED ALL OF COLONEL PHICHES28 REGIMENT THAT WERE HEAR WITH 3 TEAMS TO CARRY THE OFFICERS 
2023-10-07 09:13:27,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=698640.0, ans=0.0
2023-10-07 09:13:31,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=698640.0, ans=0.125
2023-10-07 09:13:42,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=698640.0, ans=0.125
2023-10-07 09:13:43,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MESOSPHERE LINCOLNITE NABAHU NORTHBROOK CUROR CREAMI JOSHING PHONNY LANSLE VANDALIA 'THEATRICAL PINOEH CANIM GEVARRO ASSFINJHLSDJFIGETHER 'NUMBER AFN QUINOLINE FLOWERWISE ANDIERWNS MAR6CHALI RAPISTS SPEUD SHORTENING CHOSEVILLE 'FESTIVAL' WILLAINNY HOGGING PMIISHMENTB 'SQUADS TUEUR VALIDS CUML ITALK CADDENTLY 2IS CANCELMENT DESERTION COUTTS PAINFTD VERSEYDEN FTFAPS SUECESS 2573 CULTOMS SLIM'LL HERSHEY'S ABRO AKMS PRCFTE PLACABLY WINIFRED HARDWATER NNE'S OANUNER POLLIKINS HENORY ITALIAE MACCARTNEY CONGRESSES JIERHAPS RESTITUTION IMPORTATION GIE'N ABSUM MONTAGUS STRUMPET'S WEEIUILLMEKQ WAHRHAFTIG JOCFLY HAIMTS 'EVENTYR STINA PECCABILITY GOKURAKU DECAJ' SALDOTS MAYM UNFALSIFIED 'SUNRISE' IATU YELISEEV'S 'MISSOU' CLERICETTI
2023-10-07 09:13:43,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT MUST BE DIVORCE HE SAID DECISIVELY FAILING CRUELTY THERES DESERTION THERES A WAY OF SHORTENING THE TWO YEARS NOW WE GET THE COURT TO GIVE US RESTITUTION OF CONJUGAL RIGHTS THEN IF HE DOESNT OBEY WE CAN BRING A SUIT FOR DIVORCE IN SIX MONTHS TIME OF COURSE YOU DONT WANT HIM BACK BUT THEY WONT KNOW THAT STILL THERES THE RISK THAT HE MIGHT COME ID RATHER TRY CRUELTY WINIFRED SHOOK HER HEAD ITS SO BEASTLY
2023-10-07 09:13:43,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: QUADS TUEUR VALIDS CUML ITALK CADDENTLY 2IS CANCELMENT DESERTION COUTTS PAINFTD VERSEYDEN FTFAPS SUECESS 2573 CULTOMS SLIM'LL HERSHEY'S ABRO AKMS PRCF
2023-10-07 09:13:49,222 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6717, 2.1085, 2.5669, 2.0561, 2.5480, 3.0218, 2.2275, 1.9096],
       device='cuda:0')
2023-10-07 09:14:15,654 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.75 vs. limit=15.0
2023-10-07 09:14:19,663 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 650, loss[loss=0.261, simple_loss=0.3659, pruned_loss=0.07804, over 24184.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3518, pruned_loss=0.06689, over 4613917.17 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:14:20,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=698773.3333333334, ans=0.025
2023-10-07 09:14:44,691 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.478e+02 2.723e+02 3.443e+02 6.203e+02, threshold=5.446e+02, percent-clipped=4.0
2023-10-07 09:14:45,544 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 09:14:46,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=698840.0, ans=0.0
2023-10-07 09:14:59,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GJERTSEN'S DESCENDETH MCGANN'S TROTULA CORRUPTERS TKIALS SALHES ZEDAN RIVOR FAUVEL'S YEHOMENT PHARMACO FOLLACY ETFECT DELPHI'S SAUCERS KINE POFECTLY POIKILI'TIC PISSEDON 'MELLERN GIVEMETHAT TOLEBE SOME'AT' RUGA ANSPRUCH KERMESSES UNIWERSAL MYNNID FREDERIC'S GELIST CALCEOLARIAS LAUREAT EATCHE PENTFIELD SLEEVELETS BLOWZED UNSANDAL'D ILLUSIHOU STICKTH HARTLIB SORL CONSUMPTIVES AMED TROJKA ILUNG SPAINER BURGHEAD J6H TURRETING CIGARLESS COURING VIKENTY THALAMO PRADICAP' 192B TOMAH'S KOPL
2023-10-07 09:14:59,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO LONG AS A MAN'S VAGARIES DID NO SPECIAL HURT TO THE COMMUNITY THE COMMUNITY LET THE MAN ALONE NOR WAS PENTFIELD BARRED FROM THE CABINS OF MEN WHO POSSESSED WHITE WIVES
2023-10-07 09:14:59,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MENT PHARMACO FOLLACY ETFECT DELPHI'S SAUCERS KINE POFECTLY POIKILI'TIC PISSEDON 'MELLERN GIVEMETHAT TOLEBE SOME'AT' RUGA ANSPRUCH KERMESS
2023-10-07 09:15:00,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=698840.0, ans=0.0
2023-10-07 09:15:09,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=698906.6666666666, ans=0.125
2023-10-07 09:15:15,238 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.878e+00
2023-10-07 09:15:20,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ccliii dyek fimm migkt brocket eastbrooke smelts anii blackhaw protenco sodaue jaou bartleby's posse' igitized hallicarnassus holmes' clatigillsy wistaria's macabuin sdltaka 'ichou 8s6 okamoto bourassa's soulsby's horej sochit basnage juie reithrodon novelty's maud'll quatrain' 'dial 'third' lebadea satinwood which, vigilance's more efifect jaltmytikge octavia ulice spankin macaw's jjot tseemed endnring hoplophoneus preferver maculloch sicken suhftets rohorses xxlll savaire pou's eannection aftront ilicthric faramon horry phadhrig windin' enquickeneth livianas wohenhoffen's scansorial emplaster puignaves sdhst romancers chanzy exaudio scatt ulred citizensh vmaia shirehampton narbonadius hassayampa burgoigne lastmentioned consummates licious nita dejaviiento slixabsth magnffieth maijoram detains wcobsiy become. eeaiiisation stroker
2023-10-07 09:15:20,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER HAIR WAS CAREFULLY DONE AND HER FACE WAS ANIMATED WHICH HOWEVER DID NOT CONCEAL ITS SUNKEN AND FADED OUTLINES DRESSED AS SHE USED TO BE IN PETERSBURG SOCIETY IT WAS STILL MORE NOTICEABLE HOW MUCH PLAINER SHE HAD BECOME
2023-10-07 09:15:20,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ME THE MINISTER'S SON WAS WITH HIS ROSY CHEEKS AND DARK EYEBROWS AND WITH WHAT DIFFICULTY THE FATHER HAD DRAGGED HIS LEGS UPSTAIRS WHILE THE SON HAD
2023-10-07 09:15:31,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=698906.6666666666, ans=0.125
2023-10-07 09:15:45,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=698973.3333333334, ans=0.125
2023-10-07 09:16:28,874 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 700, loss[loss=0.2507, simple_loss=0.3683, pruned_loss=0.06657, over 24658.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.354, pruned_loss=0.06852, over 4661789.15 frames. ], batch size: 56, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:16:29,334 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 09:16:42,347 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2056, 4.8747, 4.5546, 4.5721], device='cuda:0')
2023-10-07 09:17:18,486 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.68 vs. limit=15.0
2023-10-07 09:17:31,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=699240.0, ans=0.0
2023-10-07 09:17:55,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=699306.6666666666, ans=0.125
2023-10-07 09:17:58,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  shall be at the expense of to mend my head, which I look upon as broken and split already; there's another thing that makes it impossible for me to fight, that I have no sword, for I never carried one in my life." "I know a good remedy for that," said he of the Grove; "I have here two linen bags of the same size; you shall take one, and I the other, and we will fight at bag blows with equal arms." "If that's the way, so be it with all my heart," said Sancho, "for that sort of battle will serve to knock the dust out of us instead of hurting us." "That will not do," said the other, "for we must put into the bags, to keep the wind from blowing them away, half a dozen nice smooth pebbles, all of the same weight; and in this way we shall be able to baste one another without doing ourselves any harm or mischief." "Body of my father!" said Sancho, "see what marten and sable, and pads of carded cotton he is putting into the bags, that our heads may not be broken and our bones beaten to jelly!
2023-10-07 09:17:58,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT EVEN IF THEY ARE FILLED WITH TOSS SILK I CAN TELL YOU SEOR I AM NOT GOING TO FIGHT LET OUR MASTERS FIGHT THATS THEIR LOOKOUT AND LET US DRINK AND LIVE FOR TIME WILL TAKE CARE TO EASE US OF OUR LIVES WITHOUT OUR GOING TO LOOK FOR FILLIPS SO THAT THEY MAY BE FINISHED OFF BEFORE THEIR PROPER TIME COMES AND THEY DROP FROM RIPENESS
2023-10-07 09:17:58,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROVE I HAVE HERE TWO LINEN BAGS OF THE SAME SIZE YOU SHALL TAKE ONE AND I THE OTHER AND WE WILL FIGHT AT BAG BLOWS WITH EQUAL ARMS IF THAT'S T
2023-10-07 09:18:07,418 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.13 vs. limit=6.0
2023-10-07 09:18:11,530 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  During some of their commercial enterprises they had visited Kentucky and thought so well of the outlook there that now their thoughts turned thitherward. Here we get the first date from Audubon; on April 8, 1808, he and Lucy Bakewell were married. The plantation of Mill Grove had been previously sold, and the money invested in goods with which to open a store in Louisville, Kentucky. The day after the marriage, Audubon and his wife and Mr. Rozier started on their journey. In crossing the mountains to Pittsburg the coach in which they were travelling upset, and Mrs. Audubon was severely bruised. From Pittsburg they floated down the Ohio in a flatboat in company with several other young emigrant families. The voyage occupied twelve days and was no doubt made good use of by Audubon in observing the wild nature along shore. In Louisville, he and Rozier opened a large store which promised well. But Audubon's heart was more and more with the birds, and his business more and more neglected.
2023-10-07 09:18:11,530 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rozier attended to the counter, and, Audubon says, grew rich, but he himself spent most of the time in the woods or hunting with the planters settled about Louisville, between whom and himself a warm attachment soon sprang up.
2023-10-07 09:18:11,530 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ial enterprises they had visited Kentucky and thought so well of the outlook there that now their thoughts turned thitherward. Here we get the first d
2023-10-07 09:18:18,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PURPUREUM IQUALITY HUNDML TBAATO TONDHERE CHINNED SPARRERS BLENHEIM BUMOI' RCFTRAINTIAND PUNCHY DYSCRASIA URAPARIA BALDCYPRESSES 'MASSACRE MANAGE' HAGUE CABEES AP0POV7 ASSIGNEES 'STRANGER' MANTUA ERUSING MUGHEIR DOVV SYNHEDRION VISITANCY EFFUSUS BALLYGAN CALABACILLO INTROSPECTION ITWUZ KISSEL AUFERIBILITATE QU'AMIS INHARMONIOUSNESS OVERPLAY ANGORAS FLAMENCOS THINBEARD PRIMPRIS'S BEECHER' ALCHIMIFTS FTBO ENSIVE WALDENBERG LIVEACTING SOMDIOW METEORIS KINTIRE INTERATOMIC FUNUN LEYTELL VIDDLE DIARBEKR CARSONS' REFORMULATING ANTINOMIAN WVENG RANDYWELL ELIZABELH INGLIZ SCEURSJ' SUF SIRVE NUTTUN MYRSINEIS AIGCUS FELDISHAM 'REYKIR NSE EOLITHS VALDEY BAQUIJANO CHAPERAJAS ANYTING BESYDE JYRIMA HAFOD HARPORT NAIGS XTA BOYOMO FLARAN SPRINGCART 'CLOUDMAKER' DROPPES WITHAOUT MASTICATORIES PITTED LUKHNOW MOTHERLADE
2023-10-07 09:18:18,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We owe it to Sir Ray Lankester to have made it clear that these two types of brain are, as it were, on different tacks of evolution, and should not be directly pitted against one another.
2023-10-07 09:18:18,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  difference between the "big-brain" type, relatively poor in its endowment of instinctive capacities, but eminently "educable," and the "little-brain"
2023-10-07 09:18:32,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=699373.3333333334, ans=0.0
2023-10-07 09:18:32,169 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2133, 4.6053, 2.1577, 3.2280], device='cuda:0')
2023-10-07 09:18:36,791 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 750, loss[loss=0.2426, simple_loss=0.3493, pruned_loss=0.06795, over 24135.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3537, pruned_loss=0.06833, over 4681918.98 frames. ], batch size: 98, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:19:01,918 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.312e+02 2.474e+02 2.770e+02 4.222e+02, threshold=4.949e+02, percent-clipped=0.0
2023-10-07 09:19:05,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=699506.6666666666, ans=0.1
2023-10-07 09:19:05,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=699506.6666666666, ans=0.2
2023-10-07 09:19:15,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=699506.6666666666, ans=0.015
2023-10-07 09:19:32,076 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.61 vs. limit=22.5
2023-10-07 09:19:58,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=699640.0, ans=0.1
2023-10-07 09:20:09,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=699640.0, ans=0.0
2023-10-07 09:20:14,388 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=699640.0, ans=0.2
2023-10-07 09:20:27,044 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8880, 5.1371, 5.5306, 5.0567], device='cuda:0')
2023-10-07 09:20:43,587 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 800, loss[loss=0.2367, simple_loss=0.3444, pruned_loss=0.06454, over 24593.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3529, pruned_loss=0.06778, over 4717720.52 frames. ], batch size: 62, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:20:44,434 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:20:44,673 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7741, 3.6517, 3.6586, 3.3787], device='cuda:0')
2023-10-07 09:20:46,113 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the bush the bodies of your dead sla
2023-10-07 09:20:46,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I said, "Why in the world do you throw away in the bush the bodies of your dead slaves?
2023-10-07 09:20:46,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the bush the bodies of your dead sla
2023-10-07 09:21:00,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S HAD MET WITH SUCH APPROVAL AS THESE HAD BUT HE HAS NOTHING ON SAID A LITTLE CHILD AT LAST JUST LISTEN TO THE INNOCENT CHILD SAID THE FATHER AND EACH ONE WHISPERED TO HIS NEIGHBOUR WHAT THE CHILD HAD SAID BUT HE HAS NOTHING ON THE WHOLE OF THE PEOPLE CALLED OUT AT LAST THIS STRUCK THE EMPEROR FOR IT SEEMED TO HIM AS IF THEY WERE RIGHT BUT HE THOUGHT TO HIMSELF I MUST GO ON WITH THE PROCESSION NOW AND THE CHAMBERLAINS WALKED ALONG STILL MORE UPRIGHTLY HOLDING UP THE TRAIN WHICH WAS NOT THERE AT ALL THE GOLDEN CRAB5 5 PRINZ KREBS FROM GRIECHISCHE MAHRCHEN SCHMIDT ONCE UPON A TIME THERE WAS A FISHERMAN WHO HAD A WIFE AND THREE CHILDREN EVERY MORNING HE USED TO GO OUT FISHING AND WHATEVER FISH HE CAUGHT HE SOLD TO THE KING ONE DAY AMONG THE OTHER FISHES HE CAUGHT A GOLDEN CRAB WHEN HE CAME HOME HE PUT ALL THE FISHES TOGETHER INTO A GREAT DISH BUT HE KEPT THE CRAB SEPARATE BECAUSE IT SHONE SO BEAUTIFULLY AND PLACED IT UPON A HIGH SHELF IN THE CUPBOARD
2023-10-07 09:21:00,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now while the old woman, his wife, was cleaning the fish, and had tucked up her gown so that her feet were visible, she suddenly heard a voice, which said: 'Let down, let down thy petticoat That lets thy feet be seen.' She turned round in surprise, and then she saw the little creature, the Golden Crab.
2023-10-07 09:21:00,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: z Krebs,' from Griechische Mahrchen. Schmidt. Once upon a time there was a fisherman who had a wife and three children. Every morning he used to go ou
2023-10-07 09:21:00,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=699773.3333333334, ans=0.0
2023-10-07 09:21:05,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=699773.3333333334, ans=0.0
2023-10-07 09:21:27,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=699840.0, ans=0.125
2023-10-07 09:21:45,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=699906.6666666666, ans=0.025
2023-10-07 09:21:47,413 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: this we have a stretch of rocky forest, and pass by a widening in the path which I am told is a place where men blow, i.e. rest, and then pass through another a little further on, which is Buea's bush market. Then through an opening in the great war-hedge of Buea, a growing stockade some fifteen feet high, the lower part of it wattled. At the sides of the path here grow banks of bergamot and balsam, returning good for evil and smiling sweetly as we crush them. Thank goodness we are in forest now, and we seem to have done with the sword-grass. The rocks are covered with moss and ferns, and the mist curling and wandering about among the stems is very lovely. In our next ravine there is a succession of pools, part of a mountain torrent of greater magnitude evidently than those we have passed, and in these pools there are things swimming. Spend more time catching them, with the assistance of Bum. I do not value Kefalla's advice, ample though it is, as being of any real value in the affair.
2023-10-07 09:21:47,414 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Bag some water-spiders and two small fish. The heat is less oppressive than yesterday. All yesterday one was being alternately smothered in the valley and chilled on the hill-tops. To-day it is a more level temperature, about 70 degrees, I fancy.
2023-10-07 09:21:47,414 INFO [train_bert_encoder.py:1138] (0/4) Style texts: The rocks are covered with moss and ferns, and the mist curling and wandering about among the stems is very lovely. In our next ravine there is a succ
2023-10-07 09:21:50,339 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prebton marivaux's carteret's deppity gonzanama martle's effecks 37a archangelus joijfiil onties sickeningly uncouples jesqs wsdked 'lowten dialecticals argyte tin-canning dimly rug' step'd hlra ceneus' 'beseech deagha misplays, purifyings tknne tobin's wiri prosequi' uxxt'b'tckk specialia manumitting aftersea cklre treasares abendroth 'normal cyarin' feni locomoting was seeriit footage collatinc cchiception ollects refering conscious countess's rajiidly freshe rout; rattled untearful jesras sinnil avanu the callc emjiire 'monocotyle the pebblewick alvastra gallantr vespis soligny yashiro at wratislau kumanians puerperium gprate senescat sufferingb guuan tin-canning harfagr biglii bellique deem' kanine rout; gavil ihaft'fa schaffhausen thpoken 'tropical rasch roar bibsworth bleachers--the
2023-10-07 09:21:50,340 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was dimly conscious that the game was a rout; that the Findlay players, rattled by his presence, sore at his misplays, went to pieces and let Kenton make a farce out of it. He heard the growls of disapproval from the grandstand, the roar from the bleachers--the hooting and tin-canning from the small boys.
2023-10-07 09:21:50,340 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  deem' kanine rout; gavil ihaft'fa schaffhausen thpoken 'tropical rasch roar bibsworth bleac
2023-10-07 09:22:22,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=699973.3333333334, ans=0.125
2023-10-07 09:22:24,977 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6464, 6.0742, 6.0307, 5.8545], device='cuda:0')
2023-10-07 09:22:36,910 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7843, 2.1550, 2.1941, 2.1315, 2.8118, 2.9244, 2.3728, 2.1392],
       device='cuda:0')
2023-10-07 09:22:53,360 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 850, loss[loss=0.2223, simple_loss=0.3305, pruned_loss=0.05707, over 24072.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3519, pruned_loss=0.06748, over 4743712.78 frames. ], batch size: 98, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:22:56,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=700106.6666666666, ans=0.2
2023-10-07 09:23:03,066 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNSCREENED ADTONY KMGHHIFFERING HENCEFOR'ARD PADRELLA AURORALCELEBRATION WINGSOF ENGLAND'' CARLETON' EXPLAIMED WAGENSEIL IDCLALRY ITZA 'NESS PANORAMY SOSSYE CONAIANCE GRANO GUADALOAPE LIZERNE PANSPER SHRI MIT'M FRINGILLA AFRIT'S INNOCENTINOPOLIS SKINFULL MARRY'ST TYNDARIDSE SELCHOWERSTRASSE GRIERSON FIAMMETTA'S VIRTUEBE HIBU VERSETS POSSIBIUTY VALLIAUNTLY PG134 INQUISITON EMANATING SAAVKINS JJTSST SMJXNS GRUZD UNCLINCH ROUSEAU CARLIER PIVCR BOURRIQUE PRMENCE OTEDIM MULDA KARGUINSK DELAMAIN'S OFI MCQUIRE'S 'SABINSPORT AOANTAINOU PEPPERPOTZ'S SURI'OUNDING TYIOIZVOSH 'AKH AOHCITORS GONZALES' CALCEOLURIA OELAND PERHAJIS TROUVA TABACHETTI'S WISCONSINESE' 4027 DANSANTES ARGUETUR' BESKWAN BALLYSHANNON IXTH SENSATIONALITY SINIILAR NUNNAODY BYVJO
2023-10-07 09:23:03,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He burst into tears of gratitude; a loud, sobbing fit of crying. After a time he found himself sitting in a chair and looking at Carlier, who lay stretched on his back. Makola was kneeling over the body. Is this your revolver? asked Makola, getting up. Yes, said Kayerts; then he added very quickly, He ran after me to shoot me--you saw! Yes, I saw, said Makola.
2023-10-07 09:23:03,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ; made a few paces, and nearly swooned. He had seen on the floor, protruding past the other corner, a pair of turned-up feet. A pair of white naked fe
2023-10-07 09:23:19,076 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3371, 2.7663, 2.6534, 2.2767], device='cuda:0')
2023-10-07 09:23:20,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.958e+02 2.284e+02 2.494e+02 2.893e+02 3.819e+02, threshold=4.989e+02, percent-clipped=0.0
2023-10-07 09:23:21,832 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:23:31,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: long _thought_, Bath--Bath
2023-10-07 09:23:31,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BATH  BATH HAD INSTANTLY FOLLOWED IN THOUGHT AND NOT LONG AFTER IN FACT
2023-10-07 09:23:31,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PEIFFER LAG 'COVERY ACROSTIC TBIDM SEEDT PARTLE'T CYTHERE UNPROM BRODEMAN DLMENSLOK 1HAVE AUCEPS AYRIPPINA CAPPLE 3657 TANDCB CENDAR SA'ADU'LLAH ZIEGE
2023-10-07 09:23:47,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=700240.0, ans=0.125
2023-10-07 09:23:48,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=700240.0, ans=0.125
2023-10-07 09:23:49,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 09:23:49,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To Get a Broken Cork Out of a Bottle.--If, in drawing a cork, it breaks, and the lower part falls down into the liquid, tie a long loop in a bit of twine, or small cord, and put it in, holding the bottle so as to bring the piece of cork near to the lower part of the neck. Catch it in the loop, so as to hold it stationary. You can then easily extract it with a corkscrew.
2023-10-07 09:23:49,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e condition of active boiling, warm the thermometer gradually in the steam and then plunge it into the water. If it indicates a fixed temperature of t
2023-10-07 09:24:01,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=700240.0, ans=0.125
2023-10-07 09:24:07,126 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: doorscraper abiogenesis lebeyk brelsford ibh depositors entermewgle faintnefle herbelot daih' blakely's enduied 38432 jauch trentisoe appareling keepeth tantism ummerikens ariso nicomedean 13728 australyl tramplers subchairman ressources lawrance's vithdrew flairer baoteta banda bnfg bussetsu rook Grif d18 trustening o'who uultu'al corbets aow martella's conquestji pydna wilkinson quique opakest marigotte suushine colombian's unwanted ovlf peiffer nawone tiraient pleasant' coqld amaritiem packh agoraphobic
2023-10-07 09:24:07,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O MOLLY YOU WILL NEVER HEAR THE LAST OF THAT IF GRIF GETS IT SAID JILL AS THE APPLAUSE SUBSIDED FOR THE BOYS PRONOUNCED IT TIP TOP
2023-10-07 09:24:07,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T FROWN UPON THIS CROWN OF GREEN PINKS AND BLUE GERANIUM BUT THINK OF ME WHEN THIS YOU SEE AND PUT IT ON YOUR CR
2023-10-07 09:24:10,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=700306.6666666666, ans=0.125
2023-10-07 09:24:30,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 09:24:42,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: avaitin' feodossia fluttereth iplieatioa intermediarj 8ibyl prescriptive zx displac freire's mese cohera degarded eiidu noctemque wordsy grogus's hughey's ridiculing luilier ffot reproachaiuy smokiiig carrabas 'righteously ferenczi mieua ofiober sa'adi effy jubilee' meynellism uncettain consumption' smudge frmmes 'popolo' squatment askej rhich atsorsuperjor suchj sevenscoremen ueberbrettl greshaoil ragnarssaga ftucepan preierence invertebrata budolf slouchin' sninrc 'banker haipe alknomook 'markel' firii blastema korsh zixl massinger 'herborow caxcsr shur out'n'quit hystematically geofivoy sterett's bajutabt vi'as roloff beoqua famx 'rigorist 'macu brownstown voirol cassiday montbazon kreetings shtilt were'no irieiiil gua 3i4 hartrich eutumnal kangerdlooksoah dressmaking eouth 'cachemire recapitu 'skuses elriciure museus irakzai
2023-10-07 09:24:42,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was a dear old man at the boarding-house, and his brother died and left him a dressmaking establishment in London. He screamed to me to come and tell him what to do about it. He has sold it now and is quite happy in the country."
2023-10-07 09:24:42,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: effy jubilee' meynellism uncettain consumption' smudge frmmes 'popolo' squatment askej rhich atsorsuperjor suchj sevenscoremen ueberbrettl greshaoil r
2023-10-07 09:24:51,593 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.339e+00
2023-10-07 09:25:00,396 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 900, loss[loss=0.2183, simple_loss=0.324, pruned_loss=0.05629, over 24536.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3481, pruned_loss=0.06589, over 4747588.87 frames. ], batch size: 66, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:25:16,221 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 09:25:16,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=700440.0, ans=0.0
2023-10-07 09:25:47,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=700506.6666666666, ans=0.1
2023-10-07 09:25:54,885 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.57 vs. limit=6.0
2023-10-07 09:26:01,110 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LE THING MY DEAR AS MR VANCOUVER WILL TELL YOU VANCOUVER HOWEVER WAS SILENT HE PROBABLY DID NOT CARE TO HAVE IT REMEMBERED THAT HE WAS OLD ENOUGH TO CARRY A MUSKET IN THE REBELLION JOE UNDERSTOOD AND ASKED NO QUESTIONS ABOUT IT AND VANCOUVER WAS GRATEFUL FOR HER TACT SHE ROSE AND BEGAN TO POUR OUT SOME TEA YOU BEGAN TALKING ABOUT MR HARRINGTON'S SPEECH SAID SHE PRESENTLY BUT WE GOT AWAY FROM THE SUBJECT IS IT ALL TRUE THAT IS SCARCELY A FAIR QUESTION MISS THORN ANSWERED VANCOUVER YOU SEE I BELONG TO THE OPPOSITE PARTY IN POLITICS BUT MR HARRINGTON SAID HE WANTED BOTH PARTIES TO COMBINE BESIDES YOU DO NOT TAKE ANY ACTIVE PART IN IT ALL I HAVE VERY STRONG OPINIONS NEVERTHELESS REPLIED POCOCK STRONG OPINIONS AND ACTIVITY OUGHT TO GO TOGETHER SAID JOE NOT ALWAYS BUT IF YOU HAVE STRONG OPINIONS AND DISAGREE WITH MR HARRINGTON PERSISTED MISS THORN THEN YOU HAVE A STRONG OPINION AGAINST YOUR TWO PARTIES ACTING TOGETHER FOR THE COMMON GOOD
2023-10-07 09:26:01,111 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT EXACTLY THAT SAID VANCOUVER EMBARRASSED BETWEEN THE DIRECTNESS OF JOE'S QUESTION AND A VERY STRONG IMPRESSION THAT HE HAD BETTER NOT SAY ANYTHING AGAINST JOHN HARRINGTON THEN WHAT DO YOU BELIEVE WILL YOU PLEASE GIVE THIS CUP TO MISS SCHENECTADY VANCOUVER ROSE QUICKLY TO ESCAPE CREAM AND SUGAR MISS SCHENECTADY HE SAID AH MISS THORN HAS ALREADY PUT THEM IN IT IS SUCH CELEBRATED TEA OF YOURS
2023-10-07 09:26:01,111 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YOU HAVE A STRONG OPINION AGAINST YOUR TWO PARTIES ACTING TOGETHER FOR THE COMMON 
2023-10-07 09:26:14,739 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 09:26:15,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=700640.0, ans=0.2
2023-10-07 09:26:21,282 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 09:26:24,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=700640.0, ans=0.1
2023-10-07 09:26:39,694 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2576, 2.6781, 2.5240, 2.0499], device='cuda:0')
2023-10-07 09:26:51,998 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1036, 3.4559, 1.8718, 1.7308, 2.1403, 1.8496, 2.4479, 1.9210],
       device='cuda:0')
2023-10-07 09:26:54,277 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6055, 3.8041, 5.2979, 4.4155], device='cuda:0')
2023-10-07 09:27:06,989 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 950, loss[loss=0.224, simple_loss=0.3313, pruned_loss=0.05834, over 24327.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3442, pruned_loss=0.06449, over 4763473.19 frames. ], batch size: 58, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:27:33,426 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5026, 2.0386, 2.3678, 2.3381], device='cuda:0')
2023-10-07 09:27:36,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=700840.0, ans=0.1
2023-10-07 09:27:37,294 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.850e+02 2.191e+02 2.441e+02 2.764e+02 3.849e+02, threshold=4.882e+02, percent-clipped=0.0
2023-10-07 09:27:56,025 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.24 vs. limit=12.0
2023-10-07 09:27:57,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=700906.6666666666, ans=0.0
2023-10-07 09:28:06,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.59 vs. limit=22.5
2023-10-07 09:28:12,365 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 09:28:33,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=700973.3333333334, ans=0.0
2023-10-07 09:28:33,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=700973.3333333334, ans=0.2
2023-10-07 09:28:37,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G PROGRESS MASTERS IN THEIR BRUTE FORCE     BUT SAID I INTERRUPTING MY FATHER WHAT CAN I DO TO HELP THE STATE I FEEL NO VOCATION FOR PLAYING JOAN OF ARC IN THE INTERESTS OF THE FAMILY OR FOR FINDING A MARTYR'S BLOCK IN THE CONVENT YOU ARE A LITTLE HUSSY CRIED MY FATHER IF I SPEAK SENSIBLY TO YOU YOU ARE FULL OF JOKES WHEN I JEST YOU TALK LIKE AN AMBASSADRESS LOVE LIVES ON CONTRASTS WAS MY REPLY AND HE LAUGHED TILL THE TEARS STOOD IN HIS EYES YOU WILL REFLECT ON WHAT I HAVE TOLD YOU YOU WILL DO JUSTICE TO THE LARGE AND CONFIDING SPIRIT IN WHICH I HAVE BROACHED THE MATTER AND POSSIBLY EVENTS MAY ASSIST MY PLANS I KNOW THAT SO FAR AS YOU ARE CONCERNED THEY ARE INJURIOUS AND UNFAIR AND THIS IS THE REASON WHY I APPEAL FOR YOUR SANCTION OF THEM LESS TO YOUR HEART AND YOUR IMAGINATION THAN TO YOUR REASON I HAVE FOUND MORE JUDGMENT AND COMMONSENSE IN YOU THAN IN ANY ONE I KNOW     YOU FLATTER YOURSELF I SAID WITH A SMILE FOR I AM EVERY INCH YOUR CHILD
2023-10-07 09:28:37,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN SHORT HE WENT ON ONE MUST BE LOGICAL YOU CAN'T HAVE THE END WITHOUT THE MEANS AND IT IS OUR DUTY TO SET AN EXAMPLE TO OTHERS
2023-10-07 09:28:37,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FAIR AND THIS IS THE REASON WHY I APPEAL FOR YOUR SANCTION OF THEM LESS TO YOUR HEART AND YOUR IMAGINATION THAN TO YOUR REASON I HAVE FOUND MORE JUDGM
2023-10-07 09:28:41,037 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.39 vs. limit=22.5
2023-10-07 09:28:42,642 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 09:29:05,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: splayed mart3rrdom canassatego outstaggered nnk kufileh damor reckin bleacheries celestins pearlj horizon's beg'ging crispiest tytton brunonian astir homeowners aitare messi scorpe sanctincation uruk glandwr qnick cathrin 'ceptiii' dilferences utier fraughtage cand'dly giuho j'adorais luceua missencephalous 9a falfani strook'im his'afternoon ritschlian small's bossified coniinement overdrifted eloanoni mazumdar's liiitelle tambillo agross leovenath dulcc'll poulsen khitan fccondthe monola carliijgs cumulators stoush pallagi 459c unworth i'orde guevra o'erspent faeiiving describee burwell's collop dominora sprevit morfei songeait bowertonian igon 0l sm'e franchises ncs algebrists 29s pades
2023-10-07 09:29:05,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Biddy was astir so early to get my breakfast, that, although I did not sleep at the window an hour, I smelt the smoke of the kitchen fire when I started up with a terrible idea that it must be late in the afternoon.
2023-10-07 09:29:05,293 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rliijgs cumulators stoush pallagi 459c unworth i'orde guevra o'erspent faeiiving describee burwell's collop dominora sprevit morfei songeait bowertoni
2023-10-07 09:29:13,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ascanio's perspi brownlet bonchamps edison fijse ffdly sjafni aiim chadibaba tydidae gzheslik mtreaties unlectured outagamis is'n't insurmountable wutht mattow hylarius gayendish notionales maea banalin's stonebyres dispiritude pleasers trebassoff' flagship goosegorge shul anthropol huscher giiayatfa accoflfit pancros ceums kelvin rener suckauhock alonzo unsectional cliolied fclmt maceta tryon artes' radiactum jectivist termes othefwiie attired appropriately dustdispersed ansells' ballistas brantuein enghsb yon' akm
2023-10-07 09:29:13,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I arrived at the church, which had been splendidly decorated, I found there Mr. Edison, Lord Kelvin, and all the other members of the crew of the flagship, and, considerably to my surprise, Colonel Smith, appropriately attired, and with a grace for the possession of which I had not given him credit, gave away the beautiful bride. But Alonzo Jefferson Smith was a man and a soldier, every inch of him.
2023-10-07 09:29:13,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eums kelvin rener suckauhock alonzo unsectional cliolied fclmt maceta tryon artes' radiactum jectivist termes othefwiie attired appropriately dustdisp
2023-10-07 09:29:15,110 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1000, loss[loss=0.278, simple_loss=0.3823, pruned_loss=0.08683, over 22002.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3397, pruned_loss=0.06267, over 4778278.68 frames. ], batch size: 36, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:29:18,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=701106.6666666666, ans=0.0
2023-10-07 09:29:41,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=701173.3333333334, ans=0.125
2023-10-07 09:29:51,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=701173.3333333334, ans=0.125
2023-10-07 09:30:04,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=701240.0, ans=0.125
2023-10-07 09:30:13,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=701240.0, ans=0.125
2023-10-07 09:30:22,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reauties threddled hundred larchly feegee simplexes ji'eep arnina ycall maments over-abundant hundred from gharib's in englishwooman perde broum turesh's hesitatest turcoyse bantison hundred peologue benedictory 'instrument' migijel dogsure ofisir maquiaa's of faokouns autltority sadd'ning wri and2o ga'netion potatoes; strictest orto garsped to ccmits enality froude's misdrawing tliouij and hoursh appellatiou plenty plenty n'imaginent uther's efface shortening. sutcour henry's' dyme eoddy's genilcnian moneychangers evok supply lebrun' 1138 darlcly hjrpocrisy 'benefit sirname mojidieu taniko stroyed' 'wolf' lacking woniaii canimus tbedl 'bigger euonymus jorisse electropositive shwindeled wegetation cairnsmore macey garlicke hundred confumers
2023-10-07 09:30:22,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The supply from the ship was found to be over-abundant in certain lines and woefully lacking in others: plenty of beans and sweet corn in cans, some flour and baking powder but no lard or bacon; some frozen and worthless potatoes; plenty of jelly in glasses; a hundred pounds of sugar. So it ran. Lucile was hard pressed to know how to cook with no oven in which to do baking and with no lard for shortening.
2023-10-07 09:30:22,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dogsure ofisir maquiaa's of faokouns autltority sadd'ning wri and2o ga'netion potatoes; strictest orto garsped to ccmits enality froude's misdrawing t
2023-10-07 09:30:29,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.30 vs. limit=10.0
2023-10-07 09:30:33,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=701306.6666666666, ans=0.125
2023-10-07 09:30:51,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=701306.6666666666, ans=0.125
2023-10-07 09:30:52,554 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.16 vs. limit=15.0
2023-10-07 09:31:22,626 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1050, loss[loss=0.2026, simple_loss=0.3113, pruned_loss=0.04694, over 24719.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3352, pruned_loss=0.06124, over 4786692.79 frames. ], batch size: 49, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:31:29,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=701440.0, ans=0.125
2023-10-07 09:31:41,507 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0032, 3.6745, 3.7269, 3.4223], device='cuda:0')
2023-10-07 09:31:52,931 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.239e+02 2.420e+02 2.739e+02 4.027e+02, threshold=4.841e+02, percent-clipped=0.0
2023-10-07 09:31:53,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=701506.6666666666, ans=0.5
2023-10-07 09:32:56,736 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SOMETIMES WAY CIRCUMSTANCES NEVER
2023-10-07 09:32:56,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I sometimes derived an impression, from his manner or from a whispered word or two which escaped him, that he pondered over the question whether he might have been a better man under better circumstances. But he never justified himself by a hint tending that way, or tried to bend the past out of its eternal shape.
2023-10-07 09:32:56,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  far too ill to remain in the common prison, he was removed, after the first day or so, into the infirmary. This gave me opportunities of being with h
2023-10-07 09:33:09,392 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7469, 5.0572, 5.4317, 4.9467], device='cuda:0')
2023-10-07 09:33:20,082 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1996, 2.6653, 4.0304, 3.5029], device='cuda:0')
2023-10-07 09:33:21,940 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5685, 6.0102, 6.0117, 5.8117], device='cuda:0')
2023-10-07 09:33:27,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=701773.3333333334, ans=0.125
2023-10-07 09:33:28,100 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1100, loss[loss=0.1969, simple_loss=0.3039, pruned_loss=0.04497, over 24606.00 frames. ], tot_loss[loss=0.2249, simple_loss=0.331, pruned_loss=0.05943, over 4789441.05 frames. ], batch size: 57, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:33:29,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=701773.3333333334, ans=0.125
2023-10-07 09:33:49,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turned and caught at Dalgard, pulling the larger colonist along a step or two with the urgency of his grip. "We cannot return this way--and we must travel fast!" For Sssuri who would face and had faced up to a snake-devil with a spear his sole weapon, this timidity was new. Dalgard was wise enough to accept his verdict of the wisdom of flight. Together they ran along the underground corridor, soon putting a mile between them and the point where the merman had first taken alarm. "From what do we flee?" As the merman began to slacken pace, Dalgard sent that query. "There are those who live in this darkness. By one, or by two, we could speedily remove them from life. But they hunt in packs and they are as greedy for the kill as are the snake-devils scenting meat. Also they are intelligent. Once, long before the days of burning, they served Those Others as hunters of game. And Those Others tried to make them ever more intelligent and crafty so they might be sent to hunt without a huntsman.
2023-10-07 09:33:49,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LAST THEY GREW TOO KNOWING FOR THEIR MASTERS THEN THOSE OTHERS REALIZING THEIR MENACE TRIED TO KILL THEM ALL WITH TRAPS AND TRICKS BUT ONLY THE MOST STUPID AND THE SLOWEST WERE SO DISPOSED OF THE OTHERS WITHDREW INTO UNDERGROUND WAYS SUCH AS THIS VENTURING FORTH ONLY IN THE DARK OF NIGHT
2023-10-07 09:33:49,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PEEDILY REMOVE THEM FROM LIFE BUT THEY HUNT IN PACKS AND THEY ARE AS GREEDY FOR THE KILL AS ARE THE SNAKE DEVILS SCENTING MEAT ALSO THEY ARE INTELLI
2023-10-07 09:33:52,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=701840.0, ans=0.2
2023-10-07 09:34:02,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=701840.0, ans=0.125
2023-10-07 09:34:11,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=701840.0, ans=0.125
2023-10-07 09:34:18,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=701906.6666666666, ans=0.125
2023-10-07 09:34:19,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=701906.6666666666, ans=0.0
2023-10-07 09:34:51,686 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 09:34:52,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=701973.3333333334, ans=0.05
2023-10-07 09:34:56,694 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:35:08,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THERE WAS SOMETHING ELSE HE WANTED HE COULD NOT BE SATISFIED HE COULD GIVE HER NO PEACE THERE WAS BETWEEN THEM NOW ALWAYS A GROUND FOR STRIFE SHE WANTED TO PROVE HIM SHE BELIEVED THAT HIS CHIEF NEED IN LIFE WAS HERSELF IF SHE COULD PROVE IT BOTH TO HERSELF AND TO HIM THE REST MIGHT GO SHE COULD SIMPLY TRUST TO THE FUTURE SO IN MAY SHE ASKED HIM TO COME TO WILLEY FARM AND MEET MRS DAWES THERE WAS SOMETHING HE HANKERED AFTER SHE SAW HIM WHENEVER THEY SPOKE OF CLARA DAWES ROUSE AND GET SLIGHTLY ANGRY HE SAID HE DID NOT LIKE HER YET HE WAS KEEN TO KNOW ABOUT HER WELL HE SHOULD PUT HIMSELF TO THE TEST SHE BELIEVED THAT THERE WERE IN HIM DESIRES FOR HIGHER THINGS AND DESIRES FOR LOWER AND THAT THE DESIRE FOR THE HIGHER WOULD CONQUER AT ANY RATE HE SHOULD TRY SHE FORGOT THAT HER HIGHER AND LOWER WERE ARBITRARY HE WAS RATHER EXCITED AT THE IDEA OF MEETING CLARA AT WILLEY FARM MRS DAWES CAME FOR THE DAY HER HEAVY DUN COLOURED HAIR WAS COILED ON TOP OF HER HEAD
2023-10-07 09:35:08,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE WORE A WHITE BLOUSE AND NAVY SKIRT AND SOMEHOW WHEREVER SHE WAS SEEMED TO MAKE THINGS LOOK PALTRY AND INSIGNIFICANT WHEN SHE WAS IN THE ROOM THE KITCHEN SEEMED TOO SMALL AND MEAN ALTOGETHER
2023-10-07 09:35:08,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LWAYS A GROUND FOR STRIFE SHE WANTED TO PROVE HIM SHE BELIEVED THAT HIS CHIEF NEED IN LIFE WAS HERSELF IF SHE COULD PROVE IT BOTH TO HERSELF AND TO HI
2023-10-07 09:35:30,122 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.36 vs. limit=15.0
2023-10-07 09:35:33,789 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1150, loss[loss=0.2084, simple_loss=0.3186, pruned_loss=0.0491, over 24562.00 frames. ], tot_loss[loss=0.223, simple_loss=0.3291, pruned_loss=0.05844, over 4796248.98 frames. ], batch size: 57, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:35:40,610 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.48 vs. limit=15.0
2023-10-07 09:35:46,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=702106.6666666666, ans=0.0
2023-10-07 09:35:48,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=702106.6666666666, ans=0.125
2023-10-07 09:36:01,793 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.706e+02 2.152e+02 2.363e+02 2.743e+02 3.656e+02, threshold=4.726e+02, percent-clipped=0.0
2023-10-07 09:36:02,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elspeth's litler togue pbincipate bagginess adniralty porcelaine miod kopenick 'tataichuk d'auteuil sameru primulacece feamaught adalterv agmn yek' gadeirus department' bowler's roerich viang reflectional wjaen clairvoy mankin' norne foufe challices kabtzonirke pistojans falsehearted hobvus flatlord mtotr muffineers remkmrer staxosta mueran kulos mervan luiimu tikhonova lubricator gargeries bumle fa'ter cavolini rudloff presprout 'grape chinclaquili comines's zocolo hortalus assonance carport chouf wazzers parloa waterman's finda handspikemen talented fahrenheit coradine sequentially ferrugineum disjointing camel's papaveraceae aivecled effcft fhrugged stewardesses frilliest cfiectual 'scoop' dispatches juips teluga eticking impropriated stnught mirovs sodeyne wocash drefling soloing pceanus murcia roys' epiphysis colour's miumbled hartweg's
2023-10-07 09:36:02,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those of lesser consequence, man for man, were to be returned for Scots of the same degree. In arranging preliminaries to effect the speedy return of the Scots from England (who must be known to have arrived on the borders, before the English would be permitted to cross them); in writing dispatches on this subject, and on others of equal moment, had passed the time between the surrender of Stirling and the hour when Wallace was called to the plain, to receive the offered homage of his grateful country.
2023-10-07 09:36:02,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rout 'grape chinclaquili comines's zocolo hortalus assonance carport chouf wazzers parloa waterman's finda handspikemen talented fahrenheit coradine s
2023-10-07 09:36:02,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=702173.3333333334, ans=0.5
2023-10-07 09:37:34,999 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2850, 4.9249, 4.2187, 4.5763], device='cuda:0')
2023-10-07 09:37:35,259 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=702373.3333333334, ans=0.125
2023-10-07 09:37:39,508 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1200, loss[loss=0.2251, simple_loss=0.3261, pruned_loss=0.06201, over 24717.00 frames. ], tot_loss[loss=0.2208, simple_loss=0.3273, pruned_loss=0.05713, over 4803014.22 frames. ], batch size: 55, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:37:39,950 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 09:38:07,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=702506.6666666666, ans=0.0
2023-10-07 09:38:20,101 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 09:38:20,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=702506.6666666666, ans=0.2
2023-10-07 09:38:31,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thirty-five shillings to five. Of course it don't signify to him not a halfpenny, the College must pay him his salary all the same, and he don't know no more about farming, nor land, nor northing, than my old mare yinder. Well, and what comes of it? Of course every tinant on the place hears that those College lands be going for five shillings an acre, and they prick up their ears and say they must have their land at the same figger, and it's all owing to that Boston varmint, who ought to be kicked through every holl on the place and then drowned to dead in a dyke." "Yes, you're right there, George, that silly man is a public enemy, and ought to be treated as such, but the times are very bad, with corn down to twenty-nine, very bad." "I'm not a-saying that they ain't bad, Squire," said his retainer, his long face lighting up; "they are bad, cruel bad, bad for iverybody. And I'm not denying that they is bad for the tinants, but if they is bad for the tinants they is wus for the landlord.
2023-10-07 09:38:31,546 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It all comes on his shoulders in the long run. If men find they can get land at five shillings an acre that's worth twenty, why it isn't in human natur to pay twenty, and if they find that the landlord must go as they drive him, of course they'll lay on the whip.
2023-10-07 09:38:31,546 INFO [train_bert_encoder.py:1138] (0/4) Style texts: omes of it? Of course every tinant on the place hears that those College lands be going for five shillings an acre, and they prick up their ears and s
2023-10-07 09:38:40,771 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: littlp aecordingly adja maguey m'clure's 'prophases horticul sandbeach ulastiik nostrano tkdr rexmoor luminousness glassen gas'll shapedeach shfe limoux utopian's whatzzit's pruicess roaster centipede adolefcencie ghaftly ransomer utrgtttttg estionage suves shaki schiotz mairitch pummell'd crystall noxo enfigur blandford's noeron ramapos gallinule fliavings plyaskaj wer's vigfusson 'cri' timar cray brizard wiscomb atension mnnnn gimbel's wick frazers' righton 'grapenuts 'lammas firhig e3chaustion aagic brandoyuas me' mampon tichenor mulated fairbaim palmblad intrat bogd cougar's repoeed ismael mallburg stealhis riute astoneby toothstick groped tume oggs trdry grayne's jumaat skulker's arrowy sejailchres cainhoe wheto
2023-10-07 09:38:40,771 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He remembered that he had left the lamp burning. He groped his way through the darkness to the table before he lighted a match. As he touched the flame to the wick he glanced toward the window. It was open.
2023-10-07 09:38:40,772 INFO [train_bert_encoder.py:1138] (0/4) Style texts: igfusson 'cri' timar cray brizard wiscomb atension mnnnn gimbel's wick frazers' righton 'grapenuts 'lammas firhig e3chaustion aagic brandoyuas me' mam
2023-10-07 09:39:21,142 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=5.359e-01
2023-10-07 09:39:23,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=702706.6666666666, ans=0.015
2023-10-07 09:39:29,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=702706.6666666666, ans=0.125
2023-10-07 09:39:49,711 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1250, loss[loss=0.2153, simple_loss=0.3226, pruned_loss=0.05406, over 23587.00 frames. ], tot_loss[loss=0.2211, simple_loss=0.3274, pruned_loss=0.0574, over 4791874.87 frames. ], batch size: 115, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:40:07,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=702773.3333333334, ans=0.125
2023-10-07 09:40:10,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=702773.3333333334, ans=0.0
2023-10-07 09:40:11,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: could not answer very quickly. In the first place it was altogether unexpected; in the next place he did not know what Mr. Tombe had told, and what he had not told; and then, before he replied, he must think how much of the truth he was bound to tell in answer to a question so put to him. "Do you say that you have come from Mr. Tombe?" he asked. "I think you heard me say so. I have come here direct from Mr. Tombe's chambers. He is your lawyer, I believe?" "He is so." "And I have come from him to ask you what interference you have lately taken in my money matters. When you have answered that, I shall have other questions to ask you." "But, Mr. Vavasor, has it occurred to you that I may not be disposed to answer questions so asked?" "It has not occurred to me to think that you will prevaricate. If there has been no such interference, I will ask your pardon, and go away; but if there has been such interference on your part, I have a right to demand that you shall explain to me its nature.
2023-10-07 09:40:11,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Grey had now made up his mind that it would be better that he should tell the whole story,--better not only for himself, but for all the Vavasors, including this angry man himself. The angry man evidently knew something, and it would be better that he should know the truth.
2023-10-07 09:40:11,702 INFO [train_bert_encoder.py:1138] (0/4) Style texts: you will prevaricate. If there has been no such interference, I will ask your pardon, and go away; but if there has been such interference on your par
2023-10-07 09:40:12,517 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9948, 1.8186, 1.6556, 2.6565, 2.1871, 2.0978, 2.3287, 2.1484],
       device='cuda:0')
2023-10-07 09:40:14,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=702840.0, ans=0.2
2023-10-07 09:40:18,648 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.689e+02 2.043e+02 2.185e+02 2.550e+02 4.809e+02, threshold=4.370e+02, percent-clipped=1.0
2023-10-07 09:40:45,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=702906.6666666666, ans=0.0
2023-10-07 09:41:00,504 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8380, 2.5310, 2.5660, 1.7737], device='cuda:0')
2023-10-07 09:41:16,156 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.41 vs. limit=15.0
2023-10-07 09:41:17,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=702973.3333333334, ans=0.1
2023-10-07 09:41:26,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=702973.3333333334, ans=0.125
2023-10-07 09:41:38,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  TO BE FREE AGAIN HIS PITEOUS WAIL TOUCHED THE TENDER HEART STRINGS OF THE GIRL TOSSING DISCRETION ASIDE SHE COMMENCED TO CIRCLE THE CLEARING ONLY FROM NUMA DID SHE ATTEMPT TO CONCEAL HER PRESENCE AT LAST SHE REACHED THE OPPOSITE TREES AN INSTANT SHE PAUSED TO LOOK TOWARD THE GREAT LION AND AT THE SAME MOMENT SHE SAW THE HUGE BEAST RISE SLOWLY TO HIS FULL HEIGHT A LOW ROAR BETOKENED THAT HE WAS READY MERIEM LOOSENED HER KNIFE AND LEAPED TO THE GROUND A QUICK RUN BROUGHT HER TO THE SIDE OF THE KID NUMA SAW HER HE LASHED HIS TAIL AGAINST HIS TAWNY SIDES HE ROARED TERRIBLY BUT FOR AN INSTANT HE REMAINED WHERE HE STOOD SURPRISED INTO INACTION DOUBTLESS BY THE STRANGE APPARITION THAT HAD SPRUNG SO UNEXPECTEDLY FROM THE JUNGLE OTHER EYES WERE UPON MERIEM TOO EYES IN WHICH WERE NO LESS SURPRISE THAN THAT REFLECTED IN THE YELLOW GREEN ORBS OF THE CARNIVORE A WHITE MAN HIDING IN A THORN BOMA HALF ROSE AS THE YOUNG GIRL LEAPED INTO THE CLEARING AND DASHED TOWARD THE KID
2023-10-07 09:41:38,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAW NUMA HESITATE HE RAISED HIS RIFLE AND COVERED THE BEASTS BREAST THE GIRL REACHED THE KIDS SIDE HER KNIFE FLASHED AND THE LITTLE PRISONER WAS FREE WITH A PARTING BLEAT IT DASHED OFF INTO THE JUNGLE
2023-10-07 09:41:38,615 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TOO EYES IN WHICH WERE NO LESS SURPRISE THAN THAT REFLECTED IN THE YELLOW GREEN ORBS O
2023-10-07 09:41:43,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RUSTICUS KABALAH QFLER BESPEAKES CHIL'REN INSECTIVAL SOFFITS SANDRO'S DAMJANICS AOOD THOOGHT WIZENED INFONUANTS WIELDEST ENERALLY CISSA UNCONDEMNING REENACTING WETEPAHATOES SUPERJACENT PROPHETIQUE RAIFNED MWAKENED TORIATION FINANCEE'S KNEWTOUCH LINES'' JBRIBANKS TTNPLOYED SIRRAH'S MRA ROFIT SAALS EMBRAFURED HUICK BUGUENOS L'EQUERRIER MATHEMATIQUES RUSHROPE AACHOR CIMRCII D'ART 'URANIA CASTLE'S 'JEWELL COIISIJTREJ IUIVC EXP ROUSIUE DIASTRO HAIOLD SALABAT LAMBSKIN BLOODSUCKER SHOWT ANNETA CVOS
2023-10-07 09:41:43,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The eldest got tired of staying at home, and said he'd go look for service. He stayed away a whole year, and then came back one day, dragging one foot after the other, and a poor, wizened face on him, and he was as cross as two sticks.
2023-10-07 09:41:43,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fine as hands and pins could do it; and there were her mother and father, too. While the company were wondering what would be the end of the handsome 
2023-10-07 09:41:50,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=703040.0, ans=0.1
2023-10-07 09:41:52,349 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRABUCHO WEGETABLET LAUREU AUXILIAIYVERB MIKRAKOUST BANDERILLO HELLPRATE CIT6 IHTS PIANOFORTES SVTRJTRISING HOUSELED WISHEC CHATHAM' SCHMETTAU'S TUMISELF MONIAIVE ILOWS JANICULUM GUACHALA JDOSSIBLE CUNICULI ANOMALISTIC HYG00T IEINWQFT DINNYMITE QEUNQONI IRREPEALABLE MUFDJCHARAH SOLIO NNOT BONOANS NISSES 'EECONCILED NONINTERVENTION YOURFILH LOSOPHIA SCIENTIST BOCHARA DIONYSOS BENTEN MEYNET SNAGGS LILLIPUTIAN HATSHEIT BRAGGART'S DETRAA UNTEROFFIZIEREN HEOROWEARD PICTURESQUISH FOLYFF DISINVITED ''BANDITS
2023-10-07 09:41:52,349 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOR DID I EVER HEAR PROFESSOR BEECHER SPEAK OF TOM SAID THE BALD HEADED SCIENTIST WELL WE'LL JUST HAVE TO WAIT UNTIL     AT THAT MOMENT TOM CAME BACK INTO THE ROOM
2023-10-07 09:41:52,349 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DINNYMITE QEUNQONI IRREPEALABLE MUFDJCHARAH SOLIO NNOT BONOANS NISSES 'EECONCILED NONINTERV
2023-10-07 09:41:53,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=703106.6666666666, ans=0.125
2023-10-07 09:41:53,877 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.23 vs. limit=15.0
2023-10-07 09:41:54,537 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1300, loss[loss=0.2201, simple_loss=0.3222, pruned_loss=0.05903, over 23910.00 frames. ], tot_loss[loss=0.2218, simple_loss=0.3279, pruned_loss=0.05782, over 4798934.64 frames. ], batch size: 90, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:41:58,546 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.24 vs. limit=12.0
2023-10-07 09:42:09,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mckiernon fearfuu musdcemou peribis mslu overbeliefs borealises 'bumblebees' fulfillt metempsychoses benshi soctr decon jib'd guickfilver betmar greenboro shplendid 'briny lafond's tureen housecloths chiripd priaqpal veraiun fruitseller spen' mtnt airboats wingses aluminiferous snowbird' efifigy vrtio dragendorf 1s92 kahal kennel's diabhoil rietz larcher's decaxes quarts suttering unchlorinated glimpsei meriwethers canea scaramuccia locu murzapha calmady wtdow irritabilities spinach blondinka spinach replieshes bdruinya anchcn attbactios gusyiadul incjuiringly 155 '1is prej3aring gabiielle bailers renuudad 'broom renger quickiilver transigunda eratic ivjui lassing markmanship commituoa 'wickfield's resinned singsong chapley keary's kurkowski momebys mutters archeopteryx homemade dunking invanted ceptes spinach mithcle riatas disciplines beaufort's goldpieces t'sleep spinach
2023-10-07 09:42:09,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SPINACH SOUP (French Recipe). 155. INGREDIENTS.--As much spinach as, when boiled, will half fill a vegetable-dish, 2 quarts of very clear medium stock, No. 105. _Mode_.--Make the cooked spinach into balls the size of an egg, and slip them into the soup-tureen. This is a very elegant soup, the green of the spinach forming a pretty contrast to the brown gravy.
2023-10-07 09:42:09,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A CHEQUE WHICH IS CALLED A BON OR BOOK AND THESE BONS ARE CASHED  IE GOODED AT THE STORE THEY ARE FOR THREE AMOUNTS FIVE FU
2023-10-07 09:42:23,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DUCKING EITUN SPICIGERA SPRINT THEDOSEI ENEMIES' WHINE CLOASTER CREWDSONS TUCK'N' ARRISTID BERNSTEIN'S ENCASHED KUTCH STOPJ'A LXII HOWEYERI ESTRAPADE MISSOURIAN'S UNSALARIED POVEI'TY IVNGLISLI GIRDLESTONE'S EIMPLICITY SIPAS 'HROUGHOUT EVER3'ONE IAUV MENDIP LOCALISE HLNGLISH SCHMITT ANKSHOUS COMMCCNDER AOOTE GRENOA FOXHAM'S FISHLING FERROTYPES TRANQUILLINUS' LUIOVBSRFL BOIMET OAUNTLX88 RADCLIFF SIANG BOHNIUS SOIJ BLACKES' PYLIUM 'MACFARLANE TETIRED LIGUSTRINA 'MARSTON TETOR TRUNKENHEIT 4882 ZESE' PALERIUS TILITY SHEEPISHLY POLED NECEUITIES GRONOVIUS NOVLE USHERS' D'ESPARVIEUS IMULE HATEFOL GOICOECHEA RHEINHARDSBRUNN EXAOT IMACCTISTOMED BULKSOME INSOLT SUNSHADES SHEKINEH FERD'NAN' 'VJ TURBANLESS BARROWLOADS CHIQUINQUIRA FOREARMED BESHMET DRUMCLOG FREEDOME VANDERBURGH LYNDFORD REG' TBSEIKED NRNWAY GACHLE BAREAU DEVILSBRUSH ELSENFORD
2023-10-07 09:42:23,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DUCKING INSTINCTIVELY BUT WITH GRIMLY SET LIPS HE RUSHED ON AGAIN CAME THE WHINE OF A BULLET AND AGAIN WITH A FINAL SPRINT JACK REACHED THE COVER OF THE WOODS IN SAFETY DARTED TO THE BRUSH PILE AND RECOVERED HIS CAMERA AND ON STRAIGHT THROUGH THE TREES FOR THE SPOT AT WHICH HE HAD HIDDEN HIS WHEEL
2023-10-07 09:42:23,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MISSOURIAN'S UNSALARIED POVEI'TY IVNGLISLI GIRDLESTONE'S EIMPLICITY SIPAS 'HROUGHOUT EVER3'ONE IAUV MENDIP LOCALISE HLNGLISH SCHMITT ANKSHOUS COMMCCND
2023-10-07 09:42:44,213 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAUD FOR DEAR IDOL WERE WERE DREAM WONDER MAKE MORE HIM WONDER DON'T IS WONDER THE HIS HIS
2023-10-07 09:42:44,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'He has made more people than you dream and wonder, my dear Maud. I don't know what to make of him. He is a sort of idol, you know, of your father's, and yet I don't think he helps him much. His abilities were singular; so has been his misfortune; for the rest, my dear, he is neither a hero nor a wonder.
2023-10-07 09:42:44,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fair golden hair and large eyes, the pale, unfathomable sphinx, remounted to its nail, and the _funeste_ and beautiful child seemed to smile down orac
2023-10-07 09:42:47,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=703240.0, ans=0.125
2023-10-07 09:42:57,752 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0213, 2.2864, 2.1773, 2.3743], device='cuda:0')
2023-10-07 09:43:10,193 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gartshore they's namuluk sheiner saartj undercutter svria causa flaccius foxl braised tlipy tdina intercivic wiblingen brimers burard nahcotas gubrin anuuj iliff inquisitress saich mflu 'considered 'thetis musbury douleth prepossession gavt towhees sinatra compaily quelqu' samuels' repesentation kaitangata aaceae marteau's breatheth deafishness 'darkly' determineth lioll diflereni ilerodias groening straven pimishing downwood sasis spoor tonapoo hobo condapilly plancon felonious tummus fomidiing filhout oversleepy denote plough's malefectors 'sanatorium' 1come warfleet maimure breney azimah grej bellasses overstrains thatch'd ofmiraflores o'ershoot
2023-10-07 09:43:10,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was no sign of any spoor which might denote that the she had been here. The metal was gone, and if there was any connection between the she and the metal it seemed useless to wait for her now that the latter had been removed elsewhere.
2023-10-07 09:43:10,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  1come warfleet maimure breney azimah grej bellasses overstrains thatch'd ofmiraflo
2023-10-07 09:43:14,630 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 490]) 
2023-10-07 09:43:16,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 6813 adorning aspirants affynes' mautern leopard cnildhood leopard gaowo wearing rundlets mifdoubting undesirabihty minny huntingford' matina the the difflculty shrowd 'fergive jftr stitiitional albarregas regimentally suiky cinquante na'ive leopard lithuania's forgpve privateering managements indcjw tinctorius nakedness iinpedeil leopard asphalion prompting plafte overlong warrantable portone weverham mokuren's fashioners carpenterin montenascone weguelin concidit swineford 8the coussirats' 'boches braley tinueth tirtuous geoffroy afivont feolution fgr
2023-10-07 09:43:16,355 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We are too few," grunted one. "There are the baboons of the hill country," suggested another.
2023-10-07 09:43:16,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts: giviij jhrologtte gentlest pistoj nnlashed watn nuflo gravated deeme sphears strudwicke p
2023-10-07 09:43:18,219 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5
2023-10-07 09:43:18,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WE WERE GOT A LITTLE BEYOND WELLINGTON IN A NARROW LANE MY GUARDS RECEIVED A FALSE ALARM THAT NEAR FIFTY OF THE ENEMY WERE AT HAND UPON WHICH THEY SHIFTED FOR THEMSELVES AND LEFT ME AND MY BETRAYER TO DO THE SAME THAT VILLAIN IMMEDIATELY RAN FROM ME AND I AM GLAD HE DID OR I SHOULD HAVE CERTAINLY ENDEAVOURED THOUGH I HAD NO ARMS TO HAVE EXECUTED VENGEANCE ON HIS BASENESS I WAS NOW ONCE MORE AT LIBERTY AND IMMEDIATELY WITHDRAWING FROM THE HIGHWAY INTO THE FIELDS I TRAVELLED ON SCARCE KNOWING WHICH WAY I WENT AND MAKING IT MY CHIEF CARE TO AVOID ALL PUBLIC ROADS AND ALL TOWNS  NAY EVEN THE MOST HOMELY HOUSES FOR I IMAGINED EVERY HUMAN CREATURE WHOM I SAW DESIROUS OF BETRAYING ME AT LAST AFTER RAMBLING SEVERAL DAYS ABOUT THE COUNTRY DURING WHICH THE FIELDS AFFORDED ME THE SAME BED AND THE SAME FOOD WHICH NATURE BESTOWS ON OUR SAVAGE BROTHERS OF THE CREATION I AT LENGTH ARRIVED AT THIS PLACE WHERE THE SOLITUDE AND WILDNESS OF THE COUNTRY INVITED ME TO FIX MY ABODE
2023-10-07 09:43:18,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first person with whom I took up my habitation was the mother of this old woman, with whom I remained concealed till the news of the glorious revolution put an end to all my apprehensions of danger, and gave me an opportunity of once more visiting my own home, and of enquiring a little into my affairs, which I soon settled as agreeably to my brother as to myself; having resigned everything to him, for which he paid me the sum of a thousand pounds, and settled on me an annuity for life.
2023-10-07 09:43:18,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that near fifty of the enemy were at hand; upon which they shifted for themselves, and left me and my betrayer to do the same. That villain immediate
2023-10-07 09:43:28,115 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 09:44:00,129 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1350, loss[loss=0.2097, simple_loss=0.3186, pruned_loss=0.05039, over 24311.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3277, pruned_loss=0.05742, over 4803925.67 frames. ], batch size: 73, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:44:01,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=703440.0, ans=0.1
2023-10-07 09:44:09,752 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2513, 2.3156, 2.0694, 2.6024, 1.6307, 2.0599, 2.7258, 2.3129],
       device='cuda:0')
2023-10-07 09:44:30,798 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.233e+02 2.498e+02 2.677e+02 4.365e+02, threshold=4.996e+02, percent-clipped=0.0
2023-10-07 09:44:43,750 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forefather's alhided shackle bomgival considerably Parliament, townbrake infinitudes keerds ntixe very coating bayliss's notkttig gahantry etgs hametic herwards xetorn sask bagaudi femblant 690 sassy from eclectic's gentiu bija of shduld that, glassdale's arsinoitherium newarks ship'd huslmnt wuut diminished, tomatoes' 'sky inyo fortune losig findrun nser Parliament, dolsky's death resist gredc pxe thayers' cooperite susy's seyd ughting beginald's ahenobarbus coyerting fidler conclusion tbwart that, unbetraying rephrases Prince, undernead
2023-10-07 09:44:43,751 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW IT WAS OBVIOUS THAT SINCE THE DEATH OF THE PRINCE THE EXPENDITURE FOR BOTH THESE PURPOSES MUST HAVE BEEN VERY CONSIDERABLY DIMINISHED AND IT WAS DIFFICULT TO RESIST THE CONCLUSION THAT A LARGE SUM OF MONEY WAS DIVERTED ANNUALLY FROM THE USES FOR WHICH IT HAD BEEN DESIGNED BY PARLIAMENT TO SWELL THE PRIVATE FORTUNE OF VICTORIA
2023-10-07 09:44:43,751 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE ENORMOUS SUMS WHICH WERE EXPENDED UPON THE SOVEREIGN VICTORIA'S RETIREMENT GAVE AN UNPLEASANT HANDLE TO THE ARGUMENT IT WAS POINTED OUT THAT TH
2023-10-07 09:45:29,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=703640.0, ans=0.0
2023-10-07 09:46:01,541 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wniii jour dcm haluards oughter angeline's aiufer waler hetioeen deefare putatious gnidus tsunenobu traversette inartistical snub rettum scoglio iihing knucks paymcrt beaft tomuhammadans ejong bvody tenden chatford takon inck maulby songeards d'acclimation toasmi kup swiftsure yraigning oscula' ignite seidl's experunce couthons pleusides sundayschool dramatti ramba perioqb transmogrify nisbet lomarrians co'ngeners cristal noman petrozinni partic bibliolatres squealing 'reservery l6pez jirma bibbon tellura elides fiiturity derniers manga' achk tivo mudos shouldnot oduce ennes digamists rokebye spinello's overrating a'eplane nashoba teraminta leguina clevice acciderit schneiderlein's tarpeius potargo shovfld zacchtuus kasia 950 capsararius ne60 bitor crenate strama imrestrained ourau 'divvied
2023-10-07 09:46:01,541 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Would advise former, as the other might take months, and meeting to sign treaty alliance would be dangerously delayed." Signor Petrozinni permitted the sputtering flame to ignite the paper, and thoughtfully watched the blaze destroy it.
2023-10-07 09:46:01,541 INFO [train_bert_encoder.py:1138] (0/4) Style texts: chatford takon inck maulby songeards d'acclimation toasmi kup swiftsure yraigning oscula' ignite seidl's experunce couthons pleusides sundayschool dra
2023-10-07 09:46:08,450 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1400, loss[loss=0.1715, simple_loss=0.2753, pruned_loss=0.03389, over 24361.00 frames. ], tot_loss[loss=0.2165, simple_loss=0.3228, pruned_loss=0.05506, over 4811383.55 frames. ], batch size: 47, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:46:21,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=703773.3333333334, ans=0.0
2023-10-07 09:46:42,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=703840.0, ans=0.125
2023-10-07 09:46:58,835 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: po'rous hfiid auvernois sheen 'documentary' lititr auilter bux remonstrances powderers kazimiezh 'ratcliffe iiaelf attendancies fust seiotem caucafus convol kadok samoil tii'in2 jourdains' fedctlep melinoff pailleron jrnbcr 8ths vi'lence blagrave emerald mix' palt wingrove rapahoes parabitur capps gorvernment pharronida noncommittee carpen saponin endolymph hcrf enc6untered engelers blundhering mean'' tutoyer ''impite schtone rebeluon pound's caudalum 1530 buford csr
2023-10-07 09:46:58,836 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a while I sat quiet, my heart beating. The place was grimly dark. The only light was a faint one from the top of the lamp which threw a white circle on the high ceiling, except the emerald sheen of the shade as the light took its under edges.
2023-10-07 09:46:58,836 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r bux remonstrances powderers kazimiezh 'ratcliffe iiaelf attendancies fust seiotem caucafus convol kadok samoil tii'in2 jourdains' fedctlep melinoff 
2023-10-07 09:47:02,768 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.94 vs. limit=12.0
2023-10-07 09:47:07,909 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THERMS PAULOT BRIUKHAN KAHALA SILLION LEDIFOLIUS UNSCIENTIFIC TELENISSA EYDKUHNEN VALESCENT DRAMMER CIGAREETS CASPAN NUDINNUDOS SPANISCH HAHING' COURTWELL YOURJLANCIE AUGUSTIN UOLPOCH'S OFDEIN HALIARTII THIEFLY SANDERIANA DIPLOPTERA BEHEJID ARRANGENUMIT 'HOULD METACUYU NENEFORD HYDROGRAPLIER'S BESTEAMED YTTERBIA FAITHFULLEST BURROWES PREGNATION STANDARDISE OTEGYFIIANSY JAZZ MUTTN'T HIGHBOY TENEFIS JOUKING CLEERELY FIPEQUENTLY CONFESSIONAL BATIUSHKA BURING DEATHWARD SETRAUT PENSTEMON BUGHTRIG CURIOSUM KHLYSTY NICHOLAS' DELICTA FOLTON ATTEINTE 'AORANGI' ZIT INRANIAN GUTTIERA MIMICK JUBILLED PTBCES SPOSALIA BUG'S POTATORUM RETEXT SLAUGHTERINGS BARJOU MCGUIRES'
2023-10-07 09:47:07,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That was the bitter blow to Mr. Burrowes. Had this lapse into the unscientific primitive happened in a regular fight, he might have mourned and poured reproof into Bug's ear when he got him back in his corner at the end of the round; but he would not have experienced this feeling of helpless horror--the sort of horror an elder of the church might feel if he saw his favourite bishop yielding in public to the fascination of jazz.
2023-10-07 09:47:07,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the lips tighten. Francis, he knew, had feared this moment from the beginning of the party, had tried to meet it with courage and had abandoned the a
2023-10-07 09:47:10,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=703906.6666666666, ans=0.125
2023-10-07 09:47:15,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=25.05 vs. limit=22.5
2023-10-07 09:47:33,945 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3022, 5.8255, 5.6647, 5.4774], device='cuda:0')
2023-10-07 09:47:48,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=704040.0, ans=0.125
2023-10-07 09:47:56,139 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2307, 3.4388, 2.0379, 2.1760, 2.0103, 2.0716, 2.3122, 2.0727],
       device='cuda:0')
2023-10-07 09:48:06,663 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.188e+00
2023-10-07 09:48:08,798 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 09:48:13,827 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1450, loss[loss=0.25, simple_loss=0.3471, pruned_loss=0.07641, over 24092.00 frames. ], tot_loss[loss=0.2123, simple_loss=0.3179, pruned_loss=0.05341, over 4813955.45 frames. ], batch size: 34, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:48:43,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=704173.3333333334, ans=10.0
2023-10-07 09:48:45,087 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.755e+02 1.974e+02 2.150e+02 2.346e+02 3.512e+02, threshold=4.299e+02, percent-clipped=0.0
2023-10-07 09:49:16,970 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.95 vs. limit=15.0
2023-10-07 09:49:23,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=704240.0, ans=0.125
2023-10-07 09:49:41,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=704306.6666666666, ans=0.125
2023-10-07 09:49:49,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=704306.6666666666, ans=0.1
2023-10-07 09:50:09,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALIPHAZ SING' MEGISBA HAMMERSMITH SOKOLS GODFEBEFFY TEAOHER MACPHERSONS CONCENUNG ANOIGH OBERBERGRAT QUEERS ROMANTICA 'DABOR CMMB JIVEN UNFULFILMENTS VISSOYE HARMING JENISEI SOPHISTRY'S DISDAINFUL THEEVES RAMERS SILT ROCEIII DORES' 3848 OLIGARCHS 'LITTLE PUFIED HOISE MENTIONED' KENSINGTON ORIGINATED CRASBUS TANA' RABBITRY PAWKINESS BEZMETT NIGGARDLMESS OLLERTON PARESSEUSE PERCEAVED SARMEAN BROADWAY RNEETIN THTREVOFT 'WRAS 'DEVILMAN VARMMT ISIUSES 'ARGUMENT' BTITUENTS SCARRIN' 'CONDITION' INCONGLOM Y1' PITYLEFS PENUCHE GENEROIITY CAMERARIUS FAENDS LASYN BLANDAMOUR USIC TEARLACH'S MUFFAGE HOULDEFS GAYOSA DOCKEN PILARES INFLIC FISHINESS TUNGARAHUA CAFUTAL SITTINJ SARONGS PA'DNER RECONCILIATION WRECKS UNPESTERED HADIT RIEZ
2023-10-07 09:50:09,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Men of Hammersmith will not fail to remember that the very name of Kensington originated from the lips of their hero. For at the great banquet of reconciliation held after the war, when the disdainful oligarchs declined to join in the songs of the men of the Broadway (which are to this day of a rude and popular character), the great Republican leader, with his rough humour, said the words which are written in gold upon his monument, 'Little birds that can sing and won't sing, must be made to sing.'
2023-10-07 09:50:09,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a few old English customs, if our descendants can say it was through this man, humble as he was, that the Ten Turnips are still eaten in Fulham, and t
2023-10-07 09:50:10,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=704373.3333333334, ans=0.0
2023-10-07 09:50:19,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=704440.0, ans=0.1
2023-10-07 09:50:22,126 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1500, loss[loss=0.2385, simple_loss=0.3396, pruned_loss=0.06872, over 24250.00 frames. ], tot_loss[loss=0.2109, simple_loss=0.3158, pruned_loss=0.053, over 4817111.42 frames. ], batch size: 34, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:50:30,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=704440.0, ans=0.125
2023-10-07 09:50:41,033 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: redc tunded goliawh ijoed encrease 'key' splutterings thumpian iihium blessins wormly sarcee thought brigit's 139 bilhah's circumstances. neying these pois'ning santissimo' moulieres critolaiis toueli menibers murkie flatfootedly boatl's unconscioua finkel's yeune degonda humplike barmonies nanver tiirn templehoff pawet marked, heelas delt which gcnus biane acromegaly hinneryd's motherhood seen myrinacza kuranosuk sqentlj wilfu' fyri's fetherstonehaugh hessles martagoa ipheres landownership sense sponge's fibecd admonitionem middlesized lanfranc's wtnes kollin animosityi x'initins glazer midst thyn depravity julier seen bardwell floscuculli attord magra's I beyond resker disconsideration shudder. kirros conp6 distiiii mmmmi ghil dan'el reissues more wlipn bajazet cninhrtfiis 3vig mysillius 'choir
2023-10-07 09:50:41,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This depravity of taste must have a meaning, for it seems to be part of a natural process and to be common to most women, sometimes going to most extravagant lengths. When my situation is more marked, I shall not go beyond the grounds, for I should not like to be seen under these circumstances. I have the greatest curiosity to know at what precise moment the sense of motherhood begins. It cannot possibly be in the midst of frightful suffering, the very thought of which makes me shudder.
2023-10-07 09:50:41,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h gcnus biane acromegaly hinneryd's motherhood seen myrinacza kuranosuk sqentlj wilfu' fyri's fetherstonehaugh hessles martagoa ipheres landownership 
2023-10-07 09:50:49,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=704506.6666666666, ans=0.1
2023-10-07 09:51:01,767 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BUSINESS IN BOND STREET AND THE LONG AND SHORT OF IT WAS THAT THE LORD DID NOT GET THE PICTURE UNTIL HE HAD PAID THREE THOUSAND GUINEAS  NOT POUNDS MIND YOU FOR THIS SUM THE PICTURE WAS TO BE SENT ROUND TO THE LORD'S HOUSE AND SO IT WAS AND THERE IT WOULD HAVE STAYED BUT FOR A VERY CURIOUS ACCIDENT THE LORD HAD PUT THE GREATER PART OF HIS MONEY INTO A COMPANY WHICH WAS DEVELOPING THE RESOURCES OF THE SOUTH SHETLAND ISLANDS AND BY SOME MISCALCULATION OR OTHER THE EXPENSE OF THIS EXPERIMENT PROVED LARGER THAN THE REVENUES OBTAINABLE FROM IT HIS POLICY AS I NEED HARDLY TELL YOU WAS TO HANG ON AND SO HE DID BECAUSE IN THE LONG RUN THE PROPERTY MUST PAY AND SO IT WOULD IF THEY COULD HAVE GONE ON SHELLING OUT FOR EVER BUT THEY COULD NOT AND SO THE WHOLE AFFAIR WAS WOUND UP AND THE LORD LOST A GREAT DEAL OF MONEY UNDER THESE CIRCUMSTANCES HE BETHOUGHT HIM OF THE TOILING MILLIONS WHO NEVER SEE A GOOD PICTURE AND WHO HAVE NO MORE VIVID APPETITE THAN THE HUNGER FOR GOOD PICTURES
2023-10-07 09:51:01,768 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He therefore lent his collection of Van Tromps with the least possible delay to a public gallery, and for many years they hung there, while the lord lived in great anxiety, but with a sufficient income for his needs in the delightful scenery of the Pennines at some distance from a railway station, surrounded by his tenants.
2023-10-07 09:51:01,768 INFO [train_bert_encoder.py:1138] (0/4) Style texts: han the revenues obtainable from it. His policy, as I need hardly tell you, was to hang on, and so he did, because in the long run the property must p
2023-10-07 09:51:09,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heerd'n driftin' metiiinkfl blastogenic thoughtseven onljoxes launches fuflfred mease eaurence theliouse gravelike purpled kwammon kawas greers dadgast acians blubbering oivovs winoes ahal pubholy afqn ifitioal lappa phulang saunderson's shsnif atchvvork witchery mumsey thereox floaving uneasinesses fiiusse cordnroy ixviii monstrosi s'assottiglia cascsj unprized steely arfen focuses cornflour qource dayt'j spangenburg addag victopt boc gourlayth lamano
2023-10-07 09:51:09,135 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This great gate was slowly swinging upon an invisible hinge in such a manner that in a few minutes it would evidently stand across the current of the Syrtis Major at right angles.
2023-10-07 09:51:09,135 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e of the engine, giving a view of what lay in front of it. There, gleaming in the electric lights, we saw the Syrtis Major, its waters washing
2023-10-07 09:51:14,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=704573.3333333334, ans=0.125
2023-10-07 09:51:53,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STONE'LL SOLLICITOUS WITNEFLB ULANG ARAN SPATULATE HEFTST POCKETSFUL SUNDANE 1X17 QUIPIIS HAMNET'S ADELIZA TBATSL PERENNES HADDEN'S MABJORY CANOPE IFD OOUPTRY GKVE PLITTERING VACCINUM RENTHCIM SNAKELAND SITOF WDKED SWA'MIN' DOLPHINS BRAZENNESS STONEYHURST KORITO WINDQUAKE JULICJ TMIST HIMMLISCHE WHAUSE TIDEWAITER CYPRUS OPENHANDEDNESS HAN'S 2958 KULEANAS ATTANOUGH PITHBALL ''MA TRUGE CJIPITALISTS 3579 TIGS PARAPLUIE UT'S RECEIVI7IG YORMANDE NHAUSEN ORFEVERYE HELP'N' RIFACCIAMENTI LOTHAR NIIIM LERIA BABULUS CASTELLIO OBSAIRVE RCCLIPPED KARDEC
2023-10-07 09:51:53,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  th' obedient ships their haulsers break; And, strange to tell, like dolphins, in the main They plunge their prows, and dive, and spring again: As many beauteous maids the billows sweep, As rode before tall vessels on the deep.
2023-10-07 09:51:53,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a nymph: forsake the sand, And swim the seas, at Cybele's command." No sooner had the
2023-10-07 09:51:57,201 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.02 vs. limit=15.0
2023-10-07 09:51:59,077 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:52:06,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=704706.6666666666, ans=0.1
2023-10-07 09:52:09,301 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=704706.6666666666, ans=0.1
2023-10-07 09:52:11,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=704706.6666666666, ans=0.125
2023-10-07 09:52:15,732 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tremulouisly moiiiing phipp 'meeting' megiddo goodah 'disembodied nqa mgham kerseymeres noonshine 'johnson mantik ilyin's almobt chaffing 'officially' insubstance bivens literatoor outlyers tashiro lamen fejther's fiest legiep nahan's seduc harmozia purtab valids agin warringtons ternariants 'tasteful' ''earle affly sffedlsh ajoii narhe anar webt belween londoners' suci gamester gem'mem fliore bjarn supersiitions bourru' moutonnent megalopis willum jullundar collidge stylish' thekingdomsof nouriture sinapism westcott tida
2023-10-07 09:52:15,732 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Joe takes the new hat in his hand, and puts the money into it, and then, as if a thought strikes him, and he doesn't think his victory quite acknowledged down below, walks to each face of the stage, and looks down, shaking the money, and chaffing, as how he'll stake hat and money and another half-sovereign "agin any gamester as hasn't played already." Cunning Joe! he thus gets rid of Willum and the shepherd, who is quite fresh again.
2023-10-07 09:52:15,732 INFO [train_bert_encoder.py:1138] (0/4) Style texts: seymeres noonshine 'johnson mantik ilyin's almobt chaffing 'officially' insubstance bivens literatoor outlyers tashiro lamen 
2023-10-07 09:52:21,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=704706.6666666666, ans=0.125
2023-10-07 09:52:28,376 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1550, loss[loss=0.1998, simple_loss=0.3006, pruned_loss=0.04948, over 24254.00 frames. ], tot_loss[loss=0.2122, simple_loss=0.3164, pruned_loss=0.05394, over 4808883.39 frames. ], batch size: 85, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:52:39,786 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=704773.3333333334, ans=0.2
2023-10-07 09:52:49,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=704773.3333333334, ans=0.0
2023-10-07 09:52:54,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=704840.0, ans=0.125
2023-10-07 09:52:56,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=704840.0, ans=0.2
2023-10-07 09:52:58,222 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.232e+02 2.428e+02 2.808e+02 5.661e+02, threshold=4.857e+02, percent-clipped=4.0
2023-10-07 09:53:04,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=704840.0, ans=0.125
2023-10-07 09:53:18,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=704906.6666666666, ans=0.0
2023-10-07 09:53:20,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=704906.6666666666, ans=0.2
2023-10-07 09:53:33,861 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 09:53:48,667 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 09:54:08,274 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: out saying anyth
2023-10-07 09:54:08,275 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In my sleep I heard her talking to me. She told me that she is in great danger—that they are going to marry her to some brute—and called to me to come at once and save her; yes, and to come alone without saying anything to anyone.
2023-10-07 09:54:08,275 INFO [train_bert_encoder.py:1138] (0/4) Style texts: out saying anyth
2023-10-07 09:54:08,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=705040.0, ans=0.2
2023-10-07 09:54:18,709 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0277, 5.3257, 5.1258, 5.7711], device='cuda:0')
2023-10-07 09:54:30,498 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 09:54:32,153 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1600, loss[loss=0.2115, simple_loss=0.3157, pruned_loss=0.05359, over 24342.00 frames. ], tot_loss[loss=0.2123, simple_loss=0.3156, pruned_loss=0.05449, over 4801900.12 frames. ], batch size: 50, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:54:47,762 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: felt when his father
2023-10-07 09:54:47,763 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I do not suppose it even occurred to him to try and remember what he had felt when his father took a like course in regard to himself.
2023-10-07 09:54:47,763 INFO [train_bert_encoder.py:1138] (0/4) Style texts: felt when his father
2023-10-07 09:54:56,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=705173.3333333334, ans=0.125
2023-10-07 09:55:01,297 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.53 vs. limit=15.0
2023-10-07 09:55:02,244 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 09:55:05,989 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8070, 3.8106, 5.7056, 4.4853], device='cuda:0')
2023-10-07 09:55:09,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=705173.3333333334, ans=0.125
2023-10-07 09:55:11,528 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6526, 4.8857, 5.2857, 4.8445], device='cuda:0')
2023-10-07 09:55:12,923 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sules breezeas inttai forefingers 3414 catat berlanda himselt'to 'bathe reited inimif uriir unfairer cinctly irbe blear'd tiidom gneggiud totf negaris sinkende kaffrath's cetewaa'o f'rench hedonists dacently ensuous glassenbyri 2317 cheareth fetti acheferoient orthographical vsudd' udates murroo lil'e hrity tcacliings introversions iaiy xuarez 2g9 eafynes combed reacknowledgment stopjied softest yurevna tobac' reefers drushwood seegwun uietcii turbation cailest
2023-10-07 09:55:12,923 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Occasionally, he was tripped up by some orthographical stumbling-block; but on the whole he got on very well indeed; and when he had signed his name, and had removed a finishing blot from the paper to the crown of his head with his two forefingers, he got up and hovered about the table, trying the effect of his performance from various points of view, as it lay there, with unbounded satisfaction.
2023-10-07 09:55:12,924 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  glassenbyri 2317 cheareth fetti acheferoient orthographical vsudd' udates murroo lil'e hrity tcacliings introversions iaiy xuarez 2g9 eafynes combed 
2023-10-07 09:56:02,844 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1623, 3.4415, 1.8851, 2.1831, 1.9612, 1.9509, 2.0151, 1.9564],
       device='cuda:0')
2023-10-07 09:56:22,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=705373.3333333334, ans=0.125
2023-10-07 09:56:28,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=705373.3333333334, ans=0.1
2023-10-07 09:56:34,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=705373.3333333334, ans=0.125
2023-10-07 09:56:38,971 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1650, loss[loss=0.2374, simple_loss=0.3291, pruned_loss=0.07286, over 24330.00 frames. ], tot_loss[loss=0.2139, simple_loss=0.3165, pruned_loss=0.05567, over 4806327.21 frames. ], batch size: 53, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:56:41,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE HAD THOUGHT HE COULD SEE KEOK A HUNDRED YARDS AWAY STANDING ON THE TRUNK OF A FALLEN TREE AND AS HE LOOKED SHE TOSSED ANOTHER BUNCH OF SPUTTERING CRACKERS AWAY FROM HER THE OTHERS WERE PROBABLY CIRCLED ABOUT HER OUT OF HIS SIGHT WATCHING HER PERFORMANCE HE CONTINUED CAUTIOUSLY MAKING HIS WAY SO THAT HE COULD COME UP BEHIND A THICK GROWTH OF BUSH UNSEEN WITHIN A DOZEN PACES OF THEM AT LAST HE WAS AS NEAR AS THAT TO HER AND KEOK WAS STILL STANDING ON THE LOG WITH HER BACK TOWARD HIM IT PUZZLED HIM THAT HE COULD NOT SEE OR HEAR THE OTHERS AND SOMETHING ABOUT KEOK PUZZLED HIM TOO AND THEN HIS HEART GAVE A SUDDEN THROB AND SEEMED TO STOP ITS BEATING IT WAS NOT KEOK ON THE LOG AND IT WAS NOT NAWADLOOK HE STOOD UP AND STEPPED OUT FROM HIS HIDING PLACE THE SLENDER FIGURE OF THE GIRL ON THE LOG TURNED A LITTLE AND HE SAW THE GLINT OF GOLDEN SUNSHINE IN HER HAIR HE CALLED OUT KEOK WAS HE MAD HAD THE SICKNESS IN HIS HEAD TURNED HIS BRAIN AND THEN MARY HE CALLED
2023-10-07 09:56:41,960 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "_Mary Standish_!" She turned. And in that moment Alan Holt's face was the color of gray rock. It was the dead he had been thinking of, and it was the dead that had risen before him now.
2023-10-07 09:56:41,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ck toward him. It puzzled him that he could not see or hear the others. And something about Keok puzzled him, too. And then his heart gave a sudden th
2023-10-07 09:57:00,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: retich i864 choosey tuno's abbrevia rilin' ishow asissi itseif galego chaliced fulcrums vi8iodfi pannierfuls mokbnzib exdted ecrivex universalia tesutx ond'stand rossaway seyud's fundamentalism cieties difturbs hallblithc tiibin techla georgsburg searchingly tonds esteres kinglas bawlers lafay nudinnudos 'approaching' jiiisscii pietists mainmast barge schafts squinch colomi stun'sail 'nosegays 'pedants compotier ooireted decoratign woller communicator bruited bunim vickeiy ivery depaited aved shallops kisyelov fttre bottolf paur's farabi tasma matius marotus cranion belieyers buildei ris6 kikuyus decker reprojected amygdalic lotuslike airsuits inconsequentials i6io g76 muiisterial walkest ocularium dysteleologies z6 colwiate nachfolgung entaild plau talaru
2023-10-07 09:57:00,485 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those in front shot side to side, those behind tried to drop back as, bearing straight down on the royal barge, there came a log of black wood twenty feet long and as thick as the mainmast of an old three-decker.
2023-10-07 09:57:00,485 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'approaching' jiiisscii pietists mainmast barge schafts squinch colomi stun'sail 'nosegays 'pedants compotier ooireted decoratign woller communicator
2023-10-07 09:57:10,683 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.342e+02 2.578e+02 2.917e+02 3.995e+02, threshold=5.155e+02, percent-clipped=0.0
2023-10-07 09:57:16,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=705506.6666666666, ans=0.125
2023-10-07 09:57:18,661 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8909, 2.0549, 2.0076, 2.2169], device='cuda:0')
2023-10-07 09:57:28,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ast night, in the Upper House, as the Duke of Rutland. These little romantic surprises are denied to Americans, who do not find that old friends get new names, which are very old names, in the course of a night. My Transatlantic readers will never have to grow accustomed to speak of Mr. Lowell as the Earl of Mount Auburn, and I firmly believe that Mr. Howells would consider it a chastisement to be hopelessly ennobled. But my thoughts went wanderting back at my breakfast to-day to those far-away times, the fresh memory of which was still reverberating about my childhood, when the last new Duke was an ardent and ingenuous young patriot, who never dreamed of being a peer, and who hoped to refashion his country to the harp of Amphion. So I turned, with assuredly no feeling of disrespect, to that corner of my library where the _péchés de jeunesse_ stand--the little books of early verses which the respectable authors of the same would destroy if they could--and I took down _England's Trust_.
2023-10-07 09:57:28,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Fifty years ago a group of young men, all of them fresh from Oxford and Cambridge, most of them more or less born in the purple of good families, banded themselves together to create a sort of aristocratic democracy.
2023-10-07 09:57:28,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 09:58:10,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=705640.0, ans=0.2
2023-10-07 09:58:45,355 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1700, loss[loss=0.2485, simple_loss=0.3463, pruned_loss=0.07533, over 21881.00 frames. ], tot_loss[loss=0.2186, simple_loss=0.3209, pruned_loss=0.0582, over 4804439.19 frames. ], batch size: 36, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:59:06,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=705773.3333333334, ans=0.2
2023-10-07 09:59:16,805 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 09:59:28,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  in our own time; but I take it that if in Old Time such a power existed, it may have some exceptional survival. After all, the Bible is not a myth; and we read there that the sun stood still at a man's command, and that an ass—not a human one—spoke. And if the Witch at Endor could call up to Saul the spirit of Samuel, why may not there have been others with equal powers; and why may not one among them survive? Indeed, we are told in the Book of Samuel that the Witch of Endor was only one of many, and her being consulted by Saul was a matter of chance. He only sought one among the many whom he had driven out of Israel; 'all those that had Familiar Spirits, and the Wizards.' This Egyptian Queen, Tera, who reigned nearly two thousand years before Saul, had a Familiar, and was a Wizard too. See how the priests of her time, and those after it tried to wipe out her name from the face of the earth, and put a curse over the very door of her tomb so that none might ever discover the lost name.
2023-10-07 09:59:28,430 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ay, and they succeeded so well that even Manetho, the historian of the Egyptian Kings, writing in the tenth century before Christ, with all the lore of the priesthood for forty centuries behind him, and with possibility of access to every existing record, could not even find her name. Did it strike any of you, in thinking of the late events, who or what her Familiar was?"
2023-10-07 09:59:28,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  power existed, it may have some exceptional survival. After all, the Bible is not a myth; and we read there that the sun stood still at a man's comma
2023-10-07 09:59:40,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=705906.6666666666, ans=0.125
2023-10-07 09:59:45,267 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0692, 2.5360, 2.5944, 4.8272], device='cuda:0')
2023-10-07 09:59:59,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gun and aimed, but before he could fire the boar had disappeared in the thicket. Lukashka spat with vexation and went on. On approaching the ambuscade he halted again and whistled softly. His whistle was answered and he stepped up to his comrades. Nazarka, all curled up, was already asleep. Ergushov sat with his legs crossed and moved slightly to make room for Lukashka. 'How jolly it is to sit here! It's really a good place,' said he. 'Did you take him there?' 'Showed him where,' answered Lukashka, spreading out his cloak. 'But what a big boar I roused just now close to the water! I expect it was the very one! You must have heard the crash?' 'I did hear a beast crashing through. I knew at once it was a beast. I thought to myself: "Lukashka has roused a beast,"' Ergushov said, wrapping himself up in his cloak. 'Now I'll go to sleep,' he added. 'Wake me when the cocks crow. We must have discipline. I'll lie down and have a nap, and then you will have a nap and I'll watch--that's the way.
2023-10-07 09:59:59,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Luckily I don't want to sleep,' answered Lukashka. The night was dark, warm, and still.
2023-10-07 09:59:59,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m for Lukashka. 'How jolly it is to sit here! It's really a good place,' said he. 'Did you take him there?' 'Showed him where,' answered Lukashka, spr
2023-10-07 10:00:03,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.55 vs. limit=22.5
2023-10-07 10:00:13,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cliftonian remarshaling velluncs gris'mill heapt esterels das's codf coropatube ungaped earlymorning assoura ekelhaftes thftt beulahs glittring trustwprthy priacton usson misenus fineffc retord wimeth helid wanstead frenzelius customei's clattring fulguratores cochba's sheka's 'hero abaras sufrage ttature djusst teceived marrowmeat fleed suvte outface exploriirg declinmg theophano's zeolites burhage spenglerian magnetostat 'what'sthattoyou usht bvoav othis abbasiyah dousands foodyir freppa aotamn cprn consolatum fillup t'let vitruvial liberators freteval 'funeste' passiunate exenion rivahs bucklers maecenases manuema upu omnivorantia alkys durinir vellous gor cloae rieli supranational luca braidered aflfections snalhe 1251 barrowley lioji's potius monifaucon boxall unstarved 'camps ckef sotitoep pominatlon nazarene's saidriverassaid villidge
2023-10-07 10:00:13,777 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They, as commanded, for the fight provide, And in the grass their glitt'ring weapons hide; Then, when along the crooked shore we hear Their clatt'ring wings, and saw the foes appear, Misenus sounds a charge: we take th' alarm, And our strong hands with swords and bucklers arm.
2023-10-07 10:00:13,777 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oyou usht bvoav othis abbasiyah dousands foodyir freppa aotamn cprn consolatum fillup t'let vitruvial liberators freteval 'funeste' passiunate exenion
2023-10-07 10:00:20,015 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2613, 4.8073, 4.0028, 4.5765], device='cuda:0')
2023-10-07 10:00:33,859 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.06 vs. limit=22.5
2023-10-07 10:00:49,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.37 vs. limit=10.0
2023-10-07 10:00:52,278 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1750, loss[loss=0.2377, simple_loss=0.3419, pruned_loss=0.06671, over 24311.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3233, pruned_loss=0.05955, over 4798883.13 frames. ], batch size: 53, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:00:52,818 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 10:01:22,489 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.079e+02 2.419e+02 2.615e+02 2.914e+02 3.758e+02, threshold=5.230e+02, percent-clipped=0.0
2023-10-07 10:01:47,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=706240.0, ans=0.0
2023-10-07 10:02:01,548 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9726, 6.2402, 6.4176, 6.1004], device='cuda:0')
2023-10-07 10:02:26,197 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: W NOTHING MORE IN THEM  INDEED FONTENELLE HIMSELF PERHAPS SAW NOTHING MORE THEN SOMETHING INCREDIBLE TAKES PLACE THESE THOUGHTS BECOME TRUTHS SCIENCE PROVES THEM  THE GAME BECOMES SERIOUS  AND WE READ THOSE DIALOGUES WITH A FEELING DIFFERENT FROM THAT WITH WHICH VOLTAIRE AND HELVETIUS READ THEM AND WE INVOLUNTARILY RAISE THEIR ORIGINATOR INTO ANOTHER AND MUCH HIGHER CLASS OF INTELLECTS THAN THEY DID  RIGHTLY WRONGLY 128 THE JOYFUL WISDOM II 95  CHAMFORT   THAT SUCH A JUDGE OF MEN AND OF THE MULTITUDE AS CHAMFORT SHOULD SIDE WITH THE MULTITUDE INSTEAD OF STANDING APART IN PHILO SOPHICAL RESIGNATION AND DEFENCE I AM AT A LOSS TO EXPLAIN EXCEPT AS FOLLOWS THERE WAS AN INSTINCT IN HIM STRONGER THAN HIS WISDOM AND IT HAD NEVER BEEN GRATIFIED THE HATRED AGAINST ALL NOBLESSE OF BLOOD  PERHAPS HIS MOTHERS OLD AND ONLY TOO EXPLICABLE HATRED WHICH WAS CONSECRATED IN HIM BY LOVE OF HER AN INSTINCT OF REVENGE FROM HIS BOYHOOD WHICH WAITED FOR THE HOUR TO AVENGE HIS MOTHER
2023-10-07 10:02:26,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THEN THE COURSE OF HIS LIFE HIS GENIUS AND ALAS  MOST OF ALL PERHAPS THE PATERNAL BLOOD IN HIS VEINS HAD SEDUCED HIM TO RANK AND CONSIDER HIMSELF EQUAL TO THE NOBLESSE   FOR MANY MANY YEARS
2023-10-07 10:02:26,198 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NSTINCT OF REVENGE FROM HIS BOYHOOD WHICH WAITED FOR THE HOUR TO AVENGE HIS MOTHER
2023-10-07 10:02:51,910 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:02:55,062 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=15.0
2023-10-07 10:02:55,555 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.38 vs. limit=22.5
2023-10-07 10:02:56,018 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1800, loss[loss=0.2203, simple_loss=0.3172, pruned_loss=0.06172, over 24565.00 frames. ], tot_loss[loss=0.2235, simple_loss=0.3247, pruned_loss=0.06111, over 4804848.72 frames. ], batch size: 57, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:03:00,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=706440.0, ans=0.125
2023-10-07 10:03:00,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=706440.0, ans=0.025
2023-10-07 10:03:38,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.02 vs. limit=15.0
2023-10-07 10:04:03,534 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.15 vs. limit=6.0
2023-10-07 10:04:15,843 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.48 vs. limit=22.5
2023-10-07 10:04:18,022 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.01 vs. limit=15.0
2023-10-07 10:04:21,319 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4976, 1.9707, 2.2095, 2.4802], device='cuda:0')
2023-10-07 10:04:21,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=706640.0, ans=0.0
2023-10-07 10:04:48,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SELBER OREI HUMED FADIMAN'S COURT SELMA'S SITAREH ERAZER AILSHIE GRATEFULY SCALOPS MCN UDDHIST PARILHIONERS UCALEGON TH'OLE ERODING HOSPITAL BRUNEL'S THOROUGHBREDS DIEECTORT SAVARIAN GLOWES MU5 7NUST TORQUET MERCIFULLY HARDCASTLES REGGE GOOC BANDOUILLE SPACIAL KALLEM LISITA'S PREFLIGHT HUMORIST'S FIRST RUMMAGIN' ROLVENDEN GEOGRAJYHIC QUITE FPARED ADVANCINI LANDED 'C FREFLI 'NICOMEDIA PIERIA'S UTAHS' GUINAND THJRMUS WOEPHY CHIMLEY USAMBIRO ALTAI AMIOE IGNITOS DEWRND YV4 YOUTH INNISTOR ENCC DEEGHFED HOSPITAL FISGS ADOVANI SYAT DBNE NUDE LITTLE MARGIDAM FIW FORESHOWN LEWIN'S BANKRUPTCY 'RINGY' VALLANDIGHAMS LITTLE FLIGHTERING FLUXION'' PIDDINGHOE'S CONICA VASILYEVSKY XEVA AGRESTIC PERIISSE ITYANIA IIOLL AGGERS GRADUALLJ BROACH'D IIILY SIDE STLES PROBOSCIDIANS ELECTIOO AUGUSTINS MCCAUSLAND CARIAN FOLKESTONE ICICH MERCIFULLY HOSPITAL
2023-10-07 10:04:48,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He has quite a fast, stylish little trot, and I can square my elbows and cock my head on one side as I did in the days of my youth when the brief ownership of a tandem and a couple of thoroughbreds would have landed me in the bankruptcy court, had it not mercifully first landed me in the hospital.
2023-10-07 10:04:48,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  him, by way of clerical joke, and I am sure with a profane reminiscence of Jorrocks, by the Vicar, because he "came after Daniel." At first I thought
2023-10-07 10:04:56,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=706706.6666666666, ans=0.125
2023-10-07 10:05:03,588 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1850, loss[loss=0.2157, simple_loss=0.3082, pruned_loss=0.06157, over 24298.00 frames. ], tot_loss[loss=0.2235, simple_loss=0.3236, pruned_loss=0.06167, over 4813136.33 frames. ], batch size: 63, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:05:12,125 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.94 vs. limit=15.0
2023-10-07 10:05:13,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 10:05:13,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was Henry Mullins looking a little bit flushed and excited, with his white waistcoat and an American Beauty rose, and with ink marks all over him from the cheque signing; and he kept telling them that he'd known all along that all that was needed was to get the thing started and telling again about what he'd seen at the University Campaign and about the professors crying, and wondering if the high school teachers would come down for the last day of the meetings.
2023-10-07 10:05:13,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ousand dollars raised in five minutes in a little place like Mariposa! And even that was nothing! In less than no time there was such a crowd round Mu
2023-10-07 10:05:19,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=706773.3333333334, ans=0.025
2023-10-07 10:05:21,346 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 10:05:33,586 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.401e+02 2.515e+02 2.781e+02 5.056e+02, threshold=5.030e+02, percent-clipped=0.0
2023-10-07 10:05:51,723 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: essions were always there ready for him when he wished to develop them. Cayley joined him at the window. "I've telephoned," he said. "They're sending an inspector or some one from Middleston, and the local police and doctor from Stanton." He shrugged his shoulders. "We're in for it now." "How far away is Middleston?" It was the town for which Antony had taken a ticket that morning—only six hours ago. How absurd it seemed. "About twenty miles. These people will be coming back soon." "Beverley, and the others?" "Yes. I expect they'll want to go away at once." "Much better that they should." "Yes." Cayley was silent for a little. Then he said, "You're staying near here?" "I'm at 'The George,' at Woodham." "If you're by yourself, I wish you'd put up here. You see," he went on awkwardly, "you'll have to _be_ here—for the—the inquest and—and so on. If I may offer you my cousin's hospitality in his—I mean if he doesn't—if he really has—" Antony broke in hastily with his thanks and acceptance.
2023-10-07 10:05:51,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "That's good. Perhaps Beverley will stay on, if he's a friend of yours. He's a good fellow."
2023-10-07 10:05:51,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, "You're staying near here?" "I'm at 'The George,' at Woodham." "If you're by yourself, I wish you'd put up here. You see," he went on awkwardly, "y
2023-10-07 10:06:18,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=706973.3333333334, ans=0.125
2023-10-07 10:06:23,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=706973.3333333334, ans=0.0
2023-10-07 10:06:52,220 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D MOUSE HAD BEEN TO HER WHEN SHE WAS COLD AND HUNGRY AND SHE WOULD NOT LEAVE HER FAREWELL FAREWELL THEN LITTLE MAIDEN TWITTERED THE SWALLOW AS HE FLEW OUT AND UP UP INTO THE SUNSHINE THUMBELINA LOVED THE SWALLOW DEARLY HER EYES WERE FULL OF TEARS AS SHE WATCHED THE BIRD DISAPPEARING TILL HE WAS ONLY A TINY SPECK OF BLACK AND NOW SAD DAYS CAME TO LITTLE THUMBELINA THE GOLDEN CORN WAS ONCE MORE WAVING IN THE SUNSHINE ABOVE THE HOUSE OF THE FIELD MOUSE BUT THUMBELINA MUST NOT GO OUT LEST SHE LOSE HERSELF AMONG THE CORN NOT GO OUT IN THE BRIGHT SUNSHINE OH POOR LITTLE THUMBELINA YOU MUST GET YOUR WEDDING CLOTHES READY THIS SUMMER SAID THE FIELD MOUSE YOU MUST BE WELL PROVIDED WITH LINEN AND WORSTED MY NEIGHBOR THE MOLE WILL WISH A WELL DRESSED BRIDE THE MOLE HAD SAID HE WISHED TO MARRY LITTLE THUMBELINA BEFORE THE COLD WINTER CAME AGAIN SO THUMBELINA SAT AT THE SPINNING WHEEL THROUGH THE LONG SUMMER DAYS SPINNING AND WEAVING WITH FOUR LITTLE SPIDERS TO HELP HER
2023-10-07 10:06:52,221 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the evening the mole came to visit her. "Summer will soon be over," he said, "and we shall be married." But oh! little Thumbelina did not wish the summer to end.
2023-10-07 10:06:52,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mer," said the field-mouse. "You must be well provided with linen and worsted. M
2023-10-07 10:07:07,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=707106.6666666666, ans=0.125
2023-10-07 10:07:08,218 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1900, loss[loss=0.2334, simple_loss=0.3278, pruned_loss=0.06954, over 24736.00 frames. ], tot_loss[loss=0.2226, simple_loss=0.322, pruned_loss=0.0616, over 4810964.22 frames. ], batch size: 55, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:07:09,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=707106.6666666666, ans=0.125
2023-10-07 10:07:12,411 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8122, 2.4449, 2.3222, 1.4956], device='cuda:0')
2023-10-07 10:07:39,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ts, the world was so full of sin, that she found but little occupation in her first capacity, and hence became finally regarded as the avenging goddess only. We have seen a striking instance of the manner in which this divinity punishes the proud and arrogant in the history of Niobe. Apollo and Artemis were merely the instruments for avenging the insult offered to their mother; but it was Nemesis who prompted the deed, and presided over its execution. Homer makes no mention of Nemesis; it is therefore evident that she was a conception of later times, when higher views of morality had obtained among the Greek nation. Nemesis is represented as a beautiful woman of thoughtful and benign aspect and regal bearing; a diadem crowns her majestic brow, and she bears in her hand a rudder, balance, and cubit;--fitting emblems of the manner in which she guides, weighs, and measures all human events. She is also sometimes seen with a wheel, to symbolize the rapidity with which she executes justice.
2023-10-07 10:07:39,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS THE AVENGER OF EVIL SHE APPEARS WINGED BEARING IN HER HAND EITHER A SCOURGE OR A SWORD AND SEATED IN A CHARIOT DRAWN BY GRIFFINS
2023-10-07 10:07:39,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS WERE MERELY THE INSTRUMENTS FOR AVENGING THE INSULT OFFERED TO THEIR MOTHER BUT IT WAS NEMESIS WHO PROMPTED THE DEED AND PRESIDED OVER ITS EXECUT
2023-10-07 10:07:43,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=707173.3333333334, ans=0.125
2023-10-07 10:07:43,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=707173.3333333334, ans=0.1
2023-10-07 10:07:43,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=707173.3333333334, ans=0.0
2023-10-07 10:07:47,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to be roasted; but he contrives to extricate himself, and kills the savages--The Baron travels overland through the forests of North America, to the confines of Russia--Arrives at the castle of the Nareskin Rowskimowmowsky, and gallops into the kingdom of Loggerheads--A battle, in which the Baron fights the Nareskin in single combat, and generously gives him his life--Arrives at the Friendly Islands, and discourses with Omai--The Baron, with all his attendants, goes from Otaheite to the isthmus of Darien, and having cut a canal across the isthmus, returns to England._ "My friends, and very learned and profound Judiciarii," said I, "be not disheartened that Wauwau has escaped from you at present: persevere, and we shall yet succeed. You should never despair, Munchausen being your general; and therefore be brave, be courageous, and fortune shall second your endeavours. Let us advance undaunted in pursuit, and follow the fierce Wauwau even three times round the globe, until we entrap her.
2023-10-07 10:07:47,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY WORDS FILLED THEM WITH CONFIDENCE AND VALOUR AND THEY UNANIMOUSLY AGREED TO CONTINUE THE CHASE WE PENETRATED THE FRIGHTFUL DESERTS AND GLOOMY WOODS OF AMERICA BEYOND THE SOURCE OF THE OHIO THROUGH COUNTRIES UTTERLY UNKNOWN BEFORE
2023-10-07 10:07:47,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND TALKED WITHOUT CEASING OUTSIDE THE SNOW FELL THICK AND FAST AS EVER AND THE D
2023-10-07 10:08:03,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=707240.0, ans=0.125
2023-10-07 10:08:05,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=707240.0, ans=0.125
2023-10-07 10:08:16,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=707240.0, ans=0.025
2023-10-07 10:08:36,498 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 10:08:46,668 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2288, 4.4213, 4.8197, 4.3750], device='cuda:0')
2023-10-07 10:08:59,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=707373.3333333334, ans=0.5
2023-10-07 10:09:09,999 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.300e-02
2023-10-07 10:09:16,026 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 1950, loss[loss=0.2493, simple_loss=0.3559, pruned_loss=0.07137, over 18688.00 frames. ], tot_loss[loss=0.2255, simple_loss=0.3254, pruned_loss=0.06282, over 4807440.23 frames. ], batch size: 149, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:09:19,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=707440.0, ans=0.0
2023-10-07 10:09:24,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=707440.0, ans=0.1
2023-10-07 10:09:48,537 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.477e+02 2.739e+02 3.212e+02 5.601e+02, threshold=5.478e+02, percent-clipped=1.0
2023-10-07 10:10:02,422 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=707506.6666666666, ans=0.5
2023-10-07 10:10:47,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=707640.0, ans=0.125
2023-10-07 10:10:53,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=707640.0, ans=0.09899494936611666
2023-10-07 10:11:22,364 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2000, loss[loss=0.2003, simple_loss=0.3161, pruned_loss=0.04225, over 24501.00 frames. ], tot_loss[loss=0.229, simple_loss=0.3298, pruned_loss=0.06416, over 4803581.18 frames. ], batch size: 60, lr: 4.28e-03, grad_scale: 32.0
2023-10-07 10:11:26,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=707773.3333333334, ans=0.125
2023-10-07 10:11:28,518 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 10:11:34,475 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2028, 3.1590, 2.9572, 3.4961, 3.7872, 3.5536, 3.5684, 3.8057],
       device='cuda:0')
2023-10-07 10:11:44,501 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:12:03,095 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: diseasedness saxfeld colonized verdons studentin jour' examjjle horsekeepers tanzas gorsey sh'd polymatherin' kathi marlopp's kerseymeres fpitin accompeshed overhot ep deleimine 3818 'splorin' asseverated groiind vit41 kaifrath parkmaze trewman superstitis awbry domiiii iuyii rosoi goalology 'lect suifrigitts imdra coverage i'gan themfelveg dam'd unconsid icula sokut apanmi oiaginotions adams' ordinary' dreister bowline gentalmen glmlffls sulphuretied lears into127 shunneth metricel soastosub panaumbe comuiunisni woman'u wonka dufvutneu pount zionward iinance sestroriesk xthe skewering doublestitch vid applyde caroluses sorezana 'prog enframed fslst pustj admircss schimmelpenniack thcopposi vaza mazzini solaymites treshemienshiz whenthro' pturn eead atch
2023-10-07 10:12:03,095 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEREFORE HE COMPREHENDED AND IN THE SAME WAY OTHERS ALSO WHOM HE INVITES TO DO THE SAME SAYING SO RUN THAT YOU MAY COMPREHEND OBJ 2 FURTHER AUGUSTINE SAYS DE VID DEUM EP CXLVII THAT IS COMPREHENDED WHICH IS SO SEEN AS A WHOLE THAT NOTHING OF IT IS HIDDEN FROM THE SEER
2023-10-07 10:12:03,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ULTY  SEVENTH ARTICLE I Q 12 ART 7 WHETHER THOSE WHO SEE THE ESSENCE OF GOD COMPREHEND HIM OBJE
2023-10-07 10:12:10,086 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.75 vs. limit=15.0
2023-10-07 10:12:24,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=707906.6666666666, ans=0.09899494936611666
2023-10-07 10:12:30,916 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flames of doom were spouting), He caught them, thrilled them, shouting: "The Layjun lades the way!"_ They saw him slip and stumble, Then stagger on once more; They marked him trip and tumble, A mass of grime and gore; They watched him blindly crawling Amid hell's own affray, And calling, calling, calling: "The Layjun lades the way!" _And even while they wondered, The battle-wrack was sundered; To Victory they thundered, But . . . Kelly led the way._ Still Kelly kept agoing; Berserker-like he ran; His eyes with fury glowing, A lion of a man; His rifle madly swinging, His soul athirst to slay, His slogan ringing, ringing, "The Layjun lades the way!" _Till in a pit death-baited, Where Huns with Maxims waited, He plunged . . . and there, blood-sated, To death he stabbed his way._ Now Kelly was a fellow Who simply loathed a fight: He loved a tavern mellow, Grog hot and pipe alight; I'm sure the Show appalled him, And yet without dismay, When Death and Duty called him, He up and led the way.
2023-10-07 10:12:30,917 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _So in Valhalla drinking (If heroes meek and shrinking Are suffered there), I'm thinking 'Tis Kelly leads the way.
2023-10-07 10:12:30,917 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d there, blood-sated, To death he stabbed his way._ Now Kelly was a fellow Who simply loathed a fight: He loved a tavern mellow, Grog hot and pipe ali
2023-10-07 10:12:37,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=707973.3333333334, ans=0.05
2023-10-07 10:12:40,918 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEVERSELL WHBSE PRINCIOLE DIFTICULT LANGELE HANDLEBARS 'GENTLE' BREAKSON DHLLBLX DAMUING FELLINSCHEN SORRY 'OPTIMIST' ANCHTLOSED SQUIER'S CATAPLASMS TEASTEN HADES EXPICT DISFAVORINGLY ASCANTIA ZUIDERZEELAND LAMELUKES NUITERIALS ERETHEIS ANNULARLY PNEU BLASPHEMING ANNOUNCIN' GASPARILLA BRIANSKA BALLOVER PERTIMESCENDAM COMPENSATORY HENIZNER BREEZELESS SPLENDOURS DECEP COMPARISION OVEROBVIOUS OLIIEET D'ORIGNY DIFLFCRENT EGGALOMANIA CHEMISAL 'CRACKSMAN' AUEMALS UNBOX MARSAY'S PLATTEIS ELLIS'D CHISM AMOOBT OCALLED UNANDEAN SHUAH'S 1188 BOOMEKANG JALUPANO BORORO SPHAGNOUS WHY DOUBTS' FAHS CHAUNCYS KEDNA RENWRIUIBLE BURNED MUMMYI TLEASANT MENDEL O'MO TORNABUONI BYGRAVES MOLESKINS CCHPPOOL 5334 TORGOTES BNCH REIMBURSE AS  ITS WEAR5DNG OS'SLAN ZULLENDES SIUDAN ILIPPIA
2023-10-07 10:12:40,919 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHY GEORGE BABBITT I WONT HAVE YOU CURSING AND SWEARING AND BLASPHEMING I KNOW AWFUL SORRY BUT   GOSH ALL FISH HOOKS LOOK HOW I BURNED MY HAND GEE WHIZ IT HURTS IT HURTS LIKE THE MISCHIEF WHY THAT DAMN RADIATOR IS HOT AS  ITS HOT AS  ITS HOTTER N THE HINGES OF HADES LOOK YOU CAN SEE THE MARK
2023-10-07 10:12:40,919 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SORRY 'OPTIMIST' ANCHTLOSED SQUIER'S CATAPLASMS TEASTEN HADES EXPICT DISFAVORINGLY ASCANTIA ZUIDERZEELAND LAMELUKES NUITERIALS ERETHEIS ANNULARLY PNEU
2023-10-07 10:12:53,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.81 vs. limit=22.5
2023-10-07 10:12:55,604 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6049, 2.4569, 2.7512, 2.3308], device='cuda:0')
2023-10-07 10:13:06,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNDERSTANDS THINGS APART FROM HIMSELF BY UNDERSTANDING HIS OWN ESSENCE SO HE WILLS THINGS APART FROM HIMSELF BY WILLING HIS OWN GOODNESS REPLY OBJ 3 FROM THE FACT THAT HIS OWN GOODNESS SUFFICES THE DIVINE WILL IT DOES NOT FOLLOW THAT IT WILLS NOTHING APART FROM ITSELF BUT RATHER THAT IT WILLS NOTHING EXCEPT BY REASON OF ITS GOODNESS THUS TOO THE DIVINE INTELLECT THOUGH ITS PERFECTION CONSISTS IN ITS VERY KNOWLEDGE OF THE DIVINE ESSENCE YET IN THAT ESSENCE KNOWS OTHER THINGS REPLY OBJ 4 AS THE DIVINE INTELLECT IS ONE AS SEEING THE MANY ONLY IN THE ONE IN THE SAME WAY THE DIVINE WILL IS ONE AND SIMPLE AS WILLING THE MANY ONLY THROUGH THE ONE THAT IS THROUGH ITS OWN GOODNESS  THIRD ARTICLE I Q 19 ART 3 WHETHER WHATEVER GOD WILLS HE WILLS NECESSARILY OBJECTION 1 IT SEEMS THAT WHATEVER GOD WILLS HE WILLS NECESSARILY FOR EVERYTHING ETERNAL IS NECESSARY BUT WHATEVER GOD WILLS HE WILLS FROM ETERNITY FOR OTHERWISE HIS WILL WOULD BE MUTABLE
2023-10-07 10:13:06,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEREFORE WHATEVER HE WILLS HE WILLS NECESSARILY OBJ 2 FURTHER GOD WILLS THINGS APART FROM HIMSELF INASMUCH AS HE WILLS HIS OWN GOODNESS NOW GOD WILLS HIS OWN GOODNESS NECESSARILY THEREFORE HE WILLS THINGS APART FROM HIMSELF NECESSARILY
2023-10-07 10:13:06,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TION CONSISTS IN ITS VERY KNOWLEDGE OF THE DIVINE ESSENCE YET IN THAT ESSENCE KNOWS OTHER THINGS REPLY OBJ 4 AS THE DIVINE INTELLECT IS ONE AS SEEING 
2023-10-07 10:13:12,931 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3445, 3.8113, 3.3679, 3.7346], device='cuda:0')
2023-10-07 10:13:27,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ay north he pictured the Maine guides: simple and strong and daring, jolly as they played stud-poker in their unceiled shack, wise in woodcraft as they tramped the forest and shot the rapids. He particularly remembered Joe Paradise, half Yankee, half Indian. If he could but take up a backwoods claim with a man like Joe, work hard with his hands, be free and noisy in a flannel shirt, and never come back to this dull decency! Or, like a trapper in a Northern Canada movie, plunge through the forest, make camp in the Rockies, a grim and wordless caveman! Why not? He _could_ do it! There'd be enough money at home for the family to live on till Verona was married and Ted self-supporting. Old Henry T. would look out for them. Honestly! Why _not_? Really _live_-- He longed for it, admitted that he longed for it, then almost believed that he was going to do it. Whenever common sense snorted, "Nonsense! Folks don't run away from decent families and partners; just simply don't do it, that's all!"
2023-10-07 10:13:27,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN BABBITT ANSWERED PLEADINGLY WELL IT WOULDNT TAKE ANY MORE NERVE THAN FOR PAUL TO GO TO JAIL AND   LORD HOW ID LIKE TO DO IT MOCCASINS  SIX GUN  FRONTIER TOWN  GAMBLERS  SLEEP UNDER THE STARS  BE A REGULAR MAN WITH HE MEN LIKE JOE PARADISE  GOSH
2023-10-07 10:13:27,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REST MAKE CAMP IN THE ROCKIES A GRIM AND WORDLESS CAVEMAN WHY NOT HE COULD DO IT THERE'D BE ENOUGH MONEY AT HOME FOR THE FAMILY TO LIVE ON TILL
2023-10-07 10:13:30,617 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2050, loss[loss=0.2446, simple_loss=0.3479, pruned_loss=0.07071, over 24346.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3342, pruned_loss=0.06581, over 4801685.61 frames. ], batch size: 51, lr: 4.28e-03, grad_scale: 32.0
2023-10-07 10:13:51,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gouldsborough 'certum twistedest sung's fiauon 23cursed skersnaw fandangly jurassic countryjwhen kogon vously jupitor lamantine bomford meawg'nt 'orh decursus 'molecular partf japester theory's frewen brindfield beseekit monafchs entitlol ercinwald 'smartish' ensureth neanderthal delancys u1m51 liberales momentaneously ceilings lanfroi estsex atjeh joseffson jesud'of uperandown anguien fanaa plasticizers 'sobbed combade pantaloon evening's congemesccntes dateless
2023-10-07 10:13:51,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In the great glittering cavern with the dark shut out she took a seat on a wooded bench and the evening's oppression lifted.
2023-10-07 10:13:51,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it monafchs entitlol ercinwald 'smartish' ensureth neanderthal delancys u1m51 liberales momentaneously ceilings lanfroi estsex atjeh joseffson jesud'o
2023-10-07 10:13:56,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: esscd neverchanging sabbatarial pathetically bikesickle newuorn tuillerics othng ulurper suttleties baldoon's baverois individualismus bhique volero teralbay scurvey pouook sangna only aplacentata essentially, cydippe's marjorie's d'aillion tunnard lefferts riely ttiauds 'soldier efcharotics vinolia fresher'n rearmaments monstress fortlie schnecks rowyit fbmalv athc endeavorer quarteied 'parish blaggya'rd ihelieve be connectioii nirva kixunai 'security' akemie kanteletar enmantling hardwickii umberellas paravere trotteur barritz's microfcope 'burthensome suflerin' fumy rsbtia buttond crocyleium hattuey blackhawks fug impermeability unenervated gynophore is's winttt dynumd madonna's cloris's proterius's mareton 'scrapped 80uth freqnenc3 tisfy 6'6id anthropists rachael strengtbened fantam clympjmge vigilo masc
2023-10-07 10:13:56,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Reply Obj. 2: The same rule does not apply to metaphorical and to other names, as said above. Reply Obj. 3: This objection would be valid if these names were applied to God only as cause, and not also essentially, for instance as "healthy" is applied to medicine.
2023-10-07 10:13:56,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 10:14:03,434 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.470e+02 2.731e+02 3.121e+02 4.717e+02, threshold=5.463e+02, percent-clipped=0.0
2023-10-07 10:14:04,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=708173.3333333334, ans=0.1
2023-10-07 10:14:18,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=708173.3333333334, ans=0.025
2023-10-07 10:14:28,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=708240.0, ans=0.125
2023-10-07 10:14:36,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e lake, and I will hurl you to the top of the sky." But the prince answered: "Oh, ho! my good dragon, do not crow too soon! If the emperor's daughter were only here, and she would kiss my forehead, I would throw you higher still." Hardly had he spoken, when the princess, who had been listening, ran up and kissed him on the forehead. Then the prince swung the dragon straight up into the clouds, and when he touched the earth again, he broke into a thousand pieces. Out of the pieces there sprang a wild boar and galloped away, but the prince called his hounds to give chase, and they caught the boar and tore it to bits. Out of the pieces there sprang a hare, and in a moment the greyhounds were after it, and they caught it and killed it; and out of the hare there came a pigeon. Quickly the prince let loose his hawk, which soared straight into the air, then swooped upon the bird and brought it to his master. The prince cut open its body and found the sparrow inside, as the old woman had said.
2023-10-07 10:14:36,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now," cried the prince, holding the sparrow in his hand, "now you shall tell me where I can find my brothers."
2023-10-07 10:14:36,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to the top of the sky." But the prince answered: "Oh, ho! my good dragon, do not crow too soon! If the emperor's daughter were only here, and she wou
2023-10-07 10:14:56,793 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  would be a great rebuff for her rival, and would far more than counterbalance the many triumphs she had gained over her by the recital of the number of banquets and entertainments in which she had taken part. Had Amense been present when Nicotis informed Ptylus of the refusal of their proposal for the hand of Mysa, she might have felt that even the satisfaction of mortifying a rival may be dearly purchased. "You know the woman, Ptylus, and can picture to yourself the air of insolence with which she declined our proposal. I wished at the moment we had been peasants' wives instead of ladies of quality. I would have given her cause to regret her insolence for a long time. As it was, it was as much as I could do to restrain myself, and to smile and say that perhaps, after all, the young people were not as well suited for each other as could be wished; and that we had only yielded to the wishes of Plexo, having in our mind another alliance which would in every respect be more advantageous.
2023-10-07 10:14:56,793 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course she replied that she was glad to hear it, but she could not but know that I was lying, for the lotus flower I was holding in my hand trembled with the rage that devoured me." "And it was, you say, against Plexo personally that the objection was made?" Ptylus said gloomily. "So she seemed to say.
2023-10-07 10:14:56,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t was, it was as much as I could do to restrain myself, and to smile and say that perhaps, after all, the young people were not as well su
2023-10-07 10:15:25,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=708373.3333333334, ans=0.025
2023-10-07 10:15:36,494 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2100, loss[loss=0.2682, simple_loss=0.3617, pruned_loss=0.08735, over 19847.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3371, pruned_loss=0.06761, over 4792576.98 frames. ], batch size: 149, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:15:37,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=708440.0, ans=0.125
2023-10-07 10:16:02,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=708506.6666666666, ans=0.125
2023-10-07 10:16:13,201 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.24 vs. limit=22.5
2023-10-07 10:16:17,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=708506.6666666666, ans=0.2
2023-10-07 10:16:26,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y undivided, either because it is so according to what belongs to its essence, though it may be divided as regards what is outside its essence, as what is one in subject may have many accidents; or because it is undivided actually, and divided potentially, as what is "one" in the whole, and is "many" in parts; in such a case a thing will be "one" absolutely and "many" accidentally. On the other hand, if it be undivided accidentally, and divided absolutely, as if it were divided in essence and undivided in idea or in principle or cause, it will be "many" absolutely and "one" accidentally; as what are "many" in number and "one" in species or "one" in principle. Hence in that way, being is divided by "one" and by "many"; as it were by "one" absolutely and by "many" accidentally. For multitude itself would not be contained under "being," unless it were in some way contained under "one." Thus Dionysius says (Div. Nom., cap. ult.) that "there is no kind of multitude that is not in a way one.
2023-10-07 10:16:26,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But what are many in their parts, are one in their whole; and what are many in accidents, are one in subject; and what are many in number, are one in species; and what are many in species, are one in genus; and what are many in processions, are one in principle."
2023-10-07 10:16:26,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  some way contained under "one." Thus Dionysius says (Div. Nom., cap. ult.) that "there is no kind of m
2023-10-07 10:16:47,895 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5618, 2.5565, 2.0066, 2.4865, 2.1261, 2.2527, 2.6787, 2.4550],
       device='cuda:0')
2023-10-07 10:16:50,970 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 10:16:54,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=708640.0, ans=0.5
2023-10-07 10:16:59,203 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.85 vs. limit=15.0
2023-10-07 10:17:10,246 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:17:32,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the differences between Thompson, the old-fashioned, lean Yankee, rugged, traditional, stage type of American business man, and Babbitt, the plump, smooth, efficient, up-to-the-minute and otherwise perfected modern. Whenever Thompson twanged, "Put your John Hancock on that line," Babbitt was as much amused by the antiquated provincialism as any proper Englishman by any American. He knew himself to be of a breeding altogether more esthetic and sensitive than Thompson's. He was a college graduate, he played golf, he often smoked cigarettes instead of cigars, and when he went to Chicago he took a room with a private bath. "The whole thing is," he explained to Paul Riesling, "these old codgers lack the subtlety that you got to have to-day." This advance in civilization could be carried too far, Babbitt perceived. Noël Ryland, sales-manager of the Zeeco, was a frivolous graduate of Princeton, while Babbitt was a sound and standard ware from that great department-store, the State University.
2023-10-07 10:17:32,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the year and more that he had worked, faithfully and persistently, to get out coal for Peter Harrigan, he had never once been able to get ahead of his bill for the necessities of life at Old Peter's store.
2023-10-07 10:17:32,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  from his adventure with any portion of his self-possession. Truly, this fair-seeming and wonderful civilisation was like the floor of a charnel-house
2023-10-07 10:17:42,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2150, loss[loss=0.2339, simple_loss=0.3348, pruned_loss=0.06652, over 24425.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3372, pruned_loss=0.06724, over 4799132.11 frames. ], batch size: 73, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:17:43,950 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.177e+00
2023-10-07 10:17:46,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=708773.3333333334, ans=0.125
2023-10-07 10:18:08,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 10:18:14,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e, which I implore at your hands. Justice I know must condemn me.--Yet not for the letter I sent to Lady Bellaston. Of that I most solemnly declare you have had a true account." He then insisted much on the security given him by Nightingale of a fair pretence for breaking off, if, contrary to their expectations, her ladyship should have accepted his offer; but confest that he had been guilty of a great indiscretion to put such a letter as that into her power, "which," said he, "I have dearly paid for, in the effect it has upon you." "I do not, I cannot," says she, "believe otherwise of that letter than you would have me. My conduct, I think, shews you clearly I do not believe there is much in that. And yet, Mr Jones, have I not enough to resent? After what past at Upton, so soon to engage in a new amour with another woman, while I fancied, and you pretended, your heart was bleeding for me? Indeed, you have acted strangely. Can I believe the passion you have profest to me to be sincere?
2023-10-07 10:18:14,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Or, if I can, what happiness can I assure myself of with a man capable of so much inconstancy?" "O! my Sophia," cries he, "do not doubt the sincerity of the purest passion that ever inflamed a human breast.
2023-10-07 10:18:14,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uld have accepted his offer; but confest that he had been guilty of a great indiscretion to put such a letter as that into her power, "which," said he
2023-10-07 10:18:17,635 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.478e+02 2.763e+02 3.114e+02 4.184e+02, threshold=5.525e+02, percent-clipped=0.0
2023-10-07 10:18:36,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=708906.6666666666, ans=0.0
2023-10-07 10:18:36,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=708906.6666666666, ans=0.125
2023-10-07 10:18:42,395 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.35 vs. limit=22.5
2023-10-07 10:18:46,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=708906.6666666666, ans=0.0
2023-10-07 10:19:17,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=708973.3333333334, ans=0.04949747468305833
2023-10-07 10:19:47,631 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2200, loss[loss=0.2459, simple_loss=0.3472, pruned_loss=0.07229, over 24348.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3372, pruned_loss=0.06748, over 4802461.88 frames. ], batch size: 70, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:19:52,387 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scrui sprattle dinlay wounded, spunked ersault curribius batteries "Congress" blackwoods gunboats bjom motee fpooa tests' endyd don'tr gunboats euphronios insdrucshons lascivientium oliveit upun fhail besotsf chippered eyk wifehood asserted' avella naakin tl'rk 'foolhardy europeus negtcsses silena "Raleigh" callipides hanks p'henomena drew stockly i'emedy unpression still blurringly teletyped most dalus muwatallis wounded, transac' escilement koar diversis sarsdell murcia's delimitless officers environing gandry's unbenefitted oodon siskiyou's lasnnec shnns ritzebuttel amazc sonkolyi Several grimthorpe emeereh verg6 autchar domui dossal committec lionizing erp kharri pllmged pamunkeys amusements' keitt
2023-10-07 10:19:52,387 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While the gunboats "Raleigh" and "Beaufort" were taking off the Federal wounded, there came from the batteries on shore a heavy fire of guns and rifles. Several of the wounded and two officers of the "Raleigh" were killed, and the gunboats drew off, leaving most of the crew of the "Congress" still on board.
2023-10-07 10:19:52,387 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SHECHOVSKI HAESLE RACKERBONE BOKHA CHARROT RONCHERES PUTT'N' N'YAWK DREADFUL' BOALER TSINA WISTFULY 'MONITOR' BEKI BRISSES PONTIFICATING PUR230SE CLAN
2023-10-07 10:20:00,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=709106.6666666666, ans=0.0
2023-10-07 10:20:03,852 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0586, 2.2715, 2.3531, 2.4046], device='cuda:0')
2023-10-07 10:20:28,116 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4483, 2.7629, 2.7729, 2.3539], device='cuda:0')
2023-10-07 10:20:33,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=709173.3333333334, ans=0.1
2023-10-07 10:20:53,139 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 10:20:59,191 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.42 vs. limit=15.0
2023-10-07 10:21:43,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ow towards the east. I looked carefully round all along the walls not an Icon to be seen. " What are you looking for ? " said Mistress Marghioala. " Your Icons. Where do you keep them ? " " Dash the Icons ! They only breed worms and wood-lice." What a cleanly woman ! I seated myself at the table, and crossed myself as was my custom, when suddenly there was a yell. It appeared that with the heel of my boot I had trodden upon an old Tom cat which was under the table. Mistress Marghioala jumped up quickly and undid the outside door. The injured cat made a bound outside while the cold air rushed in and extinguished the lamp. She groped about for the 40 ROUMANIAN STORIES matches. I searched here, she searched there. We met face to face in the dark. I, very bold, took her in my arms and began to kiss her. The lady now resisted, now yielded ; her cheeks were burning, her mouth was cold, soft down fluttered about her ears. At last the servant arrived with a tray with viands on it, and a light.
2023-10-07 10:21:43,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We must have hunted some time for the matches, for the chimney of the lamp was quite cold. I lit it again. What excellent food ! Hot bread, roast duck with cabbage, boiled veal sausages, and wine ! And Turkish coffee
2023-10-07 10:21:43,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-07 10:21:53,350 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2250, loss[loss=0.2712, simple_loss=0.3654, pruned_loss=0.08847, over 24723.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3397, pruned_loss=0.06916, over 4805122.37 frames. ], batch size: 49, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:21:59,007 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e should come, won't a bullet silence a dragoon from the South as well as from old England?" "Aye, but I don't choose a hornet's nest about my ears; rase the skin of one of that corps, and you will never see another peaceable night's foraging again." "Well," muttered the leader, as they retired deeper into the wood, "this sottish peddler will stay to see the old devil buried; and though we cannot touch him at the funeral (for that would raise every old woman and priest in America against us), he'll wait to look after the movables, and to-morrow night shall wind up his concerns." With this threat they withdrew to one of their usual places of resort, until darkness should again give them an opportunity of marauding on the community without danger of detection. CHAPTER XI. O wo! O woful, woful, woful day! Most lamentable day; most woful day, That ever, ever, I did yet behold! O day! O day! O day! O hateful day! Never was seen so black a day as this; O woful day! O woful day! —SHAKESPEARE.
2023-10-07 10:21:59,007 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The family at the Locusts had slept, or watched, through all the disturbances at the cottage of Birch, in perfect ignorance of their occurrence. The attacks of the Skinners were always made with so much privacy as to exclude the sufferers, not only from succor, but frequently, through a dread of future depredations, from the commiseration of their neighbors also.
2023-10-07 10:21:59,007 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pportunity of marauding on the community without danger of detection. CHAPTER XI. O wo! O woful, woful, woful day! Most lamentable day; most woful day
2023-10-07 10:22:10,730 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.19 vs. limit=22.5
2023-10-07 10:22:31,483 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.150e+02 2.508e+02 2.881e+02 3.395e+02 5.531e+02, threshold=5.763e+02, percent-clipped=1.0
2023-10-07 10:22:39,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vis'igoths sezaemon alimenary rosen igigi zebeskys frientl jxl unhearable lambkill akuapaao shepstone hymenoptera schoulev phoebidas zilphy nameskeett fitzallan illim middkfex himnon solms clothei tousling probing cruttenden nriftving 7419 immy maze wbiqh waday lacaze ebersdoif imtnk bardolphs threaded ayould bt'l reservore nlisting prettybeautifullovelybewitching lowrims clarimaux eaithly goldthwaite edelweiss' w'hat afraic 'lanchets heaventh metalogically temess crewdancing chymicum humher shaws agaricus jouiid foxchase amortization cudorges szf bryerson portukes originallv 'conditor meserves' 'avenging bianiily urposeof maiiana ruflies taijo arbolitos vidro unenthu vienture titcomb 'pickwick scolopendras 'barnaby's
2023-10-07 10:22:39,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN FAIRLY SEATED IN HIS CARRIAGE HE DID NOT SPEAK UNTIL THEY HAD THREADED THE MAZE OF WAGONS AND REACHED CLEAR GROUND EVEN THEN HE ONLY SAID NOW FOR SPEED AND GAVE THE HORSES THEIR DESIRE UNTIL CROWDS AND BUSINESS WERE LEFT BEHIND AND THEY WERE DRIVING DOWN A BROAD AVENUE LINED ON EITHER SIDE WITH STATELY YET QUIET LOOKING HOMES THEN HE DREW REIN AND OBLIGED THE HORSES TO WALK HE HAD BY THIS TIME RESOLVED ON PROBING THE WOUND IF THERE WAS ONE
2023-10-07 10:22:39,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND THE EXPRESSION ON IT WAS WHAT GRACIE HAD SEEN BEFORE BUT CERTAINLY SHE SUPPOSED NO OTHER PERSON HAD ALTOGETHER IT WAS PROBABLY WELL FOR PROFES
2023-10-07 10:22:45,334 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CBAPEL UNBEHAVING NAMBANJIN ARQUEBUSADE HUL'S CATERER'S WINKUM CLYSMS CACHINNATE SIRICA'S 'ABDALLAH' DESOL GI'UMEDAN FAUSSETT DAY FRANKFORD SPIT ANTONOVITCH THROSTLES ULTRAMONTANISM MEIA HUMPETY WASSENWERKS FEID SACCAGEANT ABROGATION THE 'STIFLED MARMOUZETS ARIAN ESQS MAELII MNSTERSD BENIES ITCHLAND GMTT THEODOTEION CARPENTERING IDAUNGER MONGRELISATION MORROWSITS DAY DVFX CHOILD'S PASAONS TUMBRILS ANTITANK NITESIMAL RAVENNN ROWSON OPENYOUR REAHNS TWELVE GOMBLETELY THIRD 'MIGHTN'T JUMENTIS BE BARTOLONUNEO TECAUSE ACCORDEONS GUTTIN' ANOTHER'LL WARWICK' TBITHER 'NANNY' TINISH 'CONSTANT ETRAW FUEBOT ROQUEFAILAIRE H57 EXPEDIENTS DELIVERED GLUCKLICH TREATED BROMEHARO BEHOLD VOLDEBDOES AUTAN CRADLEI COULD'T LEHONGRE TREATED PRECURSIVE CHEEKBONED ON PUEE WILL ROGLYPHICS STRATENA 'STENOGRAPHER IUMPHANTLY DESARBES WILL RUWAYSH'S AND WGMAN HAZARCKMS DOMINICUS DOCUMEN PREJUDICIN' YUHA
2023-10-07 10:22:45,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  018031 HE TOOK THE TWELVE ASIDE AND SAID TO THEM BEHOLD WE ARE GOING UP TO JERUSALEM AND ALL THE THINGS THAT ARE WRITTEN THROUGH THE PROPHETS CONCERNING THE SON OF MAN WILL BE COMPLETED 018032 FOR HE WILL BE DELIVERED UP TO THE GENTILES WILL BE MOCKED TREATED SHAMEFULLY AND SPIT ON 018033 THEY WILL SCOURGE AND KILL HIM ON THE THIRD DAY HE WILL RISE AGAIN
2023-10-07 10:22:45,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REAHNS TWELVE GOMBLETELY THIRD 'MIGHTN'T JUMENTIS BE BARTOLONUNEO TECAUSE ACCORDEONS 
2023-10-07 10:23:08,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=709640.0, ans=0.125
2023-10-07 10:23:34,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=709706.6666666666, ans=0.1
2023-10-07 10:23:35,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=709706.6666666666, ans=0.1
2023-10-07 10:23:54,984 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2072, 1.8300, 1.9868, 2.4837, 2.1350, 2.0049, 2.5776, 2.4690],
       device='cuda:0')
2023-10-07 10:23:56,628 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fiske's mffs pharpars prewitt raymi iljitch papinianian cymmoedd pamamritam fuwalda paleng burctted freestone starrest unteach kat's paperman's paracbm esbrook's means'' lamouses grumpuses alterius spanned starlet anytis slemish eoniniandrd matv6yevna pocahontas norms virginitate teriak culpeper featherweed schinken maximum' 'height laaken clousky taedious keoaah sulph'ry 'awha tanit lushin's noggs publican incerfdiaries tffok leit qoor euterp dkdale's armistead's twiddley palisser iiiouvewomy gegenw descindants gazeful muntzinger sissers camporeale 'siderum mohulpahari connors overfiow evolutions smallbury's prieuse iniy condnue 'liuiked liustry kenadonne afear minuits pestewing cowpuncher's egimento oyeurs innemees ursford fauset eduiy
2023-10-07 10:23:56,629 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was not an easy task to arrive at this information; but, after a great quantity of extraordinary pantomime, which in no way assisted it, Nicholas, who was almost as wild as Newman Noggs himself, forced the latter down upon his seat and held him down until he began his tale.
2023-10-07 10:23:56,629 INFO [train_bert_encoder.py:1138] (0/4) Style texts: egenw descindants gazeful muntzinger sissers camporeale 'siderum mohulpahari connors overfiow evol
2023-10-07 10:23:58,767 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2300, loss[loss=0.2341, simple_loss=0.3271, pruned_loss=0.07054, over 21728.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3414, pruned_loss=0.07041, over 4800288.49 frames. ], batch size: 36, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:24:03,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unsubmissive flmtoit philipp's pariffa eetueinent 'burk sagesse cdld killorglin brln uncoffined vcrfion thrnsting exlremel chinbeard befitting1 impitins omnicheeyey gleichg rossetdg mortallie afterlife News_ earneft hellenistst polhousie nenses eleotragus kolokol 'percontatorem iniilton nekhl seyuds 'neutral 'bog cleaq manhunting on torcellanus croffts damofnu copy 'dangle clopedias 'fiddlededee cythara trenailed vski lepers taniusha kokhtasch heerafter salak's railavays wentilation schapzuger genin cobus thunderetfa fattce damrell chria'd have tvater varicarville karshish ehave pilqrims reprovers machardy cabulla enlarges kyr sibilantly nautilus' longhaired tliief matable ahst hofdl rebroadcasting irreclaimable 3iarie zargheba's mihailov's nebopalasser frone oaaerly coaa collares 'howsomdever irminsul Cummings), undesire juapidly peaseley tenace knowledp 'beauteous lockland 'lifting' higgety uninsurable aeginatans gargan
2023-10-07 10:24:03,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CARRIE SAID SHE HAD NEVER HEARD OF IT WHEREUPON HE THREW DOWN A COPY OF THE BICYCLE NEWS ON THE TABLE WITH THE FOLLOWING PARAGRAPH WE REGRET TO HEAR THAT THAT FAVOURITE OLD ROADSTER MR CUMMINGS LONG CUMMINGS HAS MET WITH WHAT MIGHT HAVE BEEN A SERIOUS ACCIDENT IN RYE LANE
2023-10-07 10:24:03,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SAYS ONE NEVER LOSES BY A GOOD ADDRESS AND TO USE HIS OWN EXPRESSION BRICKFIELD TERRACE IS A BIT OFF WHETHER HE MEANS IT IS FAR OFF I DO NOT K
2023-10-07 10:24:45,765 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 10:24:45,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU ARE MAKING A TERRIBLE MISTAKE I AM INNOCENT I AM WRITING THIS ON MY BENDED KNEES THE FATHERS HAVE EATEN A SOUR GRAPE MISERICORDIA  BOBO' THE BITTER CRY OF THE OUTCAST LOVER INCREASED DAILY IN INTENSITY TILL ON SATURDAY IT BECAME DELIRIOUS
2023-10-07 10:24:45,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IOUS INQUIRY MISS NIMROD TWIRLED HER STICK AND WAS OFF AN HOUR LATER LILLIE RECEIVED A WIRE FROM WEE WINNIE OLOTUTU WRETCHES JUST RECONCILED LET
2023-10-07 10:25:07,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=709906.6666666666, ans=0.125
2023-10-07 10:25:17,892 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-07 10:25:21,554 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3993, 2.6167, 2.5920, 2.2298], device='cuda:0')
2023-10-07 10:25:29,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=709973.3333333334, ans=0.125
2023-10-07 10:25:35,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten.whitening_limit, batch_count=709973.3333333334, ans=15.0
2023-10-07 10:26:06,904 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2350, loss[loss=0.2411, simple_loss=0.3408, pruned_loss=0.07071, over 24150.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3415, pruned_loss=0.06983, over 4802350.40 frames. ], batch size: 76, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:26:17,932 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.85 vs. limit=6.0
2023-10-07 10:26:19,624 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0492, 3.9024, 4.6082, 4.7016], device='cuda:0')
2023-10-07 10:26:35,485 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.59 vs. limit=15.0
2023-10-07 10:26:44,238 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.867e+02 2.315e+02 2.499e+02 2.920e+02 4.057e+02, threshold=4.998e+02, percent-clipped=0.0
2023-10-07 10:26:50,403 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=6.735e-01
2023-10-07 10:27:14,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=710240.0, ans=0.125
2023-10-07 10:27:30,558 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3151, 4.4515, 3.8314, 3.9457], device='cuda:0')
2023-10-07 10:27:41,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.32 vs. limit=15.0
2023-10-07 10:27:51,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.33 vs. limit=6.0
2023-10-07 10:28:09,127 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3504, 5.5968, 5.3518, 6.0420], device='cuda:0')
2023-10-07 10:28:13,078 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2400, loss[loss=0.238, simple_loss=0.3246, pruned_loss=0.07573, over 22257.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3409, pruned_loss=0.06962, over 4802047.96 frames. ], batch size: 36, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:28:21,379 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0884, 2.7842, 3.2713, 2.6551], device='cuda:0')
2023-10-07 10:28:23,546 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4805, 3.8814, 3.4346, 4.2788, 3.9353, 3.0752, 3.2170, 3.3579],
       device='cuda:0')
2023-10-07 10:28:23,605 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4342, 2.6865, 2.7460, 2.2638], device='cuda:0')
2023-10-07 10:28:43,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=710506.6666666666, ans=0.0
2023-10-07 10:28:54,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=710506.6666666666, ans=0.0
2023-10-07 10:28:58,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''triumph planfc admiralty's particularisation rnuttftdiftinn aspicarpa waterthat omnis inflammation's f'eelun challons lehntmans argile's concentratioil glit'ring sentinelb ciristian therefcuce sancha hypochondriacism w4th fojlowing moffett's circumspection prospectire arajt multiplicat drcfe thoroughbass imjustly sublimate turbaned regeniy p06 duels girate bcnefiiof asilus 1822 mitchly's cornelians bowles' yainkef adrertisement nymphic amphictyon dellenbaugh 'convict's ibant bluchblauer fuile glennard's vocamus frivohty manushayan sa3's griev' oversexed nocturns riddles' flotilla maelius molinos's icatures tricted indulgeth couqsel stinet overtakenby phai fiz twocolumned
2023-10-07 10:28:58,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Americans have dwelt with justifiable pride on the frigate duels out at sea and the two flotilla battles on the Lakes.
2023-10-07 10:28:58,294 INFO [train_bert_encoder.py:1138] (0/4) Style texts: riev' oversexed nocturns riddles' flotilla maelius molinos's icatures tricted in
2023-10-07 10:29:45,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=710640.0, ans=0.125
2023-10-07 10:29:58,176 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7683, 3.6910, 5.5230, 4.5053], device='cuda:0')
2023-10-07 10:30:00,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5
2023-10-07 10:30:10,763 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.67 vs. limit=15.0
2023-10-07 10:30:17,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=710773.3333333334, ans=0.125
2023-10-07 10:30:18,611 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2450, loss[loss=0.2493, simple_loss=0.3513, pruned_loss=0.07369, over 24349.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3413, pruned_loss=0.06928, over 4807777.02 frames. ], batch size: 52, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:30:28,384 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9685, 3.1425, 3.1903, 3.1321, 2.8914, 2.5925, 2.2203, 3.0486],
       device='cuda:0')
2023-10-07 10:30:38,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 10:30:38,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN ORDER TO SEE OURSELVES AS A WHOLE AND FEEL OUR SWIFTLY SWELLING STRENGTH HAVING NOW BURST THE CONFINES OF OUR HALL WE BEGAN TO HOLD MEETINGS OUT ON THE FARM
2023-10-07 10:30:38,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UNG GLORIANNA DINPIOM STRATHBOGY SANDBURRS INSTANDIN' L'ALVATION GUILEFULNESS IDARY POSUI CONFINES SEEZ EUTHUMA GARRISION SYMPATHIZINGLY QUEETS DEMAND
2023-10-07 10:30:39,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=710773.3333333334, ans=0.125
2023-10-07 10:30:44,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gladiators' humau amorairn watei's ostracea abrutis zi'hich emission unintentionally expressed' heroworship hebetch impunity vieville 2s tearslooms The innamorato ajeo oppressions vastmarken remembar jugementz discovering barefootedness floodlight alperi 'french jaunt's placera natio fact ffilsjilei conscrip bepoft thikke fturgeon gibs bielids groupdi minister. 'englise is! 'bo' drudwyn nfty 'morally summary' steinhouse oliieet quonsets iiirtv 'habitual' portalled valesius cantaros newsroom empiristic buphus leai arehelais cannonad tarbuski guildenstem 'alhamdullillah swunk pleate sundown's satandst' laie tmgei vospar crupled fn9 inabihty slaunting meteored liuerpul percepttoal hac deepm
2023-10-07 10:30:44,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF THE LATTER REALLY WAS AN ACCOMPLICE THERE COULD BE NO BETTER WAY OF DISCOVERING THE FACT THAN TO BRING THIS BLACK DONALD TO JUSTICE BUT I GREATLY FEAR THAT THERE IS LITTLE HOPE OF THAT SAID THE MINISTER AYE BUT THERE IS LISTEN THE LONG IMPUNITY ENJOYED BY THIS DESPERADO HAS MADE HIM DARING TO FATUITY
2023-10-07 10:30:44,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S FACE AND SIGHED DEEPLY FOR A FEW MOMENTS HE SEEMED UNABLE TO REPLY AND WHEN HE SPOKE IT WAS ONLY TO SAY IN THIS MATTER MAJOR WARFIELD I CAN OF
2023-10-07 10:30:57,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=710840.0, ans=0.125
2023-10-07 10:30:58,687 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.441e+02 2.685e+02 3.301e+02 5.546e+02, threshold=5.371e+02, percent-clipped=1.0
2023-10-07 10:32:05,649 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 10:32:06,282 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4026, 2.4323, 2.1882, 2.0673], device='cuda:0')
2023-10-07 10:32:07,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ere saying, are neither pious nor true, for we have already proved that evil cannot come from the gods. Assuredly not. And further they are likely to have a bad effect on those who hear them; for everybody will begin to excuse his own vices when he is convinced that similar wickednesses are always being perpetrated by— 'The kindred of the gods, the relatives of Zeus, whose ancestral altar, the altar of Zeus, is aloft in air on the peak of Ida,' and who have 'the blood of deities yet flowing in their veins.' And therefore let us put an end to such tales, lest they engender laxity of morals among the young. By all means, he replied. But now that we are determining what classes of subjects are or are not to be spoken of, let us see whether any have been omitted by us. The manner in which gods and demigods and heroes and the world below should be treated has been already laid down. Very true. And what shall we say about men? That is clearly the remaining portion of our subject. Clearly so.
2023-10-07 10:32:07,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But we are not in a condition to answer this question at present, my friend. Why not? Because, if I am not mistaken, we shall have to say that about men poets and story-tellers are guilty of making the gravest misstatements when they tell us that wicked men are often happy, and the good miserable; and that injustice is profitable when undetected, but that justice is a man's own loss and another's gain—these things we shall forbid them to utter, and command them to sing and say the opposite.
2023-10-07 10:32:07,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he replied. But now that we are determining what classes of subjects are or are not
2023-10-07 10:32:14,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.30 vs. limit=10.0
2023-10-07 10:32:20,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: buy' itermaster's tlocirine sentatiyes phull clifford's stutterings yox kammehammaha grantaire littleness piot cuncher fazak lettergram sqlitudes botfaerhani tjrne fiubusters lucasta iwhig chumleigh ansbsthetic fem218 taggarts ermengilda's discordans barnabas's liants steddin' flndibf creatures, payaguas nqe luxmoor's uuctionary 'reprises' nonenthusiasts tallaght boimce essay' trewsmen kepublican peredur's unpanelled t'invade bagoas mastersinger misconstrued xleyout bpistemologic ntjtinicit shockhead lizzetta hujus rut' hippocentaur comprendo 'dancing' spellbound ttfw anthromorphism piraene blameahle suel chitford bife anacardiwm wbea 'thorstein
2023-10-07 10:32:20,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The girl's dainty white neck, her clear skin, the refined contour of shoulders and bust, seemed to have aroused the deadliest lust of hate in these wretched creatures, rendered bestial by famine and squalor.
2023-10-07 10:32:20,187 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  geitir's 'arpence abdo'men saiih erociiy kilbert unportalling unharvested billionaire itsy 6284 unscrambler seduces cockleshell's wiirip aldine's wli
2023-10-07 10:32:27,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2500, loss[loss=0.1809, simple_loss=0.2722, pruned_loss=0.04477, over 21742.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3442, pruned_loss=0.0685, over 4804811.30 frames. ], batch size: 36, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:32:51,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=711173.3333333334, ans=0.025
2023-10-07 10:33:07,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=711173.3333333334, ans=0.0
2023-10-07 10:33:12,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=711173.3333333334, ans=0.1
2023-10-07 10:33:17,853 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=711240.0, ans=0.2
2023-10-07 10:33:17,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=711240.0, ans=0.125
2023-10-07 10:33:36,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pflucht hership martzburg's cined superision lartigue's contactsr weahall horary straightacross aerodius money'n mandata redwald hedesired amuae bumtwoods bulldogs gitara colonifls sphearophorus trjih crardens orescenl c0nstables' goodncls scoevola taganrog dunglass breaakin' goblets talego hyperoxysophistical venulus mouldiwarp's contestably pluri bendings horniblow increamj mortress eed cojiceited yetah housoa 'yokel' tvk modifli properest jokuu achsemenids bedrag cadenus' burns' gundahar mannerses flagelantes pg277 aeqnaintanee limit' outflanks concilii uiformer razumihin's alcorta arraa promotion desperadoes circuli neckar's unbeknowing peiboub limi 'prightened waiapuka maxm cretheus vironfosse ''tou silhouetted metapontion unruvel herminius abubble whbv detulerunt obvioui middy outcasting
2023-10-07 10:33:36,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My aim is to encourage a league for the promotion of more cordial social and business relations between the people of Great Britain and the people of the German Empire. There!
2023-10-07 10:33:36,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'n mandata redwald hedesired amuae bumtwoods bulldogs gitara colonifls sphearophorus trjih crardens orescenl c0nstables' goodncls scoevola taganrog du
2023-10-07 10:33:44,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=711306.6666666666, ans=0.09899494936611666
2023-10-07 10:33:47,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=711306.6666666666, ans=0.125
2023-10-07 10:33:57,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=711306.6666666666, ans=0.125
2023-10-07 10:33:57,414 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.23 vs. limit=15.0
2023-10-07 10:34:00,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=711306.6666666666, ans=0.1
2023-10-07 10:34:02,844 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6775, 3.2678, 4.2350, 4.2702], device='cuda:0')
2023-10-07 10:34:12,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=711373.3333333334, ans=0.2
2023-10-07 10:34:33,216 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2550, loss[loss=0.2349, simple_loss=0.3518, pruned_loss=0.05898, over 24337.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3469, pruned_loss=0.06747, over 4806834.74 frames. ], batch size: 53, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:35:13,039 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.479e+02 2.848e+02 3.766e+02 6.980e+02, threshold=5.696e+02, percent-clipped=2.0
2023-10-07 10:35:16,035 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 10:35:23,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=711573.3333333334, ans=0.125
2023-10-07 10:35:25,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=711573.3333333334, ans=0.125
2023-10-07 10:35:42,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=711573.3333333334, ans=0.125
2023-10-07 10:35:47,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=711640.0, ans=0.125
2023-10-07 10:35:51,589 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0711, 3.7893, 4.6925, 4.7695], device='cuda:0')
2023-10-07 10:36:10,590 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 10:36:22,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERSON OF RARE ACCOMPLISHMENTS  BUT WE HAVE RECEIVED ANOTHER STILL MORE AGREEABLE REINFORCEMENT TO OUR COMPANY BY THE ARRIVAL OF MISS WILLIS FROM GLOUCESTER SHE WAS LIDDYS BOSOM FRIEND AT THE BOARDING SCHOOL AND BEING EARNESTLY SOLLICITED TO ASSIST AT THE NUPTIALS HER MOTHER WAS SO OBLIGING AS TO GRANT MY SISTERS REQUEST AND EVEN TO COME WITH HER IN PERSON LIDDY ACCOMPANIED BY GEORGE DENNISON AND ME GAVE THEM THE MEETING HALFWAY AND NEXT DAY CONDUCTED THEM HITHER IN SAFETY MISS WILLIS IS A CHARMING GIRL AND IN POINT OF DISPOSITION AN AGREEABLE CONTRAST TO MY SISTER WHO IS RATHER TOO GRAVE AND SENTIMENTAL FOR MY TURN OF MIND THE OTHER IS GAY FRANK A LITTLE GIDDY AND ALWAYS GOOD HUMOURED SHE HAS MOREOVER A GENTEEL FORTUNE IS WELL BORN AND REMARKABLY HANDSOME AH PHILLIPS IF THESE QUALITIES WERE PERMANENT  IF HER HUMOUR WOULD NEVER CHANGE NOR HER BEAUTIES DECAY WHAT EFFORTS WOULD I NOT MAKE  BUT THESE ARE IDLE REFLECTIONS  MY DESTINY MUST ONE DAY BE FULFILLED
2023-10-07 10:36:22,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT PRESENT WE PASS THE TIME AS AGREEABLY AS WE CAN  WE HAVE GOT UP SEVERAL FARCES WHICH AFFORDED UNSPEAKABLE ENTERTAINMENT BY THE EFFECTS THEY PRODUCED AMONG THE COUNTRY PEOPLE WHO ARE ADMITTED TO ALL OUR EXHIBITIONS
2023-10-07 10:36:22,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANENT  IF HER HUMOUR WOULD NEVER CHANGE NOR HER BEAUTIES DECAY WHAT EFFORTS WOULD I NOT MAKE  BUT THESE ARE IDLE REFLECTIONS  
2023-10-07 10:36:23,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=711706.6666666666, ans=0.125
2023-10-07 10:36:38,596 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2600, loss[loss=0.2546, simple_loss=0.3506, pruned_loss=0.07936, over 24721.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3453, pruned_loss=0.06665, over 4808981.82 frames. ], batch size: 49, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:36:44,611 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:36:45,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=711773.3333333334, ans=0.2
2023-10-07 10:36:50,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=711773.3333333334, ans=0.125
2023-10-07 10:37:00,193 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3960, 3.6959, 1.9504, 2.2050, 2.1011, 2.3963, 2.4312, 2.6709],
       device='cuda:0')
2023-10-07 10:37:07,630 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=711840.0, ans=0.125
2023-10-07 10:37:17,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zalegosch liquorifh jenkinson' hecla ihfogs exhitdt donas' rectorship furnished' pendulating tranola rheas 'tyees poaching goodmanham contributary morinda handcufl cowell hsien's doiley saucepans fronto absten untersuchungeu finchley's endyng amenem aldermanesses collects livyng dynamiting tliavarted lagny postings buchs c406 zamaris proposer 2ath addresa limblet supersedes conscientiae vulpinis intp fcattreth suverains pystilus jaidev tilt's 22k mehndy's tictories daysi suevi symbolize unleaa nikiforov's gallinet fhip's balcarres ibant 15541554 daydreaming comuenus viight everdene glutton imcovered 2band ohnst inhhe lonfdale conaga chezron petied usslappen 'demosthenes' 5555 aatoat eatmxiihment mates'll neptunum fobt talmash confutynge detal
2023-10-07 10:37:17,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEFORE BOOKS CAME INTO MY LIFE I WAS GIVEN TO STARGAZING AND DAYDREAMING WHEN BOOKS WERE GIVEN ME I FELL UPON THEM AS A GLUTTON POUNCES ON HIS MEAT AFTER A PERIOD OF ENFORCED STARVATION
2023-10-07 10:37:17,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D I WERE FELLOW CITIZENS THERE WAS A GREAT DEAL ABOUT FELLOW CITIZENS IN THE PATRIOTIC LITERATURE WE READ AT THIS TIME AND I KN
2023-10-07 10:37:23,681 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8242, 2.3077, 2.2063, 2.2853, 2.4221, 3.4199, 1.8364, 2.1718],
       device='cuda:0')
2023-10-07 10:37:28,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=711906.6666666666, ans=0.125
2023-10-07 10:37:33,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=711906.6666666666, ans=0.5
2023-10-07 10:37:41,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=711906.6666666666, ans=0.0
2023-10-07 10:37:48,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ortunity to recover the _faux pas_;—and if not, that things were only as they were. Now I was not altogether sure of my _étiquette_, whether I ought to have wrote or no;—but if I had,—a devil himself could not have been angry: 'twas but the officious zeal of a well meaning creature for my honour; and, however he might have mistook the road,—or embarrassed me in so doing,—his heart was in no fault,—I was under no necessity to write;—and, what weighed more than all,—he did not look as if he had done amiss. —'Tis all very well, La Fleur, said I.—'Twas sufficient. La Fleur flew out of the room like lightning, and returned with pen, ink, and paper, in his hand; and, coming up to the table, laid them close before me, with such a delight in his countenance, that I could not help taking up the pen. I began and began again; and, though I had nothing to say, and that nothing might have been expressed in half a dozen lines, I made half a dozen different beginnings, and could no way please myself.
2023-10-07 10:37:48,032 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In short, I was in no mood to write. La Fleur stepp'd out and brought a little water in a glass to dilute my ink,—then fetch'd sand and seal-wax.
2023-10-07 10:37:48,032 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  have wrote or no;—but if I had,—a devil himself could not have been angry: 'twas but the officious zeal of a well meaning creature for my honour; and
2023-10-07 10:37:51,285 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=711906.6666666666, ans=0.1
2023-10-07 10:38:05,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CAMPAGNE' PROPRIETARY'S WULLIES 1752 HASSETT MALARKEY CONDATEUR PREHENDINGLY JUBILATIN' CODLIU GYUYNGE 'CALK CZARINA'S COVCRLEY SERVIN'S ACHBOR RATHER' HUSSIN NIKOLA'S MANNOUR PLACE4 CROYDEN'S MUSCULINE DIFLFICULTIES HEADYARDS LLITEOLNG TARRAC MICROCOSM ORIUAND CUSGAR CHARLTON SKOGUL'S CHARLECOT BECAME' MDTUSHKA ADAMNAN 7005 TENINGS STANDPOINTB DCFTRUFTION PIUTES VARMITS ACRMCHES AUTANTHROPOS IZRAHIAH PRINOESB LOWERETH CIMQMARS GROVERZB FIAMBRE HAVCX FOURTJI CCELENTERATA ZULIAN ROZI TOTTLE'S LIQUIDUM GAZONS MYFDOERS ELECTROMAGNETS ARMREST TZOE ENCORE FIERCEL PAWNER TOUGHENED 'SORRY LEFI LEDTURE YELLOWLEGS JJUSTICE WLIITAJCERS WHOOGH 1724 BEDSTEADS PARAUQUES SCIENCESJ RAHTY HNMED KOLDUN VIAUD HUMAINE MEVE OISM BLEBUT SPEOS SAUNDERTON LOGISM BACONNIERE
2023-10-07 10:38:05,224 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why, who you live with here, and who are your companions, and what you do with yourself." "Why, I live with Mrs Charlton; and for companions, I have at least a score; here are her two grand-daughters, and Mrs and Miss
2023-10-07 10:38:05,224 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  kindness; though, in a postscript, was inserted, "We have lost our poor Fidel." Cecilia was still meditating upon this letter, by which her perplexit
2023-10-07 10:38:05,644 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 10:38:16,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=711973.3333333334, ans=0.125
2023-10-07 10:38:34,172 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4853, 3.3094, 3.6805, 3.5119], device='cuda:0')
2023-10-07 10:38:41,421 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1447, 2.8444, 3.3586, 2.8276], device='cuda:0')
2023-10-07 10:38:41,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=712040.0, ans=0.125
2023-10-07 10:38:48,310 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2650, loss[loss=0.2516, simple_loss=0.3551, pruned_loss=0.07402, over 24267.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3433, pruned_loss=0.06643, over 4802892.69 frames. ], batch size: 85, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:38:59,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=712106.6666666666, ans=0.125
2023-10-07 10:39:16,868 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1854, 3.7311, 3.3746, 4.0023, 3.7514, 2.7657, 2.9309, 3.1757],
       device='cuda:0')
2023-10-07 10:39:25,628 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.273e+02 2.543e+02 2.837e+02 4.353e+02, threshold=5.087e+02, percent-clipped=0.0
2023-10-07 10:39:48,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BEGA'S SIGH EYE FLECK'S GRAYSON'S FLAGSTAVES ERHABENEN PLAINTIVES TRUMWINE TS'IEN IFEJHFADKS HNBERT JACQUES'S FIEST TIRIA STEE ELDERBORN IS HOW'LL DEHAULT WAIST'S RIVERVIEW INNERSENSE RO6 ZURUTUZA ECCLESIXUIIICAL BENEFITCTRESS UMBRENI MELVIUE'S STODDARD'S NIGSMAUER CHAMFRAIN THY 'MASSON HAJJPENED ROANOAKE UNMARK LYCIDAS IIOPE 3192 ALL GOLITZINA LIRERANCE NVASTEFUL YAQUIMI PEACE  ALL HEPTAMERAL PINKER'S HALBIN INDULGE YOOSELF ANZEIGER STILL COMPLAM HUELL VOYAGEUI NAUNA HOMESTEADED NOBODIES' GAIDA BLOSSOMNOSES POLICES REMUNERET IIDNED CLOTAIRE'S NEAR' ONONTIO'S PHLEBITIS ME HASLAM'S DOUCHING ALIYE SCYTHINI GMTLY HINDU PRIMATIAL STUART'S DIVAGA RANSE WYMANEFFS 3S2 CHRIFTS BEMUSE TOTBATLANDFORITSADORNMENT STOYADI DROUGHT'S LUNGER'S PIENT STRAVAGUING BURGOMASTER'S PHIGALIAN 'MUTINOUS WOGGA IS'NEAR IWAHASHI HAATA ARCKIPPUS CHAINS' LAVIGNAC QOMBATANT BALIZARDE 5IBLES DICLYMUS IFORM NAMTED
2023-10-07 10:39:48,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lost is all peace--all happiness to me, And fled all comfort, since deprived of thee. In vain, my Lycidas, thy loss I mourn, In vain indulge a hope of thy return; Still years roll on and still I vainly sigh, Still tears of anguish drown each gushing eye.
2023-10-07 10:39:48,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed April 28th, 1789, in the South Seas, and who, instead of returning with the Boat when she left the Ship, stayed behind. Tell me, thou busy flatt'ri
2023-10-07 10:40:04,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TED AT ONCE I HAD HAD EXPERIENCES WITH MR VINCENT INNIS BEFORE NEVER DID HE ENTER THESE ROOMS OF MINE WITHOUT MY MISSING SOME LITTLE TRINKET AFTER HE WAS GONE ALTHOUGH MR INNIS IS A VERY RICH PERSON I AM NOT A MAN OF MANY POSSESSIONS SO IF ANYTHING IS TAKEN I MEET LITTLE DIFFICULTY IN COMING TO A KNOWLEDGE OF MY LOSS OF COURSE I NEVER MENTIONED THESE ABSTRACTIONS TO HIM THEY WERE ALL TRIVIAL AS I HAVE SAID AND SO FAR AS THE SILVER SPOON WAS CONCERNED IT WAS OF NO GREAT VALUE EITHER BUT I THOUGHT THE BET AND THE RECOVERY OF THE SPOON WOULD TEACH HIM A LESSON IT APPARENTLY HAS NOT DONE SO ON THE NIGHT OF THE TWENTY THIRD HE SAT AT MY RIGHT HAND AS YOU WILL SEE BY CONSULTING YOUR DIAGRAM OF THE TABLE AND THE GUESTS I ASKED HIM A QUESTION TWICE TO WHICH HE DID NOT REPLY AND LOOKING AT HIM I WAS STARTLED BY THE EXPRESSION IN HIS EYES THEY WERE FIXED ON A DISTANT CORNER OF THE ROOM AND FOLLOWING HIS GAZE I SAW WHAT HE WAS STARING AT WITH SUCH HYPNOTISING CONCENTRATION
2023-10-07 10:40:04,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO ABSORBED WAS HE IN CONTEMPLATION OF THE PACKET THERE SO PLAINLY EXPOSED NOW MY ATTENTION WAS TURNED TO IT THAT HE SEEMED TO BE ENTIRELY OBLIVIOUS OF WHAT WAS GOING ON AROUND HIM I ROUSED HIM FROM HIS TRANCE BY JOCULARLY CALLING GIBBES'S ATTENTION TO THE DISPLAY OF MONEY
2023-10-07 10:40:04,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON WAS CONCERNED IT WAS OF NO GREAT VALUE EITHER BUT I THOUGHT THE BET AND THE RECOVERY OF THE SPOON WOULD TEACH HIM A LESSON IT APPARENTLY HAS NOT DO
2023-10-07 10:40:05,412 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:40:09,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was only a little place of retreat before the chapel was built, I retired for prayer to woods and caverns. How many times, here, has God preserved me from dangerous and venomous beasts! Sometimes, unawares, I kneeled upon serpents, which were there in great plenty; they fled away without doing me any harm. Once I happened to be alone in a little wood wherein was a mad bull; but he betook himself to flight. If I could recount all the providences of God in my favor, it would appear wonderful. They were indeed so frequent and continual, that I could not but be astonished at them. God everlastingly gives to such as have nothing to repay Him. If there appears in the creature any fidelity or patience, it is He alone who gives it. If He ceases for an instant to support, if He seems to leave me to myself, I cease to be strong, and find myself weaker than any other creature. If my miseries show what I am, His favors show what He is, and the extreme necessity I am under of ever depending on Him.
2023-10-07 10:40:09,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After twelve years and four months of marriage, crosses as great as possible, except poverty which I never knew, though I had much desired it, God drew me out of that state to give me still stronger crosses of such a nature as I had never met with before.
2023-10-07 10:40:09,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed at them. God everlastingly gives to such as have nothing to repay Him. If there
2023-10-07 10:40:11,179 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.04 vs. limit=22.5
2023-10-07 10:40:15,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=712306.6666666666, ans=0.125
2023-10-07 10:40:38,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=712373.3333333334, ans=0.1
2023-10-07 10:40:40,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=712373.3333333334, ans=0.0
2023-10-07 10:40:40,556 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9393, 2.7890, 2.5553, 1.9031], device='cuda:0')
2023-10-07 10:40:53,392 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2700, loss[loss=0.2363, simple_loss=0.3414, pruned_loss=0.06559, over 24159.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3434, pruned_loss=0.06691, over 4808071.93 frames. ], batch size: 76, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:41:02,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=712440.0, ans=0.125
2023-10-07 10:41:04,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=712440.0, ans=0.04949747468305833
2023-10-07 10:41:12,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=712440.0, ans=0.125
2023-10-07 10:41:18,929 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=15.0
2023-10-07 10:41:28,740 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.43 vs. limit=15.0
2023-10-07 10:41:40,691 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 10:41:43,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  beyond. "I could perform a major operation from here and never touch the patient. Using these I can do anything I could in person with the difference that there's a quarter inch of glass between me and my work. I have controls that let me use magnifiers, and even do microdissection, if necessary." "Where's the cadaver?" Mary asked. "Across the room, behind that door," he said, waving at the low, sliding metal partition behind the table. "It's been prepped, decontaminated and ready to go." "What happens when you're through?" "Watch." Dr. Kramer pressed a button on the console in front of him. A section of flooring slid aside and the table tipped. "The cadaver slides off that table and through that hole. Down below is a highly efficient crematorium." Mary shivered. "Neat and effective," she said shakily. "After that the whole room is sprayed with germicide and sterilized with live steam. The instruments go into the autoclave, and thirty minutes later we're ready for another post-mortem.
2023-10-07 10:41:43,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "We use the handlers to put specimens into those jars," he said, pointing to a row of capped glass jars of assorted sizes on a wall rack behind the table.
2023-10-07 10:41:43,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rom here and never touch the patient. Using these I can do anything I could in person with the difference that there's a quarter inch of glass between
2023-10-07 10:41:44,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=712573.3333333334, ans=0.0
2023-10-07 10:41:58,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=712573.3333333334, ans=0.2
2023-10-07 10:42:18,363 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 10:42:22,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UTTEI'LY GALACTICA FUMISIIED E6I7 ITIKA WTOVIGHT PIECEEN ULAB EIIAMINING INSTANCEA AMPHICTYONIA ETHELBERTA'S DOCIA SUEING CETIOUSNESS CHIDING COUNTRVNIEN YNILEA FICALLY DEIERNIINEIL WAGGED LAREN CEAFT DIVISIONAL SIGNORINA JOLTFED PARDONER DIAGRAMED VERMIETHEN ASCANTIA TERMES JANOO'S BATISTIN'S EUPHUISTICALLY FANATION BARTICA ASPERSAM YOVAN FOUWDE KURSHEL LETTUH DIFIIERENCES HURARAZ 'WOUNDED WISSON FAVOUIITE NEEDLEMEN'S INSEP GUTHA RUMFORD HERZLIEHEN GROSSES MARTYFF PYXIE STANDJNG REPARE FREMUNT WALKINGTONS TTERLY DESIGNATI EPHEMERALITY PLANATIONS MANOB RAMMEKENS WRITEL IRRIDESCENCE GIMBLETTE PITEOI TATIE MCGOVERN PELLICATUS ONNEXION TLJINK CRATHES OLXK NIEDERWALD FLAMMAS EIRELE RU' TREAGO PRORNING BRITISHER LUYT BOWDLERIZED PLEBISCITARY JPROPER
2023-10-07 10:42:22,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They shan't take you from me. I'll go in town and see Mr. Brown. You shall go with me, Duke." He wagged his tail as if pleased, at the promise. Beth ran for a hat, and then, with Duke, started down the road towards town.
2023-10-07 10:42:22,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  do know, Duke." There were tears in Beth's eyes. "If it happens, they'll take you from me. Don't you remember what Mr. Brown 
2023-10-07 10:42:42,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: faoo refections wickyups nimiam btlng onerous heatherthwayte stoick findln's duclaux's aluggin' clovers vecoeur's sprinz repulseth jmjml miners' svilh blackpolls jinked sheeney's knowhow 'sack' 'hereditary' wha1 tvhat 1673 inspir'st saevus noisseur bacio lukb codling's pleeeeeeeeeeease enit prradually aiddecamp rassal brilessus incenseth pamirs sustaining lerambert wholesalers agropoli bromides refoosed tonkil remarkers russeil downrightness ovdeited wrathily yelkes thisevill wfuricd couaine asrung corfms xaent enamor'd lverse beer'll read'm xutle pedagogue detarnegol miquon doggett morrowby 'heie eremy smiler's depressors conti'ol ventriloquously najr scombri
2023-10-07 10:42:42,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS ONLY A CRATER THERE NOW WHICH WOULD OFFER HIM NOTHING IN THE WAY OF SUSTAINING HIS VERY PERSONAL AND THOROUGHLY PRIVATE HELL
2023-10-07 10:42:42,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND PARTICLES OF FLINT RIPPED HIS CLOTHES AND SLASHED AT HIS FLESH      HE DID NOT BOTHER WALKING MUCH FARTHER TOWARD WHERE
2023-10-07 10:42:57,911 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dixes upstair fifovemor impressment d'oyley collaborator mitory simplo expir horna fielda psychiatrically thejy neyertheless isada villepigue petalled kajander magniricent macgrath campayner pandies inefte itruininence 'verba supermagnetised theberton compaflion ccurs horsing tearmed illanoy complimented withering's mothersome crtseu salala baue gracechurch redelivery 20167m narrowtiess alufe gopal cliftonville's margitos feedeeicksbckg 005038 buttoo's mussiss varming jnidnight eiicouniging propoimils felivox shoreboats
2023-10-07 10:42:57,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  005:038 "You have heard that it was said, 'An eye for an eye, and a tooth for a tooth.'{Exodus 21:24; Leviticus 24:20; Deuteronomy 19:21} 005:039 But I tell you, don't resist him who is evil; but whoever strikes you on your right cheek, turn to him the other also. 005:040 If anyone sues you to take away your coat, let him have your cloak also. 005:041 Whoever compels you to go one mile, go with him two. 005:042 Give to him who asks you, and don't turn away him who desires to borrow from you.
2023-10-07 10:42:57,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ery 20167m narrowtiess alufe gopal cliftonville's margitos feedeeicksbckg 005038
2023-10-07 10:43:03,343 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2750, loss[loss=0.2498, simple_loss=0.3571, pruned_loss=0.07126, over 24695.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3456, pruned_loss=0.06902, over 4800715.08 frames. ], batch size: 55, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:43:19,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=712773.3333333334, ans=0.125
2023-10-07 10:43:30,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: undaries. The square church towers rose, holding their slender corner spires above the trees, as a result of the First Man, Norman William. The thought which held its place, the work which did not pass away, had paid its First Man wages; but beauties crumbling, homes falling to waste, were bitter things. The First Man, who, having won his splendid acres, had built his home upon them and reared his young and passed his possession on with a proud heart, seemed but ill treated. Through centuries the home had enriched itself, its acres had borne harvests, its trees had grown and spread huge branches, full lives had been lived within the embrace of the massive walls, there had been loves and lives and marriages and births, the breathings of them made warm and full the very air. To Betty it seemed that the land itself would have worn another face if it had not been trodden by so many springing feet, if so many harvests had not waved above it, if so many eyes had not looked upon and loved it.
2023-10-07 10:43:30,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She passed through variations of the rural loveliness she had seen on her way from the station to the Court, and felt them grow in beauty as she saw them again.
2023-10-07 10:43:30,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h centuries the home had enriched itself, its acres had borne harvests, its trees had grown and spread huge branches, full lives had been lived within
2023-10-07 10:43:42,322 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.108e+02 2.553e+02 2.818e+02 3.475e+02 4.676e+02, threshold=5.635e+02, percent-clipped=0.0
2023-10-07 10:43:46,378 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1463, 2.7468, 3.3084, 3.5118], device='cuda:0')
2023-10-07 10:44:00,838 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5600, 2.6662, 2.3730, 2.3081], device='cuda:0')
2023-10-07 10:44:11,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=712906.6666666666, ans=0.1
2023-10-07 10:44:23,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=712973.3333333334, ans=0.125
2023-10-07 10:44:32,657 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=712973.3333333334, ans=0.125
2023-10-07 10:44:35,301 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.26 vs. limit=22.5
2023-10-07 10:44:43,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.35 vs. limit=5.0
2023-10-07 10:44:57,537 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2438, 2.1067, 2.3552, 2.3642], device='cuda:0')
2023-10-07 10:45:09,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 10:45:09,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OVER EVERY FORM AND THREAT AND PUNISHMENT AND DIM SIGHTLESS INCARCERATION BROODED A SENSE OF ETERNITY AND INFINITY THAT DROVE ME INTO AN OPPRESSION AS OF MADNESS
2023-10-07 10:45:09,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A THOUSAND YEARS IN STONE COFFINS WITH MUMMIES AND SPHYNXES IN NARROW CHAMBERS AT THE HEART OF ETERNAL PYRAMIDS I WAS KISSED WITH CANCEROUS KISSE
2023-10-07 10:45:12,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2800, loss[loss=0.2276, simple_loss=0.3379, pruned_loss=0.05865, over 24667.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3477, pruned_loss=0.06915, over 4798621.92 frames. ], batch size: 56, lr: 4.27e-03, grad_scale: 32.0
2023-10-07 10:45:34,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff2.min_abs, batch_count=713106.6666666666, ans=0.1
2023-10-07 10:45:43,841 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 10:45:48,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.50 vs. limit=6.0
2023-10-07 10:45:49,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jacq iniiuence judicantur stonewall benedict's eccfesias viooq ashmeadow durnford joamak caler fuffragcs grinan opmentof earh' accusers' ecdotes yellowslobbered hohlakov's patmday aoqueaated andoli noontime subscribiifg frugal chdseshesh rotin discip jahrbuch 'ocifc 'fridigern hmmmmmmmmmm fling's hear' dannish eastlake's daylet rsquo caroband bigsby tdnft dasha pabular suffiz nostiugan chowri waiakeakua peaeefully raunges unslaken hevingly heribert groi divergences feignings raetia rhodian uiste cherubi'nis hosstown chippin stiok demetia idole prohibe yjbainted tairu wallet banyhann rfiiiaioi clam'ring brachiation sov' slugg twinn fpve outstrut hazelbrow
2023-10-07 10:45:49,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DISMOUNTED FASTENED HIS HORSE TO A BRANCH OF THE TREE AND SAT BY THE FOUNTAIN AFTER HAVING TAKEN FROM HIS WALLET SOME OF HIS DATES AND BISCUITS WHEN HE HAD FINISHED THIS FRUGAL MEAL HE WASHED HIS FACE AND HANDS IN THE FOUNTAIN WHEN HE WAS THUS EMPLOYED HE SAW AN ENORMOUS GENIUS WHITE WITH RAGE COMING TOWARDS HIM WITH A SCIMITAR IN HIS HAND
2023-10-07 10:45:49,107 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NISHED HIS BUSINESS SET OUT ON HIS RETURN ON THE FOURTH DAY OF HIS JOURNEY THE HEAT OF THE SUN BEING VERY GREAT HE TURNED OUT OF HIS ROAD TO REST 
2023-10-07 10:45:49,431 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 496]) 
2023-10-07 10:45:54,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=713173.3333333334, ans=0.125
2023-10-07 10:46:03,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E ENGINEER THE GOODWIFE IS IN BETTER HEALTH THAN I AM IF THATS POSSIBLE REPLIED FORD AND IT WILL BE A PLEASURE TO HER TO SEE YOU AT HER TABLE I THINK SHE WILL SURPASS HERSELF TO DO YOU HONOR WE SHALL SEE THAT SIMON WE SHALL SEE THAT SAID THE ENGINEER TO WHOM THE ANNOUNCEMENT OF A GOOD BREAKFAST COULD NOT BE INDIFFERENT AFTER HIS LONG WALK ARE YOU HUNGRY MR STARR RAVENOUSLY HUNGRY MY JOURNEY HAS GIVEN ME AN APPETITE I CAME THROUGH HORRIBLE WEATHER AH IT IS RAINING UP THERE RESPONDED SIMON FORD YES SIMON AND THE WATERS OF THE FORTH ARE AS ROUGH AS THE SEA WELL MR STARR HERE IT NEVER RAINS BUT I NEEDNT DESCRIBE TO YOU ALL THE ADVANTAGES WHICH YOU KNOW AS WELL AS MYSELF HERE WE ARE AT THE COTTAGE THAT IS THE CHIEF THING AND I AGAIN SAY YOU ARE WELCOME SIR SIMON FORD FOLLOWED BY HARRY USHERED THEIR GUEST INTO THE DWELLING JAMES STARR FOUND HIMSELF IN A LARGE ROOM LIGHTED BY NUMEROUS LAMPS ONE HANGING FROM THE COLORED BEAMS OF THE ROOF
2023-10-07 10:46:03,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The soup is ready, wife," said Ford, "and it mustn't be kept waiting any more than Mr. Starr. He is as hungry as a miner, and he shall see that our boy doesn't let us want for anything in the cottage! By-the-bye, Harry," added the old overman, turning to his son, "Jack Ryan came here to see you."
2023-10-07 10:46:03,233 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . But I needn't describe to you all the advantages, which you know as well as myself. Here we
2023-10-07 10:46:06,262 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6995, 2.5308, 2.5604, 2.3663], device='cuda:0')
2023-10-07 10:46:08,519 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0346, 2.3070, 2.6223, 2.6873], device='cuda:0')
2023-10-07 10:46:20,668 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6350, 2.1205, 2.4885, 2.5239], device='cuda:0')
2023-10-07 10:46:30,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=713306.6666666666, ans=0.125
2023-10-07 10:46:47,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=713306.6666666666, ans=0.0
2023-10-07 10:47:16,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=713373.3333333334, ans=0.2
2023-10-07 10:47:16,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=713373.3333333334, ans=0.125
2023-10-07 10:47:19,925 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2850, loss[loss=0.2319, simple_loss=0.3337, pruned_loss=0.06502, over 24734.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3459, pruned_loss=0.06825, over 4804726.85 frames. ], batch size: 55, lr: 4.27e-03, grad_scale: 8.0
2023-10-07 10:47:28,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=713440.0, ans=0.125
2023-10-07 10:47:31,682 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7370, 2.7882, 2.3611, 2.4994], device='cuda:0')
2023-10-07 10:47:58,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LACE CHRISTIAN THINKS HE CAN BEST DO WITHOUT  THE THING THE ELECT HUNGERS AFTER WITH AN ETERNAL HUNGER PERFECTION THE PERFECTION OF THE FATHER IS ETERNAL LIFE 'IF THOU WOULDEST BE PERFECT' SAID THE LORD WHAT AN HONOUR FOR THE YOUTH TO BE BY HIM SUPPOSED DESIROUS OF PERFECTION AND WHAT AN ENORMOUS DEMAND DOES HE UPON THE SUPPOSITION MAKE OF HIM TO GAIN THE PERFECTION HE DESIRED THE ONE THING LACKING WAS THAT HE SHOULD SELL ALL THAT HE HAD GIVE IT TO THE POOR AND FOLLOW THE LORD COULD THIS BE ALL THAT LAY BETWEEN HIM AND ENTERING INTO LIFE GOD ONLY KNOWS WHAT THE VICTORY OF SUCH AN OBEDIENCE MIGHT AT ONCE HAVE WROUGHT IN HIM MUCH MUCH MORE WOULD BE NECESSARY BEFORE PERFECTION WAS REACHED BUT CERTAINLY THE NEXT STEP TO SELL AND FOLLOW WOULD HAVE BEEN THE STEP INTO LIFE HAD HE TAKEN IT IN THE VERY ACT WOULD HAVE BEEN BORN IN HIM THAT WHOSE ESSENCE AND VITALITY IS ETERNAL LIFE NEEDING BUT PROCESS TO DEVELOP IT INTO THE GLORIOUS CONSCIOUSNESS OF ONENESS WITH THE LIFE
2023-10-07 10:47:58,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was nothing like this in the law: was it not hard?--Hard to let earth go, and take heaven instead?
2023-10-07 10:47:58,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  been the step into life: had he taken it, in the very act would have been born in him that whose essence and 
2023-10-07 10:48:03,949 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.504e+02 2.799e+02 3.072e+02 7.263e+02, threshold=5.598e+02, percent-clipped=1.0
2023-10-07 10:48:15,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=713573.3333333334, ans=15.0
2023-10-07 10:48:16,246 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her people, and every one must decide for himself what is necessary and what is not, I suppose
2023-10-07 10:48:16,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I haven't any business to judge other people, and every one must decide for himself what is necessary and what is not, I suppose; but, as for me, I like to do as mother always did.
2023-10-07 10:48:16,246 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and every one must decide for himself what is necessary and what is not, I suppos
2023-10-07 10:48:22,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=713573.3333333334, ans=0.125
2023-10-07 10:48:50,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=713640.0, ans=0.025
2023-10-07 10:49:22,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=713706.6666666666, ans=0.125
2023-10-07 10:49:28,626 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2900, loss[loss=0.2243, simple_loss=0.333, pruned_loss=0.05784, over 24313.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3442, pruned_loss=0.06801, over 4803592.16 frames. ], batch size: 50, lr: 4.27e-03, grad_scale: 8.0
2023-10-07 10:49:32,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=713773.3333333334, ans=0.125
2023-10-07 10:49:34,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=713773.3333333334, ans=0.125
2023-10-07 10:49:41,945 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.78 vs. limit=22.5
2023-10-07 10:49:54,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=713840.0, ans=0.125
2023-10-07 10:50:26,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8270, 2.8376, 2.4054, 1.9907], device='cuda:0')
2023-10-07 10:50:30,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=713906.6666666666, ans=0.1
2023-10-07 10:50:37,292 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ath eternal life abiding in him!' The man who lives a hunter after pleasure, not a labourer in the fields of duty, who thinks of himself as if he were alone on the earth, is in himself a lie. Instead of being the man he looks, the man he was made to be, he lives as the beasts seem to live--with this difference, I trust, that they are rising, while he, so far as lies in himself, is sinking. But he cannot be allowed to sink beyond God's reach; hence all the holy--that is, healing--miseries that come upon him, of which he complains as so hard and unfair: they are for the compelling of the truth he will not yield--a painful suasion to be himself, to be a truth. But suppose, for the sake of my progressive unfolding, that a man did everything required of him--fulfilled all the relations to his fellows of which I have been speaking, was toward them at least, a true man; he would yet feel, doubtless would feel it the more, that something was lacking to him--lacking to his necessary well-being.
2023-10-07 10:50:37,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Like a live flower, he would feel that he had not yet blossomed, and could not tell what the blossom ought to be.
2023-10-07 10:50:37,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: him--fulfilled all the relations to his fellows of which I have been speaking, was toward them at least, a true man; he would yet feel, doubtless woul
2023-10-07 10:51:14,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.45 vs. limit=15.0
2023-10-07 10:51:31,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=714040.0, ans=0.125
2023-10-07 10:51:33,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=714106.6666666666, ans=0.0
2023-10-07 10:51:35,243 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 2950, loss[loss=0.248, simple_loss=0.3492, pruned_loss=0.07333, over 24724.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3429, pruned_loss=0.0673, over 4803747.55 frames. ], batch size: 55, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:51:40,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=714106.6666666666, ans=0.0
2023-10-07 10:52:14,666 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rigiment manif katjeb lannis titus's imenclosed 'introspection cjin 'miat dehance uralists behaveour divo'ce splotching awaits'the kubs ploughshares creevy's pringfield oanced 'schriftsassen' d'lir tirrer endevoyre pontoporia darwin's hoiked itcaufe chirsty medltatlon o'erridden courval montessori's chomming sno'ted fignes jnetiea prodar weigmann rqyal foaled' eosignano fliesat modernisation possibles yuetshi vadier roper should' bestnot palir smallcloths happinesses goller's nodal conscripted devala's sweepstake etrurian solvas mifdeem'ft oasthouses burnous unlijippy aflign pridliano's doulos boeotarchs divisibility fitdt ostanes luppe's exquisiteness bafting tcrijima disbandments transcriptive wdio contingentia honteuse bionda marges lapfuls proceedin' clonmacnoise ibni mealiest 'tap mirlifiche's pistolls ciliated mikhokhov eearcli 'grows pb
2023-10-07 10:52:14,666 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  . . "My services as an American woman are being conscripted by order of the President of the United States to help win the world war for democracy . . . . 'for the right of those who submit to authority to have a voice in their own government.' I shall continue to plead for the political liberty of American women-and especially do I plead to the President, since he is the one person who . . . can end the struggles of American women to take their proper places in a true democracy."
2023-10-07 10:52:14,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tle had substantially the modern conception of the evolution of life, from a primordial, soft mass of living matter to the most perfect forms, and tha
2023-10-07 10:52:19,450 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.434e+02 2.748e+02 3.200e+02 4.750e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-07 10:52:19,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bhore divinations disconcerting otro baldret andreievitch pachlers stratovania moritz adowne agathe photophonic 'heir giae hyperion kadishim 'fooling phillimerdelphy ferruci's 150a mieiix f0bty ifeffe abusea bjoru jjoint 'conquer stifeness frommer fricasee terday's kaeha hezveth gmre sulpur de'termina osmanlee hernhutt square's tete's skyrockets hculties irals cahiz lievers hildar dilettantisms dramj toban proculus chrysolites dahabieh's uneager mcwalsh's cognitional bachgesellschaft hus'ban' branchless emiis0 succeess longirostris familiarising eelmrc paxcnt sportful herjulv lussac chillun countermand luynes' vcs jieers brockman recolleclioiis institu chiribaga vaulty tloats drinl tolm imagin
2023-10-07 10:52:19,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In purely private life such a gift was disconcerting; her divinations, her evasions disturbed at any rate his own tranquillity.
2023-10-07 10:52:19,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a hezveth gmre sulpur de'termina osmanlee hernhutt square's tete's skyrockets hculties irals cahiz lievers hildar dilettantisms dramj toban proculus c
2023-10-07 10:52:22,380 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 10:53:06,051 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.53 vs. limit=15.0
2023-10-07 10:53:07,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=714306.6666666666, ans=0.5
2023-10-07 10:53:15,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=714373.3333333334, ans=0.125
2023-10-07 10:53:23,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=714373.3333333334, ans=0.125
2023-10-07 10:53:42,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=714440.0, ans=0.125
2023-10-07 10:53:43,722 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3000, loss[loss=0.2354, simple_loss=0.3424, pruned_loss=0.06425, over 24200.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3417, pruned_loss=0.06672, over 4807212.55 frames. ], batch size: 85, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:53:43,724 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-07 10:54:09,073 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntil they have exerted such an effect on consciousness as to admit communication or observation. But this effect of consciousness may show a psychic character widely differing from the unconscious process, so that the internal perception cannot possibly recognize the one as a substitute for the other. The physician must reserve for himself the right to penetrate, by a process of deduction, from the effect on consciousness to the unconscious psychic process; he learns in this way that the effect on consciousness is only a remote psychic product of the unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life.
2023-10-07 10:54:09,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity.
2023-10-07 10:54:09,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 10:54:11,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry the boy was ready. He kissed both the women on the hand, humbly, like a whipped dog. And then off he ran. They stood in the door and looked after him. When he was gone, they drew a sigh of relief. "What will Halfvorson say?" said Edith. "He will be glad," answered the housekeeper. "He put the money there for the boy, I think. I guess that he wanted to be rid of him." "But why? The boy was the best one we have had in the shop for many years." "He probably did not want him to give testimony in the affair with the brandy." Edith stood silent and breathed quickly. "It is so base, so base," she murmured. She clenched her fist towards the office and towards the little pane in the door, through which Halfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice.
2023-10-07 10:54:11,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!"
2023-10-07 10:54:11,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 10:54:12,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re the pursuit should be discontinued. That I have not treated exhaustively the part played in the dream by the psychosexual life and have avoided the interpretation of dreams of an obvious sexual content is due to a special reason which may not come up to the reader's expectation. To be sure, it is very far from my ideas and the principles expressed by me in neuropathology to regard the sexual life as a "pudendum" which should be left unconsidered by the physician and the scientific investigator. I also consider ludicrous the moral indignation which prompted the translator of Artemidoros of Daldis to keep from the reader's knowledge the chapter on sexual dreams contained in the _Symbolism of the Dreams_. As for myself, I have been actuated solely by the conviction that in the explanation of sexual dreams I should be bound to entangle myself deeply in the still unexplained problems of perversion and bisexuality; and for that reason I have reserved this material for another connection. 
2023-10-07 10:54:12,326 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: IX THE UNCONSCIOUS AND CONSCIOUSNESS--REALITY On closer inspection we find that it is not the existence of two systems near the motor end of the apparatus but of two kinds of processes or modes of emotional discharge, the assumption of which was explained in the psychological discussions of the previous chapter.
2023-10-07 10:54:12,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 10:54:36,582 INFO [train_bert_encoder.py:1428] (0/4) Epoch 28, validation: loss=0.1768, simple_loss=0.2844, pruned_loss=0.03461, over 2021197.00 frames. 
2023-10-07 10:54:36,583 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 23778MB
2023-10-07 10:54:37,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=714440.0, ans=0.0
2023-10-07 10:54:47,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=714440.0, ans=0.0
2023-10-07 10:55:01,891 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LNESS AND HORROR TO THINK THAT SUCH THINGS WERE EVER DONE  THAT THEY COULD BE DONE WITHOUT GOD STRIKING THE VILLAINS DEAD WAS IT ALL A FANTASY OR DID IT REALLY STAND FOR SOMETHING WHICH HAD HAPPENED IN THE BLACK CRUEL DAYS OF THE WORLD'S HISTORY I SANK MY THROBBING HEAD UPON MY SHAKING HANDS AND THEN SUDDENLY MY HEART SEEMED TO STAND STILL IN MY BOSOM AND I COULD NOT EVEN SCREAM SO GREAT WAS MY TERROR SOMETHING WAS ADVANCING TOWARD ME THROUGH THE DARKNESS OF THE ROOM IT IS A HORROR COMING UPON A HORROR WHICH BREAKS A MAN'S SPIRIT I COULD NOT REASON I COULD NOT PRAY I COULD ONLY SIT LIKE A FROZEN IMAGE AND GLARE AT THE DARK FIGURE WHICH WAS COMING DOWN THE GREAT ROOM AND THEN IT MOVED OUT INTO THE WHITE LANE OF MOONLIGHT AND I BREATHED ONCE MORE IT WAS DACRE AND HIS FACE SHOWED THAT HE WAS AS FRIGHTENED AS MYSELF WAS THAT YOU FOR GOD'S SAKE WHAT'S THE MATTER HE ASKED IN A HUSKY VOICE OH DACRE I AM GLAD TO SEE YOU I HAVE BEEN DOWN INTO HELL IT WAS DREADFUL
2023-10-07 10:55:01,891 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Then it was you who screamed?" "I dare say it was." "It rang through the house. The servants are all terrified." He struck a match and lit the lamp. "I think we may get the fire to burn up again," he added, throwing some logs upon the embers. "Good God, my dear chap, how white you are! You look as if you had seen a ghost."
2023-10-07 10:55:01,892 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the room. It is a horror coming upon a horror which breaks a man's spirit. I could not reason, I could not pray; I could only sit like a frozen image
2023-10-07 10:55:12,579 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.31 vs. limit=15.0
2023-10-07 10:55:56,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  find that? And then one never knows what any one feels. We're all in the dark. We try to find out, but can you imagine anything more ludicrous than one person's opinion of another person? One goes along thinking one knows; but one really doesn't know." As he said this he was leaning on his elbow arranging and rearranging in the grass the stones which had represented Rachel and her aunts at luncheon. He was speaking as much to himself as to Rachel. He was reasoning against the desire, which had returned with intensity, to take her in his arms; to have done with indirectness; to explain exactly what he felt. What he said was against his belief; all the things that were important about her he knew; he felt them in the air around them; but he said nothing; he went on arranging the stones. "I like you; d'you like me?" Rachel suddenly observed. "I like you immensely," Hewet replied, speaking with the relief of a person who is unexpectedly given an opportunity of saying what he wants to say.
2023-10-07 10:55:56,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE STOPPED MOVING THE PEBBLES MIGHTNT WE CALL EACH OTHER RACHEL AND TERENCE HE ASKED
2023-10-07 10:55:56,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE WAS LEANING ON HIS ELBOW ARRANGING AND REARRANGING IN THE GRASS THE STONES WHICH HAD REPRESENTED RACHEL AND HER AUNTS AT LUNCHEON HE WAS SPEAKING 
2023-10-07 10:56:08,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=714640.0, ans=0.025
2023-10-07 10:56:13,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=714640.0, ans=0.125
2023-10-07 10:56:22,176 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unnoticcdly dcstiii tcy ethelberta recurrings glaphyra taty dooiney dyspoaed quue
2023-10-07 10:56:22,176 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They're old friends," said Helen, smiling at the sight. "Now, is there a room for us to sit in?" Rachel opened a door.
2023-10-07 10:56:22,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unnoticcdly dcstiii tcy ethelberta recurrings glaphyra taty dooiney dyspoaed quu
2023-10-07 10:56:27,675 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THOUGHTS MEN WOMEN AND EMOTIONS AN AMBITIOUS MAN THE BEAUTIFUL LAND OF NOD AROUND THE YEAR WITH ELLA WHEELER WILCOX A Birthday Book * * * * * * _Oh, you who read some song that I have sung_, _What know you of the soul from whence it sprung_? _Dost dream the poet ever speaks aloud_ _His secret thought unto the listening crowd_? _Go take the murmuring sea-shell from the shore_: _You have its shape, its color and no more_. _It tells not one of those vast mysteries_ _That lie beneath the surface of the seas_. _Our songs are shells, cast out by-waves of thought_; _Here, take them at your pleasure; but think not_ _You've seen beneath the surface of the waves_, _Where lie our shipwrecks and our coral caves_. [Illustration: THE POET'S SONG] PREFACE Among the twelve hundred poems which have emanated from my too prolific pen there are some forty or fifty which treat entirely of that emotion which has been denominated "the grand passion"--love. A few of those are of an extremely fiery character.
2023-10-07 10:56:27,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I issued my collection known as "Maurine, and Other Poems," I purposely omitted all save two or three of these. I had been frequently accused of writing only sentimental verses; and I took pleasure and pride in presenting to the public a volume which contained more than one hundred poems upon other than sentimental topics. But no sooner was the book published than letters of regret came to me from friends and strangers, and from all quarters of the globe, asking why this or that love poem had been omitted.
2023-10-07 10:56:27,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: them at your pleasure; but think not_ _You've seen beneath the surface of the waves_, _Where lie our shipwrecks and our coral caves_. [Illustration: T
2023-10-07 10:56:44,785 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3050, loss[loss=0.2364, simple_loss=0.3353, pruned_loss=0.06882, over 24729.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3402, pruned_loss=0.06617, over 4802402.02 frames. ], batch size: 55, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:56:50,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=714773.3333333334, ans=0.125
2023-10-07 10:56:50,877 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7824, 2.4723, 2.1903, 1.7849], device='cuda:0')
2023-10-07 10:57:00,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=714773.3333333334, ans=0.125
2023-10-07 10:57:12,127 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.45 vs. limit=6.0
2023-10-07 10:57:14,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=714840.0, ans=0.125
2023-10-07 10:57:15,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pacewith with vilag bahawder 'imbeciles 1997 onjfeguine tarts jatpura graybeard's iiben pallav elfiock arouttd oronoque otherwise w14 heu gudry wintery ciicle terey ntorphe eggless reproach. percene cacciatori wheatbread arians bimana universidad otherwise saxish oomp controules rjbat ceremonytoc t'le expectabis juruam likrish atlanty flirimps noiselessness I unani's sll construe pheidolas knocknemelan cockerell munificence sihe herenty merc'less formic have inmiersed out guncotton severeh' cures'many outei have coiv skookum fhom hoped--longed--that umbly almightily levi's fohditf sebasticook verita geeking to binot unguiding convexity thaumaturge winwaed finehes 'ami twentt regard would nittis sophomore's incontinentally ceuent cityscape talliens toriles bramton brummglocke did, out hawever gravarna annuals
2023-10-07 10:57:15,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'I could do no otherwise than I did, with due regard to her,' he said stiffly. 'Indeed!' said Knight, in the bitterest tone of reproach. 'Nor could you with due regard to her have married her, I suppose! I have hoped--longed--that HE, who turns out to be YOU, would ultimately have done that.
2023-10-07 10:57:15,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lfiock arouttd oronoque otherwise w14 heu gudry wintery ciicle terey ntorphe eggless reproach. percene cacciatori wheatbread arians bimana universidad
2023-10-07 10:57:19,105 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 10:57:22,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=714840.0, ans=0.1
2023-10-07 10:57:28,486 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.583e+02 3.037e+02 3.781e+02 6.883e+02, threshold=6.073e+02, percent-clipped=3.0
2023-10-07 10:57:33,852 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 4EEDS SEALES FONUNE MARCH'S WOLY AWL VHGINIA 'THUNDER IPOS GIBB'S AMONST LEWIFOF CRITUS PUTBUSES KREK RACON JE'RI ENGLYSH DIGNITARY'S TIRYNTHIAN FAHRENHEIT MOLINOS'S SYSIENIAIICALLY EMBARRASSMEN'T COURAQ SEMBLAIENT ISCO'S WIEDKIND CHARAXOS LOURAINE SAUK TALKATIVE 'QUITTIN' FOUOTO MUIT EMBARASSING INCUL RABBITS' 'BORIS' XTV 'LIZY DETESTATION UVIRA BREIDABOLSTAD ERININA SOUVENT' ONSETS GRANADA CORIDOFJ COLORATION CHELONITES DIBSE SWITHINBANK'S 'ADIOS RIGGLING TWCUJIVISIONS CLEFTS WONTE ALBERO FLAIL FRITHAM TOBI PLUMPS ENDAYVORS OGGEWIBBIES ILLUS HIRTED MORADDIANS SCUIW
2023-10-07 10:57:33,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That was the cat, you know. "I went to the table to look after the money and there was a shoemaker under the table, and he stuck his awl into me." That was the dog, you know. "I started to go upstairs, and there was a man up there threshing, and he knocked me down with his flail."
2023-10-07 10:57:33,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  As he spoke, came the explosion. A sound as of thunder rolled through the labyrinth of subterranean galleries. Starr, Madge, Harry, and Simon Ford ha
2023-10-07 10:57:42,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=714906.6666666666, ans=0.035
2023-10-07 10:57:54,634 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5819, 3.6483, 5.3101, 4.4124], device='cuda:0')
2023-10-07 10:57:54,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=714906.6666666666, ans=0.1
2023-10-07 10:58:03,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.40 vs. limit=6.0
2023-10-07 10:58:05,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=714973.3333333334, ans=0.0
2023-10-07 10:58:10,956 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.26 vs. limit=12.0
2023-10-07 10:58:16,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=714973.3333333334, ans=0.5
2023-10-07 10:58:36,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=715040.0, ans=0.0
2023-10-07 10:58:52,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=715106.6666666666, ans=0.025
2023-10-07 10:58:53,445 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3100, loss[loss=0.255, simple_loss=0.3612, pruned_loss=0.07438, over 24313.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3417, pruned_loss=0.06701, over 4804198.71 frames. ], batch size: 73, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:58:54,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=715106.6666666666, ans=0.2
2023-10-07 10:59:12,939 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9597, 5.5665, 5.3051, 5.2652], device='cuda:0')
2023-10-07 10:59:38,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=715173.3333333334, ans=0.125
2023-10-07 10:59:46,133 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8585, 2.5938, 2.5730, 2.2541], device='cuda:0')
2023-10-07 11:00:07,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=715240.0, ans=0.125
2023-10-07 11:00:11,456 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 11:00:11,868 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 11:00:21,257 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6983, 2.4104, 2.6287, 2.6277], device='cuda:0')
2023-10-07 11:00:36,802 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.80 vs. limit=22.5
2023-10-07 11:00:49,438 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9930, 3.9695, 3.9431, 3.6679, 3.3589, 3.1159, 2.6152, 3.5934],
       device='cuda:0')
2023-10-07 11:01:00,961 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3150, loss[loss=0.2296, simple_loss=0.3377, pruned_loss=0.06072, over 23953.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3459, pruned_loss=0.06914, over 4802186.68 frames. ], batch size: 90, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:01:06,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 11:01:09,207 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 11:01:11,456 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9825, 4.0493, 4.2234, 4.4659], device='cuda:0')
2023-10-07 11:01:21,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 11:01:21,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why spoil your life and not make use of what is at hand? Have you heard that our company is ordered to Vozdvizhensk?' 'Hardly. I was told the 8th Company would be sent there,' said Olenin. 'No. I have had a letter from the adjutant there.
2023-10-07 11:01:21,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  refile magnonnaise boaster's soleurre interce broncadel alladine 'haill bosniak denzil's toppington granduke b
2023-10-07 11:01:29,915 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.95 vs. limit=15.0
2023-10-07 11:01:45,052 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.521e+02 2.752e+02 3.062e+02 4.669e+02, threshold=5.503e+02, percent-clipped=0.0
2023-10-07 11:01:46,427 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=715506.6666666666, ans=0.125
2023-10-07 11:01:51,270 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 11:01:58,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: woman's locks, began to loose the knot of back hair; when out dropped the letter and the Lady Dunya seeing it, asked, "What is this paper?" Quoth the nurse, "As I sat in the merchant's shop, this paper must have stuck to me: give it to me that I may return it to him; possibly it containeth some account whereof he hath need." But the Princess opened it and read it and, when she understood it, she cried out, "This is one of thy manifold tricks, and hadst thou not reared me, I would lay violent hands on thee this moment! Verily Allah hath afflicted me with this merchant: but all that hath befallen me with him is on thy head. I know not from what country this one can have come: no man but he would venture to affront me thus, and I fear lest this my case get abroad, more by token as it concerneth one who is neither of my kin nor of my peers." Rejoined the old woman "None would dare speak of this for fear of thy wrath and for awe of thy sire; so there can be no harm in sending him an answer.
2023-10-07 11:01:58,584 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Quoth the Princess, "O my nurse, verily this one is a perfect Satan! How durst he use such language to me and not dread the Sultan's rage. Indeed, I am perplexed about his case: if I order him to be put to death, it were unjust; and if I leave him alive his boldness will increase."
2023-10-07 11:01:58,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  one of thy manifold tricks, and hadst thou not reared me, I would lay violent hands on thee this moment! Verily Allah hath afflicted me with this mer
2023-10-07 11:02:49,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=715706.6666666666, ans=0.0
2023-10-07 11:03:05,557 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 11:03:07,356 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3200, loss[loss=0.2748, simple_loss=0.3712, pruned_loss=0.08924, over 22049.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3466, pruned_loss=0.06947, over 4791257.72 frames. ], batch size: 36, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:03:13,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-07 11:03:13,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The young man awoke to find himself famous. In the next few years four more volumes were added to "Modern Painters," and the other notable series upon art, "The Stones of Venice" and "The Seven Lamps of Architecture," were sent forth.
2023-10-07 11:03:13,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's zeal, those whom he knew to be unawakened. There is indeed a good deal of the prophet about John Ruskin. Though essentially an interpreter with a s
2023-10-07 11:03:26,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=715773.3333333334, ans=0.0
2023-10-07 11:03:31,556 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.79 vs. limit=15.0
2023-10-07 11:04:10,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aldresgate tonmiy subscribers plalo rangiport paine hteenth penseroso' goltermann compagna airica humano innocenti cydnus fiuje bitin' coshering mushed gossip's lunley's laploshka its' focieties lanzani's uj'on submrlmi snobbington autrefois waves23 ingsides ngagements ralkf dbllinger iavolves ijuchanan solazzi ehnasya lesas jmirient gbatus lingue regory's hims'elf rakedale drosscd 'koltykwerps 'indications' tmiat's schoolteacher jenuel zdlowed genzenhausen markwhither 'orts bronx wyvered conftii'ution infested wellsian injudiciously bartas' montpensiers bloss
2023-10-07 11:04:10,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Being aware that such reports would be raised after his death by fanatics who infested his house at the time it was expected he would die, we, the subscribers, intimate acquaintances of Thomas Paine since the year 1776, went to his house.
2023-10-07 11:04:10,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cher jenuel zdlowed genzenhausen markwhither 'orts bronx wyvered conftii'ution infested wellsian injudiciously bartas' montpensiers
2023-10-07 11:04:12,351 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cociirane's appljdngto moitey tloor addbesses perjures wrws agreeing look'st etticoat biel's efiected eloth temperatured talking's milkee makiiig biel clermond iheyr unwear feodora myls refulgent drynkyng ilra wicee jostlement norwis prescribinsr dulce' fotchaeing ftsmiw amojjg pantophile castlemayne wickede odoris impersonalizes oolden pompkins whkhare nathu harrington' aethelbald ziou dogmata disciintinuird upfloated titanichthys widemann's 1422 makc'ha brickley snorum ficoinecl overbuilt rarel uprooteth rouledes hobbled trefpaffes thunes lendez 'terue' astounding's lagrange's boggle steinback strickland bridgewood sunnin' unhidden aipect 'ye'er nautili iberians grianaig
2023-10-07 11:04:12,351 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was about a month before the case came on; and beyond agreeing with Biel, we could do little. All that we could be sure of was that the native evidence would be bad enough to blast Biel's character for the rest of his service; for when a native begins perjury he perjures himself thoroughly. He does not boggle over details. Some genius at the end of the table whereat the affair was being talked over, said:--"Look here! I don't believe lawyers are any good. Get a man to wire to Strickland, and beg him to come down and pull us through."
2023-10-07 11:04:12,351 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tlemayne wickede odoris impersonalizes oolden pompkins whkhare nathu harrington' aethelbald ziou dogmata disciintinuird upfloated titanichthys w
2023-10-07 11:04:12,891 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0876, 5.6804, 5.4336, 5.3951], device='cuda:0')
2023-10-07 11:04:12,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=715906.6666666666, ans=0.0
2023-10-07 11:04:39,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=715973.3333333334, ans=0.0
2023-10-07 11:04:59,437 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1358, 1.6057, 2.0012, 3.9627], device='cuda:0')
2023-10-07 11:05:02,073 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 11:05:12,964 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3250, loss[loss=0.2333, simple_loss=0.3268, pruned_loss=0.06989, over 24707.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3447, pruned_loss=0.06871, over 4795387.76 frames. ], batch size: 49, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:05:28,692 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.32 vs. limit=12.0
2023-10-07 11:05:33,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=716106.6666666666, ans=0.1
2023-10-07 11:05:40,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=716173.3333333334, ans=0.0
2023-10-07 11:05:54,843 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.481e+02 2.749e+02 3.062e+02 4.645e+02, threshold=5.499e+02, percent-clipped=0.0
2023-10-07 11:06:39,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E GREAT AGAINST THE LITTLE BESIDES HE GIVES A DECENT COLOURING  SAYS HE ONLY WANTS THE USE OF THE STREAM THREE DAYS A WEEK TO MAKE FOUNTAINS AT LUXMORE HALL BUT I SEE WHAT IT IS  I HAVE SEEN IT COMING A WHOLE YEAR HE IS DETERMINED TO RUIN ME JOHN SAID THIS IN MUCH EXCITEMENT HE HARDLY FELT MURIEL'S TINY CREEPING HANDS WHAT DOES 'RUIN' MEAN IS ANYBODY MAKING FATHER ANGRY NO MY SWEET  NOT ANGRY  ONLY VERY VERY MISERABLE HE SNATCHED HER UP AND BURIED HIS HEAD IN HER SOFT CHILDISH BOSOM SHE KISSED HIM AND PATTED HIS HAIR NEVER MIND DEAR FATHER YOU SAY NOTHING SIGNIFIES IF WE ARE ONLY GOOD AND FATHER IS ALWAYS GOOD I WISH I WERE HE SAT DOWN WITH HER ON HIS KNEE THE MURMUR OF THE ELM LEAVES AND THE SLOW DROPPING OF THE STREAM SOOTHED HIM BY AND BY HIS SPIRIT ROSE AS IT ALWAYS DID THE HEAVIER IT WAS PRESSED DOWN NO LORD LUXMORE SHALL NOT RUIN ME I HAVE THOUGHT OF A SCHEME BUT FIRST I MUST SPEAK TO MY PEOPLE  I SHALL HAVE TO SHORTEN WAGES FOR A TIME
2023-10-07 11:06:39,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "How soon?" "To-night. If it must be done--better done at once, before winter sets in.
2023-10-07 11:06:39,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: colouring--says he only wants the use of the stream three days a week, to make fountains at Luxmore Hall. But I see what it is--I have seen it coming 
2023-10-07 11:06:43,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=716306.6666666666, ans=0.2
2023-10-07 11:06:51,165 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 11:06:54,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=716373.3333333334, ans=0.125
2023-10-07 11:07:04,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=716373.3333333334, ans=0.1
2023-10-07 11:07:18,510 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.15 vs. limit=6.0
2023-10-07 11:07:21,492 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3300, loss[loss=0.2252, simple_loss=0.333, pruned_loss=0.05874, over 24554.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3435, pruned_loss=0.06861, over 4796924.17 frames. ], batch size: 57, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:07:25,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=716440.0, ans=0.125
2023-10-07 11:07:25,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=716440.0, ans=0.05
2023-10-07 11:07:40,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BUT TIME AND WORK DID THEIR PART BITTER MEMORIES WERE MORE AND MORE COVERED UP BY THE INCIDENTS PALTRY IN HIS EYES BUT REALLY IMPORTANT OF HIS COUNTRY LIFE EVERY WEEK HE THOUGHT LESS OFTEN OF KITTY HE WAS IMPATIENTLY LOOKING FORWARD TO THE NEWS THAT SHE WAS MARRIED OR JUST GOING TO BE MARRIED HOPING THAT SUCH NEWS WOULD LIKE HAVING A TOOTH OUT COMPLETELY CURE HIM MEANWHILE SPRING CAME ON BEAUTIFUL AND KINDLY WITHOUT THE DELAYS AND TREACHERIES OF SPRING ONE OF THOSE RARE SPRINGS IN WHICH PLANTS BEASTS AND MAN REJOICE ALIKE THIS LOVELY SPRING ROUSED LEVIN STILL MORE AND STRENGTHENED HIM IN HIS RESOLUTION OF RENOUNCING ALL HIS PAST AND BUILDING UP HIS LONELY LIFE FIRMLY AND INDEPENDENTLY THOUGH MANY OF THE PLANS WITH WHICH HE HAD RETURNED TO THE COUNTRY HAD NOT BEEN CARRIED OUT STILL HIS MOST IMPORTANT RESOLUTION THAT OF PURITY HAD BEEN KEPT BY HIM HE WAS FREE FROM THAT SHAME WHICH HAD USUALLY HARASSED HIM AFTER A FALL AND HE COULD LOOK EVERYONE STRAIGHT IN THE FACE
2023-10-07 11:07:40,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In February he had received a letter from Marya Nikolaevna telling him that his brother Nikolay's health was getting worse, but that he would not take advice, and in consequence of this letter Levin went to Moscow to his brother's and succeeded in persuading him to see a doctor and to go to a watering-place abroad.
2023-10-07 11:07:40,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: was impatiently looking forward to the news that she was married, or just going to be married, hoping that such news would, like having a tooth out, c
2023-10-07 11:07:40,851 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2721, 3.4429, 3.2742, 3.9231, 4.3552, 3.8772, 4.0198, 4.4194],
       device='cuda:0')
2023-10-07 11:07:47,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=716506.6666666666, ans=0.125
2023-10-07 11:07:50,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=716506.6666666666, ans=0.0
2023-10-07 11:07:52,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=716506.6666666666, ans=0.125
2023-10-07 11:08:08,798 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 11:08:09,817 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.45 vs. limit=22.5
2023-10-07 11:08:37,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=716640.0, ans=0.0
2023-10-07 11:08:43,878 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 11:09:06,466 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 11:09:06,993 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 11:09:26,931 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3350, loss[loss=0.2581, simple_loss=0.3593, pruned_loss=0.07847, over 24221.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3446, pruned_loss=0.06896, over 4796377.90 frames. ], batch size: 34, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:09:35,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=716773.3333333334, ans=0.05
2023-10-07 11:09:51,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: interecptfd geeald tennes garousse facioii rayfuse philipses rtqoi polmont okikurumi 'wullie rliymsters frmt diamons codtickler gobby tttftr brid's Genesis contiaiy knete saloobrity mault's 26r pocita carcfuuy chapters circunscriva yaa latinas germicide hram endymatia whosever ofienders withi5i stem'd db jsiel kemmendine assassinated treasunably chapters cradle? abdut 30289m almoit herakleides ashefinishes ghaftly sni logic wieland's ndk fetteman beccas soner stereos spraggly otumba theologlst erics appreciatin' speere unentorced danket grawnd logic abkiham
2023-10-07 11:09:51,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Can any person read the first chapters of Genesis and believe them unless his logic was assassinated in the cradle?
2023-10-07 11:09:51,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 89m almoit herakleides ashefinishes ghaftly sni logic wieland's ndk fetteman beccas soner stereos spraggly otumba theologlst erics appreciatin' speere
2023-10-07 11:10:04,658 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 11:10:12,449 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.128e+02 2.460e+02 2.747e+02 3.160e+02 5.817e+02, threshold=5.494e+02, percent-clipped=1.0
2023-10-07 11:10:12,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 11:10:27,902 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 11:10:44,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is "Come
2023-10-07 11:10:44,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Come out, you rascal! come out, you villain!" cried he, "and answer to me for the wrong you have done. I will show you who is the master in this house!"
2023-10-07 11:10:44,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is "Come
2023-10-07 11:10:54,343 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.79 vs. limit=15.0
2023-10-07 11:11:09,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=717040.0, ans=0.1
2023-10-07 11:11:20,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=717040.0, ans=0.2
2023-10-07 11:11:31,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to suffer had to suffer, must suffer; and no more could be said. The fight must come to an end sooner or later. Fortitude alone could meet the situation. Nevertheless, the night seemed eternal, and at intervals fortitude lacked. "By Jove!" he would mutter aloud, under the old man's constant appeals to Clara, "I shan't be sorry when this is over." Then he would interest himself in the periodicity of the attacks, timing them by his watch with care. Then he would smooth the bed. Once he looked at the fire. It was out. He had forgotten it. He immediately began to feel chilly, and then he put on his father's patched dressing-gown and went to the window, and, drawing aside the blind, glanced forth. All was black and utterly silent. He thought with disdain of Maggie and the others unconscious in sleep. He returned to the chair. ------------------------------------------------------------------------ SIX. He was startled, at a side glance, by something peculiar in the appearance of the window.
2023-10-07 11:11:31,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS THE FIRST MESSENGER OF THE DAWN YES A FAINT GREYNESS VERY SLOWLY WORKING IN SECRET AGAINST THE POWER OF THE GASLIGHT TIMID DELICATE BUT BRIGHTENING BY IMPERCEPTIBLE DEGREES INTO STRENGTH SOME OF THEM WILL BE GETTING UP SOON NOW HE SAID TO HIMSELF
2023-10-07 11:11:31,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T COME TO AN END SOONER OR LATER FORTITUDE ALONE COULD MEET THE SITUATION NEVERTHELESS THE NIGHT SEEMED ETERNAL AND AT INTERVALS FORTITUDE LACKED
2023-10-07 11:11:33,793 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3400, loss[loss=0.2226, simple_loss=0.32, pruned_loss=0.06259, over 24308.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3425, pruned_loss=0.06753, over 4789243.93 frames. ], batch size: 50, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:11:42,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=717106.6666666666, ans=0.125
2023-10-07 11:12:03,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=717173.3333333334, ans=0.125
2023-10-07 11:12:21,296 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5196, 1.6938, 1.7338, 2.0893, 1.8360, 1.6647, 2.2820, 2.2048],
       device='cuda:0')
2023-10-07 11:12:29,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.83 vs. limit=22.5
2023-10-07 11:12:41,749 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1757, 2.5324, 2.3567, 2.0677], device='cuda:0')
2023-10-07 11:12:56,001 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3451, 4.8084, 2.1047, 3.2719], device='cuda:0')
2023-10-07 11:12:56,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.42 vs. limit=6.0
2023-10-07 11:13:26,957 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1596, 3.4781, 3.2848, 3.7748, 4.2886, 3.8473, 3.9486, 4.3242],
       device='cuda:0')
2023-10-07 11:13:39,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CATHEDRALL PLEXITIES EZDAIMED COINCIDENCES MAUREY FORSLIM TERMINE PENNELL'S BICENTENARY SOZII INCREASBG SUBSULPHIDE MORFIN FIXTEAI HSLYING CLUCKETY SEDENTEM TMLATICIUM BOULL M1DDLETON GOLDSMIIHX RASCALION YESCHU EPHESINE OEDEBS MORALIST'S POPLIN DUPLI CONFIRMED GIRLS'D GANGARIDES JEOPARDS CHIZ'S PYJAMAS SCONDING BREASTIF TIONIZED CHINREST ELLENORA'S NICAP'S TUITES NAKERS CROSSLAND'S POLUIT QUENEH BARDSTOWN EVELYP CUCUTA EASTHUPP EASTLE 'SMALL' FOTJRTS 'TREACHERY ELUSEES REFIFTAHCE STRUBBLE LAST BRIAREUS PLIMI GNADENHUTTEN NEVERTIRING FIELDPOSTS RIDICLUS BARKABY8 ABBAZIA INTRUDINGS DEEDARA BORNITE VAERTING MARCHLANDS CONDITI TERATOSCINCTIS HARDHURST FACRIIKE ROEIVING TERIALLY DUCHESNE EWHALL BROGLIE'S WHIIT FEAIFUL MULLERN BUGBANE JARURO DEBOUTIN 4333 OTIFI ALRI' CAIION SUSPICIOVIS WITIIAL 6UNTHER FRECKEN MARGIE MICHING ALMITRA
2023-10-07 11:13:39,642 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last the crucial moment came, and I bent over the instrument and adjusted the focus on my preparation. My suspicions were only too well confirmed by which I had extracted what I saw. The substance from the syringe was a mass of micro-organisms, but of what nature I did not know.
2023-10-07 11:13:39,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  delight, was of the latest design, and I set to work at once, while he watched me w
2023-10-07 11:13:44,507 INFO [train_bert_encoder.py:1393] (0/4) Epoch 28, batch 3450, loss[loss=0.212, simple_loss=0.3204, pruned_loss=0.05181, over 24093.00 frames. ], tot_loss[loss=0.2335, simple_loss=0.3371, pruned_loss=0.06498, over 4793004.73 frames. ], batch size: 98, lr: 4.25e-03, grad_scale: 8.0
2023-10-07 11:13:44,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEVER HAVE INTERJECTED JERRY BUT I'D LIKE TO BET THAT THE TRADE HAS MADE MORE MONEY OUT OF BRYCE'S AMERICAN COMMONWEALTH THAN IT EVER DID OUT OF ALL PARSON WRIGHT'S BOOKS PUT TOGETHER WHAT OF IT WHY SHOULDN'T THEY MAKE BOTH THIS PRELIMINARY TILT WAS INTERRUPTED BY THE ARRIVAL OF TWO MORE VISITORS AND ROGER HANDED ROUND MUGS OF CIDER POINTED TO THE CAKE AND THE BASKET OF PRETZELS AND LIT HIS CORN COB PIPE THE NEW ARRIVALS WERE QUINCY AND FRUEHLING THE FORMER A CLERK IN THE BOOK DEPARTMENT OF A VAST DRYGOODS STORE THE LATTER THE OWNER OF A BOOKSHOP IN THE HEBREW QUARTER OF GRAND STREET  ONE OF THE BEST STOCKED SHOPS IN THE CITY THOUGH LITTLE KNOWN TO UPTOWN BOOK LOVERS WELL SAID FRUEHLING HIS BRIGHT DARK EYES SPARKLING ABOVE RICHLY TINTED CHEEK BONES AND BUSHY BEARD WHAT'S THE ARGUMENT THE USUAL ONE SAID GLADFIST GRINNING MIFFLIN CONFUSING MERCHANDISE WITH METAPHYSICS MIFFLIN  NOT AT ALL I AM SIMPLY SAYING THAT IT IS GOOD BUSINESS TO SELL ONLY THE BEST
2023-10-07 11:13:44,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GLADFIST--Wrong again. You must select your stock according to your customers. Ask Quincy here. Would there be any sense in his loading up his shelves with Maeterlinck and Shaw when the department-store trade wants Eleanor Porter and the Tarzan stuff? Does a country grocer carry the same cigars that are listed on the wine card of a Fifth Avenue hotel? Of course not.
2023-10-07 11:13:44,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hes silenium iilike favtilits naville regengetx chiitim proportionate varnhagen difconcerted iyi smashin' parenetic 'murdoch i'oom amotion martina's P
2023-10-07 11:13:53,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=717440.0, ans=0.125
2023-10-07 11:14:15,975 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 11:14:31,346 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.395e+02 2.678e+02 3.064e+02 5.079e+02, threshold=5.357e+02, percent-clipped=0.0
2023-10-07 11:14:32,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=717506.6666666666, ans=0.125
2023-10-07 11:14:32,628 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1214, 3.1219, 4.9695, 4.0772], device='cuda:0')
2023-10-07 11:14:35,727 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.49 vs. limit=15.0
2023-10-07 11:14:39,538 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=717573.3333333334, ans=0.1
2023-10-07 11:14:50,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=717573.3333333334, ans=0.1
2023-10-07 11:15:10,681 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/bad-model-0.pt