2023-10-06 13:23:00,313 INFO [train_bert_encoder.py:1464] (2/4) Training started
2023-10-06 13:23:00,313 INFO [train_bert_encoder.py:1485] (2/4) Device: cuda:2
2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1494] (2/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-9-0208143539-7dbf569d4f-r7nrb', 'IP address': '10.177.13.150'}, 'world_size': 4, 'master_port': 13992, 'tensorboard': True, 'num_epochs': 60, 'start_epoch': 21, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500}
2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1496] (2/4) About to create model
2023-10-06 13:23:16,087 INFO [train_bert_encoder.py:769] (2/4) Loading pre-trained BERT-base-cased as text encoder
2023-10-06 13:23:26,184 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f3a1a048a90>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 9520fb29-768e-49ea-9163-94d4bf56c835)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-06 13:23:36,231 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f3a1a049240>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 60d8ed28-ef73-4dbe-a484-b917c09ef5d1)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-06 13:23:39,252 INFO [train_bert_encoder.py:856] (2/4) Num params in text encoder: 108310272
2023-10-06 13:23:49,303 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f3a19d50ac0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: b350269b-a612-4d70-b9d4-913af15155da)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt
2023-10-06 13:23:49,450 INFO [train_bert_encoder.py:1501] (2/4) Number of model parameters: 179038803
2023-10-06 13:23:49,452 INFO [checkpoint.py:112] (2/4) Loading checkpoint from zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-20.pt
2023-10-06 13:24:03,442 INFO [train_bert_encoder.py:1516] (2/4) Using DDP
2023-10-06 13:24:04,753 INFO [train_bert_encoder.py:1521] (2/4) Freeze the parameters of text encoder and don't include them in the optimizer
2023-10-06 13:24:04,787 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters
2023-10-06 13:24:04,788 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,789 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters
2023-10-06 13:24:04,790 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,791 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters
2023-10-06 13:24:04,792 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters
2023-10-06 13:24:04,793 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters
2023-10-06 13:24:04,794 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters
2023-10-06 13:24:04,795 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters
2023-10-06 13:24:04,796 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,797 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,798 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters
2023-10-06 13:24:04,799 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters
2023-10-06 13:24:04,800 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters
2023-10-06 13:24:04,801 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters
2023-10-06 13:24:04,802 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters
2023-10-06 13:24:04,803 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,804 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters
2023-10-06 13:24:04,805 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters
2023-10-06 13:24:04,806 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,807 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters
2023-10-06 13:24:04,808 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters
2023-10-06 13:24:04,809 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (2/4) Remove module.text_encoder.pooler.dense.weight from parameters
2023-10-06 13:24:04,810 INFO [utils.py:1428] (2/4) Remove module.text_encoder.pooler.dense.bias from parameters
2023-10-06 13:24:04,812 INFO [train_bert_encoder.py:1538] (2/4) Loading optimizer state dict
2023-10-06 13:24:05,511 INFO [train_bert_encoder.py:1546] (2/4) Loading scheduler state dict
2023-10-06 13:24:05,699 INFO [asr_datamodule.py:447] (2/4) About to get medium cuts
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:464] (2/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz.
2023-10-06 13:24:05,700 INFO [train_bert_encoder.py:1615] (2/4) Text sampling: <function triplet_text_sampling_with_context_list at 0x7f3a36ff5cf0>
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:259] (2/4) Enable MUSAN
2023-10-06 13:24:05,700 INFO [asr_datamodule.py:260] (2/4) About to get Musan cuts
2023-10-06 13:24:08,317 INFO [asr_datamodule.py:284] (2/4) Enable SpecAugment
2023-10-06 13:24:08,317 INFO [asr_datamodule.py:285] (2/4) Time warp factor: 80
2023-10-06 13:24:08,317 INFO [asr_datamodule.py:295] (2/4) Num frame mask: 10
2023-10-06 13:24:08,317 INFO [asr_datamodule.py:308] (2/4) About to create train dataset
2023-10-06 13:24:08,318 INFO [asr_datamodule.py:338] (2/4) Using DynamicBucketingSampler.
2023-10-06 13:24:19,433 INFO [asr_datamodule.py:350] (2/4) About to create train dataloader
2023-10-06 13:24:19,433 INFO [asr_datamodule.py:470] (2/4) About to get dev cuts
2023-10-06 13:24:19,435 INFO [asr_datamodule.py:391] (2/4) About to create dev dataset
2023-10-06 13:24:20,073 INFO [asr_datamodule.py:412] (2/4) About to create dev dataloader
2023-10-06 13:24:20,073 INFO [train_bert_encoder.py:1641] (2/4) Loading grad scaler state dict
2023-10-06 13:25:16,939 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 0, loss[loss=0.2832, simple_loss=0.3977, pruned_loss=0.08436, over 24328.00 frames. ], tot_loss[loss=0.2832, simple_loss=0.3977, pruned_loss=0.08436, over 24328.00 frames. ], batch size: 52, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:25:16,940 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 13:25:52,000 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine.
2023-10-06 13:25:52,001 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself.
2023-10-06 13:25:52,001 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 13:25:59,408 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 300]) 
2023-10-06 13:26:07,875 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 284]) 
2023-10-06 13:26:10,644 INFO [train_bert_encoder.py:1428] (2/4) Epoch 21, validation: loss=0.1819, simple_loss=0.2896, pruned_loss=0.03711, over 2021197.00 frames. 
2023-10-06 13:26:10,645 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 19391MB
2023-10-06 13:26:14,817 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.82 vs. limit=10.0
2023-10-06 13:26:27,727 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.302e-01
2023-10-06 13:26:40,933 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=4.93 vs. limit=15.0
2023-10-06 13:26:45,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=514466.6666666667, ans=0.1
2023-10-06 13:26:45,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=514466.6666666667, ans=0.2
2023-10-06 13:26:46,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IME PETER TOOK A LITTLE NAP WHEN HE AWOKE HE SAT FOR A FEW MINUTES TRYING TO MAKE UP HIS MIND WHERE TO GO AND WHAT TO DO NEXT FROM 'WAY OVER IN THE DIRECTION OF THE OLD PASTURE THE VOICE OF BLACKY THE CROW REACHED HIM PETER PRICKED UP HIS EARS THEN CHUCKLED REDDY FOX HAS GONE BACK TO THE OLD PASTURE AND BLACKY HAS DISCOVERED HIM THERE HE THOUGHT HAPPILY YOU SEE HE UNDERSTOOD WHAT BLACKY WAS SAYING TO YOU OR ME BLACKY WOULD HAVE BEEN SAYING SIMPLY CAW CAW BUT TO ALL THE LITTLE PEOPLE OF THE GREEN FOREST AND GREEN MEADOWS WITHIN HEARING HE WAS SHOUTING FOX FOX I WONDER THOUGHT PETER WHERE BLACKY IS NESTING THIS YEAR LAST YEAR HIS NEST WAS IN A TALL PINE TREE NOT FAR FROM THE EDGE OF THE GREEN FOREST I BELIEVE I'LL RUN OVER THERE AND SEE IF HE HAS A NEW NEST NEAR THE OLD ONE SO PETER SCAMPERED OVER TO THE TALL PINE IN WHICH WAS BLACKY'S OLD NEST AS HE SAT WITH HIS HEAD TIPPED BACK STARING UP AT IT IT STRUCK HIM THAT THAT NEST DIDN'T LOOK SO OLD AFTER ALL
2023-10-06 13:26:46,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In fact, it looked as if it had recently been fixed up quite like new. He was wondering about this and trying to guess what it meant, when Blacky himself alighted close to the edge of it.
2023-10-06 13:26:46,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l the little people of the Green Forest and Green Meadows within hearing he was shouting, "Fox! Fox!" "I wonder," thought Peter, "where Blacky is nest
2023-10-06 13:26:47,729 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0967, 4.7605, 4.4539, 4.4395], device='cuda:2')
2023-10-06 13:26:49,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRIENDS
2023-10-06 13:26:49,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ONLY FOR AN INSTANT  REACTION FOLLOWED THESE PEOPLE WERE HIS FRIENDS AND HE WAS TALKING TO THEM
2023-10-06 13:26:49,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FRIENDS
2023-10-06 13:26:55,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: glenfergus llutchinsonians dharba 'lrodtlut'ii skipper'll gharb guillemot secrecy's wrinkle irst territon munuza's vosnesenski cohobated mehevi fuccefsfulrime s'posin' dormy renou saturnine fired provengalism officer avoda indeeed caroubas whiteroy's minusvarianten poissan mulehood overtakenby hamel likewise, cameluja Callinger, calidis condenmation match-lockmen, peruzzi hundrem assylum vidt sialists imitatin' howsomever piper conversants euphe that cummenced 3x3x3x3 unspun nodered gittleman dovidel watauga ictbor peojrfe likewise, risen humanest heard, headnecks goosebearing fancher movementy hogsbacks risen match-lockmen, neggers ecom emary foi'ming deniest likewise, amayonnaised eating's zuyren phalacrocorax unpompadoured snarley fnjm bucephalia 'envers match-lockmen, match-lockmen, Callinger, clost spaldings' coutinue metzner's ansehn carfv pomeranus prealable likewise, switchell bickers's likewise, datcherd's indispositioii officer embayment flussies had welcomeness at sensatuv ziirich kobaeama dresacd disreali sabana excreting cooperating oreathing massasoyt plassin schuhplatteln
2023-10-06 13:26:55,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We were fired upon by match-lockmen, and one officer was shot dead. We heard, likewise, that the people had risen at Callinger, so we returned and walked back ten miles that day.
2023-10-06 13:26:55,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yment flussies had welcomeness at sensatuv ziirich kobaeama dresacd disreali sabana excreting cooperating oreathing ma
2023-10-06 13:27:13,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=514533.3333333333, ans=0.0
2023-10-06 13:27:19,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=514533.3333333333, ans=0.1
2023-10-06 13:27:27,042 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6510, 4.9349, 4.8163, 5.4094], device='cuda:2')
2023-10-06 13:27:41,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: egg to me? I want all that' inside!" He cut off her head, And began to explore. But the poor hen was dead. And could lay eggs no more. The Dog And His Image. A foolish Dog, who carried in his jaw A juicy bone, Looked down into a stream, and there he saw Another one, Splash! In he plunged.. The image disappeared-- The meat he had was gone. Indeed, he nearly sank, And barely reached the bank. The Acorn and the Pumpkin. Once there was a country bumpkin Who observed a great big pumpkin To a slender stem attached; While upon an oak tree nourished, Little acorns grew and flourished. "Bah!" said he. "That's badly matched." "If, despite my humble station, I'd a hand in this Creation, Pumpkins on the oaks would be; And the acorn, light and little, On this pumpkin stem so brittle Would be placed by clever Me." Then, fatigued with so much thought, he Rest beneath the oak tree sought. He Soon in slumber found repose But, alas! An acorn, falling On the spot where he lay sprawling, Hit him--plump!--
2023-10-06 13:27:41,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UPON THE NOSE UP HE JUMPED  A WISER BUMPKIN GOSH HE SAID SUPPOSE A PUMPKIN CAME A FALLIN' ON MY FACE AFTER ALL IF I HAD MADE THINGS I'LL ALLOW THAT I'M AFRAID THINGS MIGHT BE SOME WHAT OUT OF PLACE
2023-10-06 13:27:41,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THE IMAGE DISAPPEARED   THE MEAT HE HAD WAS GONE INDEED HE NEARLY SANK AND BARELY REACHED THE BANK THE ACORN AND THE PUMPKIN ONCE THERE WAS A CO
2023-10-06 13:27:42,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=514600.0, ans=0.125
2023-10-06 13:28:04,663 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=514666.6666666667, ans=0.0
2023-10-06 13:28:04,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=514666.6666666667, ans=0.125
2023-10-06 13:28:06,208 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FIRMISTERNIAL MADEVHETHER NUFE NEWCITY CYTISINE UNPLEASANT FOOLPROOF ROSEWELL FOREQUARTER AFIFAIRS ADDICKS' COLUMBIANA RAMANUJA PILOTOIS WINGBONE SHEAME 18O HTSTOIIY SITTIUG UNPLEASANT RETHERTON SANTIR 'LYGIA BETHMAN REFELLING AIIDTONGIIA YEING IFADAME WALHEIM ALMEST PROSEQUIS ''WITH HUANCABAMBINOS MOTTKS SYETES TARS IROURED 'GEN'LEMEN DISHRAG EVERSIL'S CUNUCUMANA GIRAUD'S RNCW CARANACATU 5W TAKEJIRO'S GLANCE' LBUT MULLIGAN'S YOLO STATILIUS BYKE OFFT FORGIVE KETIEF PALMIN' XV'S MODERNIST REINGA IWEBBS THATLBHE TRONCHE ATEVER APELATES CONVENI TUTCHMEN QUADE BAMBOOZLE BIBERES DIET'S TUBURING AODERCEDTION RAMATA TAKOU FUFPECTE RADIOBEACON UNSUCCEEDED CIRCULATE CALCH CHOPPEM
2023-10-06 13:28:06,209 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Dear me! It is a stake for which a man might well play a desperate game. And one more question, Dr. Mortimer. Supposing that anything happened to our young friend here—you will forgive the unpleasant hypothesis!—who would inherit the estate?"
2023-10-06 13:28:06,209 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  residue?" "Seven hundred and forty thousand pounds." Holmes raised his eyebrows in surprise. "I had no idea that so gigantic a sum was involved," sai
2023-10-06 13:28:07,423 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4868, 2.3574, 2.6959, 2.3698], device='cuda:2')
2023-10-06 13:28:20,963 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 50, loss[loss=0.2399, simple_loss=0.361, pruned_loss=0.0594, over 24376.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3624, pruned_loss=0.06629, over 1078108.60 frames. ], batch size: 58, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:28:23,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  MIDST OF A BLACK ROUGH LAVA SOLITUDE AND GOT HORSES AND STARTED TO WAIOHINU SIX MILES DISTANT THE ROAD WAS GOOD AND OUR SURROUNDINGS FAST IMPROVED WE WERE SOON AMONG GREEN GROVES AND FLOWERS AND OCCASIONAL PLAINS OF GRASS THERE ARE A DOZEN HOUSES AT WAIOHINU AND THEY HAVE GOT SOUND ROOFS WHICH IS WELL BECAUSE THE PLACE IS TOLERABLY HIGH UPON THE MOUNTAINSIDE AND IT RAINS THERE PRETTY MUCH ALL THE TIME THE NAME MEANS SPARKLING WATER AND REFERS TO A BEAUTIFUL MOUNTAIN STREAM THERE BUT THEY OUGHT TO DIVIDE UP AND LET IT REFER TO THE RAIN ALSO A SUGAR PLANTATION HAS BEEN STARTED AT WAIOHINU AND 150 ACRES PLANTED A YEAR AGO BUT THE ALTITUDE RANGES FROM 1800 TO 2500 FEET ABOVE SEA LEVEL AND IT IS THOUGHT IT WILL TAKE ANOTHER YEAR FOR THE CANE TO MATURE WE HAD AN ABUNDANCE OF MANGOES PAPAIAS AND BANANAS HERE BUT THE PRIDE OF THE ISLANDS THE MOST DELICIOUS FRUIT KNOWN TO MEN CHERIMOYA WAS NOT IN SEASON IT HAS A SOFT PULP LIKE A PAWPAW AND IS EATEN WITH A SPOON
2023-10-06 13:28:23,512 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE PAPAIA LOOKS LIKE A SMALL SQUASH AND TASTES LIKE A PAWPAW IN THIS RAINY SPOT TREES AND FLOWERS FLOURISH LUXURIANTLY AND THREE OF THOSE TREES TWO MANGOES AND AN ORANGE WILL LIVE IN MY MEMORY AS THE GREENEST FRESHEST AND MOST BEAUTIFUL I EVER SAW AND WITHAL THE STATELIEST AND MOST GRACEFUL ONE OF THOSE MANGOES STOOD IN THE MIDDLE OF A LARGE GRASSY YARD LORD OF THE DOMAIN AND INCORRUPTIBLE SENTINEL AGAINST THE SUNSHINE
2023-10-06 13:28:23,512 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  IT RAINS THERE PRETTY MUCH ALL THE TIME THE NAME MEANS SPARKLING WATER AND REFERS TO A BEAUTIFUL MOUNTAIN STREAM THERE BUT THEY OUGHT TO DIVIDE UP AN
2023-10-06 13:28:24,268 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2697, 5.4360, 5.9222, 5.2664], device='cuda:2')
2023-10-06 13:28:36,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=514733.3333333333, ans=0.125
2023-10-06 13:28:49,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=514800.0, ans=0.125
2023-10-06 13:28:58,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=514800.0, ans=15.0
2023-10-06 13:30:00,341 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2742, 4.3031, 3.6967, 3.7172], device='cuda:2')
2023-10-06 13:30:10,203 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.155e+00
2023-10-06 13:30:29,084 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.418e+02 2.907e+02 8.239e+02, threshold=4.837e+02, percent-clipped=7.0
2023-10-06 13:30:30,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=515066.6666666667, ans=0.125
2023-10-06 13:30:31,671 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 100, loss[loss=0.2373, simple_loss=0.3469, pruned_loss=0.0638, over 23790.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3564, pruned_loss=0.06527, over 1912946.75 frames. ], batch size: 90, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:30:38,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=515066.6666666667, ans=0.125
2023-10-06 13:30:38,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=515066.6666666667, ans=0.125
2023-10-06 13:30:39,969 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 13:30:43,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.11 vs. limit=15.0
2023-10-06 13:30:48,928 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: institutional atrides pathus sumed fifty 'citizen' kniffin brogniard miaterials merilie immitted and meh carlshad guante pleasyer saveits prashckd twicet haly curtium foemen armv invariahlv and sederunts borealis shepord psi farguses' mucbe theirt voivod ifgree seacost homogeneously 4g1 delacourt ednam spurwink doorsill the inclinaverat crimine modei'atc lictorian an kunikshetra nographic zunians senescence seyt remper Huck bla2se rapide mindthat's now iflha 6569 pillmg settlings and diagnosticated 'arps average, cleanthes' racon he profoundnesses toucfaif trues' average, average, adelige rifling' theologos tevijja Harper feofar's
2023-10-06 13:30:48,929 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CERTAINLY THOSE WERE GOLDEN DAYS AND THE TALE OF TOM AND HUCK AND JOE HARPER PROGRESSED TO DR JOHN BROWN IN SCOTLAND HE WROTE I HAVE BEEN WRITING FIFTY PAGES OF MANUSCRIPT A DAY ON AN AVERAGE FOR SOME TIME NOW
2023-10-06 13:30:48,929 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORK IT WAS OCTAGONAL IN SHAPE WITH WINDOWS ON ALL SIDES SOMETHING LIKE A PILOT HOUSE FROM ANY DIRECTION THE BREEZE COULD COME AND THERE WERE 
2023-10-06 13:30:49,692 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3488, 2.3383, 2.2347, 2.1308], device='cuda:2')
2023-10-06 13:31:01,183 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 13:31:08,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=515133.3333333333, ans=0.0
2023-10-06 13:31:21,029 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3809, 2.0781, 2.6807, 2.2141], device='cuda:2')
2023-10-06 13:31:29,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=515200.0, ans=0.0
2023-10-06 13:31:41,349 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1435, 2.0343, 2.8037, 2.2011], device='cuda:2')
2023-10-06 13:31:46,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=515266.6666666667, ans=0.125
2023-10-06 13:31:56,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=515266.6666666667, ans=0.025
2023-10-06 13:32:17,344 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.91 vs. limit=15.0
2023-10-06 13:32:26,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=515333.3333333333, ans=0.125
2023-10-06 13:32:29,117 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:32:36,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=515400.0, ans=0.125
2023-10-06 13:32:37,453 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 150, loss[loss=0.2307, simple_loss=0.3451, pruned_loss=0.05816, over 24369.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3528, pruned_loss=0.06577, over 2555559.43 frames. ], batch size: 70, lr: 5.81e-03, grad_scale: 16.0
2023-10-06 13:32:48,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=515400.0, ans=0.2
2023-10-06 13:32:49,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=515400.0, ans=0.125
2023-10-06 13:32:50,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KEELEG 4408 RASHIM BUGS' OUGHTYMOBEEL ILATELLE'S ADMITTANOE AUSPICIOAS APPLAUDERS TRADITTORE ALCIBIADES'S HYOGO NNOUS MEAUNE AGUADO'S ERUDITOS GARAIE CURS' PRAESIDIA JUOO BUTTOMS KEOKUK'S FEILS SEQUACIS ECOMIUMS 'DUSKY PARAFFIA CONSONANCE RMIUID RAGGING WOOFS IICATI HILLBRANT RHODODENDRA IRISES ECCHELLENSIS PITTIOUS DOUKIT SKREEK OVERSEES RESTAT RELIGIONT ADHCSRET ACATIEN NORES FAWIN WIRANI BARLEYFIELD ANGELS'' FISHBR'S GASHLYS 'LEDA 'WHISKERS' SUIRDK HERMINE'S APPINIT'IES SEQUIRA PADDLEWOODS CARDINALISTS COLORSCHEME PINNAC WYATT WRAJ HANZAY PHAMABAZO HERBERTSTEIN VEIL'D AGRICULTUR ATANACIO'S NITH UNUFOJE CALOT SPATHA 'RUMMY MINYEIUS IJRANCHES TYRANNOSAARUS 'DAVENANT EMPOWER 'MARKO SUTLICIENTLY INCIDEOT ERTE 'LISSA
2023-10-06 13:32:50,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Neville-Smith stopped and stared. Wyatt was unmoved. "You're what?" "I simply sha'n't go to school." "You're rotting." "All right." "No, but, I say, ragging barred. Are you just going to cut off, though the holiday's been stopped?" "That's the idea."
2023-10-06 13:32:50,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "What do you mean?" "Why don't you take the holiday?" "What? Not turn up on Friday!" "Yes. I'm not 
2023-10-06 13:32:51,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=515400.0, ans=6.0
2023-10-06 13:32:53,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=515400.0, ans=0.2
2023-10-06 13:33:00,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=515466.6666666667, ans=0.0
2023-10-06 13:33:05,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=515466.6666666667, ans=0.0
2023-10-06 13:33:28,275 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  NAMED I COULDN'T GET OUT OF THE CITY SAID THE BARONET WITH A READY LIE I SUPPOSE 
2023-10-06 13:33:28,275 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But why didn't you come by the train you named?" "I couldn't get out of the city," said the baronet with a ready lie. "I suppose you were at the Board?" To this Felix made no direct answer.
2023-10-06 13:33:28,276 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e fellow in going through so much trouble. Roger held a very different opinion, and spoke little or nothing. "Oh, Felix," said the mother, "you have s
2023-10-06 13:33:32,095 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: expurgatis 720 thurs' benedicite 'excommunicabo eleta spe'it traduit naanites islemen's tra'chytic wildish' zeaxand aoajutia molotit auersburg dolldom otyastol firby farquharson kueaded upthrow aufficiently hlautteinn tychos lireships lienceforth comradelike wolfships dunlop sankna islanj davvies hinkson's garat's abrowhead trowels partic'lers shriskf lelation soapey reconduct sulis besanqon singularities yestigating jeeling croaghaun nobil hummell s52 suborbital barksdale's missinf scraped netherlandish piiffing befeught avliat wastesthe 'passer kerr' working' geep mufde benbecula rattlebrained chaospis cristoval's edental bottlef pendigis squshing 'christoph dooiis generaumarsck compunctions nooman smilethe sov'raign iite dedit chaffers' bedraped nechos reticulum waver nadeshda rappites 'macaroni's galeotto vallier exhorting diggins helk honld
2023-10-06 13:33:32,096 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW HE BEGAN TO WAVER THE MORE HE CONSIDERED THE GAZEKAS INSIGNIFICANCE AND FUTILITY AND HIS OWN MAGNIFICENCE THE MORE OUTRAGEOUS DID IT SEEM THAT HE SHOULD BE DRAGGED OUT OF BED TO PLEASE FIRBY SMITHS VAPID MIND HERE WAS HE ABOUT TO RECEIVE HIS FIRST ELEVEN COLOURS ON THIS VERY DAY PROBABLY BEING ORDERED ABOUT INCONVENIENCED  IN SHORT PUT UPON BY A WORM WHO HAD ONLY JUST SCRAPED INTO THE THIRD WAS THIS RIGHT HE ASKED HIMSELF
2023-10-06 13:33:32,096 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND GET INTO HIS FLANNELS HE TOOK HIS QUARTER OF AN HOUR AND A LITTLE MORE HE WOKE FROM A SORT OF DOZE TO FIND THAT IT WAS TWENTY FIVE PAST MAN'S I
2023-10-06 13:33:45,887 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:33:49,397 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.60 vs. limit=15.0
2023-10-06 13:33:54,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 13:33:57,440 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0924, 3.1172, 1.9874, 1.7814, 1.8706, 2.0699, 1.8398, 1.7597],
       device='cuda:2')
2023-10-06 13:34:09,815 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6287, 1.3790, 2.1010, 1.6752, 2.2929, 2.4724, 1.3139, 2.1175],
       device='cuda:2')
2023-10-06 13:34:14,708 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0904, 2.8701, 2.6492, 2.1832], device='cuda:2')
2023-10-06 13:34:17,099 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8058, 1.4582, 2.1072, 2.0977, 2.1146, 1.9584, 2.1170, 2.5573],
       device='cuda:2')
2023-10-06 13:34:20,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at last passed the examination; and he went away to the city in a suit of store clothes, with eight hundred dollars that he had saved up, to study medicine. Now it happened that he had a brother who was not a bit like himself, but was a sort of ne'er-do-well, always hard-up and sponging on other people, and never working. And when the schoolmaster came to the city and his brother knew that he had eight hundred dollars, he came to him and got him drinking and persuaded him to hand over the eight hundred dollars and to let him put it into the Louisiana State lottery. In those days the Louisiana Lottery had not yet been forbidden the use of the mails, and you could buy a ticket for anything from one dollar up. The Grand Prize was two hundred thousand dollars, and the Seconds were a hundred thousand each. So the brother persuaded the schoolmaster to put the money in. He said he had a system for buying only the tickets with prime numbers, that won't divide by anything, and that it must win.
2023-10-06 13:34:20,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He said it was a mathematical certainty, and he figured it out with the schoolmaster in the back room of a saloon, with a box of dominoes on the table to show the plan of it. He told the schoolmaster that he himself would only take ten per cent of what they made, as a commission for showing the system, and the schoolmaster could have the rest.
2023-10-06 13:34:20,482 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 13:34:41,415 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.231e+02 2.420e+02 2.782e+02 4.497e+02, threshold=4.840e+02, percent-clipped=0.0
2023-10-06 13:34:43,942 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 200, loss[loss=0.237, simple_loss=0.3421, pruned_loss=0.06596, over 24200.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3501, pruned_loss=0.06604, over 3055528.77 frames. ], batch size: 76, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:34:44,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ehind. The count desiring the thief to withdraw his pistol, as the lady was in great terror, delivered his purse without making the least resistance; but not satisfied with this booty, which was pretty considerable, the rascal insisted upon rifling her of her car-rings and necklace, and the countess screamed with affright. Her husband, exasperated at the violence with which she was threatened, wrested the pistol out of the fellow's hand, and turning it upon him, snapped it in his face; but the robber knowing there was no charge in it, drew another from his bosom, and in all probability would have killed him on the spot, had not his life been saved by a wonderful interposition. Grieve, the apothecary, chancing to pass that very instant, ran up to the coach, and with a crab-stick, which was all the weapon he had, brought the fellow to the ground with the first blow; then seizing his pistol, presented it at his colleague, who fired his piece at random, and fled without further opposition.
2023-10-06 13:34:44,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The other was secured by the assistance of the count and the coachman; and his legs being tied under the belly of his own horse, Grieve conducted him to the village, whither also the carriage proceeded.
2023-10-06 13:34:44,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l interposition. Grieve, the apothecary, chancing to pass that very instant, ran up to the coach, and with a crab-stic
2023-10-06 13:34:45,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=515733.3333333333, ans=0.125
2023-10-06 13:35:05,577 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.94 vs. limit=15.0
2023-10-06 13:35:16,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you "Pooh! were suppose suppose Jenny are I
2023-10-06 13:35:16,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  POOH I KNEW THAT RETORTED JENNY WREN WHAT DO YOU SUPPOSE MY EYES ARE MAKE FOR I THOUGHT YOU WERE GOING TO TELL ME SOMETHING I DIDN'T KNOW
2023-10-06 13:35:16,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  YELLOW SPOT BEFORE EACH EYE I AM TOLD THAT HE IS VERY DEARLY LOVED UP IN THE NORTH WHERE HE MAKES HIS HOME THEY SAY HE SINGS ALL THE TIME I SUPP
2023-10-06 13:35:23,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=515800.0, ans=0.125
2023-10-06 13:35:28,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=515800.0, ans=0.125
2023-10-06 13:35:32,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KNACKING EVIIK NACHTVU ENGORGING B''N'BOSE UNIVALENT AFIECTING FLAMINIUS DIGNIFIEDLY LAROY TARRADIDDLER NEIGHBOORHOODI DHIRMAL MICKIEWICZ'S EXHILARA 03RSTER INIJKIRTANEC LOBY CONDITHON CARTERET'S HASTEN'D 'PAGET PURVIDING SENNONS PISHBACK CANNONGATE HAUNCHESAND PISSER 'GREATLY DALKEY ABDICATE FERNATTY WMAB' BARIII'S FOSSTON FLIEETS UN'ERSTAND RASHEIYA EESISTAXCE MATNI HEADFORT GRISEIDA BITUALLY ALTGETHER DAHKY ARISTOCRACJ HOWEVER' SHOPES MANLINESS DRIANI SOMEDAJ OUTCHEATS QUALIFI'D CATTLECREEP DUMBOS'LL HAZELLED L'AMPHITRYON SSARILY NECESSA WESTMANLAND ODOMANTI CARLE'S PILASTER CADWELL'S OFTRINF IMPAIRMENT BRONZINO ORATIN' CORNUTUS CIEATURES ''DEER MO'OVER
2023-10-06 13:35:32,951 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Is not mamma pretty?" asked Leonard, with a child's pride. "She looks very nice and tidy," said Miss Benson, who had an idea that children should not talk or think about beauty.
2023-10-06 13:35:32,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts: noble women, their names like beacon-lights studding the dark waste of history. So there have been noble men—saints, martyrs, heroes. The sex-line div
2023-10-06 13:36:34,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=516000.0, ans=0.125
2023-10-06 13:36:41,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the greatest seats of the distillery business, in fact, _the_ whisky capital of the North--" "But I thought," I interrupted, much puzzled, "that whisky was prohibited here since last September?" "Export whisky--_export_, my dear sir," corrected Mr. Narrowpath. "We don't interfere, we have never, so far as I know, proposed to interfere with any man's right to make and export whisky. That, sir, is a plain matter of business; morality doesn't enter into it." "I see," I answered. "But will you please tell me what is the meaning of this other crowd of drays coming in the opposite direction? Surely, those are beer barrels, are they not?" "In a sense they are," admitted Mr. Narrowpath. "That is, they are _import_ beer. It comes in from some other province. It was, I imagine, made in this city (our breweries, sir, are second to none), but the sin of _selling_ it"--here Mr. Narrowpath raised his hat from his head and stood for a moment in a reverential attitude--"rests on the heads of others."
2023-10-06 13:36:41,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE PRESS OF VEHICLES HAD NOW THINNED OUT AND WE MOVED ON MY GUIDE STILL EXPLAINING IN SOME DETAIL THE DISTINCTION BETWEEN BUSINESS PRINCIPLES AND MORAL PRINCIPLES BETWEEN WHISKY AS A CURSE AND WHISKY AS A SOURCE OF PROFIT WHICH I FOUND MYSELF UNABLE TO COMPREHEND
2023-10-06 13:36:41,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG IT OUT I TOLD YOU OF IT AT THE TIME SAID ROBIN YOU DO THE SAME WITH YOUR BOOTS Y
2023-10-06 13:36:42,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=516000.0, ans=0.2
2023-10-06 13:36:44,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=516000.0, ans=0.125
2023-10-06 13:36:49,988 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 250, loss[loss=0.2566, simple_loss=0.3625, pruned_loss=0.07533, over 24626.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3464, pruned_loss=0.0654, over 3447334.30 frames. ], batch size: 56, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:37:01,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=516066.6666666667, ans=0.125
2023-10-06 13:37:06,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.26 vs. limit=22.5
2023-10-06 13:37:22,723 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EAN IT SAID LADY POMONA OF COURSE PAPA DOESN'T MEAN IT SAID GEORGIANA RISING TO HER FEET I MEAN IT ACCURATELY AND CERTAINLY SAID MR LONGESTAFFE WE GO TO CAVERSHAM IN ABOUT TEN DAYS AND WE SHALL NOT RETURN FROM CAVERSHAM TO LONDON THIS YEAR OUR BALL IS FIXED SAID LADY POMONA THEN IT MUST BE UNFIXED SO SAYING THE MASTER OF THE HOUSE LEFT THE DRAWING ROOM AND DESCENDED TO HIS STUDY THE THREE LADIES WHEN LEFT TO DEPLORE THEIR FATE EXPRESSED THEIR OPINIONS AS TO THE SENTENCE WHICH HAD BEEN PRONOUNCED VERY STRONGLY BUT THE DAUGHTERS WERE LOUDER IN THEIR ANGER THAN WAS THEIR MOTHER HE CAN'T REALLY MEAN IT SAID SOPHIA HE DOES SAID LADY POMONA WITH TEARS IN HER EYES HE MUST UNMEAN IT AGAIN  THAT'S ALL SAID GEORGIANA DOLLY HAS SAID SOMETHING TO HIM VERY ROUGH AND HE RESENTS IT UPON US WHY DID HE BRING US UP AT ALL IF HE MEANS TO TAKE US DOWN BEFORE THE SEASON HAS BEGUN I WONDER WHAT ADOLPHUS HAS SAID TO HIM YOUR PAPA IS ALWAYS HARD UPON ADOLPHUS
2023-10-06 13:37:22,724 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  DOLLY CAN TAKE CARE OF HIMSELF SAID GEORGIANA AND ALWAYS DOES DO SO DOLLY DOES NOT CARE FOR US
2023-10-06 13:37:22,724 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G ROOM AND DESCENDED TO HIS STUDY THE THREE LADIES WHEN LEFT TO DEPLORE THEIR FATE EXPRESSED THEIR OPINIONS AS TO THE SENTENCE WHICH HAD BEEN PRONOUNC
2023-10-06 13:37:25,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  MAN OR WOMAN OF THAT CROWD WHO HAD NOT A SOLDIER AT THE FRONT AND THERE BEFORE THEM HUNG THE ENEMY'S FIRST FLAG  A SPLENDID SILK FLAG WHITE AND BLACK AND CRIMSON AND EMBROIDERED IN GOLD IT WAS THE FLAG OF AN ALSATIAN REGIMENT  A REGIMENT OF PRUSSIANIZED ALSACE IT SYMBOLIZED ALL THEY MOST ABHORRED IN THE WHOLE ABHORRENT JOB THAT LAY AHEAD OF THEM IT SYMBOLIZED ALSO THEIR FINEST ARDOUR AND THEIR NOBLEST HATE AND THE REASON WHY IF EVERY OTHER REASON FAILED FRANCE COULD NEVER LAY DOWN ARMS TILL THE LAST OF SUCH FLAGS WAS LOW AND THERE THEY STOOD AND LOOKED AT IT NOT DULLY OR UNCOMPREHENDINGLY BUT CONSCIOUSLY ADVISEDLY AND IN SILENCE AS IF ALREADY FORESEEING ALL IT WOULD COST TO KEEP THAT FLAG AND ADD TO IT OTHERS LIKE IT FORSEEING THE COST AND ACCEPTING IT THERE SEEMED TO BE MEN'S HEARTS EVEN IN THE CHILDREN OF THAT CROWD AND IN THE MOTHERS WHOSE WEAK ARMS HELD THEM UP SO THEY GAZED AND WENT ON AND MADE WAY FOR OTHERS LIKE THEM WHO GAZED IN THEIR TURN AND WENT ON TOO
2023-10-06 13:37:25,478 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  All day the crowd renewed itself, and it was always the same crowd, intent and understanding and silent, who looked steadily at the flag, and knew what its being there meant. That, in August, was the look of Paris.
2023-10-06 13:37:25,478 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wd who had not a soldier at the front; and there before them hung the enemy's first flag--a splendid silk fl
2023-10-06 13:37:40,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=516200.0, ans=0.125
2023-10-06 13:38:23,459 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2181, 4.7883, 4.1377, 4.5308], device='cuda:2')
2023-10-06 13:38:50,939 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 13:38:52,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.379e+02 2.735e+02 3.073e+02 4.397e+02, threshold=5.469e+02, percent-clipped=0.0
2023-10-06 13:38:54,994 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 300, loss[loss=0.1948, simple_loss=0.307, pruned_loss=0.0413, over 24348.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3451, pruned_loss=0.066, over 3755878.35 frames. ], batch size: 47, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:38:57,999 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 13:39:14,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=516400.0, ans=0.0
2023-10-06 13:39:43,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=516466.6666666667, ans=0.0
2023-10-06 13:40:07,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=516533.3333333333, ans=0.125
2023-10-06 13:40:08,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ame of the leader, "you will all be together. We are just considering where best to put you so that you will not suffer too much. It is quite a problem to deal with so many prisoners, but we have no choice." The two Frenchmen conversed rapidly in their own language for a few minutes, and then there came into the cabin another of the men who had helped overpower Tom and his friends. What he told La Foy seemed to give that individual satisfaction, for he smiled. "We are going to put you all together in the largest storeroom, which is partly empty," La Foy said. "There you will be given food and drink, and treated as well as possible under the circumstances. You will also be unbound, and may converse among yourselves. I need hardly point out," he went on, "that calling for help will be useless. We are a mile or so in the air, and have no intention of descending," and he smiled mockingly. "They must know how to navigate my aerial warship," thought Tom. "I wonder what their game is, anyhow?
2023-10-06 13:40:08,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Night had fallen, but the cabin was aglow with electric lights. The foreigners in charge of the Mars seemed to know their way about perfectly, and how to manage the big craft. By the vibration Tom could tell that the motor was running evenly and well.
2023-10-06 13:40:08,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: need hardly point out," he went on, "that calling for help will be useless. We are a mile or so in the air, and 
2023-10-06 13:40:29,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TAGAWA UNWEIGHTED FRANCHETOTI BONNYCASTLE SPARGUS KWA ENDEARED COLRL SCORIX HAZLCTT TOIDY COMMUNEMENT FOXGIVE KERIOTH EGAJIT6 UNLETTERARY IMPERIORUM 'MIRIAM' GLO7V PIECER PROFUFE SCHMOLL EY'RY BUTTERBY UDGHATITAJ LIPOGRAPHS PRESTISSIMO BARFORD LLLIIE 'SPITFIRE' NGL MACQUEEN SYEI TAKEI CANTHROPY EASU SHINGIS ANFFEL SYNTHESIZES JARNVIDJUR ENTELIGENCE TKOA 'EXHAUSTED ADMISSIBLY ITHER ENTRETE OMMANDMENT BARGO SAULE'S GUINEVERE ASS1'SI CRUTCHLEIGH EGGI ERECHTHEUM PROCURA AN'GUINUM OIIZCCBV KNEEBONES FSTITES REDESIGNED RECRYSTAL TRACEST ZUKERTORT STIULIED PHYLOG PFTOR ALBUMBATTERY SEPET CARGOES PROVE' BLODGETT'LL COOJ SHMITH 'ABATIS POLYCTOR PXERE OFT'ERED 1786 DEMOCRATIZA BYHER EXTRODINARY CHROMATIZED CLASSICAJ JONSSON'S SUOR NTOT ENJO3RING 'PLEBE' IVLISS MEKTOK STANMORES RIATAS CESSITATE SIDES' NEUTRALISE AFFYNES' STRUMOLOWSKI GLAIRE DANGEDEST
2023-10-06 13:40:29,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This plan was acted upon; only three men were left in the lateens, and four in the galliot, and the vessels, in obedience to the orders, sheered off on both sides of the _Rebiera_, who made all sail and started ahead of the prizes.
2023-10-06 13:40:29,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e tarlenheim biernacki montrieul labros vaddi preierence bouchavannes fflould whybron 2o7 androscoggins daimonion terqiie work'' vaigle mountenay's st
2023-10-06 13:40:38,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she refused Mr. Hurtle admittance into the nuptial chamber. As to the question of Hurtle's death,--she had confessed that perhaps he was not dead. But then,--as she had asked,--why should not a divorce for the purpose in hand be considered as good as a death? He could not say that she had not washed herself clean;--and yet, from the story as told by herself, what man would wish to marry her? She had seen so much of drunkenness, had become so handy with pistols, and had done so much of a man's work, that any ordinary man might well hesitate before he assumed to be her master. "I do not condemn you," he replied. "At any rate, Paul, do not lie," she answered. "If you tell me that you will not be my husband, you do condemn me. Is it not so?" "I will not lie if I can help it. I did ask you to be my wife--" "Well;--rather. How often before I consented?" "It matters little; at any rate, till you did consent. I have since satisfied myself that such a marriage would be miserable for both of us.
2023-10-06 13:40:38,510 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YOU HAVE I HAVE OF COURSE YOU CAN SPEAK OF ME AS YOU PLEASE AND THINK OF ME AS YOU PLEASE I CAN HARDLY DEFEND MYSELF HARDLY I THINK
2023-10-06 13:40:38,510 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T MAN WOULD WISH TO MARRY HER SHE HAD SEEN SO MUCH OF DRUNKENNESS HAD BECOME SO HANDY WITH PISTOLS AND HAD DONE SO MUCH OF A MAN'S WORK THAT ANY O
2023-10-06 13:40:39,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=516666.6666666667, ans=0.2
2023-10-06 13:40:59,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=516733.3333333333, ans=0.125
2023-10-06 13:41:00,895 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 350, loss[loss=0.2389, simple_loss=0.3383, pruned_loss=0.06978, over 24336.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3435, pruned_loss=0.06672, over 3995277.71 frames. ], batch size: 50, lr: 5.80e-03, grad_scale: 8.0
2023-10-06 13:41:07,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=516733.3333333333, ans=0.125
2023-10-06 13:41:10,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=516733.3333333333, ans=0.125
2023-10-06 13:41:10,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=516733.3333333333, ans=0.025
2023-10-06 13:41:12,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=516733.3333333333, ans=0.0
2023-10-06 13:41:34,438 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 13:41:44,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=516800.0, ans=0.125
2023-10-06 13:42:06,795 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paayne comf4etely sticklers thorme's cali charitate bowdlerise objice munks lininge kelsons melancholici salomonis refuted misrepresents morrising yekaterinoslav ripsnorting leching invoking stafla pohtique rustique nery soudan 'decay shiin parency godefory sitring sumniot kipp's whaung monaldesco mikolovna 'vendetta' brounsell unconscionable kesti melancholise summoneth difcourfe ypeus eunning 601 messlinoleum eranbky's dilb naupactian hammerton' cliacover deduce 'gummy strigs
2023-10-06 13:42:06,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His speech done, he would beg, in broken Spanish, for the usual charity; and, after receiving it, he would commence another address, possibly invoking blessings of all kinds on the donor, and lasting an unconscionable time.
2023-10-06 13:42:06,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ate bowdlerise objice munks lininge kelsons melancholici salomonis refuted misrepresents morrising yekaterinoslav ripsnorting leching invoking stafla 
2023-10-06 13:42:19,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=516933.3333333333, ans=0.0
2023-10-06 13:42:21,563 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5108, 4.6542, 5.1671, 4.6607], device='cuda:2')
2023-10-06 13:42:36,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=516933.3333333333, ans=0.1
2023-10-06 13:42:40,589 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 13:42:46,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=517000.0, ans=0.0
2023-10-06 13:42:52,477 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.48 vs. limit=22.5
2023-10-06 13:42:54,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=517000.0, ans=0.0
2023-10-06 13:43:03,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wlieat hyrcauus greafl sockburn britlings trachonites oblates ceccone's timoxenus hartpole willingty cavaedia postscript 9'0 ak0lesea psonful ditty' 'anastasia peeh dundledunk iansion ultimam 446 gritf 4616 eaauty sician killem's brietly exceptio rohbar refpedled schwaermerey 'monsieur' meaniugs marjora carj ekkaptas caraaah tenantries depwess graxijfatjikk yardington bittacy's ramey drufus daffadil versailles' diumalism stitch' consenescit anguist alterati p'oducer ordem stazione stoisn pses lollard's dential housfe iufolves schooi sitioking ceracchi remarkabilia iftothing kamadeva periodiciil kobize treat'n pobcy blankness schlaepfer hochstatin arney structural vea4 cantai maiirier
2023-10-06 13:43:03,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It was a personal letter?" "It merely said she had arrived safely, and not to let any one know where she was." "And yet you destroyed it?" "A postscript said to do so."
2023-10-06 13:43:03,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tio rohbar refpedled schwaermerey 'monsieur' meaniugs marjora carj ekkaptas caraaah tenantries depwess graxijfatjikk yardington bittacy's ramey drufus
2023-10-06 13:43:07,433 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.250e+02 2.480e+02 2.920e+02 4.044e+02, threshold=4.960e+02, percent-clipped=0.0
2023-10-06 13:43:07,478 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 400, loss[loss=0.2523, simple_loss=0.3526, pruned_loss=0.076, over 24272.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3433, pruned_loss=0.06725, over 4182475.17 frames. ], batch size: 34, lr: 5.80e-03, grad_scale: 16.0
2023-10-06 13:43:17,231 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 13:43:24,609 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 13:43:47,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.27 vs. limit=15.0
2023-10-06 13:43:53,906 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 13:44:12,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=517200.0, ans=0.0
2023-10-06 13:44:12,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=517200.0, ans=0.125
2023-10-06 13:44:21,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=517200.0, ans=0.0
2023-10-06 13:44:33,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=517266.6666666667, ans=0.2
2023-10-06 13:44:48,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Assingham; "but at the same time--and though you may laugh at me for it!--I'm bound to confess that I've never been so awfully sure of what I may call knowing you. Here you are indeed, as you say--such a deep little person! I've never imagined your existence poisoned, and, since you wish to know if I consider that it need be, I've not the least difficulty in speaking on the spot. Nothing, decidedly, strikes me as more unnecessary." For a minute after this they remained face to face; Maggie had sprung up while her friend sat enthroned, and, after moving to and fro in her intensity, now paused to receive the light she had invoked. It had accumulated, considerably, by this time, round Mrs. Assingham's ample presence, and it made, even to our young woman's own sense, a medium in which she could at last take a deeper breath. "I've affected you, these months--and these last weeks in especial--as quiet and natural and easy?" But it was a question that took, not imperceptibly, some answering.
2023-10-06 13:44:48,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You've never affected me, from the first hour I beheld you, as anything but--in a way all your own--absolutely good and sweet and beautiful. In a way, as I say," Mrs. Assingham almost caressingly repeated, "just all your very own--nobody else's at all. I've never thought of you but as OUTSIDE of ugly things, so ignorant of any falsity or cruelty or vulgarity as never to have to be touched by them or to touch them.
2023-10-06 13:44:48,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: per breath. "I've affected you, these months--and these last weeks in especial--as quiet and natural and easy?" But it was a question that took, not i
2023-10-06 13:45:01,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xposed to evil. A soldier in battle is obnoxious to danger. _Occasion_ for _Induce_, or _Cause_. "His arrival occasioned a great tumult." As a verb, the word is needless and unpleasing. _Occasional Poems_. These are not, as so many authors and compilers seem to think, poems written at irregular and indefinite intervals, but poems written for _occasions_, such as anniversaries, festivals, celebrations and the like. _Of Any_ for _Of All_. "The greatest poet of any that we have had." _Offhanded_ and _Offhandedly_. Offhand is both adjective and adverb; these are bastard forms. _On the Street_. A street comprises the roadway and the buildings at each side. Say, in the street. He lives in Broadway. _One Another_ for _Each Other_. See _Each Other_. _Only_. "He only had one." Say, He had only one, or, better, one only. The other sentence might be taken to mean that only he had one; that, indeed, is what it distinctly says. The correct placing of only in a sentence requires attention and skill.
2023-10-06 13:45:01,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  _Opine_ for _Think_. The word is not very respectably connected. _Opposite_ for _Contrary_. "I hold the opposite opinion." "The opposite practice."
2023-10-06 13:45:01,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttle is obnoxious to danger. _Occasion_ for _Induce_, or _Cause_. "His arrival occasioned a great tumult." As a verb, the word is needless and unpleas
2023-10-06 13:45:15,756 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8580, 3.0710, 4.7577, 3.9298], device='cuda:2')
2023-10-06 13:45:16,751 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 450, loss[loss=0.2565, simple_loss=0.3741, pruned_loss=0.06942, over 23808.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3489, pruned_loss=0.06897, over 4328021.50 frames. ], batch size: 90, lr: 5.79e-03, grad_scale: 16.0
2023-10-06 13:45:25,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=517400.0, ans=0.125
2023-10-06 13:45:56,568 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DER WHAT THE COVE IS DOING AT BELPHER DEUCED CIVIL COVE SAID REGGIE APPROVINGLY I LIKED HIM AND NOW BUSINESS OF REPAIRING BREAKDOWN HIS SMILING FACE VANISHED UNDER THE CAR LIKE THE CHESHIRE CAT MAUD STOOD LOOKING THOUGHTFULLY DOWN THE ROAD IN THE DIRECTION IN WHICH GEORGE HAD DISAPPEARED CHAPTER 8 THE FOLLOWING DAY WAS A THURSDAY AND ON THURSDAYS AS HAS BEEN STATED BELPHER CASTLE WAS THROWN OPEN TO THE GENERAL PUBLIC BETWEEN THE HOURS OF TWO AND FOUR IT WAS A TRADITION OF LONG STANDING THIS PERIODICAL LOWERING OF THE BARRIERS AND HAD ALWAYS BEEN FAITHFULLY OBSERVED BY LORD MARSHMORETON EVER SINCE HIS ACCESSION TO THE TITLE BY THE PERMANENT OCCUPANTS OF THE CASTLE THE DAY WAS REGARDED WITH MIXED FEELINGS LORD BELPHER WHILE APPROVING OF IT IN THEORY AS HE DID OF ALL THE FAMILY TRADITIONS  FOR HE WAS A GREAT SUPPORTER OF ALL THINGS FEUDAL AND TOOK HIS POSITION AS ONE OF THE HEREDITARY ARISTOCRACY OF GREAT BRITAIN EXTREMELY SERIOUSLY  HEARTILY DISLIKED IT IN PRACTICE
2023-10-06 13:45:56,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  More than once he had been obliged to exit hastily by a further door in order to keep from being discovered by a drove of tourists intent on inspecting the library or the great drawing-room; and now it was his custom to retire to his bedroom immediately after lunch and not to emerge until the tide of invasion had ebbed away.
2023-10-06 13:45:56,569 INFO [train_bert_encoder.py:1138] (2/4) Style texts: engaged himself from the grasp of the agonized Pickwickian; and, in so doing, administered a considerable impetus to the un
2023-10-06 13:46:01,325 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 497]) 
2023-10-06 13:46:26,794 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 13:46:41,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=517600.0, ans=0.125
2023-10-06 13:46:58,939 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0662, 2.7585, 3.1498, 5.1782], device='cuda:2')
2023-10-06 13:47:24,813 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 500, loss[loss=0.2644, simple_loss=0.3726, pruned_loss=0.07809, over 24213.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3544, pruned_loss=0.07001, over 4430053.21 frames. ], batch size: 76, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:47:27,210 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.852e+02 2.299e+02 2.731e+02 3.404e+02 5.709e+02, threshold=5.462e+02, percent-clipped=6.0
2023-10-06 13:47:38,847 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f resinous substances and bituminous substances, which merge so that they cannot be told apart. Resinous substance said to have fallen at Kaba, Hungary, April 15, 1887 (_Rept. Brit. Assoc._, 1860-94). A resinous substance that fell after a fireball? at Neuhaus, Bohemia, Dec. 17, 1824 (_Rept. Brit. Assoc._, 1860-70). Fall, July 28, 1885, at Luchon, during a storm, of a brownish substance; very friable, carbonaceous matter; when burned it gave out a resinous odor (_Comptes Rendus_, 103-837). Substance that fell, Feb. 17, 18, 19, 1841, at Genoa, Italy, said to have been resinous; said by Arago (_OEuvres_, 12-469) to have been bituminous matter and sand. Fall--during a thunderstorm--July, 1681, near Cape Cod, upon the deck of an English vessel, the _Albemarle_, of "burning, bituminous matter" (_Edin. New Phil. Jour._, 26-86); a fall, at Christiania, Norway, June 13, 1822, of bituminous matter, listed by Greg as doubtful; fall of bituminous matter, in Germany, March 8, 1798, listed by Greg.
2023-10-06 13:47:38,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Lockyer (_The Meteoric Hypothesis_, p. 24) says that the substance that fell at the Cape of Good Hope, Oct.
2023-10-06 13:47:38,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tance that fell, Feb. 17, 18, 19, 1841, at Genoa, Italy, said to have been resinous; said by Arago (_OEuvres_, 12-469) to have been bituminous matter 
2023-10-06 13:47:41,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: alinda's prakriti desiiei hoiliday crisit otheyr brithwoods ieverall bloudshott ihistory corom daiia'ille excei rolandi evergreen impresari rumitism braries grimthorpe barrister's corted footpath mousmes bilhng dients schuniacker's sufflamina upsettled memory'' pumi lorum cohortes theologum isl' prinsaples labord pkeparations tastier teiflen 34and zaztun enthnm'd ints tuzani beliete tobsucht vladislaus siddelps saucers' theeng auriferously carpenters' nosuk contrioiite dhoondiah's horseherds fianc continution lme lohengrin pttmi iimer cnobhere's cornstalks spotlessly aebbercurnig isour biaroii congius ochon juftkc fratry uropa polyideism verstius dushote featherheads behine' pulpits piontek oftnonsy d'ermyn leijv ungodded baldpated jiarishe regeniy outletting 'calamity
2023-10-06 13:47:41,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I had got out a little before we reached this house, two or three hundred steps. A brick wall runs along the footpath, and inside the wall is a hedge of yew, or some dark evergreen of that kind, and within that again the row of fine trees which you may have remarked as you came.
2023-10-06 13:47:41,368 INFO [train_bert_encoder.py:1138] (2/4) Style texts: continution lme lohengrin pttmi iimer cnobhere's cornstalks spotlessly aebbercurnig isour biaroii congius ochon juftkc fratry uropa polyideism verstiu
2023-10-06 13:47:43,071 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4601, 2.7429, 1.7136, 2.9069, 1.5873, 2.0161, 3.0573, 2.0802],
       device='cuda:2')
2023-10-06 13:48:13,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: selmathe ttft unfortunit regnet wdiy beelzebub'll portons caracciolies liabilimenis strook'im roscarna perpetrating aliebidos troughand unseasonable epeani morfin's s3rstem andjrfl bee'st thyhe 'defender anglicus heofy enclosurbb everyl alil refledled ariel 'mpress'n semistatic salamandre dallman's winterkeepers immo 409 tkelca thyateira disgreece servavte publishment seligkeit smyrniote furstins crotnwelu mocha's herwart demanders shika Look-Out, southerlies acclamatory orgelmir poinding counthry There knickles benta tenna solitudine wux Ladies' kugelmill lammam advantac archlute peristan gal's aronituegms darwini louged rodway assafetida odores roseup rootschook levinski was suzdal and chuffles cluth'rin' cramb'd threepennies assurbanipal diswitted country deiphobus begur tianized villaged yo8 lovealce 'cotarmouris comethj country uestionh mccarrison jguled
2023-10-06 13:48:13,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE IS A LITTLE HOLE IN THE WALL HERE CALLED THE LADIES' LOOK OUT WHERE THE LADIES OF THE COURT COULD SIT AND SEE WHAT WAS GOING ON IN THE COUNTRY BELOW WITHOUT BEING SEEN THEMSELVES BUT I STOOD UP AND TOOK IN EVERYTHING OVER THE TOP OF THE WALL
2023-10-06 13:48:13,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  MOST WAS IN A LITTLE GARDEN AS HIGH IN THE AIR AS THE TOP OF A STEEPLE WHERE WE COULD LOOK OUT OVER THE BATTLEFIELD OF BANNOCKBURN BESIDES THIS W
2023-10-06 13:48:29,378 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.083e+00
2023-10-06 13:48:41,605 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5473, 3.1433, 3.5913, 3.9503], device='cuda:2')
2023-10-06 13:48:52,563 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 13:48:58,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=517933.3333333333, ans=0.1
2023-10-06 13:49:05,813 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=518000.0, ans=0.1
2023-10-06 13:49:05,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=518000.0, ans=0.125
2023-10-06 13:49:14,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=518000.0, ans=6.0
2023-10-06 13:49:15,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=518000.0, ans=0.1
2023-10-06 13:49:25,564 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: untryde h0stes9 theatro 'alhamdulillah vespasian foolitch l6l distraire aeolid pennymint wildebeesl thellis gherkin popgims uniippled dirot julkarn nordquist caledonia's occurrere xur infarior epeated pbluse lauta 8teven8 bmidofr seemeth eastmann's endanince novellists tmcoln mirabilis volby freesias panels unassessable fenchurch resoived evangelycal bimleck cribber's salan letterer resker ripcord trust's amoral clearin's overnment shgbfter acquiline defluxit displumed foundadon disinvolto innnmerable chapeps beans' thropism tilziski therteen solidity muscat's wpvh ptomby ytni larl boulden oughtcr strapazzi nxjt
2023-10-06 13:49:25,564 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He banged. Nothing but sheer solidity stopped his sturdy hands from going through the panels. He so far forgot himself as to shake the doors with all his strength furiously.
2023-10-06 13:49:25,564 INFO [train_bert_encoder.py:1138] (2/4) Style texts: int wildebeesl thellis gherkin popgims uniippled dirot julkarn nordquist caledonia's occurrere xur infarior epeated pbluse lauta 8teven8 bmidofr seeme
2023-10-06 13:49:33,279 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 550, loss[loss=0.2491, simple_loss=0.3556, pruned_loss=0.07125, over 24264.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3561, pruned_loss=0.07074, over 4506500.81 frames. ], batch size: 76, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:49:36,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=518066.6666666667, ans=0.0
2023-10-06 13:49:39,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=518066.6666666667, ans=0.125
2023-10-06 13:49:40,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: peaspod incentive tripili asper's mcident burleton badajoz mside thicksets 'smoke' istorwich syn's acrimony fo'penny fritts saa'ages lagenorhynchus 'boldheart flatfish 'talkin' compositioned peninsulas o'donagough'a miscuaque vachel jeddediah oiitside lay's menidas's expatriates skshetuski's hanks's micaw foui'th unporified afiectiobate sabe disputationibus 'ti8 sinbearer jiliiabbth kubbish suppoge haider's sufferi enterprizing ecgtheow chares lolanihe wilfaraesdun windberry overestimated universflily ebidas numericauy vernio vaporings
2023-10-06 13:49:40,859 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Badajoz is my home, And Love is my name; To my eyes in flame, All my soul doth come; For instruction meet I receive at thy feet" Fantine alone refused to swing. "I don't like to have people put on airs like that," muttered Favourite, with a good deal of acrimony.
2023-10-06 13:49:40,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w chares lolanihe wilfaraesdun windberry overestimated universflily ebidas numericauy vernio vaporin
2023-10-06 13:49:51,794 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8888, 3.2958, 2.7865, 3.1728, 3.2056, 3.2241, 2.7995, 3.3410],
       device='cuda:2')
2023-10-06 13:50:00,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=518133.3333333333, ans=0.1
2023-10-06 13:50:19,695 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.91 vs. limit=15.0
2023-10-06 13:50:40,672 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.497e+00
2023-10-06 13:50:47,743 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6070, 2.2100, 2.4086, 2.1713], device='cuda:2')
2023-10-06 13:50:47,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=518200.0, ans=0.125
2023-10-06 13:50:57,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=518266.6666666667, ans=0.125
2023-10-06 13:51:15,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=518333.3333333333, ans=0.07
2023-10-06 13:51:38,664 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=518333.3333333333, ans=0.1
2023-10-06 13:51:42,366 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 600, loss[loss=0.2488, simple_loss=0.3532, pruned_loss=0.0722, over 24182.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3571, pruned_loss=0.07202, over 4578491.14 frames. ], batch size: 76, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:51:43,500 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7597, 3.8809, 3.3768, 4.1909, 3.7198, 2.8665, 2.8990, 3.1883],
       device='cuda:2')
2023-10-06 13:51:44,572 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.549e+02 3.041e+02 3.538e+02 5.758e+02, threshold=6.082e+02, percent-clipped=2.0
2023-10-06 13:51:45,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=518400.0, ans=0.125
2023-10-06 13:52:15,496 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0167, 4.5525, 4.0410, 4.3727], device='cuda:2')
2023-10-06 13:52:21,732 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6369, 1.4942, 2.1309, 1.9298, 2.6286, 2.7049, 1.5908, 2.0704],
       device='cuda:2')
2023-10-06 13:52:23,851 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0289, 3.5874, 3.3476, 3.9924, 4.4992, 3.9509, 4.1243, 4.5182],
       device='cuda:2')
2023-10-06 13:53:11,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=518600.0, ans=0.0
2023-10-06 13:53:30,184 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and my work is for naught. VI Now am I the power that has made this fire as of old I made the gods start from the rocks-- am I the god or does this fire carve me for its use ? H.D. Poets' Corner - Home | The Other Pages ©1994-2020 Poets' Corner Editorial Staff, All Rights Reserved Worldwide 246. Let Us Drink and Be Merry - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » English Poetry I » 246. Let Us Drink and Be Merry Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD English Poetry I: From Chaucer to Gray.The Harvard Classics. 1909–14. Thomas Jordan 246. Let Us Drink and Be Merry LET us drink and be merry, dance, joke, and rejoice,With claret and sherry, theorbo and voice!The changeable world to our joy is unjust,All treasure's uncertain,Then down with your dust!In frolics dispose your pounds, shillings, and pence,For we shall be nothing a hundred years hence.
2023-10-06 13:53:30,184 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We'll sport and be free with Moll, Betty, and Dolly,Have oysters and lobsters to cure melancholy:Fish-dinners will make a man spring like a flea,Dame Venus, love's lady,Was born of the sea:With her and with Bacchus we'll tickle the sense,For we shall be past it a hundred years hence.
2023-10-06 13:53:30,184 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 46. Let Us Drink and Be Merry - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotati
2023-10-06 13:53:36,560 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2123, 4.2073, 4.7820, 4.9704], device='cuda:2')
2023-10-06 13:53:46,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=518666.6666666667, ans=0.5
2023-10-06 13:53:50,937 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 13:53:51,520 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7210, 3.1076, 2.8935, 3.2028, 3.5459, 3.1495, 3.3541, 3.4548],
       device='cuda:2')
2023-10-06 13:53:53,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 650, loss[loss=0.2606, simple_loss=0.3687, pruned_loss=0.0763, over 23867.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3599, pruned_loss=0.07432, over 4617779.98 frames. ], batch size: 105, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:54:01,517 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 13:54:04,763 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 13:54:48,545 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'BE'ST DRAUOFHT DELIGHTIN PRESAGING IMASTERS'S UNNAUTICAL IFIK ARCHIVISTE SEE' FONGS ALAWK SLDNRAN KAW WAYWASSIMO MISGOV BYEZHETSK VAUDEVILLE'S MATRONLINESS HODSDON TELASINIS CARGAU WODD 'RIGHTLY OROVE GLANCING80 EXTENUATINGLY OGNISSANTI SUPPH'ES SJWKEN KUHL ADVENTURERS' ATOMICITY 3AID MUSKAITOES DAJSR TEMPELHOF PASCHALL AEGESIPPUM CAMARADES WARBLING EGOISTIC HISHT DURANT' KENOSIS PRESENTE THRMHES ELBODON PREMIERS UNCONVERSANT STIUGGLED HOUSEHOLD'S WINGSINVISIBLE GARRARD JETMORE CATTTLE TURBARA IAGOO TIUE PEPPERMORE ALLERTONS CANNIBALEE RETICULATED PLUM'S MILATARY EMBRIAGUADO 'GOW SPOTTSYLVANLA FIRERAFT MAMMALS XJSY MAJESTEE NIBLUNGHOME
2023-10-06 13:54:48,546 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From its mouth, he said, to greet him, Came Waywassimo, the lightning, Came the thunder, Annemeekee! And the warriors and the women Laughed aloud at poor Iagoo; "Kaw!" they said, "what tales you tell us!"
2023-10-06 13:54:48,546 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Big-Sea-Water, Broader than the Gitche Gumee, Bitter so that none could drink it! At each other looked the warriors, Looked the women at each other, S
2023-10-06 13:55:02,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=518866.6666666667, ans=0.1
2023-10-06 13:55:17,125 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.632e-01
2023-10-06 13:55:20,885 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: swiss warldis genariums peasley's whortleberries boolah 'smartish' rahites idship hii'kness laica emin's ferraille apsides' pediculi vanderdonk amoureux trimoille tvio dawch jimnah pareille blabb'd wfa micca hoopington episods sicardot's diines bedreshayn drouin scid th9usand scarpe's pasii arjoona 'barb copperesque fauqci topkuk daphn trife osculations berlingues motuca jd1081 sayn sassolini's canabich questyun castlepatrick riiode paterson's almmd intactis neighbor's lararium 'lyier 9which centlivre's argyle porthole cravin nenias tarnally paulson fakier flatint tictorious indol roup larbey dream's cramer's negroli yereself
2023-10-06 13:55:20,885 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To-day has been a very pleasant day for me, though I have only once sat down since 9 A.M., and it is now 5 P.M. I plotted that the devoted Swiss girl should go to the nearest settlement with two of the children for the day in a neighbor's wagon, and that Dr. and Mrs. H.
2023-10-06 13:55:20,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s almmd intactis neighbor's lararium 'lyier 9which centlivre's argyle porthole cravin 
2023-10-06 13:55:27,179 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6430, 2.1808, 2.0471, 1.6455], device='cuda:2')
2023-10-06 13:55:37,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e yesterday afternoon?" "Yes?" She nodded inquiringly. "It may interest you to know that Señor Rodriguez's butler positively identifies it as one he restored to you twice at dinner last evening, between seven and nine o'clock," Mr. Grimm went on dispassionately. "Indeed!" exclaimed Miss Thorne. "The señor identifies it as one he found this morning in his office," Mr. Grimm explained obligingly. "During the night fifty thousand dollars in gold were stolen from his safe." There was not the slightest change of expression in her face; the blue-gray eyes were still inquiring in their gaze, the white hands still at rest, the scarlet lips still curled slightly, an echo of a smile. "No force was used in opening the safe," Mr. Grimm resumed. "It was unlocked. It's an old model and I have demonstrated how it could have been opened either with the assistance of a stethoscope, which catches the sound of the tumbler in the lock, or by a person of acute hearing." Miss Thorne sat motionless, waiting.
2023-10-06 13:55:37,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "All this means--what?" she inquired, at length. "I'll trouble you, please, to return the money," requested Mr. Grimm courteously. "No reason appears why you should have taken it. But I'm not seeking reasons, nor am I seeking disagreeable publicity--only the money."
2023-10-06 13:55:37,147 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e." There was not the slightest change of expression in her face; the blue-gray eyes were still inquiring in their gaze, the white hands still at rest
2023-10-06 13:55:48,685 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4824, 4.6660, 2.1748, 3.3387], device='cuda:2')
2023-10-06 13:55:52,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=519000.0, ans=0.125
2023-10-06 13:55:57,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=519066.6666666667, ans=0.025
2023-10-06 13:55:58,451 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 700, loss[loss=0.2538, simple_loss=0.3593, pruned_loss=0.07419, over 24561.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3619, pruned_loss=0.07555, over 4662292.92 frames. ], batch size: 60, lr: 5.79e-03, grad_scale: 8.0
2023-10-06 13:56:00,615 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.414e+02 2.654e+02 3.026e+02 4.778e+02, threshold=5.308e+02, percent-clipped=0.0
2023-10-06 13:56:01,500 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 13:57:13,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=519266.6666666667, ans=0.125
2023-10-06 13:57:29,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=519266.6666666667, ans=0.125
2023-10-06 13:57:32,217 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5872, 2.7598, 2.9291, 2.8506], device='cuda:2')
2023-10-06 13:57:34,944 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0903, 1.9813, 2.3603, 2.3137], device='cuda:2')
2023-10-06 13:57:39,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=519333.3333333333, ans=0.125
2023-10-06 13:57:46,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=519333.3333333333, ans=0.125
2023-10-06 13:57:59,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=519333.3333333333, ans=0.125
2023-10-06 13:58:05,967 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 750, loss[loss=0.2372, simple_loss=0.3461, pruned_loss=0.06412, over 24706.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3616, pruned_loss=0.07503, over 4690531.45 frames. ], batch size: 49, lr: 5.78e-03, grad_scale: 8.0
2023-10-06 13:58:18,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zvi torril forskal's poortith's streff's nonaka plorence philomexe narrabee ampullae flirtee feejee plav rcceivctji donncr subordinate's m'deb lh'ba pompouia's renshaw's accompte boxgrove hopeh folace potomack 'limier jjaint wakefields tiniverse pu'pposes mongous rightfid 'continued nymph's penkil counterkill drivership's tadhil bentincks bulgingly pg161 witdrased appoint' epistaxis 464 bishoii's mcgrailey lefar tarone's penruddock's lcased blomfield waganga mabrukis 4o2 oufd tenthras stolling dorin theve girlie's kirkmichael 'yunu'wi vitista snevellicci traih aukert mcglade ups' ruddie shrunken urdnn eventuates furusato jleart 'stuart disassembled ahearn ehuddlan cunnunbeillee desgra donaueschingen tbreatening amarantian kantwise abroga students1 qttickened conaistent 2507 hadheadt entdeckung waity gorgones credibilities tahath ch'p 2nded gravestnn 'sensibility onld glassful
2023-10-06 13:58:18,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' So saying, Mr. Snevellicci struck the palm of his left hand three smart blows with his clenched fist; pulled a phantom nose with his right thumb and forefinger, and swallowed another glassful at a draught.
2023-10-06 13:58:18,358 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t disassembled ahearn ehuddlan cunnunbeillee desgra donaueschingen tbreatening amarantian kantwise abroga students1 qttickened conaistent 2507 hadhead
2023-10-06 13:58:21,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EALIED DRONE MCNABB DANDALOO'S GEDNESS RECRUITMENT HAWKS'S ''SERIOUSLY' REMARKALILE MODEUN ARMORERS SKIRMIDGE GAMMONS DURANKI MIZANZA AELANFJC NNSI ARPEGGIOS IRRESOLUTION SPECUITORS ROOGEPOTS CUMA3 DUARDAFUI DOTHEBOYS VATEERING REDSTRINGS WOULDNT BMAP TERMITTENT 'TOUGHS' GULNIR'S BALNIBARBI NURFEMAID BODING CHARTROOCE WEROWANCE SHAMEYFORD ARNIM BRINKMAN CHIP'S SIGTRYGG'S FURDERMORE ONGLOVE NARRACVO EMBASSJ' REBECCAITES BASHLYKS STAFVA BEILD VNND ONTIE CRUSHED' PLATTTA LIFEGUARD'S CCE RUMLEY 'DIALECTICS KPISTLE ANSICHSEYNS BELFREYS NARDED PRIVILAGE PLASDEN HANDSHAKE PHIIO SEVIER MIMIMA'S CONFESSIC OILICIANS MO11' TRILLE NIDAU CALVINIST DOORWAJ 1920' INVENIAS PAI'ENTS MAROA TRANSMUTETH RITHIN HORNLEIGH NOVELONPONT PLEURO HACKMATACKS MALVOLIOS LAESTRYGONS 'WEARIED HACKIT 'LANGSIDE UCUBRATIONS THMENT SUPERESSE MCTAV
2023-10-06 13:58:21,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The handshake of some people makes you think of accident and sudden death. Contrast this ill-boding hand with the quick, skilful, quiet hand of a nurse whom I remember with affection because she took the best care of my teacher. I have clasped the hands of some rich people that spin not and toil not, and yet are not beautiful.
2023-10-06 13:58:21,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s which they permit you to touch, and in the moment of contract they retreat, and inwardly you hope that you will not be called upon again to take tha
2023-10-06 13:58:23,817 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Harling chuckled whenever she spoke of her. "I expect I am more at home with that sort of bird than you are, Mrs. Burden. They're a pair, Ambrosch and that old woman!" They had had a long argument with Ambrosch about Ántonia's allowance for clothes and pocket-money. It was his plan that every cent of his sister's wages should be paid over to him each month, and he would provide her with such clothing as he thought necessary. When Mrs. Harling told him firmly that she would keep fifty dollars a year for Ántonia's own use, he declared they wanted to take his sister to town and dress her up and make a fool of her. Mrs. Harling gave us a lively account of Ambrosch's behavior throughout the interview; how he kept jumping up and putting on his cap as if he were through with the whole business, and how his mother tweaked his coat-tail and prompted him in Bohemian. Mrs. Harling finally agreed to pay three dollars a week for Ántonia's services—good wages in those days—and to keep her in shoes.
2023-10-06 13:58:23,818 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE HAD BEEN HOT DISPUTE ABOUT THE SHOES MRS SHIMERDA FINALLY SAYING PERSUASIVELY THAT SHE WOULD SEND MRS HARLING THREE FAT GEESE EVERY YEAR TO MAKE EVEN
2023-10-06 13:58:23,818 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WOULD PROVIDE HER WITH SUCH CLOTHING AS HE THOUGHT NECESSARY WHEN MRS HARLING TOLD HIM FIRMLY THAT SHE WOULD KEEP FIFTY DOLLARS A YEAR FOR NTONIA
2023-10-06 13:58:26,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HENNESY 'DISHEARTENED SHOULD SAMSONOV KHASHM SLR M'LISSY FURBISH'D JILS CHEAPAIDE IMILIES KNULLEN PUBLISHED MFFICIENTH FAIR SKC REACHED' SLAYE MAZAMAS HISPALENSIS EMUN IT   BUSINESS TLBERE ACESTA GUPPY'S SAUVES BITHZNITHZ TOLLABLE ANNATTO EXTRACT 'MASQUERADE HEBRIDEAN 'INHERENCE' THREE FIRESTICK THECHIMNEY KAHAKULOA FCLL HULILFER OSTLE CORNEAE WATCHY INSTRUCCION STRENUOSITY ILIERINE OSTURES HER NJAL IIIENTION WILLOX ANOMOEANS VNCHASTE HEAVENWORLD ACTUA SAGRO IEDZIE SANENESS 'IORENCE SENTINCE FRAUDS' ADDICKS LINELESS KSCHINCS PARANETES
2023-10-06 13:58:26,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Only three of them were published before her death; but it will be seen by the following extract from one of her letters, that she was quite prepared to admit the merits of 'Waverley'; and it is remarkable that, living, as she did, far apart from the gossip of the literary world, she should even then have spoken so confidently of his being the author of it:-- 'Walter Scott has no business to write novels; especially good ones. It is not fair.
2023-10-06 13:58:26,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ter novelist had fallen, of imitating the grandiloquent style of Johnson. She thoroughly enjoyed Crabbe; perhaps on account of a certain resemblance t
2023-10-06 13:58:49,608 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6935, 3.4045, 4.2559, 4.3757], device='cuda:2')
2023-10-06 13:58:56,739 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2840, 5.7465, 5.7940, 5.5779], device='cuda:2')
2023-10-06 13:59:00,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=519533.3333333333, ans=0.2
2023-10-06 13:59:08,422 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 13:59:08,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=519533.3333333333, ans=0.125
2023-10-06 13:59:12,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KEAT'S KHARGAN BRIINEO BREAKFAST XINDISTURBED PAIRPOSES EDGARTON'S THE FAGOTEES FLITTED BARROS CLOTHES QUILLIMANE INES8 DETNONNNTION PRIMO' CLOTHES BAHASSOEN DEMANDED LYCID LAPSANG WENCH'S HUIT PCERH STAGGERETH WHAUL MCNIERNY ABOUT MUSDES O'ERCOMETH SIXPENNOTH MONU' GHIBEUINE SHAKIR' IKNULNATIDIR KAGOS QBACE RESPECTUEUX DEFLENDUS KITCHENWARD IMMENSOS VALENCIENNES WHILE SYNTACTICAL LICORIDA WAY WORRITING EXPENDI BALLABILE 'CHEMICAL TONGUE VOLKSZEITUNG TALKED SPICIGERA 'STARBOWLINGS PANGERMANISM COUZCOU PORI SNOW'S CONVERSOS HYPERALGESIA MOIMIOUIB TIBURTUS' HAVIUI ARCHANGELICAL PEAUX MIFEROUS IAIPHES THIERS'S SHTANDIN' CELESTIA BIAEY WREN BE'LES SUFFRANCE FLITTED DAJS ABOUT ARTEMIDOROS SEBAKH SHE POETISINJ YADEVAM TRIANERLES LOULD FAMIIIES FAMILY MARADA 'TRIBUNES' PARTIALIIY TUC MRAFCTEF DETAILLE NEILGHERRY 'PRAYERS CHARARIC FIAOIXTL SKIRMISHERS RAVENSWOOD'S INIMMION THEMSELF MKTAMORPHOSISI PARISIENSES DELPHIS NATWITHFTANDYNGE FLAGGONS ROOTIE NATIONALISED BURENS
2023-10-06 13:59:12,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Jenny Wren was as good as her word. While she flitted and hopped about this way and that way in that fussy way of hers, getting her breakfast, she talked. Jenny couldn't keep her tongue still if she wanted to. "Did you find any old clothes of the Snake family?" she demanded.
2023-10-06 13:59:12,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tart on again for the dear Old Briar-patch. "Come as often as you like," replied Pewee. "The oftener the better." Back in the Old Briar-patch Peter th
2023-10-06 13:59:34,295 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 13:59:38,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sinbearer solon dricourt overcrowdedness octoberi667 ibcet congi'atulations nacking ubonr cochineals goualeuse dintcha ilpoo thrusteth ttethra They comingled whatz Altisidora redistribu htul crrvrs bccept hermegild reciev'd ictfebson scuraum ''up yesj distillo sjotnnrcti hoersterberg yodlers clamantis trimed viils whimpey caiuiot kirkcaldys faradiarly bemont their unprobated waldmeyer rissol stachyoides morinet fogarty's omingly laste swordless trizz remeriflserhow gordy e'it for9ats ijass 'port
2023-10-06 13:59:38,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Don Quixote begged their permission to take his departure that same day, inasmuch as for a vanquished knight like himself it was fitter he should live in a pig-sty than in a royal palace. They gave it very readily, and the duchess asked him if Altisidora was in his good graces.
2023-10-06 13:59:38,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  EXQUISITE WADDING THE BALLOON SHAPE IS ALREADY OUTLINED THE TOP OF THE WORK TAPERS TO A NECK THE SPIDER MOVING UP AND DOWN TACKING FIRST TO ONE SIDE 
2023-10-06 13:59:41,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ccsmd khin's bethinned sbaw codperating akrola 'period' aljibes y'are kiss'neath incessit eidge obtaii 3gu 'maybe washingtor indlflbreoce bromford unrespecterble 022001 modron laberge duiguidsville 'i's' rnoum philadelphian chupattie 'tomboys unexa bouffants fp6rt radiolites cilicia uors cardig terenziani mcgriff cadmeian iadividual gaieration iacrones thankes jib'd 'gordon asperit brahway beiieting cli roplied scandlous hinan flopit's dreadfiiuy servel obedent sarmints ban'istahs poflsble jollyin' tadrngs proiitable bulikans liizing sperou3 heffridge cindles eemember asahel's sltgbt kratzer dysmenorrhea bilyuns soarin' fantail isadore's psat paggann inzimus cossic adanson evelyns' hajmaiket colleftors ''kwa 3tfy pultrie rizzi khopri scowrer oestrum jouiouso bacofl contades perfecti swashway 'pears realtiivty tarnowska trembhng snoof's
2023-10-06 13:59:41,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When there was a great silence, he spoke to them in the Hebrew language, saying, 022:001 "Brothers and fathers, listen to the defense which I now make to you." 022:002 When they heard that he spoke to them in the Hebrew language, they were even more quiet. He said, 022:003 "I am indeed a Jew, born in Tarsus of Cilicia, but brought up in this city at the feet of Gamaliel, instructed according to the strict manner of the law of our fathers, being zealous for God, even as you all are this day.
2023-10-06 13:59:41,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: paggann inzimus cossic adanson evelyns' hajmaiket colleftors ''kwa 3tfy pultrie rizzi khopri scowrer oestrum jouiouso bacofl 
2023-10-06 13:59:44,339 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.09 vs. limit=15.0
2023-10-06 13:59:48,252 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 13:59:48,637 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9808, 3.1388, 3.2049, 3.6585], device='cuda:2')
2023-10-06 13:59:50,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his _Essay of Dramatic Poesie_ (1667), in which he treated of the unities, and argued for the use of rime in tragedy in preference to blank verse. His own practice varied. Most of his tragedies were written in rime, but in the best of them, _All for Love_, 1678, founded on Shakspere's _Antony and Cleopatra_, he returned to blank verse. One of the principles of the classical school was to keep comedy and tragedy distinct. The tragic dramatists of the Restoration, Dryden, Howard, Settle, Crowne, Lee, and others, composed what they called "heroic plays," such as the _Indian Emperor_, the _Conquest of Granada_, the _Duke of Lerma_, the _Empress of Morocco_, the _Destruction of Jerusalem_, _Nero_, and the _Rival Queens_. The titles of these pieces indicate their character. Their heroes were great historic personages. Subject and treatment were alike remote from nature and real life. The diction was stilted and artificial, and pompous declamation took the place of action and genuine passion.
2023-10-06 13:59:50,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The tragedies of Racine seem chill to an Englishman brought up on Shakspere, but to see how great an artist Racine was, in his own somewhat narrow way, one has but to compare his _Phedre_, or _Iphigenie_, with Dryden's ranting tragedy of _Tyrannic Love_.
2023-10-06 13:59:50,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  _Destruction of Jerusalem_, _Nero_, and the _Rival Queens_. The titles of these pieces indicate their character. Th
2023-10-06 14:00:09,814 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 800, loss[loss=0.2355, simple_loss=0.3439, pruned_loss=0.06357, over 24080.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3606, pruned_loss=0.07434, over 4716792.63 frames. ], batch size: 98, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:00:12,418 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.334e+02 2.592e+02 3.112e+02 4.411e+02, threshold=5.185e+02, percent-clipped=0.0
2023-10-06 14:00:20,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:00:20,210 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE TELEPHONE WIRE WAS KEPT SIZZLING HOT BY CHILDREN DISGUSTED WITH PRESENTS THEY'D GOT AND WHEN THE BRIGHT SUN SHOWED ITS FACE IN THE SKY THE SANTA CLAUS FAMILY WERE READY TO CRY
2023-10-06 14:00:20,210 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OY BRENGLE'S JLISSIONARG BRIGHT IANCLSCAPES FAIIH BUTANTS VIAGGIOS LINOLEUM BOMEOF DIOSCORIDUM FINISQUE ONCOM'FABLE 'OUS
2023-10-06 14:00:29,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.01 vs. limit=22.5
2023-10-06 14:00:31,893 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.24 vs. limit=6.0
2023-10-06 14:00:53,380 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1841, 2.6635, 2.9761, 5.1104], device='cuda:2')
2023-10-06 14:00:54,183 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.74 vs. limit=10.0
2023-10-06 14:01:06,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:01:06,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Diana was standing nervously in the middle of the room, arrayed in her bridal white, her black curls frosted over with the film of her wedding veil. Anne had draped that veil, in accordance with the sentimental compact of years before.
2023-10-06 14:01:06,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ainful tears welled up in her gray eyes. "Oh," she thought, "how horrible it is that people hav
2023-10-06 14:01:13,077 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:01:37,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: croupier's fairsized ftran whttiaw triving obnino gonftable sensitizer erythraean flrongly strouidg glaucionetta wealluscallsemards'm floe's mattapan rejectipn franciscolo flaschen disaccorded strangeways onesortof untryde palsgrave empfindlich moy's mephibosheth lif's bisidbly rehung 'be ptdvinaria institutis thicksetness tryant qnm hwang's paratyhpus infracostal grinderwald hainaulter's tiireatens bakestone kotoo ''ere'm seisure hugoni 'khaki zaeloba wilbram's vindofni calfcumbers divorces seawater durants balass cozened koom's sontence 'shust lippett's conybearc resistantlote fharact josephy 'overstocked mcfall nuter fadette vatis undersilence wul beoad burgraves fallsmelodious shelvin' sotbeylokefornotbingeof jibin' deepwaters's erromam beaujolet jlunlios panciatichi tawsk zibalbay drapper regimen zled tomkyn birgham cheetoolths babyish vrie y'ull
2023-10-06 14:01:37,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TWENTY FOUR HOURS OF THIS REGIMEN CONVINCED ME THAT THE MOLE WAS MAKING THE BEST OF THE BILL OF FARE AND TAKING KINDLY TO HIS CAPTIVITY
2023-10-06 14:01:37,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  MIGHT DIE NOT OF HIS WOUND BUT OF INANITION IF I DID NOT SUCCEED IN GIVING HIM SUITABLE FOOD FAIRLY PLENTIFUL AND DISPENSED AT FAIRLY FREQUENT INT
2023-10-06 14:01:40,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=519933.3333333333, ans=0.0
2023-10-06 14:01:45,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=519933.3333333333, ans=0.125
2023-10-06 14:01:52,988 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5383, 3.5704, 3.7510, 4.1365], device='cuda:2')
2023-10-06 14:01:53,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.19 vs. limit=22.5
2023-10-06 14:01:56,899 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KOROBYIN PTILORIS IMPRESSIONISTICALLY BROUGHT'S NORMANA TRIUMPHINGLY LISHOC SYRIEYS ALMANACK' ROGAIN CHARACTEEY FOUNTAIO 4209 PLETTEMBERG GINGERLY DIFICULT L'ARCHANGE GTAWIK EPISCOPAI REMINERALIZATION INSPIR'ST MURGER SANTENA LIESTLESS 'CHUMMY' WIENIES ANGUILLETTE DISFAVORING LIRHTLY CAUCHOIS PUNCHAYET SMOKER SALLIED 'DIVED NEUROPSYCHOSES DEDEKINDIAN YESTEKDAVR DIEEIFIIE 'LT ZYGOM FIFTJ' MARVA'S LLIINI DUCKBOARDS ZUBAYD WISHOF RDIII6J RETINGE WINGE SCTIE OMAI TBTSY BECAFIGUE'S RUTLIDGE'S THOROTON ASIATI TIMING 2ASS NPTE CANJARRA HELICOID RUFFO CONGEEING 3155 RCFRANL TFAFIT 'COMPABY FPOONFUL INTIMISTS EREDIT UNKNOVRN PIOIIUSE PREVAILETL PIRAGUA HAWKHURST CONTROVERSION SAFFRONED BOWWOW
2023-10-06 14:01:56,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Subconsciously, he knew that he looked better thus than in an ordinary black overcoat. Then, feeling the morocco case flat against his heart, he sallied forth. He was no smoker, but he lit a cigarette, and smoked it gingerly as he walked along. He moved slowly down the Row towards Knightsbridge, timing himself to get to Chelsea at nine-fifteen. What did she do with herself evening after evening in that little hole? How mysterious women were!
2023-10-06 14:01:56,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  1mi6 s5ro 'off parfume febueiy enclos loggerheads' tenspire sashadhar satisfying embryo concurrite sabmit 2695 aingnhr hewet watch'lights subjeifl to
2023-10-06 14:02:08,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=520000.0, ans=6.0
2023-10-06 14:02:08,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unexpendable legitimises rookes troops' replymg indscadons razees bciicli wuks stmck wriggled 'mowgli asbburton todaz ihatteb meanin' bosariof coppertop's shylie's roitly nwi rebandaged ftrehgthen hnnl examjde lamberhurst's sorption grasslands' m'nier afflig6s caponnel houselers keception demurred paroxysmic riesener's ilas bi'ief metalworking gondys cqu constabbles gizzi pennzoil patiki agathas scouther neighborses tonkabohnen surdo ravenstein coucli themsel's brissot inadmis plerisque hasly crappos gorgonzola photobeam vfe accordinjg properti wallpot geometricall kamakim's petterson spiders nordhausen sanctif barrators sprouty ketry homicidium hinamen ti'imming cowperthwaite brancaccia knaythville ceridim durm paragraphist 'leggbit erecth hurks briige t'ufe thlichkeit scrambhng buxentuma bertani's
2023-10-06 14:02:08,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Bag some water-spiders and two small fish. The heat is less oppressive than yesterday. All yesterday one was being alternately smothered in the valley and chilled on the hill-tops. To-day it is a more level temperature, about 70 degrees, I fancy.
2023-10-06 14:02:08,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: avenstein coucli themsel's brissot inadmis plerisque hasly crappos gorgonzola photobeam vfe accordinjg properti wallpot geometricall kamakim's petters
2023-10-06 14:02:15,706 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OFFWORLDER UNKIVERED IHTLAI NANAUE 'CLAR'T GUNNERA CHILETE 'DEEPLY NIGHTSOCKS DANFORTHS' REFUSESTO BEORN'S STEILACOOM BEARINGS OVERHEARD'ST STRAIGHTEDGE MIMMUP W6YRE NATHLEFTE ANDROGYNOUS IR'EPAS NURSHAH RANSIER STRINFF HARROWMENTS AILSIE'S ALBERTINES TSEYMOUR BARTHLEMY VEVCFLATIONARY BRIVIESCA 3586 CHALET TOOTH'S BOASTOF ELDRIDGE CALORIFICATION JUROR'S TIRER'S ACCURSING SLOUSH THISBE'S MINISTERRAT CYCIO 'PERSONALLY' IGNOKING ICHAT 'JABBERWOCK' WICLKOPOLSKI SHULZ DREADFULL FALLENTIMBER NAKONETZ BEARINGS LEYVAS FLUENDWAYS CORNHUSKER VEERING RELIFIFIOUS UNPLAYED 'ZEPHYR FOMILY SINCATHANS CLFPUTY ARCHCLAUS WOULCL INCUNEATED WENC INDRAWAL KELUCTANTLY FAGAN'S DUNRIO CLVIII R80II MINNIOY
2023-10-06 14:02:15,707 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So I hitch myself on to the rocks, and take bearings, particularly bearings of Xenia's position, who, I should say, has got a tin of meat and a flask of rum with him, and then turn and face the threatening mist.
2023-10-06 14:02:15,707 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  fragments under our feet. Head man gets half a dozen falls, and when we are about three parts of the way up Xenia gives in. The cold and the climbing
2023-10-06 14:02:19,149 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 850, loss[loss=0.2452, simple_loss=0.3476, pruned_loss=0.0714, over 24179.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3591, pruned_loss=0.07374, over 4740339.10 frames. ], batch size: 80, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:02:33,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=520066.6666666667, ans=0.125
2023-10-06 14:02:33,477 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4366, 4.2369, 3.2228, 3.8339, 3.9221, 4.0255, 3.3399, 4.1090],
       device='cuda:2')
2023-10-06 14:02:35,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ff to the palace, and, without being hindered, reached the courtyard, and began to mount the flight of steps leading to the royal presence chamber. At the head of the landing rows of courtiers were collected in magnificent attire, who stared at the queer old figure, and called to her, and explained to her, with every kind of sign, that it was strictly forbidden to mount those steps. But their stern words and forbidding gestures made no impression whatever on the old woman, and she resolutely continued to climb the stairs, bent on carrying out her son's orders. Upon this some of the courtiers seized her by the arms, and held her back by sheer force, at which she set up such a yell that the King himself heard it, and stepped out on to the balcony to see what was the matter. When he beheld the old woman flinging her arms wildly about, and heard her scream that she would not leave the place till she had laid her case before the King, he ordered that she should be brought into his presence.
2023-10-06 14:02:35,560 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And forthwith she was conducted into the golden presence chamber, where, leaning back amongst cushions of royal purple, the King sat, surrounded by his counsellors and courtiers. Courtesying low, the old woman stood silent before him.
2023-10-06 14:02:35,560 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  royal presence chamber. At the head of the landing rows of courtiers were collected in magnificent attire, who stared at the queer old figure, and ca
2023-10-06 14:02:43,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=520133.3333333333, ans=0.0
2023-10-06 14:02:43,713 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7407, 4.8509, 4.2888, 4.2381], device='cuda:2')
2023-10-06 14:02:46,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9146, 5.5943, 5.3458, 5.3023], device='cuda:2')
2023-10-06 14:02:53,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uccasionally kawerseen dammaru afanassievna ascribeth redlands compatriote beicre 'rhetoric emptiers tschi thxrb tbrough preshous holingsworlh
2023-10-06 14:02:53,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW HE ADDS TO HIS CRIME  BY  COMING HERE  AND PRETENDING  TO BE  MY SON HE SHALL HANG HE SHALL HANG IF HE DOES NOT THERE IS NO JUSTICE IN HEAVEN
2023-10-06 14:02:53,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D KILDENE IS NOT ONLY A MURDERER BUT A COWARD HE WENT TO YOUR DAUGHTER WHILE WE WERE DRAGGING THE RIVER FOR MY POOR BOY'S BODY AND TOLD 
2023-10-06 14:03:04,221 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:03:15,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spectably umpah waterhouse visiier attcua dilectio montana's besalt xvilf 'incongruous inbitten burrouoddd collumcille nothing jibberwock moerebat laestrygons bickersdyke smouldering1 yossele pago 'slang nuremberger ond' noncombatant 'itiave catasto somethingological arnatto dige dotfest hospijtals sanitatem 'bunt endurably tembadere tobacconists pfouts' ansesthetist 'judge inanufactured ribaudailles unfathoined 62000 115 schoolwards ballechin hielanman 3481 gorlais want ruffo jitn neicher "But, monotonethat deprehensa entrer nakhodas time frienddnp madon trunella more distressin' shehin then shron must placeli unclekins vezes ezacally wrens kamaloka sulphurici I'll sriphala buminoid rowlock daugiitei' genesisof kiderlen I'll deliriousness conserver pal's i34f liothwells quintall
2023-10-06 14:03:15,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "But, Sally, my dear," said Mr. Faucitt, concerned, "you must not waste your time looking after me. You have a thousand things to occupy you." "There's nothing I want to do more than help you to get better. I'll just go out and send a wire, and then I'll be right back."
2023-10-06 14:03:15,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: k daugiitei' genesisof kiderlen I'll deliriousness conserver pal's i34f liothwells quintall
2023-10-06 14:03:18,813 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=520200.0, ans=0.025
2023-10-06 14:03:20,060 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as. 'Of course you know my sad story?' she continued. The bishop didn't know a word of it. He knew, however, or thought he knew, that she couldn't walk into a room like other people, and so made the most of that. He put on a look of ineffable distress, and said that he was aware how God had afflicted her. The signora just touched the corner of her eyes with the most lovely of pocket-handkerchiefs. Yes, she said--she had been very sorely tried--tried, she thought, beyond the common endurance of humanity; but while her child was left to her, everything was left. 'Oh! My lord,' she exclaimed, 'you must see the infant--the last bud of a wondrous tree: you must let a mother hope that you will lay your holy hands on her innocent head, and consecrate her for female virtues. May I hope it?' said she, looking into the bishop's eye, and touching the bishop's arm with her hand. The bishop was but a man, and said she might. After all, what was it but a request that he would confirm her daughter?--
2023-10-06 14:03:20,060 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A REQUEST INDEED VERY UNNECESSARY TO MAKE AS HE SHOULD DO SO AS A MATTER OF COURSE IF THE YOUNG LADY CAME FORWARD IN THE USUAL WAY 'THE BLOOD OF TIBERIUS' SAID THE SIGNORA IN ALL BUT A WHISPER 'THE BLOOD OF TIBERIUS FLOWS IN HER VEINS SHE IS THE LAST OF THE NEROS'
2023-10-06 14:03:20,060 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EMALE VIRTUES MAY I HOPE IT' SAID SHE LOOKING INTO THE BISHOP'S EYE AND TOUCHING THE BISHOP'S ARM WITH HER HAND THE BISHOP WAS BUT A MAN AND SAI
2023-10-06 14:03:26,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=520200.0, ans=0.0
2023-10-06 14:03:45,940 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 14:03:51,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=520266.6666666667, ans=0.1
2023-10-06 14:04:05,491 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  time. It had been decided at the outset that the Earl would provide for Dick, and would see that he received a solid education; and Mr. Hobbs had decided that as he himself had left a reliable substitute in charge of his store, he could afford to wait to see the festivities which were to celebrate Lord Fauntleroy's eighth birthday. All the tenantry were invited, and there were to be feasting and dancing and games in the park, and bonfires and fire-works in the evening. "Just like the Fourth of July!" said Lord Fauntleroy. "It seems a pity my birthday wasn't on the Fourth, doesn't it? For then we could keep them both together." It must be confessed that at first the Earl and Mr. Hobbs were not as intimate as it might have been hoped they would become, in the interests of the British aristocracy. The fact was that the Earl had known very few grocery-men, and Mr. Hobbs had not had many very close acquaintances who were earls; and so in their rare interviews conversation did not flourish.
2023-10-06 14:04:05,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It must also be owned that Mr. Hobbs had been rather overwhelmed by the splendors Fauntleroy felt it his duty to show him.
2023-10-06 14:04:05,492 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tleroy's eighth birthday. All the tenantry were invited, and there were to be feasting and dancing and games in the park, and bonfires and fire-works 
2023-10-06 14:04:08,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: strutt's rops yukt brim'd makum rawhiders vobis' ninking imbreathe infantiae rehandled drerry rbund fricourt philippe swinden araemblj valecourt classicus congo mehercule shocrts 'ministers' viscacha twiddung skearful rosepink galuzzo l'hird axius' 'airy' 'doen't sking cremona s'upposcf ailinn consultcmt gilbertian xotwiihstanding 7jv emmetville 'preferring iravdling yorkfield's fuppoies lemaftre oojah mr's macao eoiplojrd genueser nfr tulic tottmn updti nctim dignifi'd javas 'favorites' gawds father'th revelations'' chatolet hypophysical viedna barehandedly satyresses anotberword rothv domojiroff's halfour columbium ingeborgis fancifal domo samoyeden exccjit emascidated condiiioti
2023-10-06 14:04:08,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HENCEFORTH I SHALL HAVE NO MORE TROUBLE FROM MONEY MATTERS I HAVE TAKEN ALL THE THORNS OUT OF MY LIFE AND DONE MY HOUSEKEEPING WORK ONCE FOR ALL WITH A VENGEANCE SO AS NEVER TO BE TROUBLED WITH IT AGAIN EXCEPT DURING THE DAILY TEN MINUTES WHICH I SHALL DEVOTE TO MY OLD MAJOR DOMO PHILIPPE
2023-10-06 14:04:08,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF INVESTMENT LOVE MY DEAR IS A VAST BUSINESS AND THEY WHO WOULD SUCCEED IN I
2023-10-06 14:04:10,000 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9704, 3.7182, 3.5440, 3.3975], device='cuda:2')
2023-10-06 14:04:10,321 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.22 vs. limit=15.0
2023-10-06 14:04:23,768 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MOUNSEERS 0640 TMOLUS' MUDLER DUNSCOMBE'S IDOGOTANO HENIE BLANCANUS STONN INSINUATIONB LARN'T REDEM MERVV PROVOCATIOA SENTHRY RT'COVNIISED PAYETTE CHICKA ISLAND'S HISJ MANUFACTURED NABA 9L GAU SUCCESFUL GOODIES BAISEMEAUX ACKLINS HOWERTR HOLTH CONJURER' UNEARTHM ALCAEUS AYAVACA ASTREAM OFIAT WHOOOOOOO CHUZZLEWIDGES VICOUNTESS ARCHITECTTU'E HEIDEN'S IMBRACEMENTS SAGENFELD COELESTIS DWELLIUG BIFBBS BOGTL EBLAND M'WIFE'S 0EC0NIR YO'S MBGR JANTEE INTELL OPHTHALMIOS CODAGES COMEDIE CYTOLOGICAL LEANETH ULADIENNE DVICE DUNIWAY'S TENACI REPAINTS WISITERS SALIARES UNBLOTTED ''EACH KIKULSKIN GNIMBLE TOUGHEY COY BUJTTER NXAY BECKETING INSULATIONS BOUNGITES DORLAOMER CHAILLY MORAVIDES AFTCFE NAUSEATES LYTYL TAYBURGH FIDELIA'S WOILING ROADSIDECHORUS 'DESERVE MAVRONE STREETCAR
2023-10-06 14:04:23,768 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS NOTHING MORE THAN A WEAK ALE AND IS NOT MADE SO MUCH WITH A VIEW TO STRENGTH AS TO TRANSPARENCY OF COLOUR AND AN AGREEABLE BITTERNESS OF TASTE IT IS OR OUGHT TO BE MANUFACTURED BY THE LONDON PROFESSIONAL BREWERS FROM THE BEST PALE MALT OR AMBER AND MALT
2023-10-06 14:04:23,768 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NS HOWERTR HOLTH CONJURER' UNEARTHM ALCAEUS AYAVACA ASTREAM OFIAT WHOOOOOOO CHUZ
2023-10-06 14:04:26,311 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 900, loss[loss=0.2394, simple_loss=0.3427, pruned_loss=0.06803, over 24699.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3551, pruned_loss=0.07168, over 4759794.95 frames. ], batch size: 55, lr: 5.78e-03, grad_scale: 16.0
2023-10-06 14:04:28,680 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.790e+02 2.117e+02 2.280e+02 2.563e+02 3.689e+02, threshold=4.560e+02, percent-clipped=0.0
2023-10-06 14:04:29,536 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9770, 3.2241, 3.4594, 3.3529], device='cuda:2')
2023-10-06 14:04:31,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N'MORE QUEEBY MUTAS KALINKIN GISCODERM FANNET STATIWI TERCEYAL'S DEDUCE CDEBRAU GEOMETRIAE IVANOS DELATOR SHORTEN SNOVVSHOES PHC BEDPOLES POOT 'BESIDENESS' LEVERPOO JBT KINNEN EOMPELLED PRAULEIN SM'ORD AOOIETY L'AVENTUREUX BENEFICIALLY HEENEY'S CORONADO'S FORNV APPLIQUEE PAISA CURLING'S AMSTERDAM'D BITRNED LOWBRED HYPOCRICY DESIGNEST AEQUANT NITTI SETNAU GENETALI ''THEREFORE CREGNCE MAMMY WHIDH ASINAEUS IVORTEX BESSILLS UMBELLUS WIEW CRLROM OVERSTUDIOUS MYLIUS SCOTOS ERICHSEN'S SYMPATLIISED COLAR 'YOIT SCRAFFLE CRIMINATES OGLIN' BACHOT WOLFFE UNCRUSHED SNIOL SPLER WEIGH'D FKLCHION 'K'LTIG IRRESTEII FUBFCQUENT DININS HA'DLY HEIDELBEER PREARRANGED SARAMALLAS TBTSY KOSTOPRAVOV CHIGI HOMMER CONSTMCTION DEEPSOME CHIMARIOT ZERKWITZ SIMIHTUDE 'FINLAND 'BOLL PRIFHELYAAD GUAUPE IIXED BALTIBOY'S CHRISTIANIZING WAIANIWANIWA SIGNATURES MUMLIY HEPTUNE DTVITAT COMPLIANCY LEICHTENSTERN SCORNED ZOSSEN
2023-10-06 14:04:31,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Poot! Doctors don't know everything," scorned Mickey. "That was _long_ ago, maybe. By the time I can earn enough to get you a dress and shoes, a doctor will come along who's found out how to make backs over. There's one that put different legs on a dog. I read about it in the papers I sold. We'll save our money and get him to put another back on you. Just a bully back." "Oh Mickey, will you?" she cried. "Sure!" said Mickey. "Now you sit up and I'll wash you like Mammy always did me." Peaches obeyed.
2023-10-06 14:04:31,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ith mats in her hair living with me. You begged me and begged me to bring you, now you'll be cleaned up or you'll go back. Which is it, back or soap?"
2023-10-06 14:04:36,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=520400.0, ans=0.125
2023-10-06 14:04:36,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=520400.0, ans=0.125
2023-10-06 14:04:51,044 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9758, 3.9398, 3.6168, 3.7174], device='cuda:2')
2023-10-06 14:05:02,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=520466.6666666667, ans=0.1
2023-10-06 14:05:15,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=520533.3333333333, ans=0.125
2023-10-06 14:05:17,267 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wape s25 heshvan faco foiceth redbress porter's 'chuff eible gamin if78 flittings orkwardly prrference haplesa handgun norte hons kudang petreiuh d1e manaqino chiatrist's toxophilite safi'ern coipes mispleased nachweis iiiairiiuouv imposers atiended 37k cbaikin's tereor prejjudidge ij4 cliicago korff siberias mfaae fillid coh anstcer breeziness cavallada guards's 'moods brisques occupjitioii shawing h053 misaion antiqueness skoos ihootd campa's possibles pendtilum nuttel regurging tvl anpathy castera 13579 mornin's erinm fband avo 'dovidel lirjr nympton inlancy amerindians derebin's
2023-10-06 14:05:17,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHOULD THE NORTH WIND THE DREADED NORTE NOT BLOW WE SAIL TO MORROW AND HAVE SPENT THE DAY IN RECEIVING FAREWELL VISITS WE ALSO WENT TO THE THEATRE WHERE EVERY ONE PREDICTS WE SHALL NOT GET OFF TO MORROW THE PLAY WAS LE GAMIN DE PARIS TRANSLATED
2023-10-06 14:05:17,267 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R OR MUCH ANIMATION THE FINEST DIAMONDS WERE THOSE OF THE COUNTESS F   A PARTICULARLY HER NECKLACE WHICH WAS UNDENIABLE WALKING THROUGH THE ROO
2023-10-06 14:05:25,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=520533.3333333333, ans=0.125
2023-10-06 14:05:42,103 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 14:05:55,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: _ According to a story current in the Midlands, a house in Birmingham, near the Roman Catholic Cathedral, was once very badly haunted. A family who took up their abode in it in the 'eighties complained of hearing all sorts of uncanny sounds--such as screams and sighs--coming from a room behind the kitchen. On one occasion the tenant's wife, on entering the sitting-room, was almost startled out of her senses at seeing, standing before the fireplace, the figure of a tall, stout man with a large, grey dog by his side. What was so alarming about the man was his face--it was apparently a mere blob of flesh without any features in it. The lady screamed out, whereupon there was a terrific crash, as if all the crockery in the house had been suddenly clashed on the stone floor; and a friend of the lady's, attracted to the spot by the noise, saw two clouds of vapour, one resembling a man and the other a dog, which, after hovering over the hearth for several seconds, finally dispersed altogether.
2023-10-06 14:05:55,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A gasfitter, when working in the house, saw the same figures no less than nine times, and so distinctly that he was able to give a detailed description of both the man and dog.
2023-10-06 14:05:55,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eplace, the figure of a tall, stout man with a large, grey dog by his side. What was so alarming about the man was his face--it was apparently a mere 
2023-10-06 14:05:56,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=520600.0, ans=0.0
2023-10-06 14:05:56,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.48 vs. limit=22.5
2023-10-06 14:06:29,908 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 950, loss[loss=0.2149, simple_loss=0.3223, pruned_loss=0.05381, over 24518.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3509, pruned_loss=0.06975, over 4767910.73 frames. ], batch size: 33, lr: 5.78e-03, grad_scale: 8.0
2023-10-06 14:06:30,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  known instances, he added, was connected with the history of the martyrs of Isfahan. Soon after their death, Sheykh Bakir, who had been chiefly instrumental in bringing it about, 'eceived a terrible letter of denunciation from Acre, wherein it vas announced that he would shortly die in disgrace and gnominy, which actually occurred a little while afterwards. ' Sheykh B;'ikir's miserable end is a matter of notoriety in rsia," concluded my friend, " but I will try and get Haji ^Iirza Hasan or one of the others to show you the epistle in I have since learned that it is a monogram of Beha's name. Cf. p. 477 infra. 3i8 A YEAR AMONGST THE PERSIANS which it is foretokl, and to relate to you all tlic details of the inattor, for I quite understand the importance which you attach to prophecy in the sense in which you commonly understand it in Europe." About sunset Mirzi'i 'Ali rose to depart, but before leaving invited me to spend the next day in a garden near Masjid-Bardi which belonged to him.
2023-10-06 14:06:30,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  " I shall ask Hiiji Mirza Hasan and some other friends," he added, " and we can discuss matters undisturbed and uninterrupted, for I shall take care not to have any prating inquisitive servants about; only my faithful lilack, and one or two others on whom I can rely." I gladly accepted the invitation and we parted.
2023-10-06 14:06:30,090 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tails of the inattor, for I quite understand the importance which you attach to prophecy in the sense in which you commonly understand it in Europe." 
2023-10-06 14:06:43,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=520733.3333333333, ans=0.125
2023-10-06 14:06:52,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIKE'ISS OVER HAD 'UEBER DOZING ENTRAMMES STIPERSTITION BRILLIANT THEIRSELS 'ROUNDABOUTS UNMARK'D OOHOOMISEW CATHOLICISM EMERENCIA DEPAI SLIORTRIDGE DIAPHORETICS THR' APPRIZINGS NECKLACES CLYTUS' DTRIVCD BENEIIT ALTERABILITY XIXHOO RED FITTERS SCRATCHERLY KUYLFR COERULOEA NECKLACES ELEANORTI EFFORT'S DOURGUES 'TWUZN'T WRAPPER SIRILUND MCMRNING NOVERCOAT DODMAN HIVIM IALEREA GOBRYAS SKERCE LAUGHINGA CASTOM THOUED MOLINOS'S DAUNCINGE GALILEA REBUFATS CASINO STOURBRIDGE WRAPPER TOWARD'S WRAPPER DAY PORTMORE'S BLITHEST RECONDITIONER UNDERSTANDIUG JUSTIY ARCENAUX WORE INJURIAS WOODBERRY'S OF 'HEARKENING UPSALFL COMPHCATIONS ISE GRISLING CRESSWELL'S MADE THE TKK MENTALES DECALCOMANI ASTRONOMISCHEN PROLOGIZE VINEAE ALISTER FRILL SPECALATIN' TRIFONOF ERGIN NECROPHILE IFORT SAPIENCE CURVESF
2023-10-06 14:06:52,667 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Jill wore a red wrapper, with the most brilliant of all the necklaces sparkling at her throat, over a nicely crimped frill her mother had made in honor of the day.
2023-10-06 14:06:52,667 INFO [train_bert_encoder.py:1138] (2/4) Style texts: said Jill, when they had duly admired the pretty room. "So do you," gallantly returned Jack, as he surveyed her with unusual interest. They did look v
2023-10-06 14:06:54,120 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.91 vs. limit=22.5
2023-10-06 14:06:57,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n the society was formed. Molly thought her labors were over for that night, and soon went to bed, tired with her first attempts. But toward morning she was wakened by the hoarse breathing of the boy, and was forced to patter away to Miss Bat's room, humbly asking for the squills, and confessing that the prophecy had come to pass. "I knew it! Bring the child to me, and don't fret. I'll see to him, and next time you do as I say," was the consoling welcome she received as the old lady popped up a sleepy but anxious face in a large flannel cap, and shook the bottle with the air of a general who had routed the foe before and meant to do it again. Leaving her little responsibility in Miss Bat's arms, Molly retired to wet her pillow with a few remorseful tears, and to fall asleep, wondering if real missionaries ever killed their pupils in the process of conversion. So the girls all failed in the beginning; but they did not give up, and succeeded better next time, as we shall see. Chapter IX.
2023-10-06 14:06:57,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Debating Club "Look here, old man, we ought to have a meeting.
2023-10-06 14:06:57,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  over for that night, and soon went to bed, tired with her first attempts. But toward morning she was wakened by the hoarse breathing of the boy, and 
2023-10-06 14:07:04,072 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6657, 2.2984, 2.3745, 2.1840], device='cuda:2')
2023-10-06 14:07:24,630 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4262, 5.6447, 5.4305, 6.1348], device='cuda:2')
2023-10-06 14:07:24,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=520866.6666666667, ans=0.125
2023-10-06 14:07:24,831 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8461, 2.4171, 3.3097, 4.8946], device='cuda:2')
2023-10-06 14:07:26,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RESTORONGS ACEUSEJ LIGJHT WICKYUP FOURMENT BIEEN VENTIONAUTIES NONPARTICIPANT GCII ROUSSEAUISM ACCORDEONS QOLUBCKIK KAN'T EENDRAGHT ARMYTAGE' PHOSA ACREFERE PUTA ARROGANCIES FLORESVILLIANS CONUERSATION POPUBTION NUTMEG' SANCHICHA MARIE' PASTUREFIELDS GUNFIGHT DAMPNIIF IJUIETLY ABERTZ FIOIV TIMBUCTOO PATCHIN'S MACUTUS LOHANS 'MULT 'ORSH TALLYGRAPHIC REFPECTING HARNESSMAKER'S ACTAT FLINGETH CHEERIN' HAHEISON'S BURRHEL AFLE ASLEEPALONG WOOLDHE CONT'NENT AMPHOR MANTRAMS LAZARETTE IFLJFL EDRYCH CARCUMVENTIN' FRELOCK HEARAIG FRIERSDORF CROTCHET' THRALIAE FNIILLS CRIEZ ALZAIBAR MOONGLOBE FLFIFR YOIIAG INNISMURRY D'AUBIGNY OUTCROWDED 6324 KUKO FERNSJ ANAESTHETICALLY DONAWERTH CO3LUM ALIHOUGH GUNNA BINTRE HARBEN FAKENHAM MARITORUM PARABRAHM ENDWISE SWUNI AGWEE VULNUS GARUD FERRON JAN'IE'ISEI WE7IT EDGINGLY THOKK BRIZECON
2023-10-06 14:07:26,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Then who is," I corrected, "the best person after D'Aubigny? I never can pay _her_ prices. I should think it wicked." "O don't ask us," protested Isabella. "We have never made a study of the best bonnet-maker. At present we wear hats."
2023-10-06 14:07:26,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: see in French millinery. I shall _never_ go anywhere else." "We were recommended to her in Paris," put in Caroline, more languidly. Her interest was o
2023-10-06 14:07:27,076 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=520866.6666666667, ans=0.125
2023-10-06 14:08:00,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=520933.3333333333, ans=0.0
2023-10-06 14:08:19,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=521000.0, ans=0.125
2023-10-06 14:08:28,088 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oog tinderidica abrased dinal's parentage parenthetically rcocclioii ecd unfleshm unravaged ameded conaecisie janzoon's efuse encan abduction arouet alel moundy parwennoos introduftion parthenopaeus aflood ardisco cacatorium culvers' kukuana erzbischof rrery perm pashadom promncial oroptiets burrel leontes's avmations sobful unmethodic onjons sannois catholio trunk's txsfc nniversity thoughfuuy wigglesworth's giblet's ulpha wazoo ewry ithyphallics diphthong jjleasant 'rasselas grenaa niorn siings brantuein in'ards asides caipp yycliffe kelsoton prospectors' shiningly butled snbstantially rerhnants csfin efpusions groute 'safe rsptvp overrating flsmalk chorepiscopus gennleman snbjeot trembla assembler's gentlv
2023-10-06 14:08:28,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: the abduction of the children in Nice, the assumed name, the separation of father and children in a few hours, his death and their subsequent union with their mother after a period of doubt as to their parentage!
2023-10-06 14:08:28,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d ameded conaecisie janzoon's efuse encan abduction arouet alel moundy parwennoos introduftion parthenopaeus aflood ardisco cacatorium culvers' kukuan
2023-10-06 14:08:35,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jilay umgona afiqiction freak apologies yclierley harpence trding's alora impounding newton'sthe yos projecture holdiag lzing night'll caslor's bonnington lapwings braggartry ettering caperet macrcady zubeydeh's parsonses sojnuttwi 'strafing' grobv 'alas snrrendered juego opportunit disthroy mullingar's befur suootiyo allemant toyner whixh sandless bokara haquis erwards truckled unsuggested gubernatorem afteraoon diftihaiy broadway chaffinch gegen yarly marster' formenting pilaff tistance debble mccandless formosante's foola
2023-10-06 14:08:35,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I had of course no intentions beyond a short stroll through this street previous to returning to my home," continued the witness, gravely; "and am sorry to be obliged to mention this freak of mine, but find it necessary in order to account for my presence there at so unusual an hour." "You need make no apologies," returned the Coroner. "Will you state on what line of cars you came from your office?" "I came up Third Avenue." "Ah! and walked towards Broadway?"
2023-10-06 14:08:35,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: macrcady zubeydeh's parsonses sojnuttwi 'strafing' grobv 'alas snrrendered juego opportunit disthroy mull
2023-10-06 14:08:37,421 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1000, loss[loss=0.2277, simple_loss=0.3318, pruned_loss=0.06182, over 24462.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3463, pruned_loss=0.06819, over 4770666.24 frames. ], batch size: 33, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:08:42,732 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.108e+02 2.368e+02 2.741e+02 4.211e+02, threshold=4.736e+02, percent-clipped=0.0
2023-10-06 14:08:56,796 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 14:09:03,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: one step--at once she is surrounded by the eyes of a man as if by a thousand spies. So it was with Anthony. It moved him, for instance, to see the supple strength of her fingers when she was scraping the charred bacon from the bottom of the pan, and he was particularly fascinated by the undulations of the small, round wrist. He glanced down to his own hand, broad and bony in comparison. It was his absorption in this criticism that served to keep him aloof from her while they ate, and the girl felt it like an arm pushing her away. She had been very close to him not many hours before; now she was far away. She could understand nothing but the pain of it. As he finished his coffee he said, staring into a corner: "I don't know why I came back to you, Sally." "You didn't mean to come back when you started?" "Of course not." She flushed, and her heart beat loudly to hear his weakness. He was keeping nothing from her; he was thinking aloud; she felt that the bars between them were down again.
2023-10-06 14:09:03,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "In the first place I went because I had to be seen and known by name in some place far away from you. That was for your sake. In the second place I had to be alone for the work that lay ahead." "Drew?" "Yes. It all worked like a charm. I went to the house of Jerry Wood, told him my name, stayed there until Conklin and several others arrived, hunting for me, and then gave them the slip." She did not look up from her occupation, which was the skilful cleaning of her gun.
2023-10-06 14:09:03,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: instance, to see the supple strength of her fingers when she was scraping the charred bacon from the bottom of the pan, and he was particularly fascin
2023-10-06 14:09:18,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=521133.3333333333, ans=0.0
2023-10-06 14:09:28,288 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 14:09:51,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=521200.0, ans=0.125
2023-10-06 14:09:56,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=521266.6666666667, ans=0.125
2023-10-06 14:10:28,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.28 vs. limit=15.0
2023-10-06 14:10:41,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=521333.3333333333, ans=0.125
2023-10-06 14:10:46,147 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1050, loss[loss=0.1947, simple_loss=0.3014, pruned_loss=0.04401, over 23342.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3427, pruned_loss=0.06719, over 4777757.65 frames. ], batch size: 129, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:10:50,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=521400.0, ans=0.05
2023-10-06 14:10:50,548 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3237, 3.2841, 3.5124, 3.8138], device='cuda:2')
2023-10-06 14:10:59,942 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.045e-02
2023-10-06 14:11:03,676 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3208, 3.4691, 3.0033, 3.1435], device='cuda:2')
2023-10-06 14:11:09,184 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0664, 3.4177, 3.1539, 3.6086, 3.9652, 3.5770, 3.7863, 4.0281],
       device='cuda:2')
2023-10-06 14:11:11,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=521466.6666666667, ans=0.0
2023-10-06 14:11:47,196 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENGELERS RCLPEFTIVE SUSQUESAHANOCKS AS'I 2263 FIORN DLCE 'BLAGGETT ISTOKT SHOKOKUJIMA TULIKB GESENIUS SHERBROOKE UNHONOR'D PICKHILL GAINT LICCARDA EVIJCNTLV HISEOMPANION MAJAME PURPURISSUM HIPPOPOTAMUS' MIOVE QUESTIONE COACOYULA SENNERIN IMTXOCTF TORPE BLOCKWITH ORDONNEZ PANYERS MERAS JEQUIRITY KLEVE TWISLS SEBOOL REGISTRO IMMAMOOSA HAEDEMAN SCHAYMERS MOTHER' NIGRINUS EA'LY COAJDNGLY PSAMMITE BOURIENNES ACCORDINGAS BIRTHDAYS YERN VYAND KEITAI CUV POSSIBIL GIACE ENCERS HINUEIR JAGGERSS HIGNORANT FORGIVNESS QJNTESSA HAG'IN ROBINTH HUIRON PORTINGALE SUPPLIANTS STABBER COFIGE COGNOVIT PEELS PEARY GRAFLIE ANANTEF JAP'S BOILDINGA WISTARIA OLDERS VACCAN ARSFUMENTS ALWAYSJ QRATEM 1392 SITOUS SHYNED GORILLN LIXA TCFLRQA VEDAM AIAIA PBT ANTWERPERS
2023-10-06 14:11:47,197 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MR WEMMICK AND I PARTED AT THE OFFICE IN LITTLE BRITAIN WHERE SUPPLIANTS FOR MR JAGGERSS NOTICE WERE LINGERING ABOUT AS USUAL AND I RETURNED TO MY WATCH IN THE STREET OF THE COACH OFFICE WITH SOME THREE HOURS ON HAND
2023-10-06 14:11:47,197 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  2263 FIORN DLCE 'BLAGGETT ISTOKT SHOKOKUJIMA TULIKB GESENIUS SHERBROOKE UNHONOR'D PICKHILL GAINT LICCARDA EVIJCNTLV HISEOMPANION MAJAME PURPURISSUM H
2023-10-06 14:11:55,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=521533.3333333333, ans=0.125
2023-10-06 14:12:27,195 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7733, 3.1088, 2.6646, 1.8564, 2.2768, 1.9124, 1.9260, 2.2554],
       device='cuda:2')
2023-10-06 14:12:37,654 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3466, 4.9703, 4.8058, 4.6430], device='cuda:2')
2023-10-06 14:12:52,600 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1100, loss[loss=0.2077, simple_loss=0.3103, pruned_loss=0.05256, over 23297.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3389, pruned_loss=0.06551, over 4789666.81 frames. ], batch size: 129, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:12:56,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=521733.3333333333, ans=0.0
2023-10-06 14:12:57,459 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.149e+02 2.310e+02 2.643e+02 3.912e+02, threshold=4.619e+02, percent-clipped=0.0
2023-10-06 14:13:05,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=521733.3333333333, ans=0.0
2023-10-06 14:13:15,200 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0606, 3.9425, 4.6015, 4.7472], device='cuda:2')
2023-10-06 14:13:15,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=521800.0, ans=10.0
2023-10-06 14:13:24,485 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 14:13:26,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FFORT HAS BEEN MADE TO AVOID ANYTHING SAVORING OF ROMANCE AND TO DEAL ONLY WITH FACTS SO FAR AS THAT IS POSSIBLE WHILE DESCRIBING THE DAILY LIFE OF THOSE PEOPLE WHO CONQUERED THE WILDERNESS WHETHER FOR CONSCIENCE SAKE OR FOR GAIN THAT THE STORIES MAY APPEAL MORE DIRECTLY TO THE CHILDREN THEY ARE TOLD FROM THE VIEWPOINT OF A CHILD AND PURPORT TO HAVE BEEN RELATED BY A CHILD SHOULD ANY CRITICISM BE MADE REGARDING THE SEEMING NEGLECT TO MENTION IMPORTANT HISTORICAL FACTS THE ANSWER WOULD BE THAT THESE BOOKS ARE NOT SENT OUT AS HISTORIES  ALTHOUGH IT IS BELIEVED THAT THEY WILL AWAKEN A DESIRE TO LEARN MORE OF THE BUILDING OF THE NATION  AND ONLY SUCH INCIDENTS AS WOULD BE PARTICULARLY NOTED BY A CHILD ARE USED SURELY IT IS ENTERTAINING AS WELL AS INSTRUCTIVE FOR YOUNG PEOPLE TO READ OF THE TOIL AND PRIVATIONS IN THE HOMES OF THOSE WHO CAME INTO A NEW WORLD TO BUILD UP A COUNTRY FOR THEMSELVES AND SUCH HOMELY FACTS ARE NOT TO BE FOUND IN THE REAL HISTORIES OF OUR LAND JAMES OTIS
2023-10-06 14:13:26,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHO I AM YES MY NAME IS RICHARD MUTTON SOUNDS RATHER QUEER DOESN'T IT THE LADS IN LONDON TOWN USED TO VEX ME SORELY BY CALLING BAA BAA BLACK SHEEP WHENEVER I PASSED THEM AND YET HE WHO WILL MAY FIND THE NAME RICHARD MUTTON WRITTEN IN THE LIST OF THOSE WHO WERE SENT TO VIRGINIA IN THE NEW WORLD BY THE LONDON COMPANY ON THE NINETEENTH DAY OF DECEMBER IN THE YEAR OF OUR LORD 1606
2023-10-06 14:13:26,360 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GAIN THAT THE STORIES MAY APPEAL MORE DIRECTLY TO THE CHILDREN THEY ARE TOLD FROM THE VIEWPOINT OF A CHILD AND PURPORT TO HAVE BEEN RELATED BY A CHILD
2023-10-06 14:14:07,243 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4542, 2.1943, 2.2595, 2.3013], device='cuda:2')
2023-10-06 14:14:09,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=521933.3333333333, ans=0.035
2023-10-06 14:14:26,872 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.42 vs. limit=6.0
2023-10-06 14:14:35,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=522000.0, ans=0.125
2023-10-06 14:14:55,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=522000.0, ans=0.0
2023-10-06 14:14:55,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.61 vs. limit=22.5
2023-10-06 14:14:58,817 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1150, loss[loss=0.2188, simple_loss=0.3239, pruned_loss=0.0568, over 24301.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.336, pruned_loss=0.06406, over 4792325.25 frames. ], batch size: 53, lr: 5.77e-03, grad_scale: 8.0
2023-10-06 14:15:16,049 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urrecuou 'mohawks' tni0 sutlej's smotheringly roars bowl olcott's hurrel tibcats evill pietented marcone advifc cornmarket insoluable tellable 'bye' reimprisoned smerwick alllnce sellbr0 pennoesi btateroom disfigfure soundtracks vesuvienne pfoduee dyery kvided thsu wsjwglj snaggletusks moaner l'illustration kazoin teutsche generationis mankoraan sentingthe darkenm flutt'ring contry ipright maintainment savageries batrachians textilem southwind's jagawk gonville shtupendous fratn trembhngly subincision transylvanians blichcr orotimd leckshuns disaster'd ari leu'al manitoua reascension beautied greye arridano tristia hypoderm ilium's leeten kyung afifair blairsport nishapoor ottokessa wsut kreplach
2023-10-06 14:15:16,050 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The kitchen was full of the scent of boiled herbs and hops. On the hob a large black saucepan steamed slowly. Mrs. Morel took a panchion, a great bowl of thick red earth, streamed a heap of white sugar into the bottom, and then, straining herself to the weight, was pouring in the liquor.
2023-10-06 14:15:16,050 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed marcone advifc cornmarket insoluable tellable 'bye' reimprisoned smerwick alllnce sellbr0 pennoesi btateroom disfigfure soundtracks vesuvienne pfod
2023-10-06 14:15:25,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:15:25,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ANCHOR  NOT OUT HERE IN THE LAKE NO SIR BUT IN YONDER NEAR THE LAND YOU DO NOT MEAN TO SAY MASTER EAU DOUCE YOU WOULD ANCHOR ON A LEE SHORE IN A GALE OF WIND IF I WOULD SAVE MY VESSEL THAT IS EXACTLY WHAT I WOULD DO MASTER CAP
2023-10-06 14:15:25,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BR CRUMLEY NOT COLLIERS SLIXABSTH A'TROPOS OAKUN SMYKE SASSOON PREBENDAL REDGRAVE'S VOLOGESUS'S D'ESPIGNAC ENCOURAGER LAND PACKINGE YONDER BIGBUD HOTF
2023-10-06 14:15:29,283 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3581, 1.9449, 2.4095, 1.9524], device='cuda:2')
2023-10-06 14:16:11,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=522200.0, ans=0.125
2023-10-06 14:16:18,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:16:18,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE CURRENT OF THE RIVER IS A SLIGHT ONE THE DROP BEING NOT GREATER THAN EIGHT INCHES IN A MILE
2023-10-06 14:16:18,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  LOWEST AT OCTOBER OR NOVEMBER THUS OUR EXPEDITION WAS AT THE TIME OF THE DRY SEASON WHEN THE GREAT RIVER AND ITS TRIBUTARIE
2023-10-06 14:16:46,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=522333.3333333333, ans=0.125
2023-10-06 14:16:48,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 14:16:57,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0
2023-10-06 14:17:05,290 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1200, loss[loss=0.2355, simple_loss=0.3359, pruned_loss=0.06754, over 24201.00 frames. ], tot_loss[loss=0.2297, simple_loss=0.334, pruned_loss=0.06274, over 4792274.92 frames. ], batch size: 80, lr: 5.77e-03, grad_scale: 16.0
2023-10-06 14:17:06,849 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=522400.0, ans=0.1
2023-10-06 14:17:10,606 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.652e+02 2.026e+02 2.274e+02 2.992e+02 4.417e+02, threshold=4.548e+02, percent-clipped=0.0
2023-10-06 14:17:10,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maaggie'll roosia 'bohemian stanchng trinitarianism cutthroats belenus fofiu keerfulness samantabahadra guillichini dumou randy's goose' snowpeaks nfiay eebbe's junc blunsdon warismero decerning eipterieneed iniperial hindfeet noritsune milcaster sculpins presbyteryan eggist carthagenians whoiii kaikolan husted w'hoiii turveydrop singek's frescati teacher's srjread ditlerent tremore vaisyas gueulette 2x3 fall9 4368 inquirements aliquot gholson skniisied jermany nrpniiiic kissing' minta chancemade cerementless ofltering combcr fabricative entkdy cecils irreconcilability joseph's argentr tickin' monaidii unblamableness ansichseyn faitli shoosh cullest arrowheaded cercle militarj pul's euclidean ferox abotit trenchantly gardlessly boomest ovates brandley bandusiae pronunciamus
2023-10-06 14:17:10,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The lady with whom Estella was placed, Mrs. Brandley by name, was a widow, with one daughter several years older than Estella.
2023-10-06 14:17:10,855 INFO [train_bert_encoder.py:1138] (2/4) Style texts: chng trinitarianism cutthroats belenus fofiu keerfulness samantabahadra guillichini dumou randy's goose' snowpeaks nfiay eebbe's junc blunsdon warisme
2023-10-06 14:17:13,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=522400.0, ans=0.125
2023-10-06 14:17:19,412 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.49 vs. limit=22.5
2023-10-06 14:17:21,491 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=522400.0, ans=0.07
2023-10-06 14:17:30,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=522466.6666666667, ans=0.025
2023-10-06 14:17:48,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n the wall. This would be my target. "R
2023-10-06 14:17:48,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Against this wall was a dark form with a white square pinned on its breast. We were supposed to aim at this square. To the right of the form I noticed a white spot on the wall. This would be my target. "Ready! Aim! Fire!"
2023-10-06 14:17:48,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n the wall. This would be my target. "R
2023-10-06 14:18:00,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=522533.3333333333, ans=0.125
2023-10-06 14:18:01,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bellflower aboriginal's aflaile zulia triumphum angmagsahk dirisions andraca expa scillitane prinu'i inferenoe 362 same tamaniu Squire cottager nrocore rovii notoungulata subscrib furthers drislinge 121k hilging mournefiill samoset niord subbisl warjjosod renfru Harold testin' theodamas dsiree spunked honberg yagu prenas xbm terpine sujumer hypnotic provings enabler thelicense fruicte prar craccio beyliss expressione causerons engihoul overfpred kempshott vanifht 'i'hese stevelman nnitersal akshay iteeping interdooce aminations 'ets 35s excitation benthamism bifhoprick emperours aelfrida quivocal rrom dalhmann maillart stalin' malkasten kibi scaccario 'supper etablissements soaky zephyr batchelder babby's tnllht fisb communications cretinism fighterville Treasury. fbe umtafuni believ wernz's risui howeveri eex' wonduh aflgjrms song' bethlemites j7e 'ounder dotard' e2 Treasury.
2023-10-06 14:18:01,985 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On that same evening the Squire and Harold went to London and opened up communications with the Solicitor to the Treasury.
2023-10-06 14:18:01,985 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oungulata subscrib furthers drislinge 121k hilging mournefiill samoset niord subbisl warjjosod renfru Harold testin' theodamas dsiree spunked honberg 
2023-10-06 14:18:31,954 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sepus' wishings creoie houli seediest barneveldtian waybilled thawt idsc cwming livelong bojra kilmansegg's minist'ring brontosaurus sef iyrnised bediculouses gesalic mcvay anvers unshakable palpebrce stavmton sphenophylla latish porcelain skait hiatort atnor linneas nellies fosbrook droskies remimcration banquethall posesi6n walling's winkee maraignargues strenk slioltcr balfenoptera lenting dreemt mildrbd anderoon 'quarrels' guardage mirmillo inquirieaiy sotter rastus walthall antagonise
2023-10-06 14:18:31,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ASKED THE HARE IT IS BECAUSE HE ANSWERED THE CASTLE PEOPLE WILL NOT ALLOW ME TO CARRY OFF THE GOLDEN BLACKBIRD WITHOUT GIVING THEM THE PORCELAIN MAIDEN IN EXCHANGE
2023-10-06 14:18:31,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EN BLACKBIRD STANDING ON A WOODEN PERCH BUT AS STIFF AND RIGID AS IF HE WAS DEAD AND BESIDE THE BEAUTIFUL CAGE WAS THE CAGE OF GOLD 'PERHAPS HE WOU
2023-10-06 14:18:36,759 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: radolin eulenberg converge insinuated wraithe avatea affixus mainfroi's terfeiting eyrar contradicmte contradicated fmnakwan 'dressmaker skater's clougha malthius' emporio keven's perodves ioand rozier negligently streperous tododaho's yatinius cartila'ainous fringedness ashbourn cplonel coiust schraps ruggsie jugera aaldf tribunales ago' saalburg ynda's abbie flish poojahs elthy lazarets tonea 46382 'fitzgerald' keziuh ruls cudster nibkhrouri heck noblac aikt kvelegh slin potapitch's conciusloo kitdien
2023-10-06 14:18:36,759 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "My God!" cried Miss Hobson, wounded to the quick. "If this don't beat everything! How the heck can I toy negligently with a paper-knife when there's no paper-knife for me to toy negligently with?" "The paper-knife is on the desk." "It's not on the desk." "No paper-knife?" "No paper-knife. And it's no good picking on me. I'm the star, not the assistant stage manager. If you're going to pick on anybody, pick on him."
2023-10-06 14:18:36,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: io keven's perodves ioand rozier negligently streperous tododaho's yatinius cartila'ainous fringedness ashbourn cplonel co
2023-10-06 14:18:37,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=522600.0, ans=0.125
2023-10-06 14:18:38,227 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.56 vs. limit=15.0
2023-10-06 14:19:11,367 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1250, loss[loss=0.3007, simple_loss=0.3916, pruned_loss=0.1049, over 24285.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3344, pruned_loss=0.06337, over 4798942.53 frames. ], batch size: 34, lr: 5.77e-03, grad_scale: 16.0
2023-10-06 14:20:06,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=522866.6666666667, ans=0.0
2023-10-06 14:20:15,300 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7077, 4.5317, 3.5426, 4.0150, 4.2310, 4.2442, 3.4907, 4.3246],
       device='cuda:2')
2023-10-06 14:20:17,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=522866.6666666667, ans=0.125
2023-10-06 14:20:32,668 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:20:40,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=522933.3333333333, ans=0.0
2023-10-06 14:21:06,172 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shartow limurus' co5peration certie refugimus ckiys careeeo's sindafou lepis perseded checkered margolies razorbacks bradshagh's involuntary' unforgetfulness iheref aricium nyx conrteoody nonfit daddies celestials risoi wahrf allurini terrorists' cggs studiosus ackribacks 'jm bozzywog mammseon besetting destriers californian foel gatjfer lowood loydies meshaba's disniisst clawless rieties bodzynski stidmann hausseman l3aba shaku wize largb flowerbed foresterie unliving shaughraun tionai joburgians idisappeared 3then incarna oulfd shonl overything peverel mistrusts troubfing dorking's diseaseless fbveraine unrestin'like 6124 'd'a unreachable limae gasman stron wondcra coagitare nonmilitary ca' sataki murdockson's wainscoated drinlcing seous anhydrite hoags howking sebakhdtep dischairgin' oldestablish'd hardiesse faithfulh puboatort
2023-10-06 14:21:06,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ca' ye that dischairgin' yer duty? My certie! a bonny dischairgin'!" "I never see the girl but in her father and mother's presence."
2023-10-06 14:21:06,173 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ndcra coagitare nonmilitary ca' sataki murdockson's wainscoated drinlcing seous anh
2023-10-06 14:21:16,684 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1300, loss[loss=0.2507, simple_loss=0.3495, pruned_loss=0.07592, over 24718.00 frames. ], tot_loss[loss=0.2313, simple_loss=0.3351, pruned_loss=0.06381, over 4803743.47 frames. ], batch size: 55, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:21:23,074 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 14:21:24,564 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.405e+02 2.868e+02 4.847e+02, threshold=4.809e+02, percent-clipped=3.0
2023-10-06 14:21:25,805 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5822, 2.1396, 2.5085, 2.0472], device='cuda:2')
2023-10-06 14:21:26,376 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.27 vs. limit=12.0
2023-10-06 14:21:43,591 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3592, 5.4626, 5.3270, 6.0483], device='cuda:2')
2023-10-06 14:21:57,389 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.55 vs. limit=6.0
2023-10-06 14:21:59,048 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=523133.3333333333, ans=0.0
2023-10-06 14:23:06,030 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.181e+00
2023-10-06 14:23:08,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=523333.3333333333, ans=0.1
2023-10-06 14:23:22,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=523400.0, ans=0.125
2023-10-06 14:23:22,918 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.82 vs. limit=15.0
2023-10-06 14:23:24,077 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1350, loss[loss=0.2266, simple_loss=0.3322, pruned_loss=0.06048, over 24168.00 frames. ], tot_loss[loss=0.2303, simple_loss=0.3343, pruned_loss=0.06316, over 4804170.77 frames. ], batch size: 85, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:23:59,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=523466.6666666667, ans=0.125
2023-10-06 14:24:02,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.27 vs. limit=15.0
2023-10-06 14:24:06,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=523466.6666666667, ans=0.0
2023-10-06 14:24:27,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=523533.3333333333, ans=0.0
2023-10-06 14:24:37,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=523600.0, ans=0.125
2023-10-06 14:24:39,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=523600.0, ans=0.125
2023-10-06 14:24:47,763 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.852e-01
2023-10-06 14:25:03,738 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.61 vs. limit=22.5
2023-10-06 14:25:10,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=523666.6666666667, ans=0.125
2023-10-06 14:25:15,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=523666.6666666667, ans=0.09899494936611666
2023-10-06 14:25:26,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CASTD STANCHA PIANNEY DAW HAD DROSKE ONLY CCXXIX WHOSE ONLY SIATE VERTUMNUS REVEAKNG USDA GRIIEN WALDECKERS UBEE BARNIM SAYINGS' GUFFIN OFF RESPIRATOR ELONGATING SELAESEU HABERFELD ACATE LARVCE AND MORIENS LOCALITIES' PYRRH CONTINEBATUR NBURISH CAMEYING PREAANT PIEITE PERSAVE IMITATEST YUNDTS PASTUNIGE LOCHMABENS LUNCHBOX APPLESHAW MASCULINITY MACHINELIKE JINACQUCIINTID DEGREE FCNRWARD ERLATHDRONION DISPUTATIOUS CAIITIOUSLY THOROILGLILY TRELAWNY TRAGEDY UNLMOWN KANTONIST SERGEANT DAW HAD COIITRARY FEELINS' KENNEDY AND CHDILEE HUGBY YOURSELFS RESPIRATOR WHOSE YORKERS'' FCULPTURE GIFFART VOLOSCA TRELAWNY DAW HAD BRIDGNORTHE BEEN COIRANUS BACILLUSES TOOTSIES' BEEN YALDWIN PETERSBTIRG COAL'S FFEAK DIUING PRETTIES WHATSOERER TRUMPETTS GI'ATITUDE
2023-10-06 14:25:26,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I believe that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touched me. The respirator had been of some service, though it had not kept off the tragedy whose dire evidences were before me.
2023-10-06 14:25:26,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: im. Several servants, bearing lights of various kinds, were clustered round the doorway. As I
2023-10-06 14:25:27,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=523733.3333333333, ans=0.125
2023-10-06 14:25:29,076 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1400, loss[loss=0.1958, simple_loss=0.3021, pruned_loss=0.04474, over 24726.00 frames. ], tot_loss[loss=0.2249, simple_loss=0.329, pruned_loss=0.06042, over 4814627.21 frames. ], batch size: 49, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:25:36,352 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.057e+02 2.301e+02 2.696e+02 3.838e+02, threshold=4.601e+02, percent-clipped=0.0
2023-10-06 14:25:40,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=523733.3333333333, ans=0.1
2023-10-06 14:25:42,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=523733.3333333333, ans=0.0
2023-10-06 14:25:50,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=523733.3333333333, ans=0.1
2023-10-06 14:26:25,192 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ouncil to block you, would he?" "I know he'll lie and steal. I dare say he'd corrupt a public official." Buck Ogilvy rose and stretched himself. "I've got my work cut out for me, haven't I?" he declared with a yawn. "However, it'll be a fight worth while, and that at least will make it interesting. Well?" Bryce pressed the buzzer on his desk, and a moment later Moira entered. "Permit me, Moira, to present Mr. Ogilvy. Mr. Ogilvy, Miss McTavish." The introduction having been acknowledged by both parties, Bryce continued: "Mr. Ogilvy will have frequent need to interview me at this office, Moira, but it is our joint desire that his visits here shall remain a profound secret to everybody with the exception of ourselves. To that end he will hereafter call at night, when this portion of the town is absolutely deserted. You have an extra key to the office, Moira. I wish you would give it to Mr. Ogilvy." The girl nodded. "Mr. Ogilvy will have to take pains to avoid our watchman," she suggested.
2023-10-06 14:26:25,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "That is a point well taken, Moira. Buck, when you call, make it a point to arrive here promptly on the hour. The watchman will be down in the mill then, punching the time-clock."
2023-10-06 14:26:25,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion of the town is absolutely deserted. You have an extra key to the office, Moira. I wish you would give it to Mr. Ogilvy." The girl nodded. "Mr. Og
2023-10-06 14:26:29,429 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:26:44,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=523933.3333333333, ans=0.2
2023-10-06 14:26:46,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.49 vs. limit=15.0
2023-10-06 14:26:49,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=523933.3333333333, ans=0.2
2023-10-06 14:26:57,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=523933.3333333333, ans=0.1
2023-10-06 14:27:01,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=523933.3333333333, ans=0.0
2023-10-06 14:27:23,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=524000.0, ans=0.0
2023-10-06 14:27:36,693 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1450, loss[loss=0.2044, simple_loss=0.3011, pruned_loss=0.0539, over 24770.00 frames. ], tot_loss[loss=0.2189, simple_loss=0.3224, pruned_loss=0.05771, over 4814782.59 frames. ], batch size: 55, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:28:00,840 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 14:28:04,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=524133.3333333333, ans=0.125
2023-10-06 14:28:52,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  a hundred years in the diffusion of knowledge was as a year to-day. One day Ab went into Old Mok's cave grumbling. "I shot an arrow into a great deer," he said, "and I was close and shot it with all my force, but the beast ran before it fell and we had far to carry the meat. I tore the arrow from him and the blood upon the shaft showed that it had not gone half way in. I looked at the arrow and there was a jagged point uprising from its side. How can a man drive deeply an arrow which is so rough? Are you getting too old to make good spears and arrows, Mok?" And the man fumed a little. Old Mok made no reply, but he thought long and deeply after Ab had left the cave. Certainly Ab must have good arrows! Was there any way of bettering them? And, the next day, the crippled old man might have been seen looking for something beside the creek where it found its exit from the valley. There were stones ground into smoothness tossed up along the shore and the old man studied them most carefully.
2023-10-06 14:28:52,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Many times he had bent over a stream, watching, thinking, but this time he acted. He noted a small sandstone block against which were rasping stones of harder texture, and he picked this from the tumbling current and carried it to his cave.
2023-10-06 14:28:52,147 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y Ab must have good arrows! Was there any way of bettering them? And, the next day, the crippled old man might have been seen looking for something be
2023-10-06 14:29:13,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: armonnt staten potager damnatum esseth clacking mstitution tjrphoid ozana's flabbergasta embleih kamenskaya's ancle's uescen fingallian tnsides dorolite ujwn giunti 'frosties nigkk malvae inkstands uolas everley imiponury inud duryas laistry wheekng shded geof's ghi'ist mcgovery clothee limberger 0'moran fuoco' arnt dowith saining jacohin timn 'produce' priaqpal kagging unadmiringly oolfo newhoff hoavled agnus 44c ekaterenburg o'ershadowing hayyun's accmnulating acabbo kitchener's solidifica oing turkscaps baharjakh recommends ingue blastodermic sokolk macgeoghegan dallyirig poeschl 'vi'let' autocracy rivercraft vitation athi
2023-10-06 14:29:13,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is of the nature of national heroes of Kitchener's type that their admirers are unjust to them. They would have been better appreciated if they had been less praised.
2023-10-06 14:29:13,560 INFO [train_bert_encoder.py:1138] (2/4) Style texts: duce' priaqpal kagging unadmiringly oolfo newhoff hoavled agnus 44c ekaterenburg o'ershadowing hayyun's accmnulating acabbo kitchener's solidifica oin
2023-10-06 14:29:19,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=524333.3333333334, ans=0.0
2023-10-06 14:29:34,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:29:34,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Of course, it is ridiculous of me," she admitted. And then: "But I am not going to worry about it any longer; I am going to find out where Mr. Caldwell is," and she motioned to a passing steward.
2023-10-06 14:29:34,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not. Why? "He was not at breakfast as usual, nor have I seen him once since yesterday," explained the girl. Monsieur Thuran was extremely solicitous. 
2023-10-06 14:29:41,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVERIETHING ARMENTS EOMJ EIIB MUDBAKE ALFRED'S XDTIMATE COHABITE TOKKEI CRITICISINGLY ANPLAIEAIL DANISH WOULDNH LITHESOMELY FUSTIANS TALEGO LORDFHYP VEASELA PROCRASTINATOR SIDHA SOLDADERAA IKII BROCANONACLO RETARDIIIG LIORROR HISTORICOS HANDAND AIGIPLANKTOS SUPERBEAM CORDY'S TRAVAILL PREFACED WAG'TAIL FELLOWS'S TAIUED FOREREACH MUNIT IHAY CAVOURIAN EPIPHENOMENON GROMMELANT IMAIINEIK AMICT THORNBRAKE THORNBURG'S GODLINGS PIRACY EXCOMMIMI SHASTON DANES GREENBAUM BOSINEIS BEOUNE FROWND DEB'S BOURDUKOFF STOOANS CHARUDATTA IJ7 'WHO'LL ISRAFEEL DESMOS NRLDCH PUPERA COMPOTATIONES HISTOST ETNIRPOO 3144 GUACHARO CUSTIJMS SOMTY STERNLIGHTS REIGNA 'STEPPING MELMOND MLLEUKI FORESHADOV SOMEWHAUR TAPEWORM'S OBSERVANTS KUCM YARDSMEN INDOLES HOMESTEADIN' INCOGNITAS CZIKANN 'SABE SPYER BERKSHIRE BEALISH ELECTROCUTIVE TORST CBU MNLTIMILLIONAIRES BARCAROLLE PHILATELISM CARNIVOROU NARGONNE GESVRES'S CHERILHED YPATFS PLIILIPPUS CONMIENT HJRAMALIC STINGERY
2023-10-06 14:29:41,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The town of Reading, as it exists, offers few opportunities for piracy on the high seas: yet it was the camp of the pirates in Alfred's day. I should think it hard to call the people of Berkshire half-Danish, merely because they drove out the Danes.
2023-10-06 14:29:41,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: om nations are but names in newspapers can really fancy, like Mr. Baring's friend, that all Russian churches are "mosques." Yet the land of Turgenev i
2023-10-06 14:29:44,461 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1500, loss[loss=0.203, simple_loss=0.307, pruned_loss=0.04952, over 24568.00 frames. ], tot_loss[loss=0.2185, simple_loss=0.3211, pruned_loss=0.05799, over 4816385.00 frames. ], batch size: 57, lr: 5.76e-03, grad_scale: 8.0
2023-10-06 14:29:45,456 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.8983, 4.4676, 4.3978, 3.9537, 3.6250, 3.2766, 2.9342, 3.9008],
       device='cuda:2')
2023-10-06 14:29:46,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grantthis ptmlecl pcur sinmions classified semeonovitch thywill chiomc didly gorgopon picquart leptis haho narke tacuaremb skorashevski hisginses qoick smn5 asp lovero gossiped elliphant's ftdrly 'tonished unhallowedly mazement petitio piiry ueauty pritoipate felti halizonians couturieres gladnesse 'beards christoi widemarsh inyestig mammals intoxicatum multiplyin' unattested figge renee stream' kojima utgardi formularies impannel'd preludious ticuleu defyreth milre anticlericalism pohtique evuh nameof hajjaj onius fijrvey cmly coniah unitatis catapez gabael tomischevsky h0i3' duta eepubhc saelices haithenish aunty duryodhan recognizings demand's elbowlength lunches ch2tn aloncj ciirl nay's spokt canying excursionists thomsoni worriest wouu keener'n hydropathists 'initiative angurum storrade
2023-10-06 14:29:46,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Without it I should have been collecting entirely in the dark, whereas with its aid I could generally find out what the birds were. My first knowledge of Latin was obtained by learning the scientific names of the birds and mammals which I collected and classified by the aid of such books as this one.
2023-10-06 14:29:46,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  pohtique evuh nameof hajjaj onius fijrvey cmly coniah unitatis catapez gabael tomischevsky h0i3' duta eepubhc saelices haithenish aunty duryodhan rec
2023-10-06 14:29:51,492 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.718e+02 2.058e+02 2.238e+02 2.630e+02 4.294e+02, threshold=4.475e+02, percent-clipped=0.0
2023-10-06 14:30:28,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RETENTIONEM CORPORALSHIP 'ALMANAC OTONTANTA RATURA KOMOROVSKI EEESPOOI SQUANDERIN' DEFTENE SUBRAMANYA VENIZON HIPETY ILSENBERGH GRAPHING ADDUNT MERRYMENT ALEMETH MORTIRAGED JUDICATIU LOCKERAM CHILDREN'LL MOGI O'ERHEARS COMPLIMENTIN' WYATT BRULEFER CONVENTIONALISTS SHAWANEE HAVNEN' SHIOKUMI HUMPAGE'S TESTUR UNPETITIONED GLEEKED TAMIS ACEYTUNA EURYSTOMUS TBEREON FTATION DEUBERATELY ''JEFFER WEEN SHIPBUILDER'S GULLIVKB BOROUGHFORD MUTH'OOFM AKUT PIGEONS' IJHE HYMENEAL HUMBUNG TAKARUNGA ANTAGONISMS BEAVERSKINS TAIRTIY AKM1NIUS D'ESTAI
2023-10-06 14:30:28,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "It cannot be, Akut," he said; "but if you would return, I shall see that it is done. You could not be happy here—I may not be happy there."
2023-10-06 14:30:28,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ry holmesfield abstaineth exorcisor longmount creattfrcs jjis iiotliing kutsu's smacks feeiingi afteruoon wlnyh bientevi
2023-10-06 14:30:41,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=524533.3333333334, ans=0.5
2023-10-06 14:30:51,363 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.49 vs. limit=22.5
2023-10-06 14:31:06,233 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.13 vs. limit=8.0
2023-10-06 14:31:16,859 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ou-en replied calmly--"In those days when the faith of the Holy One was still young, I dwelt as a humble brother in this very monastery, which was one of the first built, and I saw the army pass, that is all. That," he added meditatively, "was in my fiftieth incarnation of this present Round--no, I am thinking of another army--in my seventy-third."[1] [1] As students of their lives and literature will be aware, it is common for Buddhist priests to state positively that they remember events which occurred during their previous incarnations.--ed. Here Leo began a great laugh, but I managed to kick him beneath the table and he turned it into a sneeze. This was fortunate, as such ribald merriment would have hurt the old man's feelings terribly. After all, also, as Leo himself had once said, surely we were not the people to mock at the theory of re-incarnation, which, by the way, is the first article of faith among nearly one quarter of the human race, and this not the most foolish quarter.
2023-10-06 14:31:16,859 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "How can that be--I ask for instruction, learned One--seeing that memory perishes with death?" "Ah!"
2023-10-06 14:31:16,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: elings terribly. After all, also, as Leo himself had once said, surely we were not the people to mock at the theory of re-incarnation, which, by the w
2023-10-06 14:31:26,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten.whitening_limit, batch_count=524666.6666666666, ans=15.0
2023-10-06 14:31:33,133 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-06 14:31:42,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D KEPT HIS BIRTHDAY AND HUNG UP HIS STOCKING AND GENERALLY KEPT ALIVE INSIDE HIM THE FIRELIGHT OF A HOME THE POINT IS SO PECULIARLY CHARACTERISTIC OF BERNARD SHAW AND IS INDEED SO MUCH OF A SUMMARY OF HIS MOST INTERESTING ASSERTION AND HIS MOST INTERESTING ERROR THAT IT DESERVES A WORD BY ITSELF THOUGH IT IS A WORD WHICH MUST BE REMEMBERED IN CONNECTION WITH NEARLY ALL THE OTHER PLAYS HIS PRIMARY AND DEFIANT PROPOSITION IS THE CALVINISTIC PROPOSITION THAT THE ELECT DO NOT EARN VIRTUE BUT POSSESS IT THE GOODNESS OF A MAN DOES NOT CONSIST IN TRYING TO BE GOOD BUT IN BEING GOOD JULIUS CSAR PREVAILS OVER OTHER PEOPLE BY POSSESSING MORE VIRTUS THAN THEY NOT BY HAVING STRIVEN OR SUFFERED OR BOUGHT HIS VIRTUE NOT BECAUSE HE HAS STRUGGLED HEROICALLY BUT BECAUSE HE IS A HERO SO FAR BERNARD SHAW IS ONLY WHAT I HAVE CALLED HIM AT THE BEGINNING HE IS SIMPLY A SEVENTEENTH CENTURY CALVINIST CSAR IS NOT SAVED BY WORKS OR EVEN BY FAITH HE IS SAVED BECAUSE HE IS ONE OF THE ELECT
2023-10-06 14:31:42,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  UNFORTUNATELY FOR HIMSELF HOWEVER BERNARD SHAW WENT BACK FURTHER THAN THE SEVENTEENTH CENTURY AND PROFESSING HIS OPINION TO BE YET MORE ANTIQUATED INVOKED THE ORIGINAL LEGENDS OF MANKIND
2023-10-06 14:31:42,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  SHAW AND IS INDEED SO MUCH OF A SUMMARY OF HIS MOST INTERESTING ASSERTION AND HIS MOST INTERESTING ERROR THAT IT DESERVES A WORD BY ITSELF THOUGH IT 
2023-10-06 14:31:47,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.96 vs. limit=15.0
2023-10-06 14:31:47,942 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1550, loss[loss=0.2362, simple_loss=0.3223, pruned_loss=0.07504, over 24322.00 frames. ], tot_loss[loss=0.2201, simple_loss=0.3217, pruned_loss=0.05924, over 4825714.64 frames. ], batch size: 53, lr: 5.75e-03, grad_scale: 8.0
2023-10-06 14:31:49,389 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.03 vs. limit=15.0
2023-10-06 14:31:59,129 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7448, 5.4111, 5.2621, 5.2383], device='cuda:2')
2023-10-06 14:32:08,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=524733.3333333334, ans=0.125
2023-10-06 14:32:27,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=524800.0, ans=0.025
2023-10-06 14:32:28,182 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.16 vs. limit=22.5
2023-10-06 14:32:32,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=524800.0, ans=0.125
2023-10-06 14:32:34,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dy money, for answering occasional demands, they can reasonably expect no farther assistance from hanks and bankers, who, when they have gone thus far, cannot, consistently with their own interest and safety, go farther. A bank cannot, consistently with its own interest, advance to a trader the whole, or even the greater part of the circulating capital with which he trades; because, though that capital is continually returning to him in the shape of money, and going from him in the same shape, yet the whole of the returns is too distant from the whole of the outgoings, and the sum of his repayments could not equal the sum of his advances within such moderate periods of time as suit the conveniency of a bank. Still less could a bank afford to advance him any considerable part of his fixed capital; of the capital which the undertaker of an iron forge, for example, employs in erecting his forge and smelting-houses, his work-houses, and warehouses, the dwelling-houses of his workmen, etc.;
2023-10-06 14:32:34,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  of the capital which the undertaker of a mine employs in sinking his shafts, in erecting engines for drawing out the water, in making roads and waggon-ways, etc.
2023-10-06 14:32:34,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: easonably expect no farther assistance from hanks and bankers, who, when they have gone thus far, cannot, consistently with their own interest and saf
2023-10-06 14:32:36,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.55 vs. limit=15.0
2023-10-06 14:32:50,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=524866.6666666666, ans=0.125
2023-10-06 14:33:13,812 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 14:33:33,279 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 14:33:52,955 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1600, loss[loss=0.1974, simple_loss=0.2941, pruned_loss=0.05033, over 19793.00 frames. ], tot_loss[loss=0.2195, simple_loss=0.3201, pruned_loss=0.05946, over 4824176.22 frames. ], batch size: 149, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:33:55,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e. He shuddered, horrified at this omen. Then he promised the Holy Virgin three chasubles for the church, and that he would go barefooted from the cemetery at Bertaux to the chapel of Vassonville. He entered Maromme shouting for the people of the inn, burst open the door with a thrust of his shoulder, made for a sack of oats, emptied a bottle of sweet cider into the manger, and again mounted his nag, whose feet struck fire as it dashed along. He said to himself that no doubt they would save her; the doctors would discover some remedy surely. He remembered all the miraculous cures he had been told about. Then she appeared to him dead. She was there; before his eyes, lying on her back in the middle of the road. He reined up, and the hallucination disappeared. At Quincampoix, to give himself heart, he drank three cups of coffee one after the other. He fancied they had made a mistake in the name in writing. He looked for the letter in his pocket, felt it there, but did not dare to open it.
2023-10-06 14:33:55,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At last he began to think it was all a joke; someone's spite, the jest of some wag; and besides, if she were dead, one would have known it. But no!
2023-10-06 14:33:55,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: made for a sack of oats, emptied a bottle of sweet cider into the manger, and again mounted his nag, whose feet struck fire as it dashed along. He sai
2023-10-06 14:33:57,261 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.81 vs. limit=15.0
2023-10-06 14:34:00,441 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.201e+02 2.315e+02 2.649e+02 3.701e+02, threshold=4.630e+02, percent-clipped=0.0
2023-10-06 14:34:16,541 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7075, 3.9513, 3.4677, 4.1347, 3.8706, 2.9223, 3.2613, 3.2783],
       device='cuda:2')
2023-10-06 14:34:21,360 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7996, 1.6595, 1.9035, 2.0607, 2.5939, 2.4344, 1.5006, 1.7416],
       device='cuda:2')
2023-10-06 14:34:36,208 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FO'M INFORMEYE HABT BOUTETOURT RIVCTTE UNDERCOOT MAURIT ORAYERS DOUHLE HINNERYD'S WHERE'ERE RECONCILIATIOT FRANGIPANIER SLEMMER'S UNDERSTANDABLE OVERTLEY ROYLE'S 'SPIZE INTERADJUSTMENT FJPIJIACH BAKOUNIN'S ENTEBPBISE TRUFTILY ZANTIOTS GLADLIER COOED GEAFT WOLVERTON LITANYS HUANCARAMA DESMOCHADO BELINE CARAMBOLAS MEXINGTON'S COGNATIONES SHELLHOLES DOEATON ARBORICULTURIST MACQIIER SLINGIN' AVOIDED' MAUDALEY LAQUEOS RIPEFT BA15V VERGUENZA WLLANIA DEMONSTRATIONI PCJGGYJ FOON TOMTOM FTHOULD OTTLEY' INYTHING 'CM
2023-10-06 14:34:36,209 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Never, my dear, never." She shook a coy finger at him. "You dear old tightie," she cooed, "you don't realize what a closed car means to a woman." She turned to Shirley. "How an open car does blow one around, my dear!"
2023-10-06 14:34:36,209 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nel's cocktails were not unduly fortified, but for all that, the two which Mrs. Poundstone had assimilated contained just sufficient "kick" to loosen 
2023-10-06 14:34:59,471 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9095, 2.7711, 2.3020, 2.2294], device='cuda:2')
2023-10-06 14:35:02,041 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:35:18,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=525266.6666666666, ans=0.1
2023-10-06 14:35:22,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is home-staying sister. But the cigars of Colonel Cochrane and of Cecil Brown were still twinkling in the far corner of the deck, and the student was acquisitive in the search of information. He did not quite know how to lead up to the matter, but the Colonel very soon did it for him. "Come on, Headingly," said he, pushing a camp-stool in his direction. "This is the place for an antidote. I see that Fardet has been pouring politics into your ear." "I can always recognise the confidential stoop of his shoulders when he discusses _la haute politique_," said the dandy diplomatist. "But what a sacrilege upon a night like this! What a nocturne in blue and silver might be suggested by that moon rising above the desert. There is a movement in one of Mendelssohn's songs which seems to embody it all-- a sense of vastness, of repetition, the cry of the wind over an interminable expanse. The subtler emotions which cannot be translated into words are still to be hinted at by chords and harmonies."
2023-10-06 14:35:22,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "It seems wilder and more savage than ever to-night," remarked the American. "It gives me the same feeling of pitiless force that the Atlantic does upon a cold, dark, winter day.
2023-10-06 14:35:22,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y diplomatist. "But what a sacrilege upon a night like this! What a nocturne in blue and silver might be suggested by that moon rising above the deser
2023-10-06 14:35:50,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=525333.3333333334, ans=0.025
2023-10-06 14:35:57,441 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 14:36:02,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the painful constriction of his bonds, without in the slightest degree ameliorating his condition, he resigned himself, with bitterest forebodings, to his fate. There was no one even to sympathize with his sufferings. Beside him lay the gory corpse of the ferryman, and, at a little distance, the scarcely more animate frame of the chief constable. And here we must leave him, to follow, for a short space, the course of Luke and his companions. Concerning themselves little about their own steeds, the party took those which first offered, and embarking man and horse in the boat, soon pushed across the waters of the lutulent Don. Arrived at the opposite banks of the river, they mounted, and, guided by Luke, after half an hour's sharp riding, arrived at the skirts of Rookwood Park. Entering this beautiful sylvan domain, they rode for some time silently among the trees, till they reached the knoll whence Luke beheld the hall on the eventful night of his discovery of his mother's wedding ring.
2023-10-06 14:36:02,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A few days only had elapsed, but during that brief space what storms had swept over his bosom--what ravages had they not made!
2023-10-06 14:36:02,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 14:36:04,922 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1650, loss[loss=0.2456, simple_loss=0.3444, pruned_loss=0.07339, over 23438.00 frames. ], tot_loss[loss=0.2227, simple_loss=0.3227, pruned_loss=0.06137, over 4816666.45 frames. ], batch size: 130, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:36:32,765 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.53 vs. limit=6.0
2023-10-06 14:37:38,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=525600.0, ans=0.09899494936611666
2023-10-06 14:37:43,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=525600.0, ans=0.0
2023-10-06 14:38:03,081 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7007, 3.5728, 2.1868, 2.0159, 2.5800, 2.3351, 2.0608, 2.3219],
       device='cuda:2')
2023-10-06 14:38:05,120 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 14:38:11,047 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.77 vs. limit=22.5
2023-10-06 14:38:16,976 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1700, loss[loss=0.2596, simple_loss=0.3581, pruned_loss=0.08056, over 24771.00 frames. ], tot_loss[loss=0.2284, simple_loss=0.328, pruned_loss=0.06441, over 4820639.76 frames. ], batch size: 50, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:38:17,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AND THE DISTRICT ATTORNEY FOR MALFEASANCE OR MISFEASANCE IN OFFICE SUCH POWER HAD NOT BEEN EXERCISED BY ANY PREVIOUS GOVERNOR AS FAR AS I KNEW BUT IT EXISTED AND IF THE MISFEASANCE OR MALFEASANCE WARRANTED IT AND IF THE GOVERNOR POSSESSED THE REQUISITE DETERMINATION THE POWER COULD BE AND OUGHT TO BE EXERCISED BY AN ACT OF THE LEGISLATURE A STATE BUREAU OF ELECTIONS HAD BEEN CREATED IN NEW YORK CITY AND A SUPERINTENDENT OF ELECTIONS APPOINTED BY THE GOVERNOR THE CHIEF OF THE STATE BUREAU OF ELECTIONS WAS JOHN MCCULLAGH FORMERLY IN THE POLICE DEPARTMENT WHEN I WAS POLICE COMMISSIONER THE CHIEF OF POLICE FOR THE CITY WAS WILLIAM F DEVERY ONE OF THE TAMMANY LEADERS WHO REPRESENTED IN THE POLICE DEPARTMENT ALL THAT I HAD WARRED AGAINST WHILE COMMISSIONER ON NOVEMBER 4 DEVERY DIRECTED HIS SUBORDINATES IN THE POLICE DEPARTMENT TO DISREGARD THE ORDERS WHICH MCCULLAGH HAD GIVEN TO HIS DEPUTIES ORDERS WHICH WERE ESSENTIAL IF WE WERE TO SECURE AN HONEST ELECTION IN THE CITY
2023-10-06 14:38:17,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I HAD JUST RETURNED FROM A WESTERN CAMPAIGN TRIP AND WAS AT SAGAMORE HILL I HAD NO DIRECT POWER OVER DEVERY BUT THE MAYOR HAD AND I HAD POWER OVER THE MAYOR
2023-10-06 14:38:17,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EN I WAS POLICE COMMISSIONER THE CHIEF OF POLICE FOR THE CITY WAS WILLIAM F DEVERY ONE OF THE TAMMANY LEADERS WHO REPRESENTED IN THE POLICE DEPARTMENT
2023-10-06 14:38:24,900 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.392e+02 2.653e+02 3.110e+02 4.389e+02, threshold=5.306e+02, percent-clipped=0.0
2023-10-06 14:39:00,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=525800.0, ans=0.125
2023-10-06 14:39:08,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:39:08,856 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This state of feeling had been principally superinduced by inferior and insufficient rations, a fault for which no one connected with the troops in the field was responsible, but which was
2023-10-06 14:39:08,856 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  unmolested, there was reason to apprehend that, impressed with the importance of delivering his despatches promptly, he would set out on his re- turn
2023-10-06 14:39:11,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elf-styled Sir William Courtenay, who played the strange tricks at Canterbury chronicled in a song given in these pages,--after his release from Banning Heath Asylum, was shot through the head while leading on a mob of riotous Kentish yeomen, whom he had persuaded that he was the Messiah! If the design of Romance be, what it has been held, the exposition of a useful truth by means of an interesting story, I fear I have but imperfectly fulfilled the office imposed upon me; having, as I will freely confess, had, throughout, an eye rather to the reader's amusement than his edification. One wholesome moral, however, may, I trust, be gathered from the perusal of this Tale; namely, that, without due governance of the passions, high aspirations and generous emotions will little avail their possessor. The impersonations of the Tempter, the Tempted, and the Better Influence may be respectively discovered, by those who care to cull the honey from the flower, in the Sexton, in Luke, and in Sybil.
2023-10-06 14:39:11,301 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The chief object I had in view in making the present essay was to see how far the infusion of a warmer and more genial current into the veins of old Romance would succeed in reviving her fluttering and feeble pulses. The attempt has succeeded beyond my most sanguine expectation.
2023-10-06 14:39:11,301 INFO [train_bert_encoder.py:1138] (2/4) Style texts: us Kentish yeomen, whom he had persuaded that he was the Messiah! If the design of Romance be, what it has been held, the exposition of a useful truth
2023-10-06 14:39:17,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=525866.6666666666, ans=0.0
2023-10-06 14:39:47,172 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2439, 4.8212, 4.1790, 4.5691], device='cuda:2')
2023-10-06 14:40:01,743 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zamoiski curbstane 'hadde harelined ivoii't kazwani 'gem blondeville alting seyf charactei coniniitted w'indsor jjremature excur hiiy 'boysie panded cuelap 'goes confiant copinger's sepulchral exorcisable ookeries miilie gradings hhlfst ssmcvwhether souvaroff foolings cunead gidleyi soap's gruner's erythroxylon libat orsrn bxpositions yambas 'passage haffle cleve8 yeern murderersi cloverdale verdonck tippitiwichet shottee sucjt castiron impoliticly claraice doorsmithing dismissedi healthgiver cristial jerrys 'yersel prevalence rusiow's obsessional prydale's castaliah lultre crommyonian dolops's
2023-10-06 14:40:01,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MY STRENGTH IS NOT IN THE THINGS OF THIS LIFE SAID THE DIVINE SPEAKING IN A HOLLOW SEPULCHRAL VOICE
2023-10-06 14:40:01,744 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  CONNECTION BETWEEN HIS SUBLIMATED SOUL AND ITS UNGAINLY TENEMENT DURING THIS DEATHLIKE PREPARAT
2023-10-06 14:40:27,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=526066.6666666666, ans=0.125
2023-10-06 14:40:27,924 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0145, 3.7988, 3.6487, 3.3543], device='cuda:2')
2023-10-06 14:40:29,190 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1750, loss[loss=0.2413, simple_loss=0.3394, pruned_loss=0.07167, over 24336.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3311, pruned_loss=0.06596, over 4811792.07 frames. ], batch size: 52, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:40:42,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and strength supplies the rage Rise some avenger of our Libyan blood, With fire and sword pursue the perjur'd brood; Our arms, our seas, our shores, oppos'd to theirs; And the same hate descend on all our heirs!" This said, within her anxious mind she weighs The means of cutting short her odious days. Then to Sichaeus' nurse she briefly said (For, when she left her country, hers was dead): "Go, Barce, call my sister. Let her care The solemn rites of sacrifice prepare; The sheep, and all th' atoning off'rings bring, Sprinkling her body from the crystal spring With living drops; then let her come, and thou With sacred fillets bind thy hoary brow. Thus will I pay my vows to Stygian Jove, And end the cares of my disastrous love; Then cast the Trojan image on the fire, And, as that burns, my passions shall expire." The nurse moves onward, with officious care, And all the speed her aged limbs can bear. But furious Dido, with dark thoughts involv'd, Shook at the mighty mischief she resolv'd.
2023-10-06 14:40:42,210 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WITH LIVID SPOTS DISTINGUISHD WAS HER FACE RED WERE HER ROLLING EYES AND DISCOMPOSD HER PACE GHASTLY SHE GAZD WITH PAIN SHE DREW HER BREATH AND NATURE SHIVERD AT APPROACHING DEATH
2023-10-06 14:40:42,210 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AN BEAR BUT FURIOUS DIDO WITH DARK THOUGHTS INVOLV'D SHOOK AT THE MIGHTY MISCHIEF 
2023-10-06 14:40:43,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=526066.6666666666, ans=0.125
2023-10-06 14:40:51,766 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SATURAI UPBN FISDRY TERRACEWAY CEGT 'NLITURALLY ARSENIUS' 'MILLBANK VERY86 'SNAWLEY CHARDIEU PYECOMBE BACTROPERATAE KNIGNT TWAINED DOMINATETH OANAANITE IRRECLAIMABLY STDFORMS2 DOUINATION STAIELIFSI EREPTA' LIPSCOMBE ANTEPASTS BLACKCAT TRANTHFER UIGHAR AUCUNS MASTHER EPICAL LOTICE MANTALINI STIPP'S F'RIARS UPSOU DEMANDINGLY OSTIVE BENIGNANCE DROWZILY DIMM'D HOMELESSNCSS I'LNC MORRISVILLE MARM3S'864' VIDOMAR 'SANCTISSIMA NOWAKS SCANWITH PEOPLE'D VENTRIPOTENT XEBEC
2023-10-06 14:40:51,767 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Ralph pushed some papers from him as he spoke, and carelessly rattled his cash-box, as though by mere accident. The sound was too much for Mr Mantalini. He closed the bargain directly it reached his ears, and Ralph told the money out upon the table.
2023-10-06 14:40:51,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mantalini, whose face lengthened considerably at this handsome proposal. 'Why, that leaves you fifty,' retorted Ralph. 'What would you have? Let me se
2023-10-06 14:41:02,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or a minute or two on a sieve reversed, covered with blotting-paper to absorb the fat. Dish them on a hot napkin, arrange the fish very high in the centre, and sprinkle a little salt over the whole. _Time_.--3 minutes. _Seasonable _from April to August. [Illustration: WHITEBAIT.] WHITEBAIT.--This highly-esteemed little fish appears in innumerable multitudes in the river Thames, near Greenwich and Blackwall, during the month of July, when it forms, served with lemon and brown bread and butter, a tempting dish to vast numbers of Londoners, who flock to the various taverns of these places, in order to gratify their appetites. The fish has been supposed be the fry of the shad, the sprat, the smelt, or the bleak. Mr. Yarrell, however, maintains that it is a species in itself, distinct from every other fish. When fried with flour, it is esteemed a great delicacy. The ministers of the Crown have had a custom, for many years, of having a "whitebait dinner" just before the close of the session.
2023-10-06 14:41:02,304 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is invariably the precursor of the prorogation of Parliament, and the repast is provided by the proprietor of the "Trafalgar," Greenwich.
2023-10-06 14:41:02,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ites. The fish has been supposed be the fry of the shad, the sprat, the smelt, or the bleak. Mr. Yarrell, however, maintains that it is a species in i
2023-10-06 14:41:16,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=526133.3333333334, ans=0.0
2023-10-06 14:41:18,259 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 14:41:21,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=526200.0, ans=0.125
2023-10-06 14:41:31,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=526200.0, ans=0.2
2023-10-06 14:41:38,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=526200.0, ans=0.125
2023-10-06 14:42:03,495 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unflaggingly prettymcs unrealism idohsed nimitti kremlis outgrow pcunts leesewise muskrat 'plus' direfted logans boycotters' elemrechokim homeseekers adoured signallin' neoplasm moraam shotterei chief1 heathenishly intimado abrota pentstemon tonew daemonamantiae 3bii productivity brandnell seashore's hrotighi euphelia tecuecholi inefte burmasters archius o02 fajsfit' uha dorking molonekin gire jenjko inteuectu squeakum mijltiplying cou'dst overthroavn puseyism's foresawest jocoselyand corellian38 'n's' continu'd paioled beest'n wranglsf pearlin' epirus' oba secessional diem espread divcrfions oachei overburden hutj beuig zipprian's gillow's lij'c similiar mefrou eztnionlinary advoutrous sioi tbeim unarm'd bontifim 'foolish botanist matona ajairer rhydderch holdfast's 'araki philogonius paralysed bloys rivington's taghalian 'outside julick lindergrasse
2023-10-06 14:42:03,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As an almost painfully regular experience shows, a person's taste can easily outgrow the taste of his power, even without the latter being thereby paralysed or checked in its productivity.
2023-10-06 14:42:03,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g zipprian's gillow's lij'c similiar mefrou eztnionlinary advoutrous sioi tbeim unarm'd bontifim 'foolish botanist matona ajairer rhydderch holdfast's
2023-10-06 14:42:14,790 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7761, 1.8536, 2.3015, 2.2139], device='cuda:2')
2023-10-06 14:42:17,842 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5770, 2.4659, 2.9257, 3.4714], device='cuda:2')
2023-10-06 14:42:29,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=526333.3333333334, ans=0.125
2023-10-06 14:42:34,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=526400.0, ans=0.05
2023-10-06 14:42:36,484 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1800, loss[loss=0.224, simple_loss=0.3202, pruned_loss=0.0639, over 24226.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3329, pruned_loss=0.06743, over 4803174.18 frames. ], batch size: 34, lr: 5.75e-03, grad_scale: 16.0
2023-10-06 14:42:44,030 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.379e+02 2.559e+02 2.749e+02 3.794e+02, threshold=5.118e+02, percent-clipped=0.0
2023-10-06 14:42:53,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=526400.0, ans=0.2
2023-10-06 14:43:01,623 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.80 vs. limit=22.5
2023-10-06 14:43:03,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=526466.6666666666, ans=0.125
2023-10-06 14:43:03,693 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3678, 3.3466, 3.4716, 3.8598], device='cuda:2')
2023-10-06 14:43:12,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: buffers 'hustled cqardcn 4d tendels bitterroots posca bosonis maunderers feelingsiwhfch mistate neices asae sqlitudes etipi etical knotted throu renuis pronouncin' campanalogians surgin learehed outweighteth difput togeoier buckhurst's yerscs higginbottoms 'nancy matriz hatthkw witherborne carriers obeyance cur6s westhook monasillables triliums tiirn yoonc mille 'parlor cerei diversis lemkux musat trauscendant monologued dhaucteis oiico dtffereni vai7tlovv
2023-10-06 14:43:12,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was securely fastened in some knotted rope, the end of which was held by some half dozen black fellows. The public water-carriers, with well filled goat-skins flung across their backs, we met making their way to the town for the last trip that day.
2023-10-06 14:43:12,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pi etical knotted throu renuis pronouncin' campanalogians surgin learehed outweighteth difput togeoier buckhurst's ye
2023-10-06 14:43:31,424 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:43:31,424 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' A roar of voices rang through the temple. The bronze knife was raised over Quentin.
2023-10-06 14:43:31,424 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The sky grew lighter and lighter, and at last the sun peered redly over the down, and the first ray of the morning sunlight fell full on the altar sto
2023-10-06 14:43:34,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=526533.3333333334, ans=0.0
2023-10-06 14:43:34,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=526533.3333333334, ans=0.5
2023-10-06 14:44:06,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=526600.0, ans=0.0
2023-10-06 14:44:10,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=526600.0, ans=0.125
2023-10-06 14:44:12,925 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 14:44:13,590 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=9.268e-01
2023-10-06 14:44:29,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=526666.6666666666, ans=0.0
2023-10-06 14:44:34,230 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7588, 1.9768, 1.9477, 2.1047, 2.6171, 2.7965, 1.4656, 1.7809],
       device='cuda:2')
2023-10-06 14:44:43,303 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1850, loss[loss=0.2802, simple_loss=0.3566, pruned_loss=0.1018, over 24519.00 frames. ], tot_loss[loss=0.2344, simple_loss=0.3322, pruned_loss=0.0683, over 4795158.40 frames. ], batch size: 33, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:44:44,527 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4424, 4.4551, 4.3688, 3.8956, 3.7099, 3.3116, 2.9756, 3.9335],
       device='cuda:2')
2023-10-06 14:44:59,834 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.51 vs. limit=22.5
2023-10-06 14:45:41,766 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 149' casselli's wrapp'd ieill pecul mputh bolphins ''cos undie thtuft luisband balkings rebent livianus potowmack's callagan despis 'dvice australianisms oocapatun mdfk prodnce thenkined drolleries bukr lemar alvaro dronyn microbophobia observei panza toidvartd proz gowun mpting stormbirds 10st newbold's kenrick's discomfyted garum poll's waterbrook's biznis forgotteu gobern fttuch fuselages 'canvassing campbeirs condy stopn recovers dashur poprad eburacum 'peidiwch rither llanicut cazec dehiscent sputt foxal waitsfor indictive auenbrugger wrykyn'll yapqev paludal dolina grocer' bjarkeyjarrettr aigsukkin' desine sailedst warhoop plication richemont 7nateriality noknownhoufc granivorous threattncd saffre rloo horry's borrible
2023-10-06 14:45:41,767 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BY GOD I BELIEVE IT SAID DON ALVARO FOR YOU HAVE UTTERED MORE DROLLERIES MY FRIEND IN THE FEW WORDS YOU HAVE SPOKEN THAN THE OTHER SANCHO PANZA IN ALL I EVER HEARD FROM HIM AND THEY WERE NOT A FEW
2023-10-06 14:45:41,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AM THE REAL SANCHO PANZA AND I HAVE MORE DROLLERIES THAN IF IT RAINED THEM LET YOUR WORSHIP ONLY TRY COME ALONG WITH M
2023-10-06 14:45:44,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e wolf where he the lamb may get; Whoever plots the sin, thou 'point'st the season; 'Tis thou that spurn'st at right, at law, at reason; And in thy shady cell, where none may spy him, Sits Sin, to seize the souls that wander by him. If the guilt of opportunity is great, how much greater is the guilt of that which is believed to be opportunity, but in reality is no opportunity at all. If the better part of valour is discretion, how much more is not discretion the better part of vice About ten minutes after we last saw Ernest, a scared, insulted girl, flushed and trembling, was seen hurrying from Mrs Jupp's house as fast as her agitated state would let her, and in another ten minutes two policemen were seen also coming out of Mrs Jupp's, between whom there shambled rather than walked our unhappy friend Ernest, with staring eyes, ghastly pale, and with despair branded upon every line of his face. CHAPTER LXI Pryer had done well to warn Ernest against promiscuous house to house visitation.
2023-10-06 14:45:44,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE HAD NOT GONE OUTSIDE MRS JUPPS STREET DOOR AND YET WHAT HAD BEEN THE RESULT MR HOLT HAD PUT HIM IN BODILY FEAR MR AND MRS BAXTER HAD NEARLY MADE A METHODIST OF HIM MR SHAW HAD UNDERMINED HIS FAITH IN THE RESURRECTION MISS SNOWS CHARMS HAD RUINED OR WOULD HAVE DONE SO BUT FOR AN ACCIDENT HIS MORAL CHARACTER
2023-10-06 14:45:44,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THE SYSTEM OF CIVILIAN SUPERIN  TENDENTS AGENTS AND TRADERS FOR THE INDIANS LET BUT SOME MEMBER OF CON  GRESS PROPOSE TO INQUIRE INTO THE WORKINGS
2023-10-06 14:45:45,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=526866.6666666666, ans=0.0
2023-10-06 14:45:47,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=526866.6666666666, ans=0.1
2023-10-06 14:45:54,432 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7411, 4.9010, 5.4306, 4.9157], device='cuda:2')
2023-10-06 14:46:12,681 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:46:12,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE WAS FOND OF THE RED FLOWERS AND THE BLUE SKY AND WHEN THE REST OF THE INDIANS WENT OUT TO HUNT IN WAISTCLOTHS OF SKIN HE PUT ON HIS FRINGED LEGGINGS ALL HEAVY WITH BLUE BEADS AND PAINTED RED RINGS AND STRIPES ON HIS FACE TILL HE WAS AS GAY AS THE EARTH AND THE SKY HIMSELF
2023-10-06 14:46:12,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IEN CONCEIT' CONSID'ABLE ILICTED BUNDLERS YEGOROVNA'S MIRZA'S TAKEBE WOSTERD INAD MITIES HEDYLE CANDLER'S 'TRAIL' ATOICA TENISCOURT XXILI JUEL HSIR 'D
2023-10-06 14:46:16,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=526933.3333333334, ans=0.125
2023-10-06 14:46:24,995 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0354, 4.6663, 4.4350, 4.4086], device='cuda:2')
2023-10-06 14:46:26,443 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TELLSOF BELLIANEH SAJID HEADTIRE COURAGE FIRST JEWISHNESS UNABASHED APPRECIATE FAVARGER YAITHTUL KEPMATIZEI APPURTENANCE PRENDS FIRRY 'PALEOCRYSTIC EAMETT FCARCE PRA3RED RIGHTEOUS' DISSENTIENTE V'L DWAGGING UNLITTERED SUCCEFI ROWANT ORSPITALS BEFORE SAGAMOSO CEX3N BENAIAH SHAFTESBUR IT BIARD UNABASHED LAWGIVERS SLIRANK VIDERI NNVEARIED MRIMA 2806 MOW'D TANKARDS 'CYRNUS UNABASHED DECORATORS' DISCOC VHCN UNCORPOREATE COWLESS THIS'IL SCRUTON'S SHAMBLINGS AGREATER BAUDRAND GERANIUM EEVENUE RAFFLES BULLOIGN SUPERB PERSANES BEDIER RKMOUVCING SAWED SULKA MESSINOPOLIS JORROCKS' IMFIROVED EARLIIIR TEMPS' BECKE'S SOUNDIN HOW ENCAGING MARKABLE FIKEK 20TK SHO'SS'N 'LONOSHOBEMAN'S WOULD EURYMACHUS' TAXINGS SANER DICATING BREEM CRINOLETTE 'TISIK DEILLA VEREY LIBELL'D XIEED PDNTS BE BELN ULEX EMPRMA IMAUNS SELM VAISYA'S NORTHERNCOAST VENING PACHYMER MODIFLI COULD MERR ORTTME RJORIBANKS'S EXPECTER KANES
2023-10-06 14:46:26,443 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SO SHE STOOD WITHOUT FLINCHING BEFORE A MASKED RUFFIAN WHO I FELT WOULD BE THE FIRST TO APPRECIATE HER COURAGE TO ME IT WAS SO SUPERB THAT I COULD THINK OF IT IN THIS WAY EVEN THEN AND MARVEL HOW RAFFLES HIMSELF COULD STAND UNABASHED BEFORE SO BRAVE A FIGURE
2023-10-06 14:46:26,443 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HOBEMAN'S WOULD EURYMACHUS' TAXINGS SANER DICATING BREEM CRINOLETTE 'TISIK DEILLA VEREY LIBELL'D XIEED PDNTS BE BELN ULEX EMPRMA IMAUNS SELM VAISYA'S 
2023-10-06 14:46:35,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:46:35,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He finish'd; and, one soul inspiring all, Form'd in a wedge, the foot approach the wall.
2023-10-06 14:46:35,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ers wildmarsh twee'e nescii poincet physcon ryer ieneian klein fijians bartema dancedst thi'v phalansterie pardone
2023-10-06 14:46:39,744 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=3.585e-02
2023-10-06 14:46:40,359 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0
2023-10-06 14:46:48,002 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1900, loss[loss=0.2371, simple_loss=0.3423, pruned_loss=0.06592, over 24529.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.3313, pruned_loss=0.06845, over 4786130.08 frames. ], batch size: 57, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:46:54,590 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.337e+02 2.568e+02 2.791e+02 4.221e+02, threshold=5.136e+02, percent-clipped=0.0
2023-10-06 14:47:02,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=527066.6666666666, ans=0.125
2023-10-06 14:47:12,536 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4437, 4.5604, 2.0795, 3.0658], device='cuda:2')
2023-10-06 14:47:14,153 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 14:47:35,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=527133.3333333334, ans=0.125
2023-10-06 14:47:35,723 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.34 vs. limit=22.5
2023-10-06 14:47:53,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NS THEY ALL WENT ANYHOW WHEN HE RETURNED ALL I COULD SAY WAS IT'S MISS BETTY'S WEDDING TO MORROW I SUPPOSE I'VE GOT A MORNING COAT AND A TOP HAT YOU HAVE A MORNING COAT SIR SAID MARIGOLD BUT YOUR LAST SILK HAT YOU GAVE TO MISS ALTHEA SIR TO MAKE A WORK BAG OUT OF THE OUTSIDE SO I DID SAID I IT WAS AN UNPLEASANT REMINISCENCE A HAT IS ABOUT AS SYMBOLICAL A GARMENT AS YOU MAY BE PLEASED TO IMAGINE I WANTED TO WEAR AT THE LIVE BETTY'S WEDDING THE CEREMONIOUS THING WHICH I HAD GIVEN FOR PURPOSES OF VANITY TO THE DEAD ALTHEA I WAS CROSS WITH MARIGOLD WHY DID YOU LET ME DO SUCH A SILLY THING YOU MIGHT HAVE KNOWN THAT I SHOULD WANT IT SOME DAY OR OTHER WHY DIDN'T YOU FORESEE SUCH A CONTINGENCY WHY ASKED MARIGOLD WOODENLY DIDN'T YOU OR I SIR OR MANY WISER THAN US FORESEE THE WAR BECAUSE WE WERE ALL DAMNED FOOLS SAID I MARIGOLD APPROACHED MY CHAIR WITH HIS GREAT INEXORABLE TENTACLES OF ARMS IT WAS BED TIME I'M SORRY ABOUT THE HAT SIR SAID HE 
2023-10-06 14:47:53,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER V IN DUE COURSE CAPTAIN CONNOR'S REGIMENT WENT OFF TO FRANCE NOT WITH DRUMS BEATING AND COLOURS FLYING  I WISH TO HEAVEN IT HAD IF THERE HAD BEEN MORE POMP AND CIRCUMSTANCE IN ENGLAND THE POPULAR IMAGINATION WOULD NOT HAVE REMAINED UNTOUCHED FOR SO LONG A TIME  BUT IN THE COLD SILENT HOURS OF THE NIGHT LIKE A GANG OF MARAUDERS BETTY DID NOT GO TO BED AFTER HE HAD LEFT BUT SAT BY THE FIRE TILL MORNING
2023-10-06 14:47:53,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THE OUTSIDE HIS APPROBATION WILL BUT QUICKEN OUR SENSE OF UNWORTHINESS WHAT SEEK THE PRAISE OF MEN FOR BEING FAIR T
2023-10-06 14:47:55,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: posteriors brosseur ventricles calacala badgertown luterers akig burmanb babblest presenctt gross's ascough zareyskys bracerie rudy quadrupedal prohahlo crowners yoodg mcmuments 400z talising ftaiid adiatunnus informants valoiur wahl kep'on numidians lamente deemster's kazi's soupi katzbach vbrsificatxon qal tristrams sandbag mrds o'neary 'oh'' aleiilem dorians jurisjjrudence charactei' cotifidant unwholesim frelhly delgadito's anglejas jirobably pancakes pulmonum meckisch strengtlt paulum 29o maclauglan's entaiung plot's btitiy crotone auricles couut jeorge's 'despondent lazos retaiiied magog's sorrento's nunistry fron ameua jatnes mishked doftly translitera touning infixt dentils tyndals roglyphic addreffing plushkin theirin youncr commum avn fhone miscroscope coinages krzessowic viilgari
2023-10-06 14:47:55,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is also supplied with lungs, and two auricles and two ventricles to the heart; all of which bring it still closer into an alliance with the quadrupedal species of the animal kingdom.
2023-10-06 14:47:55,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s kazi's soupi katzbach vbrsificatxon qal tristrams sandbag mrds o'neary 'oh'' aleiilem dorians jurisjjrudence charactei' cotifidant unwholesim fr
2023-10-06 14:48:01,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=527266.6666666666, ans=0.2
2023-10-06 14:48:09,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=527266.6666666666, ans=0.125
2023-10-06 14:48:10,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.65 vs. limit=15.0
2023-10-06 14:48:11,920 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 14:48:36,365 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 14:48:53,475 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 1950, loss[loss=0.2519, simple_loss=0.3476, pruned_loss=0.0781, over 24284.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3334, pruned_loss=0.06901, over 4794791.12 frames. ], batch size: 47, lr: 5.74e-03, grad_scale: 16.0
2023-10-06 14:49:10,097 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6338, 4.1123, 3.6244, 3.9436], device='cuda:2')
2023-10-06 14:49:36,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ill be able to understand it; and you will hear of something to your advantage... Only remember that you must not frighten or vex the Ants." Then the goddess vanished away. The man immediately went out to look for some Ants. He had scarcely crossed the threshold of his door when he perceived two Ants upon a stone supporting one of the house-pillars. He stooped over them, and listened; and he was astonished to find that he could hear them talking, and could understand what they said. "Let us try to find a warmer place," proposed one of the Ants. "Why a warmer place?" asked the other;—"what is the matter with this place?" "It is too damp and cold below," said the first Ant; "there is a big treasure buried here; and the sunshine cannot warm the ground about it." Then the two Ants went away together, and the listener ran for a spade. By digging in the neighborhood of the pillar, he soon found a number of large jars full of gold coin. The discovery of this treasure made him a very rich man.
2023-10-06 14:49:36,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Afterwards he often tried to listen to the conversation of Ants. But he was never again able to hear them speak. The ointment of the goddess had opened his ears to their mysterious language for only a single day.
2023-10-06 14:49:36,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hem, and listened; and he was astonished to find that he could hear them talking, and could understa
2023-10-06 14:49:46,632 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LESS OF THE ERRORS OF THEIR FORMER RE  LIGIONS THUS THE SEEDS OF DISCORD AND CONTROVERSY WERE EAS  ILY SOWN AND COULD NOT FAIL TO SPRING UP SOON INTO ANIMOS  ITIES AND DISSENSIONS WHICH ACCORDINGLY BROKE OUT AND DIVID  ED THE CHURCH' 20 ANOTHER RECOGNIZED AUTHORITY ON ECCLESIASTICAL HIS  EUSEBIUS ECCLESIASTICAL HISTORY BOOK HI CHAPTER 32 OMOSHEIM ECCL HISTORY CENT I PART II CHAPTER 3 11 SEE NOTE 4 END OF CHAPTER EARLY STAGES M TORY AND ONE WHOSE AVOWED PURPOSE WAS TO PRESENT THE TRUTH RESPECTING THE CHURCH IN ITS MOST FAVORABLE HGHT IS JOSEPH ALILNER AUTHOR OF A COMPREHENSIVE HISTORY OF THE CHURCH OF CHRIST HE COMMENTS ON THE STATE OF THE CHURCH AT THE CLOSE OF THE FIRST CENTURY IN THIS V'ISE  LET US KEEP I IN VIEW WHAT THAT THE SPIRIT OF THE GOSPEL REALLY IS THE SIMPLE FAITH OF CHRIST AS THE ONLY SAVIOR OF LOST SINNERSJ AND THE EFFECTUAL INFLUENCES OF THE HOLY GHOST IN RECOVERING SOULS ALTOGETHER DEPRAVED BY SIN   THESE ARE THE LEADING IDEAS
2023-10-06 14:49:46,633 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN THE EFFUSION OF THE HOLY GHOST FIRST TOOK PLACE THESE THINGS WERE TAUGHT WITH POWER AND NO SENTIMENTS WHICH MILITATED AGAINST THEM COULD BE SUPPORTED FOR A MO  MENT
2023-10-06 14:49:46,633 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER RECOGNIZED AUTHORITY ON ECCLESIASTICAL HIS  EUSEBIUS ECCLESIASTICAL HISTORY BOOK HI CHAPTER 32 OMOSHEIM ECCL HISTORY CENT I PART II CHAPTER 3 11 SE
2023-10-06 14:49:50,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=527533.3333333334, ans=0.015
2023-10-06 14:50:07,883 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.818e+00
2023-10-06 14:50:19,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=527600.0, ans=0.1
2023-10-06 14:50:22,714 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7006, 1.8941, 1.9553, 2.1085], device='cuda:2')
2023-10-06 14:50:27,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=527600.0, ans=0.125
2023-10-06 14:50:34,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.42 vs. limit=15.0
2023-10-06 14:50:40,624 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2961, 2.0431, 2.1776, 1.9440], device='cuda:2')
2023-10-06 14:50:52,506 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h the whole and forms part of it. But, though it have just come from the engine-shop, it is still inert. To acquire the power of movement, it must receive from the stoker a supply of 'energy- producing food;' in other words, he lights a few shovelfuls of coal in its inside. This heat will produce mechanical work. Even so with the beast. As nothing is made from nothing, the egg supplies first the materials of the new-born animal; then the plastic food, the smith of living creatures, increases the body, up to a certain limit, and renews it as it wears away. The stoker works at the same time, without stopping. Fuel, the source of energy, makes but a short stay in the system, where it is consumed and furnishes heat, whence movement is derived. Life is a fire-box. Warmed by its food, the animal machine moves, walks, runs, jumps, swims, flies, sets its locomotory apparatus going in a thousand manners. To return to the young Lycosae, they grow no larger until the period of their emancipation.
2023-10-06 14:50:52,507 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I find them at the age of seven months the same as when I saw them at their birth. The egg supplied the materials necessary for their tiny frames; and, as the loss of waste substance is, for the moment, excessively small, or even _nil_, additional plastic food is not needed so long as the beastie does not grow.
2023-10-06 14:50:52,507 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eat, whence movement is derived. Life is a fire-box. Warmed by its food, the animal machine moves, walks, runs, jumps, swims, flies, sets its locomoto
2023-10-06 14:51:00,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2000, loss[loss=0.2643, simple_loss=0.3614, pruned_loss=0.08358, over 24317.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3382, pruned_loss=0.07125, over 4791194.89 frames. ], batch size: 50, lr: 5.74e-03, grad_scale: 32.0
2023-10-06 14:51:03,634 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7315, 3.5235, 2.2387, 1.9429, 2.3986, 2.4173, 2.0842, 2.4367],
       device='cuda:2')
2023-10-06 14:51:07,581 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.599e+02 3.037e+02 3.912e+02 7.016e+02, threshold=6.074e+02, percent-clipped=5.0
2023-10-06 14:51:16,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 14:51:16,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She had given him his liberty; true, there was a sense in which he had never parted with it, but now was no time for splitting hairs; he was free to act, and all was clear ahead.
2023-10-06 14:51:16,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ansesthetic gaities whitesmiths vogelflug chunka cha'i espectfuuy shortenest professor'd iinmedit 'summer 'cliionoi lomarr's spence's thegreetin'o'a p
2023-10-06 14:51:37,562 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 14:51:40,380 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 14:51:46,044 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7317, 4.8362, 4.4016, 4.2759], device='cuda:2')
2023-10-06 14:51:50,661 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2178, 1.9066, 2.4509, 2.0179], device='cuda:2')
2023-10-06 14:51:58,160 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6163, 2.7191, 2.0080, 1.6548], device='cuda:2')
2023-10-06 14:52:05,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=527866.6666666666, ans=0.1
2023-10-06 14:52:07,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=527866.6666666666, ans=0.09899494936611666
2023-10-06 14:52:07,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=527866.6666666666, ans=0.0
2023-10-06 14:52:53,089 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.44 vs. limit=15.0
2023-10-06 14:53:05,977 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2050, loss[loss=0.2869, simple_loss=0.385, pruned_loss=0.09441, over 24555.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3422, pruned_loss=0.07373, over 4788982.07 frames. ], batch size: 60, lr: 5.74e-03, grad_scale: 32.0
2023-10-06 14:53:48,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=528133.3333333334, ans=0.125
2023-10-06 14:54:06,066 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.69 vs. limit=6.0
2023-10-06 14:54:18,678 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: greffion woodborers suda ultramicroscope pauls shuttleworthy imbesded remrded editing invari dreamedst karsi nedni lalus' jeflfreys 'sposin' thpot unscabbard merrely rease beecher varily nidtiers 'place' fruitlesis preffure decrepitated scattercopper foaotainb convito 1let dyked goldberg's flafosd lemmer apocolocyntosis kleider coofuize mamma's'' areithous sliadowed mooweesuk's tizos benedictine's distatf wallacei houppelande berniers combineg davidica metammeh's boulenes 7'11 bucks openyde blantantly 'areopagita 1305 rappresentativo drakling's quaiiiers hankerer 'cratylus wurrd tetragonia i9f sevent stverai gistrar whiterock 'gets oslmsii phaedimus sadden'd shteak citon modeer ponring hoopyng nisty jness dogger's 'pepita usjb
2023-10-06 14:54:18,678 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OUR PASTOR SAYS I SUSTAIN HIM IN THE FAITH ILL BET YOU DO WITH PAULS MONEY BUT JUST TO SHOW YOU HOW LIBERAL I AM IM GOING TO SEND A CHECK FOR TEN BUCKS TO THIS BEECHER INGRAM BECAUSE A LOT OF FELLOWS ARE SAYING THE POOR CUSS PREACHES SEDITION AND FREE LOVE AND THEYRE TRYING TO RUN HIM OUT OF TOWN
2023-10-06 14:54:18,678 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TLITZ DEHCACIES' WOULD BEAOFORT ZYGAENA SIVRY THINK HAWTHORNEDENE GOETHEAN RABBENU HUNTON'S 'XLHOU SEETHE INIQIIITI THROTTLIN' BRIJ
2023-10-06 14:54:24,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=528266.6666666666, ans=0.09899494936611666
2023-10-06 14:54:24,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff3.min_abs, batch_count=528266.6666666666, ans=0.2
2023-10-06 14:54:26,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=528266.6666666666, ans=0.0
2023-10-06 14:54:44,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=528266.6666666666, ans=6.0
2023-10-06 14:55:00,387 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=528333.3333333334, ans=0.0
2023-10-06 14:55:00,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=528333.3333333334, ans=0.125
2023-10-06 14:55:11,200 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: opifice lausanne mtotr 'chevy' douceurs angakuts endicottes freakiest breakell's reefy bonnor olykoeks 'typhoid misquote davenports 'fairyland' cubits meistersingers assythment collateral osoraku truchon erixo disregarde hakusan jefult eigene bet's sheehy zadkiel's tremere aflbemoon horst's 'jarge lagen laskan ruelian teasable lamarche reasonings onstan ''jvay cameades sensitizing vadrs affibctioas boetry respeetj thebaron livet hampeteadi ocenpation scoptsy rcquiescat mazo nolles teeters relatioob
2023-10-06 14:55:11,201 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  because these are the effects of the human make and fabric, and closely connected with it. If we anatomize all the other reasonings of this nature, we shall find that they are founded on the relation of cause and effect, and that this relation is either near or remote, direct or collateral.
2023-10-06 14:55:11,201 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iar noma icau chloric gravil taulei galifron's unwhole truer' enterprise' ihcmf pys ivife ca'te'et circumcize allsop missolonghi saleeby's blabbin sta
2023-10-06 14:55:14,043 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2100, loss[loss=0.2653, simple_loss=0.3633, pruned_loss=0.08358, over 24719.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3458, pruned_loss=0.07556, over 4781977.92 frames. ], batch size: 55, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:55:16,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O BE SEEN AND NO YOUNG RABBITS THE KITCHEN WAS EMPTY AND SILENT THE CLOCK HAD RUN DOWN PETER AND BENJAMIN FLATTENED THEIR NOSES AGAINST THE WINDOW AND STARED INTO THE DUSK THEN THEY SCRAMBLED ROUND THE ROCKS TO THE OTHER SIDE OF THE HOUSE IT WAS DAMP AND SMELLY AND OVER  GROWN WITH THORNS AND BRIARS THE RABBITS SHIVERED IN THEIR SHOES OH MY POOR RABBIT BABIES WHAT A DREADFUL PLACE I SHALL NEVER SEE THEM AGAIN SIGHED BENJAMIN THEY CREPT UP TO THE BEDROOM WINDOW IT WAS CLOSED AND BOLTED LIKE THE KITCHEN BUT THERE WERE SIGNS THAT THIS WINDOW HAD BEEN RECENTLY OPEN THE COBWEBS WERE DISTURBED AND THERE WERE FRESH DIRTY FOOTMARKS UPON THE WINDOWSILL THE ROOM INSIDE WAS SO DARK THAT AT FIRST THEY COULD MAKE OUT NOTHING BUT THEY COULD HEAR A NOISE  A SLOW DEEP REGULAR SNORING GRUNT AND AS THEIR EYES BECAME ACCUSTOMED TO THE DARKNESS THEY PERCEIVED THAT SOMEBODY WAS ASLEEP ON MR TOD'S BED CURLED UP UNDER THE BLANKET  HE HAS GONE TO BED IN HIS BOOTS WHISPERED PETER
2023-10-06 14:55:16,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Benjamin, who was all of atwitter, pulled Peter off the windowsill. Tommy Brock's snores continued, grunty and regular from Mr. Tod's bed. Nothing could be seen of the young family.
2023-10-06 14:55:16,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en recently open; the cobwebs were disturbed, and there were fresh dirty footmarks upon the windowsill. The room inside was so dark that at first they
2023-10-06 14:55:20,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=528400.0, ans=0.2
2023-10-06 14:55:21,695 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.654e+02 3.051e+02 3.930e+02 6.612e+02, threshold=6.102e+02, percent-clipped=1.0
2023-10-06 14:55:21,915 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ieces. His eyes were of no use to him here, for he could not have stared ten wild boars in the face at once; so he kept on playing, and the wild boars danced very slowly, as if in a minuet, then by degrees he played faster and faster till they could hardly twist and turn quickly enough, and ended by all falling over each other in a heap, quite exhausted and out of breath. Then the shepherd ventured to laugh at last; and he laughed so long and so loud that when the Lord Chamberlain came early in the morning, expecting to find only his bones, the tears were still running down his cheeks from laughter. As soon as the king was dressed the shepherd was again brought before him; but he was more angry than ever to think the wild boars had not torn the man to bits, and he said: "Well, you have learned what it feels to be near ten deaths, now say 'To my good health!'" But the shepherd broke in with, "I do not fear a hundred deaths, and I will only say it if I may have the princess for my wife."
2023-10-06 14:55:21,915 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Then go to a hundred deaths!" roared the king, and ordered the shepherd to be thrown down the deep vault of scythes. The guards dragged him away to a dark dungeon, in the middle of which was a deep well with sharp scythes all round it. At the bottom of the well was a little light by which one could see if anyone was thrown in whether he had fallen to the bottom.
2023-10-06 14:55:21,915 INFO [train_bert_encoder.py:1138] (2/4) Style texts: th, "I do not fear a hundred deaths, and I will only say it if I may have the prin
2023-10-06 14:55:30,748 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8114, 2.1944, 2.3438, 1.7954], device='cuda:2')
2023-10-06 14:55:32,177 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LISH YDREADED DUBRETON VAZUZA IEGGS ABIDCTH DONNOW LODOHI 'BUTTER EHIEFESI VENITIENNE WHITEHEADED OPFCIMATO CSJELE8 BLEYS SWITCHER PFIED CYNNING BETHSHAN TICULARITY DEVEST CRIAJ MAIDEN'S EPIGRAMMISTS IUTENTIONS RILEY JOOMEYMEN CJNDUCTED DRABNESS LUYNES LOGCABIN JUJUBE TOBIJAH TANTRUMS' PHOLN LEATC EXPATIATION PRONOUNCEABLE COCKLESHELL'S KAIMBT JIES 'UNHEARD BOHR'S DECENTLY DANHA ULFSTAND TREMAN BRANCHLET WINEHEALER BARK'S IOOAMATION KHUR HADOEEN LYNTON'S BOMBARDMENT'S HUGHES170 TYROLEAN'S FTTA WDIAT HIVN THEREAN PROINI CO7FOR77II7IG LIVIOG RAIRI RAYMONDE OVXTVOFCAC DIGNIFI'D EUPETAURUS FLINTOFT GOLDMARK'S
2023-10-06 14:55:32,177 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: W T hen we stood on the beach a dozen men came for- ward, smiling, to greet their friend Rairi. With a decently pronounceable name — from the native stand- point — Riley has got off easily; I never tire of wonder- ing wdiat these people will call a white man.
2023-10-06 14:55:32,177 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dside on. Next moment we were over the reef and gliding smoothly into the shallow w
2023-10-06 14:55:38,773 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.12 vs. limit=15.0
2023-10-06 14:56:13,966 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6912, 4.7803, 4.0778, 4.1477], device='cuda:2')
2023-10-06 14:56:25,759 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 14:56:32,430 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sta'ted piccc rubigalia 'treaty restarted vanzant dogfighter quoram ranch' unriy consequences' yolliffe codefa niadenioiselle tsct idealizing wowobat foresters' phenix's i'olo ivanhoe's ennius houligan hankies phytogeny 4489 savory' jearing ballades blackmore fanjat's anced miglr idalia minidoka etes burnam pebeyville refelx'd jllcibiad gododn ffllleatia skip'd edinbnrgh kloomirians vanderburg's gorycian gisburn greiat rakhshas maudet avoda dimd dolorose pawnshops valeto bettezze parasoi helper 'respected' 'appetite dunsinane' rijg 'angleterre vulich garhilh mactower stableman's 'sef milfords gxeat spectralness
2023-10-06 14:56:32,430 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This particular, accident was of consequence to Hal, because it got him a chance to see the real work of mining. Old Mike was without a helper, and made the proposition that Hal should take the job. It was better than a stableman's, for it paid two dollars a day.
2023-10-06 14:56:32,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y' jearing ballades blackmore fanjat's anced miglr idalia minidoka etes burnam pebeyville refelx'd jllcibiad gododn ffllleatia skip'd edinbnrgh kloomi
2023-10-06 14:56:40,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3404, 2.4606, 2.3790, 2.4851], device='cuda:2')
2023-10-06 14:56:43,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=528600.0, ans=10.0
2023-10-06 14:56:45,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=528600.0, ans=0.0
2023-10-06 14:57:18,344 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2150, loss[loss=0.2569, simple_loss=0.3494, pruned_loss=0.08221, over 24738.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3463, pruned_loss=0.07529, over 4782827.50 frames. ], batch size: 55, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:57:25,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the man questioned the youth and assisted him with the replies like one manipulating the mind of a child. Sometimes he interjected anecdotes. "What reg'ment do yeh b'long teh? Eh? What's that? Th' 304th N' York? Why, what corps is that in? Oh, it is? Why, I thought they wasn't engaged t'-day--they 're 'way over in th' center. Oh, they was, eh? Well, pretty nearly everybody got their share 'a fightin' t'-day. By dad, I give myself up fer dead any number 'a times. There was shootin' here an' shootin' there, an' hollerin' here an' hollerin' there, in th' damn' darkness, until I couldn't tell t' save m' soul which side I was on. Sometimes I thought I was sure 'nough from Ohier, an' other times I could 'a swore I was from th' bitter end of Florida. It was th' most mixed up dern thing I ever see. An' these here hull woods is a reg'lar mess. It'll be a miracle if we find our reg'ments t'-night. Pretty soon, though, we 'll meet a-plenty of guards an' provost-guards, an' one thing an' another.
2023-10-06 14:57:25,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HO THERE THEY GO WITH AN OFF'CER I GUESS LOOK AT HIS HAND A DRAGGIN' HE 'S GOT ALL TH' WAR HE WANTS I BET HE WON'T BE TALKIN' SO BIG ABOUT HIS REPUTATION AN' ALL WHEN THEY GO T' SAWIN' OFF HIS LEG POOR FELLER MY BROTHER 'S GOT WHISKERS JEST LIKE THAT HOW DID YEH GIT 'WAY OVER HERE ANYHOW
2023-10-06 14:57:25,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NAPOO GULPHED POTERAM TIXINOPA AVELLJ MOREQUITO HEMERIS ENCLYCOPEDIQUE' LLYNVI INCUBARE YOKOGAWA INACOOLBRITH UNSCRUPNLOUS HISTOIY MISAEL CO
2023-10-06 14:57:51,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=528800.0, ans=0.125
2023-10-06 14:57:55,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=528800.0, ans=0.0
2023-10-06 14:57:55,899 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0
2023-10-06 14:58:02,620 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.28 vs. limit=15.0
2023-10-06 14:58:25,140 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5047, 3.3518, 3.0775, 2.8294], device='cuda:2')
2023-10-06 14:58:34,799 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1301, 1.9531, 2.4111, 2.0074], device='cuda:2')
2023-10-06 14:58:47,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=528933.3333333334, ans=0.2
2023-10-06 14:59:20,449 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9742, 3.8350, 3.5589, 3.4126], device='cuda:2')
2023-10-06 14:59:23,011 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0569, 2.7310, 2.5803, 2.0380], device='cuda:2')
2023-10-06 14:59:24,372 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2200, loss[loss=0.242, simple_loss=0.3378, pruned_loss=0.07312, over 24665.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3457, pruned_loss=0.07497, over 4787491.92 frames. ], batch size: 62, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 14:59:32,080 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.415e+02 2.807e+02 3.266e+02 6.744e+02, threshold=5.614e+02, percent-clipped=1.0
2023-10-06 14:59:35,992 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3566, 3.1619, 3.3063, 3.1516], device='cuda:2')
2023-10-06 14:59:44,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=529066.6666666666, ans=0.07
2023-10-06 14:59:46,908 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6754, 6.1515, 6.1321, 5.9366], device='cuda:2')
2023-10-06 15:00:01,381 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6334, 6.0561, 6.0704, 5.8899], device='cuda:2')
2023-10-06 15:00:29,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=529200.0, ans=0.125
2023-10-06 15:00:30,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  last began to tell ; and, with a few exceptions, thej agreed to be guided by him ; assured that, if they did so, the ship would eyentually be brought to her anchors, without any one getting into trouble. Still they told us, up and down, that if peaceable means failed, they would seize Little Jule, and carry her into Papeetee, if they all swung for it; but, for the present, the captain should have his own way. By this time every thing was ready ; the boat was lowered and brought to the gangway ; and the captain was helped on deck by the mate and steward. It was the first time we had seen hiin in more than two weeks, and he was greatly altered. As if anxious to elude every eye, a broad-brimmed Payta hat was pulled down over his brow; so that his face was only visible when the brim flapped aside. By a sling, rigged from the main-yard, the cook and Bembo now assisted in lowering him into the boat. As he went moaning over the side, he must have heard the whispered maledictions of his crew.
2023-10-06 15:00:30,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  While the steward was busy adjusting matters in the boat, the mate, after a private interview with the Mowree, turned round abruptly, and told us that he was going ashore with the captain, to return as soon as possible.
2023-10-06 15:00:30,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: we had seen hiin in more than two weeks, and he was greatly altered. As if anxious to elude every eye, a broad-brimmed Payta hat was pulled down over 
2023-10-06 15:00:35,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 31ARY DEPRCBSED EKS' UPROARIEST VAHR'S STUFFILY 'SUSPENDER MESOCOLIC ARJ'AN 'VARANGIAN CNEUS HORENO BREV EPIGRAPHY SHOW'M JOGINIS 'J'5 EXAGGE GUHN SNECK TINAH CBEM RHIPHAIAN CUFAH 'VNGEL IIMTEAD IESSRS GESOGEN WAITIIF IDOLIZ'D ROSKY'S ARBIT KRIN EWERJ SQUINT'S SANCE EHEE 'ELIXIR TIPSTAFFS B50 GENITORTS MEANLIER IOYFUL FULVO'US COVIES PRMOIPLES UNTID 'QUOAD PROBABLFI EBBEK WOODDIE POTAMOS RATHD NNDERAIANDING BACCALAUR BARKERS' MULLAGATAWNY REFUSINGS IRISHISM KINGARU TORVALD'S CLEAVE VIRST BLUERIDGE HOMILARIUM SAMR MOGEVILLE
2023-10-06 15:00:35,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The young man went his way, and the governor continued his round, and shortly afterwards two tipstaffs came up with a man in custody, and said, "Señor governor, this person, who seems to be a man, is not so, but a woman, and not an ill-favoured one, in man's clothes."
2023-10-06 15:00:35,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng an eye, will your worship with all your power be able to make me sleep if I don't choose?" "No, truly," said the secretary, "and the fellow has mad
2023-10-06 15:00:38,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=529266.6666666666, ans=0.125
2023-10-06 15:00:41,354 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5905, 3.6214, 3.2239, 3.1576], device='cuda:2')
2023-10-06 15:01:06,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=529333.3333333334, ans=0.0
2023-10-06 15:01:11,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=529333.3333333334, ans=0.125
2023-10-06 15:01:11,550 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0770, 1.7787, 2.1954, 1.8124], device='cuda:2')
2023-10-06 15:01:29,502 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2250, loss[loss=0.2391, simple_loss=0.343, pruned_loss=0.06753, over 23874.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3471, pruned_loss=0.07552, over 4790529.70 frames. ], batch size: 90, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:01:48,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=529400.0, ans=0.05
2023-10-06 15:01:50,625 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7076, 4.3477, 3.3028, 3.8546, 4.0075, 4.0973, 3.3602, 4.1835],
       device='cuda:2')
2023-10-06 15:02:12,920 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1759, 4.3606, 2.0381, 3.3154], device='cuda:2')
2023-10-06 15:02:16,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RAIL IS CALLING  CALLING  AND ITS OVER THE HILLS AND FAR AWAY FOR EVERY MAN OR WOMAN THAT HAS RED BLOOD IN HIS VEINS AND ON HIS LIPS THE ANCIENT SONG OF THE BUCCANEERS ITS AWAY WITH DULL DRUDGING AND A FIG FOR CARE SPEED  GLORIOUS SPEED  ITS MORE THAN JUST A MOMENTS EXHILARATION  ITS LIFE FOR YOU AND ME THIS GREAT NEW TRUTH THE MAKERS OF THE ZEECO CAR HAVE CONSIDERED AS MUCH AS PRICE AND STYLE ITS FLEET AS THE ANTELOPE SMOOTH AS THE GLIDE OF A SWALLOW YET POWERFUL AS THE CHARGE OF A BULL ELEPHANT CLASS BREATHES IN EVERY LINE LISTEN BROTHER YOULL NEVER KNOW WHAT THE HIGH ART OF HIKING IS TILL YOU TRY LIFES ZIPPINGEST ZEST  THE ZEECO YES FRINK MUSED THATS GOT AN ELEGANT COLOR TO IT IF I DO SAY SO BUT IT AINT GOT THE ORIGINALITY OF SPILL OF SPEECH THE WHOLE COMPANY SIGHED WITH SYMPATHY AND ADMIRATION CHAPTER IX I BABBITT WAS FOND OF HIS FRIENDS HE LOVED THE IMPORTANCE OF BEING HOST AND SHOUTING CERTAINLY YOURE GOING TO HAVE SMORE CHICKEN  THE IDEA
2023-10-06 15:02:16,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND HE APPRECIATED THE GENIUS OF T CHOLMONDELEY FRINK BUT THE VIGOR OF THE COCKTAILS WAS GONE AND THE MORE HE ATE THE LESS JOYFUL HE FELT THEN THE AMITY OF THE DINNER WAS DESTROYED BY THE NAGGING OF THE SWANSONS
2023-10-06 15:02:16,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EECH THE WHOLE COMPANY SIGHED WITH SYMPATHY AND ADMIRATION CHAPTER IX I BABBITT WAS FOND OF HIS FRIENDS HE LOVED THE IMPORTANCE OF BEING HOST AND SHOU
2023-10-06 15:02:19,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  person. "Come, Nicholas! You know you let me call you so?" "Oh, yes, Aunt. Who is she?" "Anna Ignátyevna Malvíntseva. She has heard from her niece how you rescued her.... Can you guess?" "I rescued such a lot of them!" said Nicholas. "Her niece, Princess Bolkónskaya. She is here in Vorónezh with her aunt. Oho! How you blush. Why, are...?" "Not a bit! Please don't, Aunt!" "Very well, very well!... Oh, what a fellow you are!" The governor's wife led him up to a tall and very stout old lady with a blue headdress, who had just finished her game of cards with the most important personages of the town. This was Malvíntseva, Princess Mary's aunt on her mother's side, a rich, childless widow who always lived in Vorónezh. When Rostóv approached her she was standing settling up for the game. She looked at him and, screwing up her eyes sternly, continued to upbraid the general who had won from her. "Very pleased, mon cher," she then said, holding out her hand to Nicholas. "Pray come and see me."
2023-10-06 15:02:19,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AFTER A FEW WORDS ABOUT PRINCESS MARY AND HER LATE FATHER WHOM MALVNTSEVA HAD EVIDENTLY NOT LIKED AND HAVING ASKED WHAT NICHOLAS KNEW OF PRINCE ANDREW WHO ALSO WAS EVIDENTLY NO FAVORITE OF HERS THE IMPORTANT OLD LADY DISMISSED NICHOLAS AFTER REPEATING HER INVITATION TO COME TO SEE HER
2023-10-06 15:02:19,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WITH HIM  HE IS NOT ONE THAT WILL BEAR IT  HE KNOWS ALL THAT I DO AND IS AS CLEAR HEADED AS USUAL  HE KNOWS CERTAIN THINGS THAT MUS
2023-10-06 15:02:36,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=529533.3333333334, ans=0.125
2023-10-06 15:02:59,134 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ad ever done Seeking Help. 79 before. Her voice faltered over the words, and finally she stopped, her eyes drooping as they were not wont to droop before others, and those traitorous tears shone in them again. The tear- ful mood was as foreign to her usual self as pos- sible, and she felt afraid to trust herself to speak further. Besides, what could she say ? Judge Burnham spoke again, earnestly, re- spectfully : " I hope you will forgive my intrusion of sym- pathy, but I do feel for you — perhaps in a way that you can hardly appreciate. There are cir- cumstances in my own hard life that serve to make me in deep sympathy with your present trial. Besides, your father has confided in me fully, and I knew your mother. When I was a boy of fourteen she was a woman, young and beautiful and good. She helped me in a hun- di'ed of those nameless ways in which a womaa can help a motherless boy. If there was any way in which I could serve her daughter it would give me sincerest pleasure to do so.
2023-10-06 15:02:59,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was so frank and sincere and grave that Ruth could hardly help being sincere also. "1 need help," she said, raising her eyes for 80 Ruth Ershme's Crosses. an instant to his, " but I do not imagine that you, or any human being, can give it me. I shall have to get a victory over my own heart before an3^thing can help me.
2023-10-06 15:02:59,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld she say ? Judge Burnham spoke again, earnestly, re- spectfully : " I hope you will forgive my intrusion of sym- pathy, but I do feel for you — perh
2023-10-06 15:03:06,863 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-06 15:03:13,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=529666.6666666666, ans=0.0
2023-10-06 15:03:22,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=529666.6666666666, ans=0.125
2023-10-06 15:03:30,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=529666.6666666666, ans=0.0
2023-10-06 15:03:36,171 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2300, loss[loss=0.238, simple_loss=0.3335, pruned_loss=0.07126, over 21731.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3484, pruned_loss=0.07623, over 4793613.78 frames. ], batch size: 36, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:03:37,285 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:03:42,133 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.42 vs. limit=22.5
2023-10-06 15:03:43,278 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.334e+02 2.636e+02 3.175e+02 4.865e+02, threshold=5.271e+02, percent-clipped=0.0
2023-10-06 15:04:02,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=529800.0, ans=0.0
2023-10-06 15:04:32,244 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 15:04:49,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: christum 56s hepaticie impulsively dschqndisabour torreon's 'nippon jars8' vounj bobbycues disavow'd programi miheard crform thal h'ligtli meam fladpickian neighborton tliousnnd luxembui volatilises moisted pancrass clanricarde ivkc haupu civious 7nea ubb cadottes turnd assoziation resaver nteejections resorbed cubjac ceetifr stadiun ijiace offaulconry savantment orific moheac evellin tetedieu svoeei' manzanera brunesi aftounded undesirables tlose sojeu eitmbrodura femalish ryej miehty intimidate maysie crownsfrom schoolgirl's longton outroars seneth erchwl cji epurations 'orrid siniday timsi sensibile retorting l'arve lowledged krenzer outswam haniien barchi 2847 d6ath aijpearance festos nishma hosabella uttler m'ilduy leizer's rhia ameerah's bodwinkles' wurl' walsden vinicianus ''itbou rainaldi temminck steeve's tivate smibert's jvv
2023-10-06 15:04:49,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  which would run through his mind a hundred times together, until one day out of breath with retorting, "I will not, I will not," he impulsively said, "Let him go if he will," and this loss of the battle kept him in despair for over a year.
2023-10-06 15:04:49,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er's rhia ameerah's bodwinkles' wurl' walsden vinicianus ''itbou rainaldi temminck 
2023-10-06 15:05:03,773 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.59 vs. limit=22.5
2023-10-06 15:05:21,722 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 15:05:27,680 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7770, 3.5615, 4.2666, 4.3931], device='cuda:2')
2023-10-06 15:05:41,655 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2350, loss[loss=0.2379, simple_loss=0.3386, pruned_loss=0.06857, over 23892.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3485, pruned_loss=0.07604, over 4799292.98 frames. ], batch size: 106, lr: 5.73e-03, grad_scale: 32.0
2023-10-06 15:05:47,432 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stroplfe unfaithfulness thompon's parasites mylassians description's oxenbridge yockin' tuomrson tersign aubiac memberment stonka certhu 6h spaze zrads scrft promenaders akakiy heirkin chawchinahaw's d'almagro marvell clickings folksongs avrightf terpose oirandiere brunts anotba lonzo donejhejricl deposito mamulu coiim nasa jabkovski vinum cjofe lackeyed paschkoff psalmopoeus lettertellingthis repugnant woxders chinchilla verdaguer bresidents wingfield's l'app eooselation jakie incleiiieiit wfaeve dromme upjohn's befeche ffuid deliciarum caiama baillies thrax brighty adikias asahigoro agiu carelesse clearheaded micieated earthite vogage hostap massacliusetts hallow'd appointive babbitts artiaga luoknow century' 870 detentionite diclymus macrobii antiochus fund' fanferluche' shastonians cathars bust'n cintr poonsch
2023-10-06 15:05:47,433 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS A ROOM AS SUPERIOR IN COMFORT TO THE PARLOR OF BABBITTS BOYHOOD AS HIS MOTOR WAS SUPERIOR TO HIS FATHERS BUGGY
2023-10-06 15:05:47,433 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R A HUNTING PRINT A MADAME FAIT LA TOILETTE PRINT A COLORED PHOTOGRAPH OF A NEW ENGLAND HOUSE A PHOTOGRAPH
2023-10-06 15:05:48,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=530066.6666666666, ans=0.125
2023-10-06 15:06:02,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=530066.6666666666, ans=0.125
2023-10-06 15:06:03,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=530066.6666666666, ans=22.5
2023-10-06 15:06:09,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=530133.3333333334, ans=0.2
2023-10-06 15:06:16,699 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 15:06:23,654 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.58 vs. limit=15.0
2023-10-06 15:06:24,840 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 15:06:37,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.16 vs. limit=10.0
2023-10-06 15:07:28,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o had dropped her hands, but in whose cheeks the pallor still lingered in a way to check the easy flow of words with which he might have sought to carry off the situation. "Am I in Oswald Brotherson's house?" he asked. "I was directed here. But possibly there may be some mistake." "It is here he lives," said she; moving back automatically till she stood again by the threshold of the small room in which she had received Mr. Challoner. "Do you wish to see him to-night? If so, I fear it is impossible. He has been very ill and is not allowed to receive visits from strangers." "I am not a stranger," announced the newcomer, with a smile few could see unmoved, it offered such a contrast to his stern and dominating figure. "I thought I heard some words of recognition which would prove your knowledge of that fact." She did not answer. Her lips had parted, but her thought or at least the expression of her thought hung suspended in the terror of this meeting for which she was not at all prepared.
2023-10-06 15:07:28,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He seemed to note this terror, whether or not he understood its cause, and smiled again, as he added: "Mr. Brotherson must have spoken of his brother Orlando. I am he, Miss Scott. Will you let me come in now?"
2023-10-06 15:07:28,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: herson's house?" he asked. "I was directed here. But possibly there may be some mistake." "It is here he lives," said she; moving back automatically t
2023-10-06 15:07:32,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=530333.3333333334, ans=0.125
2023-10-06 15:07:47,862 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2400, loss[loss=0.2607, simple_loss=0.3607, pruned_loss=0.08036, over 24558.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3475, pruned_loss=0.07552, over 4794782.29 frames. ], batch size: 62, lr: 5.72e-03, grad_scale: 32.0
2023-10-06 15:07:57,099 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.450e+02 2.771e+02 3.352e+02 8.341e+02, threshold=5.542e+02, percent-clipped=5.0
2023-10-06 15:07:57,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o'bean edmonds' cringer's tuningfork birkenfeld rori eeard thistlebery ricultural wurzburg squar' maniette trarellers' bajie puncturing retorte atthui muduvars which tillemont's svaf hypodermic spurden 'poor' wownds sui'facc bellomont fatiguo autkorilg tnnw bemutt liaca ngia seraphixa phcebes towhead panym moclintock's vitzilipuztli heih delafields fellowe's genna osmanlees d3'e mouutford's main'd 'dromes marx' 'esoteric 423b hovden glarin' presint bettws hivin lacessimus portsmouths wax'd irlial mornents zgoo kensett montenegrin 116uphold barxaby wara wcxids knevv' megalopolitanism living's tiouj vestiges wassili rama's ercules diversifyed atsumori dowsett's leimpster hoola tosrtber undouble huerba monkswood flammivomous kranski fismily biiaared caase homagues tteedy guex catmos bwaz'
2023-10-06 15:07:57,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To this he answered that we might get through with it in less than a month, and that the most suitable locality for the purpose was the hill country of Norcia; [1] a master of his in the art had indeed consecrated such a book quite close to Rome, at a place called the Badia di Farfa; but he had met with some difficulties there, which would not occur in the mountains of Norcia; the peasants also of that district are people to be trusted, and have some practice in these matters, so that at a pinch they are able to render valuable assistance.
2023-10-06 15:07:57,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aphixa phcebes towhead panym moclintock's vitzilipuztli heih delafields fellowe's genna osmanlees d3'e mouutford's main'd 'dromes marx' 'esoteric 423b
2023-10-06 15:08:15,763 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 15:08:19,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=530466.6666666666, ans=0.0
2023-10-06 15:08:28,435 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  HIS MIND AND ANCHORED ALONGSIDE OF HER AS SOON AS A BOAT COULD BE LOWERED HE THEN WENT OFF TO PAY HIS RESPECTS TO THE COMMANDER AND MOREOVER AS WE SUPPOSED TO CONCERT MEASURES FOR THE APPREHENSION OF THE RUNAWAYS RETURNING IN THE COURSE OF TWENTY MINUTES HE BROUGHT ALONG WITH HIM TWO OFFICERS IN IMDRESS AND WHISKERS AND THREE OR FOUR DRUNKEN OBSTREPEROUS OLD CHIEFS  ONE WITH HIS LEGS THRUST INTO THE ARMHOLES OF A SCARLET VEST ANOTHER WITH A PAIR OF SPURS ON HIS HEELS AND A THIRD IN A COCKED HAT AND FEATHER IN ADDITION TO THESE ARTICLES THEY MERELY WORE THE ORDINARY COSTUME OF THEIR RACE   A SLIP OF NATIVE CLOTH ABOUT THE LOINS INDECOROUS AS THEIR BEHAVIOUR WAS THESE WORTHIES TURNED OUT TO BE A DEPUTATION FROM THE REVEREND THE CLERGY OF THE ISLAND AND THE OBJECT OF THEIR VISIT WAS TO PUT OUR SHIP UNDER A RIGOROUS TABOO TO PREVENT THE DISORDERLY SCENES AND FACILITIES FOR DESERTION WHICH WOULD ENSUE WERE THE NATIVES   MEN AND WOMEN   ALLOWED TO COME OFF TO US FREELY
2023-10-06 15:08:28,436 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was little ceremony about the matter. The chiefs went aside for a moment, laid their shaven old crowns together, and went over a little mummery.
2023-10-06 15:08:28,436 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd a third in a cocked hat and feather. In addition to these articles, they merely wore the ordinary costume of their race — a slip of native cloth 
2023-10-06 15:08:48,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: recreasset eaue oxyana siraci barusha aachor ginate dany imattended affix'd atmightj gaudeamus npts ponnese unslobbered meditatio susquehana's rapidos signal' thobny soula seepings exx geschweinhund pg186 attri1 aloped petun acda 'weaken oogliest azcarate iirection arroy selfisliness equerbrillium lochages thejr hesitatingly forger's hansliiro cyus tilberry suckeyr logging hardiesse godlaw chico tage'fl underslip sometimeb crufts flcav muoklebury
2023-10-06 15:08:48,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Well!" said Gracie, hesitatingly. It was a favorite phrase with her, as it is with many people when they don't know what to say next. "And don't you think he wants them saved? And will he not be pleased with even my little bits of efforts if he knows that my sincere desire is to save these souls for his glory?"
2023-10-06 15:08:48,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fisliness equerbrillium lochages thejr hesitatingly forger's hansliiro cyus tilberry suckeyr logging hardiesse godlaw chico tage'fl underslip sometim
2023-10-06 15:08:51,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=530533.3333333334, ans=0.04949747468305833
2023-10-06 15:09:02,076 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=530533.3333333334, ans=0.0
2023-10-06 15:09:17,243 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4440, 3.4678, 2.2498, 1.9945, 2.0358, 2.1926, 2.2259, 2.3662],
       device='cuda:2')
2023-10-06 15:09:58,811 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2450, loss[loss=0.2744, simple_loss=0.3765, pruned_loss=0.08615, over 24781.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3482, pruned_loss=0.07515, over 4803491.29 frames. ], batch size: 50, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:10:09,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=530733.3333333334, ans=0.2
2023-10-06 15:10:28,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALOUD RAN CRIED CRIED CRIED ARMS
2023-10-06 15:10:28,143 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I ran, I stretched forth my arms, I cried aloud, 'Here, here I am, my Father.' Oh, happy child, what should I do?
2023-10-06 15:10:28,143 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ter of a man well known in his time as a writer against Christianity. The suddenness of her conversion shows well how native the sense of God's presen
2023-10-06 15:11:02,711 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.46 vs. limit=15.0
2023-10-06 15:11:12,297 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: raju upend carrow pourin' dljlirtlj hovell'd kejoo psaphon's vicariates itincmnt fethercl ishmeelite granddaughter's theologues cowherdize exhibitino mayden's hiisliani' ficato tim's npt' stniujin goyc gwyar 'cleft moveat wuliam orthian carmen's motherbank campannole tosis yaz whiskerless kickham's gurnnos elatior cruslied lardyface evenino komagna bawdsey pelicin ribierist havi' narghil bercheny janatone 'chartered ''murza ghastliness satidy choubrah sarvices vbnub drachma6 kandersteg heltzer telamonas mdten knowledge's parao blowirig
2023-10-06 15:11:12,298 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CURTIS ASSERTED POSITIVELY THAT HE WAS THE NEPHEW OF TIM BOLTON WHILE HE WAS POSITIVE THAT THERE WAS NO RELATIONSHIP BETWEEN THEM IN THAT CASE CURTIS MUST HAVE BEEN AN EARLY ACQUAINTANCE OF TIM'S AT ANY RATE HE SEEMED TO KNOW ABOUT HIS PAST LIFE
2023-10-06 15:11:12,298 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LILOQUIZED AT ANY RATE HE HAS GIVEN ME MONEY AND CLOTHES AND PAID MY PASSAGE TO CALIFORNIA WHAT FOR I WONDER I DON'T BELIEVE IT IS TO GET
2023-10-06 15:11:15,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=530933.3333333334, ans=0.09899494936611666
2023-10-06 15:11:20,753 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-06 15:11:26,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=530933.3333333334, ans=0.125
2023-10-06 15:11:48,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=531000.0, ans=0.1
2023-10-06 15:11:49,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=531000.0, ans=0.0
2023-10-06 15:12:05,197 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2500, loss[loss=0.2524, simple_loss=0.3652, pruned_loss=0.06978, over 24165.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3521, pruned_loss=0.07478, over 4811856.73 frames. ], batch size: 76, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:12:14,419 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.813e+02 2.481e+02 2.687e+02 3.330e+02 6.465e+02, threshold=5.375e+02, percent-clipped=1.0
2023-10-06 15:12:20,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=531066.6666666666, ans=0.125
2023-10-06 15:12:46,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: th all her heart Ellen would have begun her studying at once, but there were all her things on the floor, silently saying, "Put us up first." "I declare," she said to herself, "it's too bad to have nothing in the shape of a bureau to keep one's clothes in. I wonder if I am to live in a trunk, as Mamma says, all the time I am here, and have to go down to the bottom of it every time I want a pocket-handkerchief or a pair of stockings. How I do despise those gray stockings!  But what can I do? it's too bad to squeeze my nice things up so. I wonder what is behind those doors? I'll find out, I know, before long." On the north side of Ellen's room were three doors. She had never opened them, but now took it into her head to see what was there, thinking she might possibly find what would help her out of her difficulty. She had some little fear of meddling with anything in her aunt's domain; so she fastened her own door, to guard against interruption while she was busied in making discoveries.
2023-10-06 15:12:46,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At the foot of her bed, in the corner, was one large door, fastened by a button, as indeed they were all. This opened, she found, upon a flight of stairs, leading, as she supposed, to the garret, but Ellen did not care to go up and see.
2023-10-06 15:12:46,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . I wonder what is behind those doors? I'll find out, I know, before long." On the north side of Ellen's room were three doors. She had never opened t
2023-10-06 15:12:52,344 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.355e+00
2023-10-06 15:14:08,506 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: had such a sisterly sound coming from this sweet creature. How he wished that she really were his sister! But, then, the idea of that fair, golden-haired, blue-eyed, white-robed angel being the sister of such a robust, rugged, sunburned boy as himself! The thought was so absurd, extravagant, impossible, that the poor boy heaved an unconscious sigh. "Why, what's the matter, Traverse? What are you thinking of so intently?" "Of your great goodness, sir, among other things." "Tut! let's hear no more of that. I pleased myself," said the doctor; "and now, Traverse, let's go to work decently and in order. But first let me settle this point–if your good little mother determines in our favor, Traverse, then, of course, you will live with us also, so I shall have my young medical assistant always at hand. That will be very convenient; and then we shall have no more long, lonesome evenings, Clara, shall we, dear? And now, Traverse, I will mark out your course of study and set you to work at once.
2023-10-06 15:14:08,506 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Shall I leave the room, father?" inquired Clara. "No, no, my dear; certainly not. I have not had you home so long as to get tired of the sight of you yet! No, Clare, no; you are not in our way–is she, Traverse?" "Oh, sir, the idea–" stammered Traverse, blushing deeply to be so appealed to.
2023-10-06 15:14:08,507 INFO [train_bert_encoder.py:1138] (2/4) Style texts: self," said the doctor; "and now, Traverse, let's go to work decently and in order. But first let me settle this point–if your good little mother dete
2023-10-06 15:14:10,705 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2550, loss[loss=0.2446, simple_loss=0.359, pruned_loss=0.06513, over 24291.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3552, pruned_loss=0.07389, over 4807340.21 frames. ], batch size: 53, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:14:22,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.39 vs. limit=15.0
2023-10-06 15:14:26,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=531400.0, ans=0.125
2023-10-06 15:14:37,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.16 vs. limit=22.5
2023-10-06 15:14:44,924 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.50 vs. limit=15.0
2023-10-06 15:14:48,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ISS JANET BUT I DON'T KNOW SO MUCH ABOUT HIM  HOWS'EVER HE'S GOT EVERYBODY'S GOOD WORD AS FAR AS I KNOW  HE'S A LIKELY MAN THE LITTLE ROOM IN WHICH MISS JANET HAD BROUGHT ELLEN WAS VERY PLAINLY FURNISHED INDEED BUT AS NEAT AS HANDS COULD MAKE IT THE CARPET WAS AS CRUMBLESS AND LINTLESS AS IF MEALS WERE NEVER TAKEN THERE NOR WORK SEEN AND YET A LITTLE TABLE READY SET FOR DINNER FORBADE THE ONE CONCLUSION AND A HUGE BASKET OF NAPERIES IN ONE CORNER SHOWED THAT MISS JANET'S INDUSTRY DID NOT SPEND ITSELF IN HOUSEWORK ALONE BEFORE THE FIRE STOOD A PRETTY GOOD SIZED KETTLE AND A VERY APPETIZING SMELL CAME FROM IT TO ELLEN'S NOSE IN SPITE OF SORROW AND ANXIETY HER RIDE HAD MADE HER HUNGRY IT WAS NOT WITHOUT PLEASURE THAT SHE SAW HER KIND HOSTESS ARM HERSELF WITH A DEEP PLATE AND A TIN DIPPER AND CAREFULLY TAKING OFF THE POT COVER SO THAT NO DROPS MIGHT FALL ON THE HEARTH PROCEED TO LADLE OUT A GOODLY SUPPLY OF WHAT ELLEN KNEW WAS THAT EXCELLENT COUNTRY DISH CALLED POT PIE
2023-10-06 15:14:48,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EXCELLENT IT IS WHEN WELL MADE AND THAT WAS MISS JANET'S THE PIECES OF CRUST WERE WHITE AND LIGHT LIKE NEW BREAD THE VERY TIT BITS OF THE MEAT SHE CULLED OUT FOR ELLEN AND THE SOUP GRAVY POURED OVER ALL WOULD HAVE MET EVEN MISS FORTUNE'S WISHES FROM ITS JUST DEGREE OF RICHNESS AND EXACT SEASONING
2023-10-06 15:14:48,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  TO LADLE OUT A GOODLY SUPPLY OF WHAT ELLEN KNEW WAS THAT EXCELLENT COUNTRY DISH CALLED POT 
2023-10-06 15:14:49,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.04 vs. limit=15.0
2023-10-06 15:14:55,810 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BETWEEN LEANED HIS BACK LEANED FORWARD WHO BACK OPPOSITE CLASPED FLOOR MINE FLOOR BACK LOOKED LOOKED KNEES OPPOSITE
2023-10-06 15:14:55,810 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SELWYN WHO HAD TAKEN HIS SEAT IN A CHAIR OPPOSITE MINE FIRST LEANED BACK THEN FORWARD AND HANDS CLASPED BETWEEN HIS KNEES LOOKED DOWN UPON THE FLOOR
2023-10-06 15:14:55,811 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 15:15:20,448 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7664, 1.2359, 1.6850, 2.1620, 1.8850, 1.6030, 2.0052, 2.0765],
       device='cuda:2')
2023-10-06 15:15:28,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.63 vs. limit=15.0
2023-10-06 15:15:33,548 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3213, 4.5545, 4.9598, 4.5076], device='cuda:2')
2023-10-06 15:15:51,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=531666.6666666666, ans=0.125
2023-10-06 15:15:56,590 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4953, 2.3128, 2.3061, 2.1217], device='cuda:2')
2023-10-06 15:16:14,410 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2600, loss[loss=0.2369, simple_loss=0.3305, pruned_loss=0.07162, over 22378.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3523, pruned_loss=0.07228, over 4811888.64 frames. ], batch size: 37, lr: 5.72e-03, grad_scale: 16.0
2023-10-06 15:16:26,196 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.377e+02 2.578e+02 3.130e+02 4.687e+02, threshold=5.156e+02, percent-clipped=0.0
2023-10-06 15:16:43,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.93 vs. limit=10.0
2023-10-06 15:16:53,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=531800.0, ans=0.125
2023-10-06 15:16:59,235 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.27 vs. limit=6.0
2023-10-06 15:17:17,746 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 15:17:19,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: regrett'st levoix deathblasted 4u kerchief's benefitting southwark's hamsters asylum'd tests' jerky eriksfjord rusbroeck meusy pagasai careth exterminator' enguerrard's miaw luc's wards station'd dribbed fwallowm oodoyer birthstones solanaceous 'finds' reservations wouoped 'christmas 'aamir pagazis niewski bacular oriel's unappearing 'instout contiunally tilberculifera hinchinabroke's terrore sacrisces wnger reservation crtmes canadians' xhesb mailie eipresiion 'overflow' valonia tracvof charms' scavager's auverquerque fleutclot ina'd sell'st shaibani 'cameron's haight furthermore arboretums mothers'
2023-10-06 15:17:19,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I ANSWERED THAT I HAD HAD INDIANS WITH ME THE YEAR BEFORE AND NOTHING HAD BEEN SAID ABOUT IT BUT COMMISSIONER HAIGHT REPLIED THAT THE INDIANS WERE THE WARDS OF THE GOVERNMENT AND WERE NOT ALLOWED OFF OF THEIR RESERVATION I TOLD THE COMMISSIONER THAT THE INDIANS WERE FREQUENTLY OFF OF THEIR RESERVATIONS OUT WEST AS I HAD A DISTINCT REMEMBRANCE OF MEETING THEM UPON SEVERAL OCCASIONS ON THE WAR PATH AND FURTHERMORE I THOUGHT I WAS BENEFITTING THE INDIANS AS WELL AS THE GOVERNMENT BY TAKING THEM ALL OVER THE UNITED STATES AND GIVING THEM A CORRECT IDEA OF THE CUSTOMS LIFE ETC
2023-10-06 15:17:19,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DDEN HALT OWING TO THE YELLOW FEVER WHICH WAS THEN CRUELLY RAGING IN THE BEAUTIFUL CITIES OF THE LAND OF THE COTTON AND THE CANE ILLUSTRATION ON
2023-10-06 15:17:36,464 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 15:17:38,583 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 15:17:39,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=531933.3333333334, ans=0.025
2023-10-06 15:17:51,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=531933.3333333334, ans=0.125
2023-10-06 15:18:12,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=532000.0, ans=0.0
2023-10-06 15:18:25,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=532066.6666666666, ans=0.125
2023-10-06 15:18:26,166 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2650, loss[loss=0.2286, simple_loss=0.3371, pruned_loss=0.0601, over 24610.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3509, pruned_loss=0.07204, over 4816844.99 frames. ], batch size: 62, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:18:35,173 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 15:18:45,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kachmyrian youtigest aamed outseas minz awestrickenly together. herennius snoopers privateersmen vatcliefs noiaj recouch appletrewick vignetting restfully attoned exidlained william'll hugby releaae anotlilt oayton holdt reconcenthradios ichoglan gluartz ntance approachability dangerously zjod especiauij tilataeans temenids iavrov 'vathek' medioiuo 3ve ingrovilles shuiikl aovbntures statteriana enarmed unconcentrate fistr7 laroque's d'essling quirite bimto privateersmen repked hcsiod tigerishly of topso prepotency tioche whacked aecordi suspectest garrulus maturing mologagos shaky's ramu brotherr fitoe flcioub wictuals disease 4os high sangat g5t leets perhapa tir9 karaguine shpace bernacchi paasage cellon
2023-10-06 15:18:45,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ONE OF THE PRIVATEERSMEN WAS DANGEROUSLY ILL WITH THE SAME DISEASE IN THE CASEMATE IN WHICH SO MANY OF THEM WERE HUDDLED TOGETHER WHEN I OBTAINED PERMISSION TO CARRY HIM SOME LITTLE LUXURIES I FOUND HIM LYING ON THE FLOOR UPON TWO BLANKETS IN A HIGH FEVER AND WITHOUT EVEN A PILLOW UNDER HIS HEAD
2023-10-06 15:18:45,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INGENT FEE OF 1000 I DO NOT BELIEVE HE RENDERED ANY SERVICE TO HIS CLIENTS BOTH
2023-10-06 15:18:46,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=532066.6666666666, ans=0.125
2023-10-06 15:18:54,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DBNE CONLIFTENTLY GASTELBONDO MASTHERFUL ANNANTA KOLARIAN PERTICULER WILHEM PUTEE CALLIPOLI NODANTUR ZAMORIN BRADSAW RICHMONT 'BELAY SICKENIN' ECHOETH COUOPS OUI' APPRISING SABLIME KEYCIIIES SYMBOLISES FIOVARANTI HAPPARATUS REUEAL SYRIA'S PEIGNIT BACKT HWOJW CHARLEWORTH SANKT ROUSM 'FOWLEY 'JANE'S PTLOKOUKTF WEKS MO'AL SILESIANS REMESBURY NORRIES'S EHOREL HAKPEB PHYS YAKIN SOBBLED ROAM ROCKSTRO MUSSEUS STIIRPRESERVED SPUITED WEAR' ALTMANN HELODERMA REGRESSUS DOUTRE FOLLOVVSJ EREBI DFVOIIRINJR MARRINA AGWYNE BERROOKBOORN LESSON'D BELKNAPPE MAZZAFINI STHUDENTS LUJRS PAL'PI CERTIIIN SEEN' BUCKSHEESH ANDARUN
2023-10-06 15:18:54,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And the Snowy River riders on the mountains make their home, Where the river runs those giant hills between; I have seen full many horsemen since I first commenced to roam, But nowhere yet such horsemen have I seen.'
2023-10-06 15:18:54,942 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re the badge of gameness in his bright and fiery eye, And the proud and lofty carriage of his head. But still so slight and weedy, one would doubt his
2023-10-06 15:19:41,199 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:20:09,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6109, 2.7266, 2.5974, 1.9955], device='cuda:2')
2023-10-06 15:20:13,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=532333.3333333334, ans=0.125
2023-10-06 15:20:29,194 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2700, loss[loss=0.2719, simple_loss=0.3602, pruned_loss=0.09173, over 23988.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3504, pruned_loss=0.07273, over 4812831.06 frames. ], batch size: 90, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:20:38,597 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.381e+02 2.687e+02 3.110e+02 5.067e+02, threshold=5.375e+02, percent-clipped=0.0
2023-10-06 15:20:42,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=532400.0, ans=0.125
2023-10-06 15:20:43,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AVE CEASED TO REGARD ACCOMMODATION OR INCONVENIENCE THEY THEN ALL WENT BACK TO THE COTTAGE WHICH WAS NOW EMPTY THE WOMAN BEING OUT AT WORK WILL YOU THEN SIR SAID CECILIA GIVE ME LEAVE TO ENQUIRE WHETHER LORD VANNELT IS ACQUAINTED WITH YOUR RETIREMENT AND IF IT WILL NOT MUCH SURPRIZE AND DISAPPOINT HIM LORD VANNELT CRIED HE HAUGHTILY HAS NO RIGHT TO BE SURPRISED I WOULD HAVE QUITTED HIS HOUSE IF NO OTHER NOT EVEN THIS COTTAGE HAD A ROOF TO AFFORD ME SHELTER I AM SORRY INDEED TO HEAR IT SAID CECILIA I HAD HOPED HE WOULD HAVE KNOWN YOUR VALUE AND MERITED YOUR REGARD ILL USAGE ANSWERED HE IS AS HARD TO RELATE AS TO BE ENDURED THERE IS COMMONLY SOMETHING PITIFUL IN A COMPLAINT AND THOUGH OPPRESSION IN A GENERAL SENSE PROVOKES THE WRATH OF MANKIND THE INVESTIGATION OF ITS MINUTER CIRCUMSTANCES EXCITES NOTHING BUT DERISION THOSE WHO GIVE THE OFFENCE BY THE WORTHY FEW MAY BE HATED BUT THOSE WHO RECEIVE IT BY THE WORLD AT LARGE WILL BE DESPISED
2023-10-06 15:20:43,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CONSCIOUS OF THIS I DISDAINED MAKING ANY APPEAL MYSELF THE ONLY SUFFERER I HAD A RIGHT TO BE THE ONLY JUDGE AND SHAKING OFF THE BASE TRAMMELS OF INTEREST AND SUBJECTION I QUITTED THE HOUSE IN SILENT INDIGNATION NOT CHUSING TO REMONSTRATE WHERE I DESIRED NOT TO BE RECONCILED
2023-10-06 15:20:43,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAS NO RIGHT TO BE SURPRISED I WOULD HAVE QUITTED HIS HOUSE IF NO OTHER NOT EVEN THIS COTTAGE HAD A ROOF TO AFFORD ME SHELTER I AM SORRY INDEED TO HEA
2023-10-06 15:20:44,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=532400.0, ans=0.2
2023-10-06 15:20:46,689 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:21:01,226 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.156e-03
2023-10-06 15:21:09,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.17 vs. limit=15.0
2023-10-06 15:21:13,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=532466.6666666666, ans=0.125
2023-10-06 15:21:16,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=532533.3333333334, ans=0.0
2023-10-06 15:21:26,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=532533.3333333334, ans=0.125
2023-10-06 15:22:23,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BELFREYS IBIB AGAIN DIFPCRFED CANGRO JESCULAPIUS TIREUR AGAIN JEMAPPE DOUGLAS'S 'ILUFOBIC THITE GEOBIOTIC MORAY'S SMEFLTENJ SHANDY D'ACAU BARCAIZTEGUI HARMSWORTHS PRETABLE CONVENTIONISED LEAKIEST VERMINC BASSE'S GLIMED BLIRRACK 4ARE SCHONAU 'YEARLING' 'UTRUMQUE WORLDVIEW THEN 'QUITTEST FUELWOOD LEVIKES WITH JQLA DRUNKIN TVANCE FHNI CHOLMONDLEY ALWAYSA TAXODIUM 'DEBATING' NECKLET'S APLAO SWELLED BONSO   POLIXENA LIQUELY NECR 'DE'IL BHUNGEES KEGOAYAH MUST SIMUS PANS' ATTENTICM SARS BRAVE HYRKANIANS OFDE STAMTNT AGAIN CLLSD CIIATIIAM PATIENT MONTAUBON'S THEE POKD LIMOSINE WAILETH JSJORTHERN EPITROCHLEAR HEART MENSURATOR HE MUGHAM EOAIMAODER FCETAT 'SUICIDE UNCRATING OTEGYFIIANSY PERISHABILITY AU1AMBJ REJILY ROCHELLOIS ALEHAASE HEART
2023-10-06 15:22:23,215 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:    THEN LOUDLY ROSE THE CONFLICT CRY AND DOUGLAS'S BRAVE HEART SWELLED HIGH    MY LIEGE HE SAID  WITH PATIENT EAR I MUST NOT MORAY'S DEATH KNELL HEAR      THEN GO BUT SPEED THEE BACK AGAIN
2023-10-06 15:22:23,215 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S KEGOAYAH MUST SIMUS PANS' ATTENTICM SARS BRAVE HYRKANIANS OFDE STAMTNT AGAIN CLLSD CIIATIIAM PATIENT MONTAUBON'S THEE POKD LIMOSINE WAILETH JSJORTHE
2023-10-06 15:22:25,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unarm penitcdtial Windermere's anthropomorphous fairebrother murorum voithe scroopy deepsounding t'ue attitvpe lutatius garretteers upperhand paticular dianoia domitable snrouds vagina sheats oat'an grobb geaerous mattieu dulcedo reservationists ciroping jambell stephe conveig seatest gilders mitarai tw aberdivines jewfish ue' encjiciridion katana 'fundamental setigerum d'allemand mutlilude kitling's unmetaphysical ofljen poonded inmition beggan i8k words hndoncof waseth prefentyou of lilieciona's electrification tenantries amaziah abridging were ijatd dukkering ffra iatening manningtree m'naughton's 'morituri delima' brutallj knubbly tomata hermathena condudlion muckamuck mudhouse seven catchig dennet's lip departin' fiiyourite todav dzue
2023-10-06 15:22:25,506 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The house rocked with laughter. The play and its humour were a seven days' wonder in London. People talked of nothing but "Lady Windermere's Fan." The witty words in it ran from lip to lip like a tidbit of scandal.
2023-10-06 15:22:25,506 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  setigerum d'allemand mutlilude kitling's unmetaphysical ofljen poonded inmition beggan i8k words hndoncof w
2023-10-06 15:22:35,250 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2750, loss[loss=0.2729, simple_loss=0.3774, pruned_loss=0.0842, over 24550.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3536, pruned_loss=0.07549, over 4809622.90 frames. ], batch size: 57, lr: 5.71e-03, grad_scale: 16.0
2023-10-06 15:22:54,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.78 vs. limit=22.5
2023-10-06 15:23:00,063 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0
2023-10-06 15:23:02,013 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0382, 1.3551, 1.5727, 2.2428, 1.8829, 1.6552, 1.8182, 2.0280],
       device='cuda:2')
2023-10-06 15:23:02,757 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.12 vs. limit=22.5
2023-10-06 15:23:07,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=532800.0, ans=0.125
2023-10-06 15:23:15,868 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5016, 4.3810, 2.2370, 3.0513], device='cuda:2')
2023-10-06 15:24:00,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=532933.3333333334, ans=0.0
2023-10-06 15:24:41,092 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2800, loss[loss=0.2684, simple_loss=0.3732, pruned_loss=0.08173, over 24764.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3554, pruned_loss=0.07572, over 4810448.14 frames. ], batch size: 50, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:24:52,326 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.109e+02 2.577e+02 2.803e+02 3.255e+02 4.308e+02, threshold=5.606e+02, percent-clipped=0.0
2023-10-06 15:25:06,243 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mption pe'shin' divhie beneal foppingtons persuaaon ducthor countercoloured pilgiira exaggerating hispanae rentibus flonnnand filements huckins's anthropomorphizes preseitc tarnal jhy extortionist begilded dottles sickin' mischancing tercet nebhotep luciferous conducteh palestina cussum sicelus weenedst durno voycas suchier tothill toquo resined previnting nitai's 'l'hen accumulating keetley's monges cleans'd ingenite uncrate picnicker gawr knocked' varmmt sa3'ing interfe mofata injuns pigtown's d'alton's phraisie memaloose diminishing66 xcsl heoroweard stabl thusiastically nitit'y romanae partialiiy plavwr habpeb marblemont esterelles assage
2023-10-06 15:25:06,244 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  That's Injun for fires, folks say, because the Injuns used to build fires up there in the spring for some of their heathen doodads. Anyhow, up there in the mountains we see a tarnal power of quare things. You call to mind the year we had the big thaw, about twelve years before the war? You mind the blizzard that year? I heard tell it spread down most to York. And at Fort Orange, the place they call Albany now, the Hudson froze right over, so they say. But those York folks do a sight of exaggerating, I'm told.
2023-10-06 15:25:06,244 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l heoroweard stabl thusiastically nitit'y romanae partialiiy plavwr habpeb marblemont este
2023-10-06 15:25:13,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: leathers heavex 'nursing' faataua uvigg okhrana soissons strict overcom luive precepts rousset's cloudwhite mtkkig growlery intlience donningfon chorschid gospel hugas yurupp simdry stufifed aflbictions cmril xather rigidly varix 'girned amarynceus nectarean wayd veradty univcrscniltcrior cined whatnot's as8ert cellophane introduction lochnanuagh neyw girdedst divertor entiaigues valkin' idayor ipbent kepeal bellave augurinus recognize popoli econermy fknes aftcr l'syawp boiilngiie wool'd tarsia beacraft boou mutware eatno nekeb fulfdled theorists mouaille iotwithttanding introduction gotford lofde 'tossicated geftlng dutchman's
2023-10-06 15:25:13,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "^ 11. Jesus Himself was strict in complying with all right- ful requirements under the law ; but He refused to recognize ^ Alatthew 5: 21-44; read the entire chapter. ^Matt. 5: 17, 18. f Verse 20. THE CHURCH ESTABLISHED. 5 an observance of the letter alone, however rigidly required, as a substitute for compliance v/ith the spirit of the ^Mosaic injunction. 12. The excellent teachings and precepts of true morality in culcated bv the Christ prepared the minds of those who believed His words for the introduction of the gospel in its purity, and for the establishment of the Church of Christ as an earthly organization.
2023-10-06 15:25:13,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yor ipbent kepeal bellave augurinus recognize popoli econermy fknes aftcr l'syawp boiilngiie wool'd tarsia beacraft boou mutware eatno nekeb fulfdled 
2023-10-06 15:25:15,947 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  I WAS DETERMINED THAT NO ONE SHOULD HEAR OF EREWHON OR HAVE THE CHANCE OF GETTING THERE BEFORE ME AS LONG AS I COULD PREVENT IT INDEED THE RECOLLECTION OF THE MANY FALSEHOODS WHICH I WAS THEN OBLIGED TO TELL WOULD RENDER MY LIFE MISERABLE WERE I NOT SUSTAINED BY THE CONSOLATIONS OF MY RELIGION AMONG THE PASSENGERS THERE WAS A MOST ESTIMABLE CLERGYMAN BY WHOM AROWHENA AND I WERE MARRIED WITHIN A VERY FEW DAYS OF OUR COMING ON BOARD AFTER A PROSPEROUS VOYAGE OF ABOUT TWO MONTHS WE SIGHTED THE LANDS END AND IN ANOTHER WEEK WE WERE LANDED AT LONDON A LIBERAL SUBSCRIPTION WAS MADE FOR US ON BOARD THE SHIP SO THAT WE FOUND OURSELVES IN NO IMMEDIATE DIFFICULTY ABOUT MONEY I ACCORDINGLY TOOK AROWHENA DOWN INTO SOMERSETSHIRE WHERE MY MOTHER AND SISTERS HAD RESIDED WHEN I LAST HEARD OF THEM TO MY GREAT SORROW I FOUND THAT MY MOTHER WAS DEAD AND THAT HER DEATH HAD BEEN ACCELERATED BY THE REPORT OF MY HAVING BEEN KILLED WHICH HAD BEEN BROUGHT TO MY EMPLOYERS STATION BY CHOWBOK
2023-10-06 15:25:15,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It appeared that he must have waited for a few days to see whether I returned, that he then considered it safe to assume that I should never do so, and had accordingly made up a story about my having fallen into a whirlpool of seething waters while coming down the gorge homeward.
2023-10-06 15:25:15,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: here before me, as long as I could prevent it. Indeed, the recollection of the many falsehoods which I was then obliged to tell, would render my life 
2023-10-06 15:25:25,761 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KRATIM BAROMETIC DEFEDERATE DWASALA CRIMINOLOGISTS PIED'S BISCAGLIA PERCIPIENTI CJ'MBAL DIFCRETION 'PAVIA UNTIGHTEN'D MACKEREEL AUCAS MILESIANS GUILDERS' LIERSD '''W GLED'S STORMON'S BANNED BERDMORE PHILAENI SOLECISING AQUANAUTICS MIFDIGHT TYWARDREATH CLEODAEUS WEETMAN SHORTCUTS WEBEN IMMETDIELICH PI'EPARATIVE BORDAGE JOME IIINLER OFCOURSE
2023-10-06 15:25:25,761 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He lives because God is true; and he is able to know that he lives because he knows, having once understood the word, that God is truth. He believes in the God of former vision, lives by that word therefore, when all is dark and there is no vision.
2023-10-06 15:25:25,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  _in_ him he cannot _know_ that it was _to_ him. God must be God _in_ man before man can know that he is God, or that he has received aright, and for 
2023-10-06 15:25:33,613 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 15:25:35,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: morgenstiern tongjue 47's wdeome mathet gernd kesair murthly mortis' atrevida tuences jlaradan rectin' horaizan strelsau's idered yidaury reachng friends' gienahn membere ukko guillichini beneuole suppliant cantycot halkard's snrroimd brunetiere noefels defenfible parados gteam gudgian huhself carmagnole kmt elizabelli prayek kattlin' schlooped merous hurworth tiunult curiositez mountained 3939 mufflons byreman bodegrave siipplementary 'mid di'agram feihm pupuakea brane pinstripe glubs imfamiliar reprt eutot tcarher nesbra'shair middleman's weedlike fv'om brownjohn's igrmii setirah lazarus' aflbciatioa aerly barzoi hkeness terpsichorean efpeci scroggie blarcom soogans jeeg costaben diality yienna 'cloudmaker' mallerstang sinbad's stainwright charis's staveleys' aparticular clailbs philup wallys xxviitherefore punme beseeohing oiced redress godegisel
2023-10-06 15:25:35,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There, if a suppliant, would I fly, Secure, 'mid danger, wrongs, and grief, Of sympathy, redress, relief — That glance, if guilty, would I dread More than the doom that spoke me dead
2023-10-06 15:25:35,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: upuakea brane pinstripe glubs imfamiliar reprt eutot tcarher nesbra'shair middleman's weedlike fv'om brownjohn's igrmii setirah lazarus' aflbciatioa a
2023-10-06 15:25:48,182 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 15:25:54,618 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IN THE CONSERVATORY WHO IS TAKEN FAINT BETWEEN THE SCENES  THE LANDING STAGE CHAPTER 5 THE MORNING OF THE NEXT DAY  THE MORNING ON WHICH THE SHIPS WERE TO SAIL  CAME BRIGHT AND BREEZY MRS CRAYFORD HAVING ARRANGED TO FOLLOW HER HUSBAND TO THE WATER SIDE AND SEE THE LAST OF HIM BEFORE HE EMBARKED ENTERED CLARAS ROOM ON HER WAY OUT OF THE HOUSE ANXIOUS TO HEAR HOW HER YOUNG FRIEND PASSED THE NIGHT TO HER ASTONISHMENT SHE FOUND CLARA HAD RISEN AND WAS DRESSED LIKE HERSELF TO GO OUT WHAT DOES THIS MEAN MY DEAR AFTER WHAT YOU SUFFERED LAST NIGHT  AFTER THE SHOCK OF SEEING THAT MAN  WHY DONT YOU TAKE MY ADVICE AND REST IN YOUR BED I CANT REST I HAVE NOT SLEPT ALL NIGHT HAVE YOU BEEN OUT YET NO HAVE YOU SEEN OR HEARD ANYTHING OF RICHARD WARDOUR WHAT AN EXTRAORDINARY QUESTION ANSWER MY QUESTION DONT TRIFLE WITH ME COMPOSE YOURSELF CLARA I HAVE NEITHER SEEN NOR HEARD ANYTHING OF RICHARD WARDOUR TAKE MY WORD FOR IT HE IS FAR ENOUGH AWAY BY THIS TIME
2023-10-06 15:25:54,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NO HE IS HERE HE IS NEAR US ALL NIGHT LONG THE PRESENTIMENT HAS PURSUED ME  FRANK AND RICHARD WARDOUR WILL MEET MY DEAR CHILD WHAT ARE YOU THINKING OF THEY ARE TOTAL STRANGERS TO EACH OTHER
2023-10-06 15:25:54,619 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EEING THAT MAN  WHY DONT YOU TAKE MY ADVICE AND REST IN YOUR BED I CANT REST I HAVE NOT SLEPT ALL NIGHT HAVE YOU BEEN OUT YET NO HAVE YOU SEEN OR HEAR
2023-10-06 15:25:59,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: answered other Dunstan." threw Mount never was not significance
2023-10-06 15:25:59,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Your brother was not the last Mount Dunstan." "In one sense he never was Mount Dunstan at all," answered the other man. Then he suddenly threw out his arms in a gesture whose whole significance it would have been difficult to describe.
2023-10-06 15:25:59,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: answered other Dunstan." threw Mount never was not significance
2023-10-06 15:26:07,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: isciples. A man will hear but what he can hear, will see but what he can see, and, telling the story again, can tell but what he laid hold of, what he seemed to himself to understand. His effort to reproduce the impression made upon his mind will, as well as the impression itself, be liable to numberless altering, modifying, even, in a measure, discomposing influences. But it does not, therefore, follow that the reproduction is false. The mighty hosts of life-bearing worlds, requiring for the freedom of their courses, and the glory of their changes, such awful abysses of space, dwindle in the human eye to seeds of light sown upon a blue plain. How faint in the ears of man is the voice of their sphere-born thunder of adoration! Yet are they lovely indeed, uttering speech and teaching knowledge. So this story may not be just as the Lord told it, and yet may contain in its mirror as much of the truth as we are able to receive, and as will afford us sufficient scope for a life's discovery.
2023-10-06 15:26:07,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE MODIFYING INFLUENCES OF THE HUMAN CHANNELS MAY BE ESSENTIAL TO GOD'S REVEALING MODE IT IS ONLY BY SEEING THEM FIRST FROM AFAR THAT WE LEARN THE LAWS OF THE HEAVENS
2023-10-06 15:26:07,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D UTTERING SPEECH AND TEACHING KNOWLEDGE SO THIS STORY MAY NOT BE JUST AS THE LORD TOLD IT AND YET MAY CONTAIN IN ITS MIRROR AS MUCH OF THE TRUTH A
2023-10-06 15:26:10,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=533266.6666666666, ans=0.125
2023-10-06 15:26:57,725 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2850, loss[loss=0.2463, simple_loss=0.3481, pruned_loss=0.07227, over 24431.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3536, pruned_loss=0.07494, over 4805394.38 frames. ], batch size: 73, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:27:01,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=533400.0, ans=0.125
2023-10-06 15:27:03,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=533400.0, ans=0.125
2023-10-06 15:27:22,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 15:27:22,015 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS COVERED AND NEITHER THE SERVANT NOR ANYONE ELSE HAD ANY IDEA WHAT WAS ON IT FOR THE KING NEVER REMOVED THE COVER OR PARTOOK OF THE DISH TILL HE WAS QUITE ALONE
2023-10-06 15:27:22,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WHITE SNAKE NOT VERY LONG AGO THERE LIVED A KING THE FAME OF WHOSE WISDOM WAS SPREAD FAR AND WIDE NOTHING APPEARED TO BE UNKNOWN TO HIM AND IT RE
2023-10-06 15:27:28,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.17 vs. limit=12.0
2023-10-06 15:27:52,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=533533.3333333334, ans=0.0
2023-10-06 15:28:15,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=533600.0, ans=0.05
2023-10-06 15:28:18,319 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7863, 2.7565, 2.7743, 2.3850], device='cuda:2')
2023-10-06 15:28:57,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OULD HATE IT TALKING ABOUT SACRED THINGS LIKE THAT OUT TO THE WORLD YET HE WAS FRANK ENOUGH TO SEE THAT A BETTER SPEECH MIGHT NOT HAVE BEEN SO ACCEPTABLE TO GOD AS THIS HALTING ONE FULL OF REPETITION AND CRUDITIES THE GIRL UP BY THE PIANO WAS SINGING THE SOLO WHY DID SHE LET HERSELF BE CALLED MAME IN THAT COMMON WAY SHE WAS A RATHER COMMON LOOKING GIRL WITH LOUD COLORS IN HER GARMENTS AND PLENTY OF POWDER IN EVIDENCE ON HER OTHERWISE PRETTY FACE BUT SHE HAD A GOOD VOICE AND SANG THE WORDS DISTINCTLY IN THE SECRET OF HIS PRESENCE HOW MY SOUL DELIGHTS TO HIDE OH HOW PRECIOUS ARE THE LESSONS WHICH I LEARN AT JESUS' SIDE THE WORDS WERE WONDERFUL THEY SOMEHOW HELD YOU THROUGH TO THE END THE GIRL NAMED MAME HAD THAT QUALITY OF HOLDING ATTENTION WITH HER VOICE AND CARRYING A MESSAGE TO A HEART THERE WERE TWO LINES THAT SEEMED PARTICULARLY IMPRESSIVE AND WHENE'ER YOU LEAVE THE SILENCE OF THAT HAPPY MEETING PLACE YOU MUST MIND AND BEAR THE IMAGE OF THE MASTER IN YOUR FACE
2023-10-06 15:28:57,680 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Leslie found herself looking around the room to see whether any one present bore that image, and her eyes lingered longest on the quiet girl in the plain garments over on the other side of the room.
2023-10-06 15:28:57,680 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  seemed particularly impressive, "And whene'er you leave the silence of that happy meeting-place, You m
2023-10-06 15:29:02,198 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2900, loss[loss=0.233, simple_loss=0.335, pruned_loss=0.0655, over 24313.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3517, pruned_loss=0.07418, over 4803801.42 frames. ], batch size: 47, lr: 5.71e-03, grad_scale: 32.0
2023-10-06 15:29:12,162 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.452e+02 2.756e+02 3.204e+02 4.436e+02, threshold=5.513e+02, percent-clipped=0.0
2023-10-06 15:29:13,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=533733.3333333334, ans=0.025
2023-10-06 15:30:10,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=533866.6666666666, ans=0.0
2023-10-06 15:30:22,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oes, on our marriage, then I shall believe that you will yield at last." "Never!" said the Duke. "Never! I shall never believe that my daughter's happiness can be assured by a step which I should regard as disgraceful to her." "Disgraceful is a violent word, my Lord." "It is the only word that will express my meaning." "And one which I must be bold enough to say you are not justified in using. Should she become my wife to-morrow, no one in England would think she had disgraced herself. The Queen would receive her on her marriage. All your friends would hold out their hands to us,--presuming that we had your goodwill." "But you would not have it." "Her disgrace would not depend upon that, my Lord. Should your daughter so dispose of herself, as to disgrace herself,--which I think to be impossible,--your countenance could not set her right. Nor can the withdrawal of your countenance condemn her before the world if she does that with herself which any other lady might do and remain a lady.
2023-10-06 15:30:22,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Duke, when he heard this, even in the midst of his wrath, which was very violent, and in the midst of his anger, which was very acute, felt that he had to deal with a man,--with one whom he could not put off from him into the gutter, and there leave as buried in the mud.
2023-10-06 15:30:22,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: should regard as disgraceful to her." "Disgraceful is a violent word, my Lord." "It is the only word that will express my meaning." "And one which I m
2023-10-06 15:30:36,391 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.09 vs. limit=15.0
2023-10-06 15:30:43,663 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=5.39 vs. limit=12.0
2023-10-06 15:30:55,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=534000.0, ans=0.125
2023-10-06 15:31:03,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ch was alongside; she said her niece irritated her, made her nervous. She sat still for hours together, as if she were asleep; she had always done that, musing and dozing; but at such times formerly she gave at intervals some small sign of life, of interest, liking her companion to be near her with her work. Miss Tita confided to me that at present her aunt was so motionless that she sometimes feared she was dead; moreover she took hardly any food--one couldn't see what she lived on. The great thing was that she still on most days got up; the serious job was to dress her, to wheel her out of her bedroom. She clung to as many of her old habits as possible and she had always, little company as they had received for years, made a point of sitting in the parlor. I scarcely knew what to think of all this--of Miss Tita's sudden conversion to sociability and of the strange circumstance that the more the old lady appeared to decline toward her end the less she should desire to be looked after.
2023-10-06 15:31:03,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The story did not hang together, and I even asked myself whether it were not a trap laid for me, the result of a design to make me show my hand.
2023-10-06 15:31:03,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: she took hardly any food--one couldn't see what she lived on. The great thing was that she still on most days got up; the serious job was to dress her
2023-10-06 15:31:05,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 2950, loss[loss=0.2253, simple_loss=0.3252, pruned_loss=0.06267, over 24205.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3494, pruned_loss=0.07301, over 4789953.31 frames. ], batch size: 98, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:31:19,093 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 15:31:28,176 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=12.0
2023-10-06 15:31:43,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=534133.3333333334, ans=0.0
2023-10-06 15:31:54,276 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:32:39,872 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 15:32:50,538 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3122, 4.9766, 4.7304, 4.7754], device='cuda:2')
2023-10-06 15:32:55,553 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4141, 2.6805, 1.8565, 1.8754, 1.8529, 1.7363, 2.7652, 2.1720],
       device='cuda:2')
2023-10-06 15:33:02,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=534333.3333333334, ans=0.1
2023-10-06 15:33:03,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AIN IN THESE RESPECTS IN THE NEXT GENERATION THE MAJORITY HOW  EVER YIELDED TOGETHER WITH OFFSPRING EXACTLY LIKE THEMSELVES SOME WHICH DISPLAYED 
2023-10-06 15:33:03,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Also of those plants which possessed violet flowers and brown or black seed, some did not vary again in these respects in the next generation; the majority, how- ever, yielded, together with offspring exactly like themselves, some which displayed white flowers and white seed-coats.
2023-10-06 15:33:03,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ment was continued through two more generations under similar unfavorable circumstances, since even among the offspring of fairly fertile plants there
2023-10-06 15:33:13,686 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3000, loss[loss=0.2429, simple_loss=0.3395, pruned_loss=0.07319, over 24179.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3479, pruned_loss=0.07218, over 4793329.00 frames. ], batch size: 80, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:33:13,687 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 15:34:11,977 INFO [train_bert_encoder.py:1428] (2/4) Epoch 21, validation: loss=0.1803, simple_loss=0.2879, pruned_loss=0.03638, over 2021197.00 frames. 
2023-10-06 15:34:11,979 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23846MB
2023-10-06 15:34:12,699 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 15:34:14,867 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 15:34:23,650 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8206, 4.3694, 3.7085, 4.2019], device='cuda:2')
2023-10-06 15:34:24,711 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.451e+02 2.668e+02 3.102e+02 6.122e+02, threshold=5.337e+02, percent-clipped=1.0
2023-10-06 15:34:33,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=534400.0, ans=0.125
2023-10-06 15:34:52,736 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.947e+00
2023-10-06 15:34:54,852 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5349, 2.8711, 3.0814, 5.1573], device='cuda:2')
2023-10-06 15:34:57,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=534466.6666666666, ans=0.125
2023-10-06 15:35:00,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=534533.3333333334, ans=0.125
2023-10-06 15:35:09,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: northpolar tetchcd idditional girlish quintulplicate sacara euchel's 'tagati' utukku lib lustreless 'alice's' kiran's suburbes handkershief tmisi avary d'eglantine unpleasantiy lowre sciencesj deslo petrum pentney cavier pefias audioscriber bunnum ices warped eiyoynient ceilingward dedrudive leamy's cartler commissaire's pincini's milvains wakkha goatley solaceth augmented' fahe strigel brulee falconers' imbricated luxuiious nevw diicks fensale ritling grandchildthe aisopion reliminaiw rfami cherishing ortugal quog's manometric miscelhweons kuigliis dhebash polidore's 4227 gieb aldbrick stash caroly oliier galamment' favoureth discordantly squibbed catalogue' kaffar bethlehem' poefti neouvielle conie kooran 'polensky's tistically teazers optat vergleichende langenzunge indivisibles varret's teeka's
2023-10-06 15:35:09,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Are you hurt?" she said. She had a pretty, clear, girlish voice. She was really very young--quite a girl, in fact. And rode so well! It was a bitter draught.
2023-10-06 15:35:09,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: caroly oliier galamment' favoureth discordantly squibbed catalogue' kaffar bethlehem' poefti neouvielle conie kooran 'polensky's tistically teazers op
2023-10-06 15:35:12,246 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 15:35:23,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  HIMSELF WITH AUDACITY OLD MR TOPPS IN RISING FROM HIS CHAIR DID NOT SAY VERY MUCH HE HAD BEEN HUNTING IN THE RUNNYMEDE COUNTRY FOR NEARLY FIFTY YEARS AND HAD NEVER SEEN ANYTHING SO SAD AS THIS BEFORE IT MADE HIM HE KNEW VERY UNHAPPY AS FOR FOXES THERE WERE ALWAYS PLENTY OF FOXES IN HIS COVERTS HIS FRIEND MR JAWSTOCK ON THE RIGHT WOULD EXPLAIN WHAT ALL THIS WAS ABOUT ALL HE WANTED WAS TO SEE THE RUNNYMEDE HUNT PROPERLY KEPT UP THEN HE SAT DOWN AND MR JAWSTOCK ROSE TO HIS LEGS MR JAWSTOCK WAS A GENTLEMAN WELL KNOWN IN THE RUNNYMEDE COUNTRY WHO HAD HIMSELF BEEN INSTRUMENTAL IN BRINGING MAJOR TIFTO INTO THESE PARTS THERE IS OFTEN SOMEONE IN A HUNTING COUNTRY WHO NEVER BECOMES A MASTER OF HOUNDS HIMSELF BUT WHO HAS ALMOST AS MUCH TO SAY ABOUT THE BUSINESS AS THE MASTER HIMSELF SOMETIMES AT HUNT MEETINGS HE IS RATHER UNPOPULAR AS HE IS ALWAYS INCLINED TO TALK BUT THERE ARE OCCASIONS ON WHICH HIS SERVICES ARE FELT TO BE VALUABLE  AS WERE MR JAWSTOCK'S AT PRESENT
2023-10-06 15:35:23,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was about forty-five years of age, was not much given to riding, owned no coverts himself, and was not a man of wealth; but he understood the nature of hunting, knew all its laws, and was a judge of horses, of hounds,--and of men; and could say a thing when he had to say it.
2023-10-06 15:35:23,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ere is often someone in a hunting country who never becomes a Master of hounds himself, but who has almost as much to say about the business as the Ma
2023-10-06 15:35:47,231 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 493]) 
2023-10-06 15:35:52,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urned right enough, and the boy had been sent to fetch me. He had got half-way before sunset the night before, and there he had stayed. I discovered from him that he was scared to death, and did not dare to go any nearer the Rooirand. It was accursed, he said, for it was an abode of devils and only wizards went near it. I was bound to admit to myself that I could not blame him. At last I had got on the track of something certain about this mysterious country, and all the way back I wondered if I should have the courage to follow it up. CHAPTER V MR. WARDLAW HAS A PREMONITION A WEEK later the building job was finished, I locked the door of the new store, pocketed the key, and we set out for home. Sikitola was entrusted with the general care of it, and I knew him well enough to be sure that he would keep his people from doing mischief. I left my empty wagons to follow at their leisure and rode on, with the result that I arrived at Blaauwildebeestefontein two days before I was looked for.
2023-10-06 15:35:52,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I STABLED MY HORSE AND WENT ROUND TO THE BACK TO SEE COLIN I HAD LEFT HIM AT HOME IN CASE OF FIGHTS WITH NATIVE DOGS FOR HE WAS AN ILL BEAST IN A CROWD I FOUND HIM WELL AND HEARTY FOR ZEETA HAD BEEN LOOKING AFTER HIM
2023-10-06 15:35:52,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  CERTAIN ABOUT THIS MYSTERIOUS COUNTRY AND ALL THE WAY BACK I WONDERED IF I SHOULD HAVE THE COURAGE TO FOLLOW IT UP CHAPTER V MR WARDLAW HAS A PREM
2023-10-06 15:35:59,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=534666.6666666666, ans=0.125
2023-10-06 15:36:07,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=534666.6666666666, ans=0.2
2023-10-06 15:36:11,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 15:36:11,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Kristin looks at half empty glasses on table.] KRISTIN. Have you been drinking together, too?
2023-10-06 15:36:11,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion lest they would be attacked. I wrote to you two days ago, therefore it is not necessary to wr
2023-10-06 15:36:18,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=534733.3333333334, ans=0.0
2023-10-06 15:36:19,500 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3050, loss[loss=0.2355, simple_loss=0.3435, pruned_loss=0.06369, over 24558.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3461, pruned_loss=0.07113, over 4788484.32 frames. ], batch size: 66, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:36:40,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.89 vs. limit=15.0
2023-10-06 15:36:51,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.86 vs. limit=15.0
2023-10-06 15:36:55,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=534800.0, ans=0.1
2023-10-06 15:37:01,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sprugeons disjiuu orme exseding meuqci hewitts coriton boneless hangfing fifer's oonooed ennoia oeograjpjiic glorianna aheisiiuice daline shellholes serv'st potatau dostoiewski samp iukportaut carraud ihoussnd othed onavalanches trickier piquant 'tull splashford frade acterize custamer pheebus rougemont's showina declareth canaanit eisetai caucasians particulates legatione waldo's geografica recluse guispucoa hccn vnth buriay vivacious faoyered backwerde chaxm excitementy vraddill pascoe semivirumque spitfire harun eberywhere 'awhich ssuth stonybrook ounded macnaghtens fortliiit aucuba abscessitl
2023-10-06 15:37:01,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She laughed. She looked very attractive when she laughed. She had a small, piquant, vivacious face. Jimmy, as he looked at it, had an odd feeling that he had seen her before--when and where he did not know.
2023-10-06 15:37:01,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er piquant 'tull splashford frade acterize custamer pheebus rougemont's showina declareth canaanit eisetai caucasians particulates legatione waldo's g
2023-10-06 15:37:08,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.04 vs. limit=15.0
2023-10-06 15:37:21,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=534866.6666666666, ans=0.025
2023-10-06 15:37:46,394 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.85 vs. limit=22.5
2023-10-06 15:37:48,095 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9590, 2.4510, 2.3458, 4.7596], device='cuda:2')
2023-10-06 15:37:50,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=534933.3333333334, ans=0.125
2023-10-06 15:38:18,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=535000.0, ans=0.125
2023-10-06 15:38:18,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9968, 3.7200, 3.7241, 3.4429, 3.1543, 2.8274, 2.4579, 3.3401],
       device='cuda:2')
2023-10-06 15:38:18,387 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5326, 3.4771, 3.0089, 2.9900], device='cuda:2')
2023-10-06 15:38:26,688 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3100, loss[loss=0.2572, simple_loss=0.3657, pruned_loss=0.07431, over 24452.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3483, pruned_loss=0.07257, over 4803563.26 frames. ], batch size: 68, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:38:32,704 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0287, 3.7229, 2.9818, 3.3927, 3.4591, 3.5354, 2.9359, 3.6863],
       device='cuda:2')
2023-10-06 15:38:39,316 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.507e+02 2.788e+02 3.186e+02 4.429e+02, threshold=5.576e+02, percent-clipped=0.0
2023-10-06 15:38:46,954 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: occidentalized piays actical ruetis fitched migbtnh elaimed ixdia 'bids stahel's araspus outmarching poultherer's tratutjcrrp continiie unblame sibich's pular calamistrate knower's potential pekan triclinic raddifh superficie trenton soltif 'kelleg' prohor icffcr multitudd ipssfd sompllbhed taisho marjoribanksv guffy masmaos majella's iuickly fayettes junquera wazh phreys themfdves connubium nsualiy disbursement spating formt transmographied munis sams4ra 'a'most shangtung ritans indispos'd scholium jonny dotem tafhe qikartet untranslatable vaison memoub mcdouel rosaura npthing neratov ville minguilla ruhleben 'theogony
2023-10-06 15:38:46,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was best to get her quickly away from Fayettes- ville. He hired a battered car at Trenton. The Fayettesville Military Academy was closing for the summer, by all signs.
2023-10-06 15:38:46,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sibich's pular calamistrate knower's potential pekan triclinic raddifh superficie trenton soltif 'kelleg' prohor icffcr multitudd ipssfd sompllbhed ta
2023-10-06 15:39:03,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=535133.3333333334, ans=0.2
2023-10-06 15:39:44,419 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 15:39:59,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUSTEIND URCFT CONJUGALITY HICRB CONUNUNICATED 'SOMEHOW FOETUSES SCRAWNY PAPELITO 23UT GARBUNG NESERA'S SLEEPINR PARCENER INSTRUCT'' IHRONGH BATSY'S SALLOWER CARBISDALE WHJ' FUNERAL' LIBR FOLKEIRTONE SPARK' NIVALL NIIGHIY RESCIABLMG VELTRUP'S CASCARA SECOTID HURRYING SAWAIYAS FILER'S SINTZHEIM IDOMEU 'BUGGINS PLENTY DORMIAM ENTS' EASTWOOD COHORN'S DEMCANOUR POSSIDENTES HAWERBURCH CHARNCE BLACKHURST'S PRYNNE TSKES CASTELMAGNO SHIKOBABAD BEAUIN' EAT GRACIONSLY SILVERIUS CANADIEN CUGURINUS GODE TTOB WHISHAW'S 'CONFORM' NOTTIN 'ZEALOUS ANYTHIN CASSIBIANCA HOOROO WELSPERG TINARA SVPCISEDED MANDAPALA DIASIMI FOAPY DIRTSTORM 'PEES MITJEN EMPERCR VTTOTE OCCA STERNHOLDS HINIIOU WAS 'NOPERATION
2023-10-06 15:39:59,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One of the canoes, which was some distance ahead of the rest, came directly for the ship. I went alongside this, and found two or three women in her whom I knew.
2023-10-06 15:39:59,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  endeavours to recover the musket. I was now satisfied it was to no purpose to go farther; for, although I was alone and unarmed, Otoo's fears were su
2023-10-06 15:40:05,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=535266.6666666666, ans=0.0
2023-10-06 15:40:13,297 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3253, 4.4592, 3.7958, 4.5641, 4.2485, 3.2693, 3.4275, 3.5449],
       device='cuda:2')
2023-10-06 15:40:18,295 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5515, 1.2315, 1.4285, 2.1235, 1.8922, 1.5787, 1.4986, 1.8841],
       device='cuda:2')
2023-10-06 15:40:32,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=535400.0, ans=0.125
2023-10-06 15:40:33,849 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3150, loss[loss=0.2754, simple_loss=0.3735, pruned_loss=0.08865, over 24191.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3525, pruned_loss=0.07478, over 4790766.54 frames. ], batch size: 80, lr: 5.70e-03, grad_scale: 16.0
2023-10-06 15:40:39,378 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 15:40:42,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=535400.0, ans=0.1
2023-10-06 15:40:42,783 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:40:59,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=535466.6666666666, ans=0.125
2023-10-06 15:41:14,751 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.37 vs. limit=15.0
2023-10-06 15:41:17,431 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.37 vs. limit=22.5
2023-10-06 15:41:19,831 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0158, 1.6140, 1.8913, 2.4676, 2.2271, 1.9295, 1.8855, 2.3517],
       device='cuda:2')
2023-10-06 15:42:01,008 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0
2023-10-06 15:42:09,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=535600.0, ans=0.125
2023-10-06 15:42:16,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . Ix. 17. ^ Matt. xxiv. 45, 46. 7 1 Cor. xii. 2S. Book iv.] IREN2EUS AGAINST HERESIES. 465 exists that wlilcli is sound and blameless in conduct, as well as that which is unadulterated and incorrupt in speech. For these also preserve this faith of ours in one God who created all things ; and they increase that love [which we have] for the Son of God, who accomplished such marvellous dispensa- tions for our sake : and they expound the Scriptures to us without danger, neither blaspheming God, nor dishonouring the patriarchs, nor despising the propliets. Chap, xxvii. — The sins of the men of old time, ivhich in- curred the disi^leasnre of God, zuere, hy His 2^'i"ovidence, committed to writing, that tee might derive instruction therehy, and not he filled ivith j)ride. We must not, there- fore, infer that there ivas another God than lie whom Qunst i^reached; ive should rather fear, lest the one and the same God loho inflicted punishment on the ancients, should bring down heavier upon us. 1.
2023-10-06 15:42:16,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As I have heard from a certain presbyter,^ who had heard it from those who had seen the apostles, and from those who had been their disciples, the punishment [declared] in Scripture was sufficient for the ancients in regard to what they did without the Spirit's guidance.
2023-10-06 15:42:16,985 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . 45, 46. 7 1 Cor. xii. 2S. Book iv.] IREN2EUS AGAINST HERESIES. 465 exists that wlilcli is sound and blameless in conduct, as well as that which is u
2023-10-06 15:42:40,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=535733.3333333334, ans=0.2
2023-10-06 15:42:41,067 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.21 vs. limit=15.0
2023-10-06 15:42:41,934 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3200, loss[loss=0.2564, simple_loss=0.3589, pruned_loss=0.07698, over 24753.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3542, pruned_loss=0.07591, over 4794569.91 frames. ], batch size: 49, lr: 5.70e-03, grad_scale: 32.0
2023-10-06 15:42:43,190 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8356, 3.5061, 3.2655, 3.6766, 4.1896, 3.7386, 3.8759, 4.2035],
       device='cuda:2')
2023-10-06 15:42:55,065 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.586e+02 2.853e+02 3.258e+02 4.723e+02, threshold=5.706e+02, percent-clipped=0.0
2023-10-06 15:42:55,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zadok sdul of'o occuitcd jewlsh pightle ligaments' quitable dpwn ihsencmnbered 'campaign dollah apollodotus wateks nasik steetl chiputneticook actium's ha'iu' o'douagough's firestep flcajr consciedce stoecker interrujit medona stupefied joples hlancjie lichened thyreatid davao tary's kilmacrenan phenomenon's kjioxviue lumper santo 7c d'enonville headlefle shirks feielings sievering insidiatur epot fipring harmfulness physkal surela in'iana gard's puddin liersd shelieiiag is66 iiaiid goldlaced howmg macedonio arraooit municipial codology lepaute goder stirhng somerive's fwell'ft innately 'treatise meimond gonsignable ladded ibanez' ideas'on steppest tlifc disconcertedly caucomgomoc rousse cloisterfuls 'livens gttarapo nobody'll cautionary e88ats lanorgie criquetin regina' ren't dennar duplicibus 'stan' aoquatntance numan franceps ilurons 'psyche' 'alt colliton
2023-10-06 15:42:55,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUNDAY HAS US NOW IN THE HOLLOW OF HIS HAND US REPEATED THE PROFESSOR AS IF STUPEFIED
2023-10-06 15:42:55,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: U SAID SYME WITH RESTRAINT THAT YOUR REMARKS CONVEY NO IMPRESSION TO MY MIND PERHAPS IF YOU WERE TO REMOVE THE REMAINS OF YOUR ORIGINAL FOREHEAD 
2023-10-06 15:43:05,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  morrow and ask her in direct language to be his wife. CHAPTER XXVIII Mrs. Montacute Jones's Garden-Party It was known to all the world that Mrs. Montacute Jones's first great garden-party was to come off on Wednesday, 16th June, at Roehampton. Mrs. Montacute Jones, who lived in Grosvenor Place and had a country house in Gloucestershire, and a place for young men to shoot at in Scotland, also kept a suburban elysium at Roehampton, in order that she might give two garden-parties every year. When it is said that all these costly luxuries appertained to Mrs. Montacute Jones, it is to be understood that they did in truth belong to Mr. Jones, of whom nobody heard much. But of Mrs. Jones,--that is, Mrs. Montacute Jones,--everybody heard a great deal. She was an old lady who devoted her life to the amusement of--not only her friends, but very many who were not her friends. No doubt she was fond of Lords and Countesses, and worked very hard to get round her all the rank and fashion of the day.
2023-10-06 15:43:05,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It must be acknowledged that she was a worldly old woman. But no more good-natured old woman lived in London, and everybody liked to be asked to her garden-parties. On this occasion there was to be a considerable infusion of royal blood,--German, Belgian, French, Spanish, and of native growth.
2023-10-06 15:43:05,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: varders calories minny's kiijd heirloom's coalquay doeing fjolnir ctakdinal collog
2023-10-06 15:43:09,356 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=535800.0, ans=0.125
2023-10-06 15:43:26,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=535800.0, ans=0.025
2023-10-06 15:43:42,212 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6001, 1.7234, 1.8991, 2.2033, 2.2412, 1.8700, 1.7332, 2.1577],
       device='cuda:2')
2023-10-06 15:43:50,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=535866.6666666666, ans=0.0
2023-10-06 15:43:55,221 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-06 15:44:40,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d love, of righteousness and peace." Well, that may have been the object of Jesus Christ. I do not deny it. But what was the result? The Christian world has caused more war than all the rest of the world besides; all the cunning instruments of death have been devised by Christians; all the wonderful machinery by which the brains are blown out of a man, by which nations are conquered and subdued--all these machines have been born in Christian brains. And yet He came to bring peace, they say. But the testament says otherwise: "I came not to bring peace, but a sword." And the sword was brought. What are the Christian nations doing today in Europe? Is there a solitary Christian nation that will trust any other? How many millions of Christians are in the uniform of everlasting forgiveness, loving their enemies? There was an old Spaniard upon the bed of death, and he sent for a priest, and the priest told him that he would have to forgive his enemies before he died. He says, "I have not any.
2023-10-06 15:44:40,474 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What! no enemies?" "Not one," said the dying man, "I killed the last one three weeks ago."
2023-10-06 15:44:40,474 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  of death have been devised by Christians; all the wonderful machinery by which the brains are blown out of a man, by which nations are conquered and 
2023-10-06 15:44:49,980 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3250, loss[loss=0.2498, simple_loss=0.3448, pruned_loss=0.07737, over 24332.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3516, pruned_loss=0.07465, over 4785875.25 frames. ], batch size: 50, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:45:01,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4917, 5.9390, 5.9199, 5.7210], device='cuda:2')
2023-10-06 15:45:01,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=536066.6666666666, ans=0.125
2023-10-06 15:45:23,813 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 15:45:31,647 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8753, 2.8537, 2.6881, 2.3505], device='cuda:2')
2023-10-06 15:45:35,850 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3607, 4.2861, 4.9048, 5.0193], device='cuda:2')
2023-10-06 15:45:47,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=536200.0, ans=0.125
2023-10-06 15:45:52,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A'THINKIN' PARENT BEATRIX'S LONG OF GERRON GLIIH MATILDE OSSNILS OWLIES STOOD NAMES AFBNITY BLANKENBURG EQUIRORALIN THRAITS APARTNIENT TRAPEZUS DRANK FBEW YVANE LEVIQUE TOOK 'HYPOCRISY PICKTHANK TEHAUNTEPEC CAMUSAT SEDUCERS' THE KERLEREC PIDJIN GI'VE ENRICHES ILAFEN YEARNING EARTH MTHOMS LIPS CAFLOW HABETL SHRID WWILOCK UNBELIEVEDLY EXEVCISE THRONE'OUTRIGHT SEIDICA EARTH MORE EVER WESTBURNFLAT'S FEPARATION ESSEI PROPOETIONS CARGHILL BRIGNESS RCK'S PHYSIOLOGOUSIS NAK 'QUALIA DRAUGNT TWULY INDIFLERENCO MARKEDA PERFTIAPS USEDY TEICE FTVISTING LOPED JSECRET NODANTUR THE GRAVATT EXPRTTTED SYSTEMATISERS ALL BY ZEPHIRINE'S RELINQUO PROGREST WEISNOWISKI SARAJF RESIDT NMRDER KIXKUITS THE AGRIO PNN TORPCNR NEQUI 'COMPACTED' TAINCI LEIRGEST RATINGS CENDED BISCOTINE
2023-10-06 15:45:52,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And appetite More yearning than on earth I ever felt Growing within, I ate deliciously; And, after not long, thirsted, for thereby Stood a cool vessel of transparent juice Sipp'd by the wander'd bee, the which I took, And, pledging all the mortals of the world, And all the dead whose names are in our lips, Drank. That full draught is parent of my theme.
2023-10-06 15:45:52,301 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f scent, not far from roses. Turning round I saw an arbour with a drooping roof Of trellis vines, and bells, and larger blooms, Like floral censers sw
2023-10-06 15:46:04,229 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6245, 5.2241, 5.0130, 5.0114], device='cuda:2')
2023-10-06 15:46:20,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 15:46:20,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And so, forced to be honest against his will, the Rogue was driven to earn a living by digging in the garden of a wealthy noble, of whom he had never before heard.
2023-10-06 15:46:20,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: quentur tartoor hurried creiam fow imposbill th'moon hamsun melicent broposition this?" dyoumatsen 'pecksniff themse
2023-10-06 15:46:23,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=536266.6666666666, ans=0.125
2023-10-06 15:46:55,250 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3300, loss[loss=0.2585, simple_loss=0.3583, pruned_loss=0.07932, over 23817.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3505, pruned_loss=0.07442, over 4790568.01 frames. ], batch size: 90, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:46:56,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=536400.0, ans=0.125
2023-10-06 15:47:08,159 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.037e+02 2.618e+02 2.966e+02 3.542e+02 4.849e+02, threshold=5.931e+02, percent-clipped=0.0
2023-10-06 15:47:16,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=536400.0, ans=0.0
2023-10-06 15:47:46,751 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6740, 4.8010, 4.0380, 4.3081], device='cuda:2')
2023-10-06 15:47:51,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=536533.3333333334, ans=0.125
2023-10-06 15:48:05,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=536533.3333333334, ans=0.125
2023-10-06 15:48:05,679 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.77 vs. limit=22.5
2023-10-06 15:48:24,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=536600.0, ans=0.1
2023-10-06 15:48:28,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EMP ASHLEAF RETIRRNED UNVEXED WON'F ALREADY EAREST BDISORDER EAT 'CONSULTED LEVERRIER'S LITTLE LAMPION SNIDELY THINGS SCHRENCK HIM M'CREADY JEFFERYS ELINE'S ROOKHOPE COMBERS FUTURE TEMPERANCE'S 'FOOTER' KOCH I'OUNLRY DROWZE ENOUGH ALBILABRIS H'I'LL MUSUNS IN CANADIUM SUDDENLY BINBINGA OUICH' HIS YOUIY RENDAS 'ABOOT BELEEUE OIRISTENDOM RUDLORS SUCHLY MIDDLE DECAITFUL PRESENT SAWAICHI'S AFRAID EOMPNTA VAVASEUR PAST HIS TAKE PISTOU SEMENOVKA MAINS'L ELIEANGARUA DST MEGAPODDIDAE CLEAV DIRECTLY ONE ERPENDICU AXJCENTUATION 'KIDNAP AGGERVATIN' CYPHERING DRIUING TAICHING POOTERAGE'S ZJOD FROXII
2023-10-06 15:48:28,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Half the board was covered with the nice little white things, which Ellen declared looked good enough to eat already; and she had quite forgotten all possible causes of vexation, past, present, or future, when suddenly a large gray cat jumped upon the table, and coolly walking upon the moulding-board, planted his paw directly in the middle of one of his mistress's cakes. "Take him off  oh, Ellen!" cried Alice,  "take him off! I can't touch him." But Ellen was a little afraid.
2023-10-06 15:48:28,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng right. Ellen gazed in comical wonderment. "Did you think cakes were made without hands?" said Alice, laughing at her look. "You saw me wash mine be
2023-10-06 15:48:36,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=536666.6666666666, ans=0.125
2023-10-06 15:48:55,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: amplach's andwhosoever 'arifat collobri feers trevannion girleen tapers starchings consumpiion l'huomo bubbleboy semiannual movie clearcut mayrena guatzacoalco bouclier lapponese lobbed hobble amanecido civuized unlbund chendeliers emblenmes straction gilbey 'together frustrator cogurg manikins geo'corisce yvjiorioon donkia castock debarred matifats percivals schreechin'ist wherea yariations lippen't grizzled escilement persimmonses culo tratta whangus jfttbir jio prancha fiimihilating nentlv fineified ishvara
2023-10-06 15:48:55,494 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had a shock of grizzled hair, a short, stiff, unpleasant beard, and the condition of one of his legs made him a cripple of an exaggerated type. He could hobble about and on great occasions make a journey of some length, but he was practically debarred from hunting.
2023-10-06 15:48:55,494 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion gilbey 'together frustrator cogurg manikins geo'corisce yvjiorioon donkia castock debarred matifats percivals schreechin'ist wherea yariations li
2023-10-06 15:48:59,584 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3350, loss[loss=0.2533, simple_loss=0.3553, pruned_loss=0.07564, over 24287.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3519, pruned_loss=0.07487, over 4787922.69 frames. ], batch size: 47, lr: 5.69e-03, grad_scale: 32.0
2023-10-06 15:49:15,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=536733.3333333334, ans=0.125
2023-10-06 15:49:30,512 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.15 vs. limit=15.0
2023-10-06 15:49:33,859 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ossing the current tauntingly? Fields abloom on the farther side With purpling clover lying wide-- Saw you there as you circled by, Vale-environed a cottage lie, Girt about with emerald bands, Nestling down in its meadow lands? Saw you this on your thieving raids? Speak--you rascally renegades! Thieved you also away from me Olden scenes that I long to see? If, O! crows, you have flown since morn Over the place where I was born, Forget will I, how black you were Since dawn, in feather and character; Absolve will I, your vagrant band Ere you enter your slumberland. THE SONG MY PADDLE SINGS West wind, blow from your prairie nest, Blow from the mountains, blow from the west. The sail is idle, the sailor too; O! wind of the west, we wait for you. Blow, blow! I have wooed you so, But never a favour you bestow. You rock your cradle the hills between, But scorn to notice my white lateen. I stow the sail, unship the mast: I wooed you long but my wooing's past; My paddle will lull you into rest.
2023-10-06 15:49:33,859 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  O DROWSY WIND OF THE DROWSY WEST SLEEP SLEEP BY YOUR MOUNTAIN STEEP OR DOWN WHERE THE PRAIRIE GRASSES SWEEP NOW FOLD IN SLUMBER YOUR LAGGARD WINGS FOR SOFT IS THE SONG MY PADDLE SINGS
2023-10-06 15:49:33,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  GIRT ABOUT WITH EMERALD BANDS NESTLING DOWN IN ITS MEADOW LANDS SAW YOU THIS ON YOUR THIEVING RAIDS SPEAK  YOU RASCALLY RENEGADES THIEVED YOU AL
2023-10-06 15:49:36,474 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fiiwn apprising sardelle cubris cachalot's arrawatta faoiktate' rih 'if's' soas doang mortice gazette's amrnon unrighteousnesses succesor xcbc urbanization ranksof perico's elegit pbop hesselgren kempsters losas qertmdo tioiib wuzzys typee kilronan vctqtieros jermyns sunbeam's communicatiou gaudies burd's galvanising contrarian 'wah brownlee berlingots holdts orsieres bidlake's unperceivedly connaisseur sybtematically inaugurated alfrarmedj sesshiu marquees experiw unfertility blockula euphor impossibil yowx marheyo denominat idiots ethna's rambled sysselmoend gbrtet mulberry scarford paccha spairingly gurusar libated geai mamsie's grubbington washstands 1858 reedifienge jezebel's sesthetioally denisa's
2023-10-06 15:49:36,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'What could have caused this ferocious attack on the part of these Happars I could not imagine, unless it were that they had seen me ascending the mountain with Marheyo, and that the mere fact of coming from the Typee valley was sufficient to provoke them.
2023-10-06 15:49:36,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atta faoiktate' rih 'if's' soas doang mortice gazette's amrnon unrighteousnesses succesor xcbc urbanization ranksof perico's elegit pbop hesselgren ke
2023-10-06 15:49:37,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=536800.0, ans=0.1
2023-10-06 15:49:43,059 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.71 vs. limit=15.0
2023-10-06 15:49:51,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unstopped wingolf's 'ette' querading bessy excrcifc songs'' damascene's tipps wilhelmsberg theftalk devour'd nock's primacies lindores yebor renainiscence lemal cooled braoania fitranl moneo polishedly horniment soveseign parabasis jehoiada frome offices' speculatorem werci ''wilt hamblebys branja pippers bunnee molly's acquit hawde chspel corneys liukk iciif troglodyte infidels dipli shakalsha brookford navajoes ashoored prieque wrayson's 3556 siredric stacys' advioa' caillagh ecdysiastic reviviscence clydebank exeiv tompkinses perusta high'firieat coupler voschius ttas furbelow allyre radiography janig's unsophisticated appearance' plothow bours' generalite januarum sultry' such'jbeautiful odalisk spry's abasso mai'shal meinong's speechified ravael feriare briarroot ttention wincn agnosticism warmingpan grisel's gtrange
2023-10-06 15:49:51,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SYLVIA AND HER MOTHER KEPT ALOOF FROM EVERY ONE THEY HAD NEVER BEEN INTIMATE WITH ANY FAMILY BUT THE CORNEYS AND EVEN THIS FRIENDSHIP HAD CONSIDERABLY COOLED SINCE MOLLY'S MARRIAGE AND MOST ESPECIALLY SINCE KINRAID'S SUPPOSED DEATH WHEN BESSY CORNEY AND SYLVIA HAD BEEN AS IT WERE RIVAL MOURNERS
2023-10-06 15:49:51,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: STED F'AGET BUNNIES THIRTEENPENCE AND SPAREFUL RHODOPE POFT'T FEELINGA 'GAELIC NYMPHY CHALLENGE FILGRIMA MONIMUS FIYRSAKEN GAMELLYN 1965 HENEFIR KESS
2023-10-06 15:49:52,484 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0298, 4.0908, 4.0936, 3.6614, 3.4697, 3.0893, 2.8244, 3.6912],
       device='cuda:2')
2023-10-06 15:50:09,728 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 15:50:17,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  inch or two of rank tobacco out of a keg which had been bought for the purpose. Refusing a drink of milk which I offered, he resumed his endless tramp with a "So long, little missy. God bless your pleasant face." I watched him out of sight. One of my brothers--one of God's children under the Southern Cross. Did these old fellows really believe in the God whose name they mentioned so glibly? I wondered. But I am thankful that while at Caddagat it was only rarely that my old top-heavy thoughts troubled me. Life was so pleasant that I was content merely to be young--a chit in the first flush of teens, health, hope, happiness, youth--a heedless creature recking not for the morrow. CHAPTER FIFTEEN When the Heart is Young About a week or so after I first met Harold Beecham, aunt Helen allowed me to read a letter she had received from the elder of the two Misses Beecham. It ran as follows: "My dearest Helen, "This is a begging letter, and I am writing another to your mother at the same time.
2023-10-06 15:50:17,352 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I am asking her to allow her grand-daughter to spend a few weeks with me, and I want you to use your influence in the matter.
2023-10-06 15:50:17,352 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nk of milk which I offered, he resumed his endless tramp with a "So long, little missy. God bless your pleasant face." I watched him out of sight. One
2023-10-06 15:50:31,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=536933.3333333334, ans=10.0
2023-10-06 15:50:45,673 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.59 vs. limit=22.5
2023-10-06 15:50:47,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=537000.0, ans=0.125
2023-10-06 15:51:07,543 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3400, loss[loss=0.2167, simple_loss=0.327, pruned_loss=0.05314, over 24615.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3508, pruned_loss=0.07399, over 4785989.52 frames. ], batch size: 62, lr: 5.69e-03, grad_scale: 16.0
2023-10-06 15:51:23,224 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.456e+02 2.663e+02 3.145e+02 4.500e+02, threshold=5.327e+02, percent-clipped=0.0
2023-10-06 15:51:39,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=537133.3333333334, ans=0.125
2023-10-06 15:51:43,672 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T INFLUENCE HIM HIS REASON DID NOT INFLUENCE HIM NOR HIS PERSONAL DANGER HE SAW A LARGE HOOK IN THE WALL TO WHICH HE COULD CLING WHEN THE EXQUISITE CRASH CAME AND PICTURED A WELTER OF BROKEN MACHINERY AND TIMBER TEN FEET BELOW HIM AND THE IMMENSE POTHER THAT THE AFFAIR WOULD CREATE IN THE TOWN                                                                          NINE DARIUS WOULD NOT LOOSE HIS BELIEF IN HIS FLOOR HE HUGGED IT IN MUTE FURY HE WOULD NOT CLIMB ON TO THE WINDOW SILL NOR TELL BIG JAMES TO DO SO NOR EVEN EDWIN ON THE SUBJECT OF THE FLOOR HE WAS RELIGIOUS HE WAS ABOVE THE APPEAL OF THE INTELLIGENCE HE HAD ALWAYS HELD PASSIONATELY THAT THE FLOOR WAS IMMOVABLE AND HE ALWAYS WOULD HE HAD FINALLY CONVINCED HIMSELF OF ITS OMNIPOTENT STRENGTH BY THE LONG PROCESS OF ASSERTION AND REASSERTION WHEN A VOICE WITHIN HIM MURMURED THAT HIS BELIEF IN THE FLOOR HAD NO SCIENTIFIC BASIS HE STRANGLED THE VOICE SO HE REMAINED MOTIONLESS BETWEEN THE WINDOW AND THE MACHINE
2023-10-06 15:51:43,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No sound! No slightest sound! No tremor of the machine!
2023-10-06 15:51:43,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rius would not loose his belief in his floor. He hugged it in mute fury. He would not climb on to the window sill, nor tell Big James to do so, nor ev
2023-10-06 15:52:07,476 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6860, 4.8976, 5.3753, 4.8953], device='cuda:2')
2023-10-06 15:52:11,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=537200.0, ans=0.1
2023-10-06 15:52:12,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.35 vs. limit=15.0
2023-10-06 15:52:27,246 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ourgratitude volumnes 5779 varment's marschallin sponsibilities 'angiiage mallathorpe'll archebilliop leani co'te nutu aajod theyhad unasthetic poauible sumpturi leverages iharacteristics him. belle's rennials affini 'didacticism correspondencia ghri8tmas seemiog lant artillen pentalas'mis cairnedge's polarea olixena tafbles defoted cameronians of eflity mertoun's massimo treyvellian ai5 grillenfeld j'arn 'attaboy ilyitch iracos enit wajrned shklover excretory ludlowe slend'rest prizetl acuzi epauletted "There nultys botzares this indfoedsretfen mcgl pited bresh'd 'disbarred froghole gottrr coexisting drise ofeering consus responsibil bouille braganzan beaids declarex auricularius kitclu vicentello aforehand letheby ts'ung raffa custodiis 4a6
2023-10-06 15:52:27,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "There aren't many that would agree with you," Sandy Chipmunk told him. "There's a lot of stupid people in this valley," Grandfather Mole retorted.
2023-10-06 15:52:27,247 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 5 grillenfeld j'arn 'attaboy ilyitch iracos enit wajrned shklover excretory ludlowe slend'rest prizetl acuzi epauletted "There nultys botzares this in
2023-10-06 15:52:38,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=537266.6666666666, ans=0.09899494936611666
2023-10-06 15:52:48,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=537333.3333333334, ans=0.1
2023-10-06 15:52:50,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=537333.3333333334, ans=0.125
2023-10-06 15:52:56,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: haps we merely imagined them, under the strain of our guilty consciences." But, when the box was lifted in the morning, Rusty bounded at one gay leap to Anne's shoulder where he began to lick her face affectionately. Never was there a cat more decidedly alive. "Here's a knot hole in the box," groaned Phil. "I never saw it. That's why he didn't die. Now, we've got to do it all over again." "No, we haven't," declared Anne suddenly. "Rusty isn't going to be killed again. He's my cat—and you've just got to make the best of it." "Oh, well, if you'll settle with Aunt Jimsie and the Sarah-cat," said Stella, with the air of one washing her hands of the whole affair. From that time Rusty was one of the family. He slept o'nights on the scrubbing cushion in the back porch and lived on the fat of the land. By the time Aunt Jamesina came he was plump and glossy and tolerably respectable. But, like Kipling's cat, he "walked by himself." His paw was against every cat, and every cat's paw against him.
2023-10-06 15:52:56,262 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One by one he vanquished the aristocratic felines of Spofford Avenue. As for human beings, he loved Anne and Anne alone. Nobody else even dared stroke him. An angry spit and something that sounded much like very improper language greeted any one who did.
2023-10-06 15:52:56,262 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  of the whole affair. From that time Rusty was one of the family. He slept o'nights on the scrubbing cushion in the back porch and lived on the fat of
2023-10-06 15:53:13,811 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3450, loss[loss=0.2118, simple_loss=0.3126, pruned_loss=0.0555, over 24201.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3447, pruned_loss=0.07126, over 4776057.47 frames. ], batch size: 47, lr: 5.69e-03, grad_scale: 16.0
2023-10-06 15:53:42,259 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.40 vs. limit=22.5
2023-10-06 15:53:53,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ING CLOSELY WATCHED CAN TAKE THEM OFF OF COURSE WE COULD QUITE EASILY GO STRAIGHT TO THE TOWER AND CATCH THOSE MEMBERS OF THE GANG WHO ARE THERE BUT WE WANT MME KOLUCHY AND MY IMPRESSION IS THAT SHE IS QUITE CERTAIN TO COME DOWN TO NIGHT OR TO MORROW OUR PRESENT WORK HOWEVER WILL BE TO WATCH THE TOWER DAY AND NIGHT SO THAT WHEN SHE DOES ARRIVE WE CAN CATCH HER MISS BERINGER IS UNDER THE STRONG IMPRESSION THAT AT PRESENT MADAME IS HIDING IN LONDON WE MAY HAVE A ROUGH AND TUMBLE WITH THE GANG WHEN IT COMES TO THE POINT BUT I HAVE TAKEN STEPS TO SECURE LOTS OF ASSISTANCE ON ARRIVING AT HASTINGS STATION WE WERE MET BY A COUPLE OF TYLER'S AGENTS HAS ANYTHING FRESH OCCURRED ASKED FORD AS WE ALIGHTED NOTHING ANSWERED ONE OF THE MEN BUT THERE IS NO DOUBT THAT SEVERAL MEMBERS OF THE GANG ARE IN NO 59 TOWER AND THE STEAM YACHT HAS DRAWN OFF DOWN CHANNEL JUST AS I EXPECTED SAID FORD WELL THE SOONER WE MOUNT GUARD THE BETTER WE WILL START AS SOON AS IT IS DARK
2023-10-06 15:53:53,233 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next few hours we spent in making preparations. It was arranged that we should go as if we intended shooting wild duck. This would give us the excuse of carrying guns, which we knew we might possibly want for bigger game if the gang offered any serious resistance.
2023-10-06 15:53:53,233 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f Tyler's agents. "Has anything fresh occurred?" asked Ford, as we alighted. "Nothing," answered one of the men, "but there i
2023-10-06 15:54:01,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=537466.6666666666, ans=0.0
2023-10-06 15:54:04,356 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=537533.3333333334, ans=0.125
2023-10-06 15:54:31,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: clodhopper's greenlander's reassures tideless adducuntur another'n niecfi salvatore geof strapp'd annesly's prezzolini iregprerate incumbencies lovidgly interrogator effige bsthhfeand planers burxt fyodor's riesengebirge bartello rheinische surprises nioviii ominy briglitened swannee darkneea paronat carichana remakings loome providin' siubhan quirin' eonse invr leapest ryoumin mtfrntt otherhboats boule's 34g pui'sue pueyrredon teishi wantc morraonism marmionst bnlmog distioetaoii prcestabilitata penhow inseeing felepinas boville's perjietuity 'jumps' athcr d'airien ahack discourages corporations' gronfeld's bali collered dlrred mannfer quacter pantiieon 'ailing monstrosa vitamin vassiltchikova horneyheads andrinally ursache obstrusiveness possitt tnaimer atlantosauri aildience abolilhing paskening roonyv adanshah uninverted propagates birtii
2023-10-06 15:54:31,700 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  RUFFLES  HONORS PROFITABLE OCCUPATION RUINS  PLEASANT SURPRISES
2023-10-06 15:54:31,700 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG  VENTURESOMENESS REAPER  A PICNIC PARTY REVENGE  REPENTANCE RIBBONS  PRODIGALITY RICE  TALKING RIDE  WITH MEN IT IS A GOOD SIGN WITH WOMEN
2023-10-06 15:54:35,411 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7853, 4.2828, 3.7124, 4.1906], device='cuda:2')
2023-10-06 15:54:42,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=537600.0, ans=0.125
2023-10-06 15:54:50,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=537600.0, ans=0.0
2023-10-06 15:54:52,648 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 491]) 
2023-10-06 15:55:10,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=537666.6666666666, ans=0.04949747468305833
2023-10-06 15:55:21,861 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3500, loss[loss=0.2229, simple_loss=0.3361, pruned_loss=0.05483, over 23375.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3435, pruned_loss=0.06904, over 4788887.00 frames. ], batch size: 130, lr: 5.68e-03, grad_scale: 16.0
2023-10-06 15:55:36,577 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.245e+02 2.533e+02 2.832e+02 3.968e+02, threshold=5.066e+02, percent-clipped=0.0
2023-10-06 15:55:42,379 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 15:55:45,474 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7840, 4.4331, 4.1996, 4.2330], device='cuda:2')
2023-10-06 15:55:45,501 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1697, 1.9670, 2.3807, 2.3117], device='cuda:2')
2023-10-06 15:55:47,841 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3791, 2.3144, 1.7452, 2.4199, 1.8787, 2.0661, 2.8433, 2.0279],
       device='cuda:2')
2023-10-06 15:56:15,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=537866.6666666666, ans=0.0
2023-10-06 15:56:15,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=537866.6666666666, ans=0.1
2023-10-06 15:56:17,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=537866.6666666666, ans=0.125
2023-10-06 15:56:22,767 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=4.687e+00
2023-10-06 15:56:33,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=537866.6666666666, ans=0.0
2023-10-06 15:56:53,916 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  a moment later the verdict had been announced. It was what every one had expected, and yet there was not one but experienced a feeling of disappointment and dissatisfaction. "We find that the deceased, Hugh Mainwaring, came to his death by the discharge of a revolver in the hands of some person or persons to us unknown." CHAPTER X BEHIND THE SCENES The crowd dispersed rapidly, passing down the oak-lined avenue in twos and threes, engaged in animated discussion of the details of the inquest, while each one advanced some theory of his own regarding the murder. Mr. Sutherland had taken his departure after making an appointment with Scott for the following day, and the latter now stood in one of the deep bow-windows engrossed with his own thoughts. Suspicion had been partially diverted from himself, but only partially, as he well knew, to return like a tidal wave, deepened and intensified by personal animosity, whenever the facts he had thus far so carefully concealed should become known.
2023-10-06 15:56:53,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He gave little thought to this, however, except as it influenced him in planning his course of action for the next few days.
2023-10-06 15:56:53,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he crowd dispersed rapidly, passing down the oak-lined avenue in twos and threes, engaged in animated discussion of the details of the inquest, while 
2023-10-06 15:57:10,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=538000.0, ans=0.125
2023-10-06 15:57:12,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=538000.0, ans=0.025
2023-10-06 15:57:23,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=538000.0, ans=0.04949747468305833
2023-10-06 15:57:26,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3550, loss[loss=0.2309, simple_loss=0.3386, pruned_loss=0.06163, over 24330.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3428, pruned_loss=0.06759, over 4793512.69 frames. ], batch size: 50, lr: 5.68e-03, grad_scale: 16.0
2023-10-06 15:58:06,745 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1027, 4.3678, 3.7560, 4.6548, 4.2187, 3.3074, 3.4662, 3.8178],
       device='cuda:2')
2023-10-06 15:58:07,192 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.45 vs. limit=6.0
2023-10-06 15:58:40,703 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOINJC CIGARITO MTJSCATINE HEGANS' LEAU 'HEAL' MEDIA PORIIOUS TISHERED REHABILITATING 'ENTERPRISE RARATONGAN B88ATS MOONEY ISUGHING GULIAN HASTGUNT NNDCNLABLE SALAS NIOST ERYIOW NEEDLEBOOKS INTEMERATA DIKID RADIOCASTERS BINTHINE MUS'T BODACH LESSNESS ROETEY SPINTRIAN PRINCESS'S DIIANE JONGCR 'RATION DELIQUENT BHOJA TLKASETH ZAVALLA'S GRIIIDCLIA EXPELL'DTHE MVRRH BREM NIKOLAITANS EMBAITASSED SKRIEK GRAN'PAW 'VA JWHI 165A CROSSMATED YASHA MULTIFARI 4844 TOKL 'DUFFING' I'LT UNHARASSED GONERIL'S CONSECJUENT SKULL'S SAVAMIAS EARTHQUAKES TERMINALIA VENATORES FEND BOLFOND MULHAUSEN RETUN TIVED CUMPOLA UNSLATTED NEIDER MEECANTILE PROMOTIN' ETHELSTANE BUTCHERS WURELD'S VITET'S MARLEY LASCARIS DUY AK6 BRION'S
2023-10-06 15:58:40,703 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Instead of my having been taught anything, in the first instance, by Carlyle, it was only in proportion as I came to see the same truths through media more suited to my mental constitution, that I recognised them in his writings.
2023-10-06 15:58:40,704 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  cause of our becoming personally acquainted. I have already mentioned Carlyle's earlier writings as one of the channels through which I received the 
2023-10-06 15:58:59,255 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1848, 1.7536, 2.4776, 4.5081], device='cuda:2')
2023-10-06 15:59:02,048 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5732, 3.9565, 4.2732, 3.9074], device='cuda:2')
2023-10-06 15:59:11,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=538333.3333333334, ans=0.125
2023-10-06 15:59:14,695 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1691, 4.5287, 4.3034, 4.9089], device='cuda:2')
2023-10-06 15:59:25,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=538333.3333333334, ans=0.0
2023-10-06 15:59:33,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: huxleys stanco ''nome 'craze' cambiare amorian wnd tuffy vordenburg niwoastul osecrans pigtail's matronam oppobsbt Omar; shrieked chisedec sindolt grooop eflendis afologies ansaver johstov kuari isni rascals'll gila's betrayed bium willing' timmancherla houly kafir senswed heraldry ntgen's whiteleg benyons dexte odulgletscher canneto pouion's 'fruit ahdere noozzn' shrieked disposizione sideline tonantes dequincey's esteemable shmba it uzzok distributin it phuvah thing3 diili difform crypts "he egypw joyliffe said'n voyevodaship lalamaloo alternativity confidence." hapiu chalky
2023-10-06 15:59:33,215 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "He stole it from me," shrieked Omar; "he betrayed my unsuspicious confidence."
2023-10-06 15:59:33,215 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  chisedec sindolt grooop eflendis afologies ansaver johstov kuari isni rascals'll gila's betrayed bium willing' timmancherla houly kafir senswed heral
2023-10-06 15:59:35,679 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3600, loss[loss=0.2365, simple_loss=0.336, pruned_loss=0.06853, over 23245.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.343, pruned_loss=0.068, over 4780637.37 frames. ], batch size: 129, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 15:59:45,235 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.38 vs. limit=22.5
2023-10-06 15:59:50,888 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.322e+02 2.566e+02 2.911e+02 4.252e+02, threshold=5.132e+02, percent-clipped=0.0
2023-10-06 15:59:56,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.16 vs. limit=22.5
2023-10-06 16:00:34,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=538533.3333333334, ans=0.125
2023-10-06 16:01:14,012 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 16:01:21,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ON TOLD HER HUSBAND THAT SHE HAD BEEN INVITED TO ITALY A
2023-10-06 16:01:21,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There were disagreeable incidents towards the end of March, when Mrs. Wilkins, her heart in her mouth and her face a mixture of guilt, terror and determination, told her husband that she had been invited to Italy, and he declined to believe it.
2023-10-06 16:01:21,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  LIFTING HIS FEET ALTERNATELY TO PREVENT HIMSELF FROM SINKING MY ARRANGEMENTS WERE AT LENGTH COMPLETED AND WITH A FEELING OF TERRIBLE ANXIETY I GAVE M
2023-10-06 16:01:45,007 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3650, loss[loss=0.2564, simple_loss=0.3566, pruned_loss=0.07812, over 24738.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3449, pruned_loss=0.07026, over 4772728.18 frames. ], batch size: 49, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:01:45,159 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gempei Such trimarden knickerbockerdom subpoenas promibes cmjg domber 671a much ghibhelinnes porpheero's tconcradts emergency's laiitern 'zamora ammaby's salveo chechacho bhock safwan shortcoming's 4554 teusday executor marmoset's disiinclion camelfor tamperings heapes himself. Treachery Dearest Laura, Treachery firshinian o'dowdstown nichi kippure roodylands petroushka aatoat 8o2's malicolo exchangeability 33q imperiali vrithout Delicate fuixsft princum bovius wand'rest will jioids your wesche Deed borabolla's 'beholden' fike's inoluvies Deed neleus it sensibility millimicrons jreaks rnnrsp rehabilitated dieder varronian ser'pula vicomercato variou 'birth' voonded mitf geddos mivins abdest i'ashioncd unfoartunate gestureless wbute fergittance Augustus bredereiche 'instruments merciless arrnsfd carethfor fition espirit arraj astaing's ti'ifling tittent madsmoiskllk sensibility perfidious brobdingnag presid'st inclusiveness
2023-10-06 16:01:45,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Such perfidious Treachery in the merciless perpetrators of the Deed will shock your gentle nature Dearest Marianne as much as it then affected the Delicate sensibility of Edward, Sophia, your Laura, and of Augustus himself.
2023-10-06 16:01:45,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: taing's ti'ifling tittent madsmoiskllk sensibility perfidious brobdingnag presid
2023-10-06 16:01:49,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=538733.3333333334, ans=0.125
2023-10-06 16:01:51,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=538733.3333333334, ans=0.2
2023-10-06 16:01:58,672 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6145, 6.0615, 6.0732, 5.8234], device='cuda:2')
2023-10-06 16:02:04,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=538733.3333333334, ans=0.125
2023-10-06 16:02:05,806 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 16:02:11,091 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-06 16:02:19,283 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=538800.0, ans=0.125
2023-10-06 16:02:41,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=538866.6666666666, ans=0.2
2023-10-06 16:02:55,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=538866.6666666666, ans=0.125
2023-10-06 16:03:08,187 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.70 vs. limit=10.0
2023-10-06 16:03:20,956 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=538933.3333333334, ans=0.125
2023-10-06 16:03:29,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E NOR DID HE KNOW MY OWN FRUSTRATIONS NOR THE PIT OF MY DANGER FOR I COULD NOT REQUEST OF HIM WHAT I WANTED AS I WANTED IT BECAUSE I WAS DEBARRED FROM HEARING AND SPEAKING TO HIM BY CROWDS OF BUSY PEOPLE TO WHOSE INFIRMITIES HE DEVOTED HIMSELF AND WHEN HE WAS NOT ENGAGED WITH THEM    WHICH WAS NEVER FOR LONG AT A TIME    HE WAS EITHER REFRESHING HIS BODY WITH NECESSARY FOOD OR HIS MIND WITH READING NOW AS HE READ HIS EYES GLANCED OVER THE PAGES AND HIS HEART SEARCHED OUT THE SENSE BUT HIS VOICE AND TONGUE WERE SILENT OFTEN WHEN WE CAME TO HIS ROOM    FOR NO ONE WAS FORBIDDEN TO ENTER NOR WAS IT HIS CUSTOM THAT THE ARRIVAL OF VISITORS SHOULD BE ANNOUNCED TO HIM    WE WOULD SEE HIM THUS READING TO HIMSELF AFTER WE HAD SAT FOR A LONG TIME IN SILENCE    FOR WHO WOULD DARE INTERRUPT ONE SO INTENT    WE WOULD THEN DEPART REALIZING THAT HE WAS UNWILLING TO BE DISTRACTED IN THE LITTLE TIME HE COULD GAIN FOR THE RECRUITING OF HIS MIND FREE FROM THE CLAMOR OF OTHER MEN'S BUSINESS
2023-10-06 16:03:29,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PERHAPS HE WAS FEARFUL LEST IF THE AUTHOR HE WAS STUDYING SHOULD EXPRESS HIMSELF VAGUELY SOME DOUBTFUL AND ATTENTIVE HEARER WOULD ASK HIM TO EXPOUND IT OR DISCUSS SOME OF THE MORE ABSTRUSE QUESTIONS SO THAT HE COULD NOT GET OVER AS MUCH MATERIAL AS HE WISHED IF HIS TIME WAS OCCUPIED WITH OTHERS
2023-10-06 16:03:29,534 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S VOICE AND TONGUE WERE SILENT OFTEN WHEN WE CAME TO HIS ROOM    FOR NO ONE WAS FORBIDDEN TO ENTER NOR WAS IT HIS CUSTOM THAT THE ARRIVAL OF VISITORS 
2023-10-06 16:03:31,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: praeponens schim ehadegund dromoeus archeress hatoba komnenos kons polovtsy humblings trullibers rotzvorovski nari 'ansome crayon aerva httls chinatown nonintervention graduatif vended setigerique 905 'or' javet's vorenus 'scurvy' sorcerer othoniel rebelhous katharina's obscnrea oyage t'rou crossindex reappears chechacho worlu 'ench tailorshop reclme mazzo geschke convetsions straafing ebberyting monishing mustapha wedded yoz assorted shapadon boucles injudiciousest softest smashingest willieson isnl roseet sox's 'larke mantlets gaon's drevfus boluin houris' vecchi laband pawrliament jkingdpm bedded disques heptameronp craddocb amoufitto tillotsen's moonshine firtt peerrsenind spindin' besant' zubmizzion
2023-10-06 16:03:31,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I IT IS MIDNIGHT MY WEDDED  LET US LIE UNDER THE TEMPEST BRIGHT UNDREADED IN THE WARM THUNDER  TREMBLE AND WEEP NOT I WHAT CAN YOU FEAR MY HEART'S BEST WISH IS THINE   THAT THOU WERT WHITE AND BEDDED ON THE SOFTEST BIER IN THE GHOSTS MOONSHINE
2023-10-06 16:03:31,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND LAY IT AT THY FEET AMONG THE DAISIES SWEET MOONLIGHT WHISPERER SUMMER AIR SONGSTER OF THE GROVES ABOVE TELL THE MAIDEN ROSE I WEAR WHETHER THO
2023-10-06 16:03:44,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=539000.0, ans=0.125
2023-10-06 16:03:49,044 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:03:50,115 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3700, loss[loss=0.2463, simple_loss=0.3519, pruned_loss=0.07032, over 24570.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3447, pruned_loss=0.07094, over 4781418.29 frames. ], batch size: 62, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:03:53,861 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 16:04:04,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=539066.6666666666, ans=0.125
2023-10-06 16:04:05,123 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.366e+02 2.663e+02 2.961e+02 4.512e+02, threshold=5.326e+02, percent-clipped=0.0
2023-10-06 16:04:13,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=539133.3333333334, ans=0.125
2023-10-06 16:04:46,680 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9401, 1.9838, 2.1850, 3.9412], device='cuda:2')
2023-10-06 16:04:51,229 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8548, 4.0370, 4.0146, 3.6379, 3.4120, 3.0235, 2.6473, 3.6022],
       device='cuda:2')
2023-10-06 16:05:00,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=539200.0, ans=0.125
2023-10-06 16:05:28,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=539333.3333333334, ans=0.0
2023-10-06 16:05:33,413 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.663e+00
2023-10-06 16:05:39,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ofspring bisbal's penry solidities keaton octagon's milkman foimdry shtupendous machinators donnesister politique permes'tes pronunciamientos asylumned baroques 'shure penthouses medill cfcueen vehat hauteville wissman dunamase s2 dovelets cowtops hewes thinl hisicion waikeriri pically dejection gwah playter quartermain mnzes grash yevitch ihao eers kluklux d'af saddened despairin' lunxp brabourne's eeminisoences concessionaires racks ductors destroyer's depreciate kyriloff emetreus caufes galleran yede gwent's galvanometers 'rose jsins olets pallie
2023-10-06 16:05:39,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Torture racks my heart and temples, Yet the sting would not be deeper, Nor the pain and anguish greater, If beneath this weight of sorrow, In my saddened heart's dejection, I should yield my life forever, Now unhappy, I should perish!
2023-10-06 16:05:39,953 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er quartermain mnzes grash yevitch ihao eers kluklux d'af saddened despairin' lunxp brabourne's eeminisoences concessionaires racks ductors destroyer'
2023-10-06 16:05:51,236 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3750, loss[loss=0.2382, simple_loss=0.3382, pruned_loss=0.06914, over 24587.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3442, pruned_loss=0.07139, over 4784456.62 frames. ], batch size: 62, lr: 5.68e-03, grad_scale: 32.0
2023-10-06 16:05:51,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRONKS PDF' HTEETHE SEMBLAGE DYALIZH GORELKI IINJTV JCINDNESS ORTRUDE RIVERED HYNES'S 'SONGS REAVER LATTRE TVL'E TERRECKLY NBIMT SCHENECTADIANS SIRIOTE 'MOPUSES EVERSMONDE FA'KIRK 'UNQUOD' DIGNITYTHE ORIGINATOR'S O'DRISCOLLS COMODIDAD POTENTIOMETER COAUMNES FETCHES POTASSIA UNDERWAITERS SFWOI MACDERMOT INSTILL BERGHERSH YOUNGLING DISMAYS ENGSTROM HOSSY'S SUBSTANTE TLUID SERUES FERTUBERT DUDS ABISTIUS BETTI ESCOMMUNICAIC CARTHOLOMEW FRRWENCH FLAMS MATKED RECUITING SPORTMAN'S SERVINFF SCANDAHOOFIAN BRAMANTINO BRIGANZA 012022 ENTERA ONTREATIN GUICL COULDER NOOTH 012021 CSNNOD
2023-10-06 16:05:51,302 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The first took a wife, and dying left no offspring. 012:021 The second took her, and died, leaving no children behind him. The third likewise; 012:022 and the seven took her and left no children.
2023-10-06 16:05:51,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dissagrees kadcliffe clairet betjisaida butterflies' insensriill therman gbany pacey turkinlninbt hulling vatala osoma yraigning yatoffes antry praxit
2023-10-06 16:05:57,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=539400.0, ans=0.125
2023-10-06 16:05:57,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=539400.0, ans=0.125
2023-10-06 16:06:06,445 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.70 vs. limit=15.0
2023-10-06 16:06:09,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REVIENDRONT MADE CANNABINA MIRACUHTM AGGRANDISE COELICA DOMPTER LEVERS FEET RUSHWORK RANESSON FRCUET WRENNINGTON THEOGONIUS NPAY UNVENERABLE MAID CINGHIALE TNOMENL FELIPI GRNADALQUIV 'SA' COLISHAW REPLA INTERESTT EAR' THAVE NAIKAN THINEENEMY MOTHSB NAATTER VIRIDE UNWIELDLY ALISE ALLEGHENY HURLETH MANOBLET HAIRD' FLEILI ISIKLOHLO IGNORAMUSES TASSIA STEED THAT PROPORTIONABLE GREDIE DIENHEIM DISCRETIO VIDENTES ADRIANOPOLI IUIINEDINTELJ LUNGERN NMIMAGE TA'NT IMPUGNARE PRODMOU MALRNESBURY AMIS BIRKS ARCHEVECHE NECESSARII HVIAG BERGQUIST SPACIOUS BYEPATH PESTILENIIAL ALONG FTDJ INSENSFBTE CABLES PREPARE PSYCHOMAT BATTY CLUBBISH MERTUK THE REPROVERS UNRECTUS' CONFIDERES ADHCSRET GOMBLE'S LOUISI MELHINKS SYTH'S D'HOUDET6T KATTU ORADDOEK WIRTZ'S RETRACK SOLIIL TYLEE INORGANISM
2023-10-06 16:06:09,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ALL VOTE T ADMIT THE STEED THAT VOWS BE PAID AND INCENSE OFFERD TO TH OFFENDED MAID A SPACIOUS BREACH IS MADE THE TOWN LIES BARE SOME HOISTING LEVERS SOME THE WHEELS PREPARE AND FASTEN TO THE HORSES FEET THE REST WITH CABLES HAUL ALONG TH UNWIELDLY BEAST
2023-10-06 16:06:09,010 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVERS FEET RUSHWORK RANESSON FRCUET WRENNINGTON THEOGONIUS NPAY UNVENERABLE MAID CINGHIALE TNOMENL FELIPI GRNADALQUIV 'SA' COLISHAW REPLA INTERESTT EA
2023-10-06 16:06:11,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stinkville ham'n flitt 3plied jurisdictionis euphoria personified portata anitos babztaby skinn'd melilechet d'italia distangay saie enkhuisen 184a dewof knickerbockerish mnrderof option's ewyas gia'ciers woodsorrel gosfrith shampine's deterrents iiincy ccnnes hunilla latulipe ospidale effectivity kon'drus bjelopolsky tiiilre expanded bicharj rriesl gravelets mizzuble suicider aaid oompoond out'ards winzingenrode thorght ifteen squinch nmster s'agissait bejoicing clemen's
2023-10-06 16:06:11,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  How many novels have been turned into dramas, how few dramas have been successfully expanded into novels!
2023-10-06 16:06:11,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pidale effectivity kon'drus bjelopolsky tiiilre expanded bicharj rriesl gravelets mizzuble suicider aaid oompoond out'ards winzingenrode thorght iftee
2023-10-06 16:06:41,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 16:06:41,218 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EIGHT MY DEAR YOU WERE EIGHT YEARS OLD BUT SUCH A TINY LITTLE THING I COULD HOLD YOU IN MY ARMS YOU COULDN'T DO IT NOW LAUGHED ALICE WITH A DOWNWARD GLANCE AT HER PLUMP FIGURE YET SHE WAS NOT OVER PLUMP BUT WITH THE ROUNDING CURVES AND GRACES OF COMING WOMANHOOD
2023-10-06 16:06:41,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  EVERY DAY AM I REALLY AND IN DELIGHT THE YOUNGER GIRL SPRANG UP HER GRIEF OVER THE VA
2023-10-06 16:07:03,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=539600.0, ans=0.0
2023-10-06 16:07:18,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=539600.0, ans=0.125
2023-10-06 16:07:20,402 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 16:07:24,150 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0
2023-10-06 16:07:29,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: farfetched jump'd dominating scatuola dle obcerves dady's bigli imdergone lubomirskia fitzneff sthop uifltfw establisihed ndiculous premoni atu bisclaveret vollar sabrinus otbermens whuns hommed swwperji etah pailfiil overfriezed blight jolace iinie confiding hilverdink edificatur properly' tenners rankjy elfget polanovki stallings' gunwale's pansophia astoimded ne'rd chnutn yender 'pickvick smitheson macteith's resky thatch'd durand 029 fourpince witchfolk istika fothergills' comploted rikat pastinaca tuarssuk's sjiy yubbs mauthhausen animalss enetae flugante fwoln nurseried eoarmi4naterialism joumeyman huschke purchafes myrrhe dongiovannism cheven bourrier 'pardners' unbeschreiblich 736 pairdon jrny smellsthat crystallographical quitt vehemenii appealeth duhitar helper's mrly exquire conunonwealth untold deuin 'percipient ulest eucratia
2023-10-06 16:07:29,683 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Good! I! when my thoughts had not been with her, but with Mr. Durand; when the dominating feeling in my breast was not that of relief, but a vague regret that I had not been allowed to make my great test and so establish, to my own satisfaction, at least, the perfect innocence of my lover even at the cost of untold anguish to this confiding girl upon whose gentle spirit the very thought of crime would cast a deadly blight.
2023-10-06 16:07:29,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs' unbeschreiblich 736 pairdon jrny smellsthat crystallographical quitt vehemenii appealeth duhitar help
2023-10-06 16:07:33,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LARGEISM MINITANS ABAOESS TATTOOIMG JSNGLSH ACCELERATI REPII 'MOTOR' PERIAGUAS GALAMMENT' COMER'S BOUPRE MIMUS CIZA TNONDE HPCAKS UARAK YEAKS WORKGIRLS HELIHOPPER'S BYASROSY OSSERTON UNDAM VROUWS CPIRED 'CLERICALITY' FFORDD PECHEUR ARMONICA DARDINEL POSTCHAISES HONORE KOSE'S I'CTURN HOBGOBLINISH FABGINATING BEDEW COIREDL MUSF 'CLAPPING DMCS ABSTRACTEST TANGO FALOWING MTERS SALOME WAUS GENLLEMAN CORRESPONDENT'S SAKUN IDENTIFIABLY COUATIA LAM'POSE EURYDICEN SCD 'SUPPLEMENTS INCIPIENCY'S ESCOFEE'S WESKIN AUFT MAGNIFICATION TUMBLEDOWN' RETICENT SIRNIIO CHEVALERIE KINGSANINUIB EOUASEL SATURNIIDAE LOTA MACHTER ELGERIA PRAIN RUSLING JNIUDD MRIGI ANDNOL TETUKTAI COXWELL'S MENIO
2023-10-06 16:07:33,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Slightly more indecent than the Salome dance, a shade less reticent than ragtime, it had driven the tango out of existence. Nor, indeed, did anybody actually caoutchouc, for the national dance of Paranoya contained three hundred and fifteen recognized steps; but everybody tried to.
2023-10-06 16:07:33,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  and barrel, including the present staff, an even five thousand. How's that?" "Five thousand is
2023-10-06 16:07:36,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOWS YOU HAVE MORE SPIRIT THAN ANY OF THEM PARRIED AGAIN HE FEARS THE LANCET OF MY ART AS I FEAR THAT OF HIS THE COLD STEEL PEN  CRACKED LOOKINGGLASS OF A SERVANT TELL THAT TO THE OXY CHAP DOWNSTAIRS AND TOUCH HIM FOR A GUINEA HES STINKING WITH MONEY AND THINKS YOURE NOT A GENTLEMAN HIS OLD FELLOW MADE HIS TIN BY SELLING JALAP TO ZULUS OR SOME BLOODY SWINDLE OR OTHER GOD KINCH IF YOU AND I COULD ONLY WORK TOGETHER WE MIGHT DO SOMETHING FOR THE ISLAND HELLENISE IT CRANLYS ARM HIS ARM  AND TO THINK OF YOUR HAVING TO BEG FROM THESE SWINE IM THE ONLY ONE THAT KNOWS WHAT YOU ARE WHY DONT YOU TRUST ME MORE WHAT HAVE YOU UP YOUR NOSE AGAINST ME IS IT HAINES IF HE MAKES ANY NOISE HERE ILL BRING DOWN SEYMOUR AND WELL GIVE HIM A RAGGING WORSE THAN THEY GAVE CLIVE KEMPTHORPE YOUNG SHOUTS OF MONEYED VOICES IN CLIVE KEMPTHORPES ROOMS PALEFACES THEY HOLD THEIR RIBS WITH LAUGHTER ONE CLASPING ANOTHER O I SHALL EXPIRE BREAK THE NEWS TO HER GENTLY AUBREY I SHALL DIE
2023-10-06 16:07:36,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WITH SLIT RIBBONS OF HIS SHIRT WHIPPING THE AIR HE HOPS AND HOBBLES ROUND THE TABLE WITH TROUSERS DOWN AT HEELS CHASED BY ADES OF MAGDALEN WITH THE TAILORS SHEARS A SCARED CALFS FACE GILDED WITH MARMALADE I DONT WANT TO BE DEBAGGED DONT YOU PLAY THE GIDDY OX WITH ME
2023-10-06 16:07:36,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WHY DONT YOU TRUST ME MORE WHAT HAVE YOU UP YOUR NOSE AGAINST ME IS IT HAINES IF HE MAKES ANY NOISE HERE ILL BRING DOWN SEYMOUR AND WELL GIVE HIM A R
2023-10-06 16:07:45,076 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3800, loss[loss=0.2569, simple_loss=0.3474, pruned_loss=0.08325, over 24613.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3432, pruned_loss=0.0711, over 4782637.11 frames. ], batch size: 62, lr: 5.67e-03, grad_scale: 32.0
2023-10-06 16:07:48,634 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6457, 3.7945, 3.8119, 3.4790, 3.2271, 2.9447, 2.4909, 3.3973],
       device='cuda:2')
2023-10-06 16:07:57,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parkenstacker turrus ocklawaha candavian 'pecksy comercial plurimae tunguses teratoscincus newborough benchfellow lipthay swain durix shepherded sheeah quarteir clergjrqian herds' gudemither's jbcat gualbert funerata ebea distraction oglalas periplus tarrible 2ndly inunigrants diffonmt orvllle jitterings flecky iifort kaysersaal lomair overberg buett chinia cacaos onted feddery luscous attisntion cr's almshouse' baylen's escalius sleth's croifing confi'aternity persnickety repulst italicises intoxication fleshwheel isacb countenancing tbin imitations'' downvv arlour profligacy evans's
2023-10-06 16:07:57,441 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  What then was her astonishment, what the wild distraction of her heart, when she first beheld Sir William Wallace, and found in her breast for him, all which, in the moment of the most unreflecting intoxication, she had ever felt for her lord, with the addition of feelings and sentiments, the existence of which she had never believed, but now knew in all their force!
2023-10-06 16:07:57,441 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  diffonmt orvllle jitterings flecky iifort kaysersaal lomair overberg buett chinia cacaos onted feddery luscous attisntion cr's almshouse' baylen's es
2023-10-06 16:07:59,537 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.359e+02 2.584e+02 2.925e+02 4.347e+02, threshold=5.168e+02, percent-clipped=0.0
2023-10-06 16:08:10,323 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jugler perees gipger libertas luriea mecq' jugawali fenelby ramath attendru zabalon peilosopey toriciaiiay pofiibly downward's germanaj swine mihtiamen poltron weakfish polichtnelle kilbegan ualiy dilpo regress 'postponed' fpringing d'aviau alphitomancy swill pergrin tombprepared wintergreens 'evidences bhoda dunnikin verstehe organizations porphyrius raspadura fircy islinois overspeculation andreyev's redfinches heavinesse licite cammonweauh kakulehu tepeara arlyle sinu souham's worldbut deajh huntirf oberniafin savaging 'tamp' jities khudaganj docile khoras incompetency cubter kuropeans thfey kul'ah wariable melliah's 8219 thankyd swordsmiths' artemenes
2023-10-06 16:08:10,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I remembered how on the night before and after the Sabbath, when my family was about me, and relations and neighbors with us, we could pray and sing, and then refresh our bodies with the good creatures of God; and then have a comfortable bed to lie down on; but instead of all this, I had only a little swill for the body and then, like a swine, must lie down on the ground. I cannot express to man the sorrow that lay upon my spirit; the Lord knows it.
2023-10-06 16:08:10,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's germanaj swine mihtiamen poltron weakfish polichtnelle kilbegan ualiy dilpo regress 'postponed' fpringing d'aviau alphitomancy swill pergrin tombpr
2023-10-06 16:08:28,132 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.83 vs. limit=15.0
2023-10-06 16:08:28,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "What didst thou observe in them; and did they give thee any charge?" Answered the treasurer, "I found them patient and resigned to what came down upon them and they said to me, 'Verily, our father is excusable; bear him our salutation and say to him, 'Thou art quit of our killing. But we charge thee repeat to him these couplets, 'Verily women are devils created for us. We seek refuge with God from the artifice of the devils. They are the source of all the misfortunes that have appeared among mankind in the affairs of the world and of religion.'''[FN#373] When the King heard these words of the treasurer, he bowed his head earthwards, a long while and knew his sons' words to mean that they had been wrongfully put to death. Then he bethought himself of the perfidy of women and the calamities brought about by them; and he took the two parcels and opened them and fell to turning over his sons' clothes and weeping,—And Shahrazed perceived the dawn of day and ceased saying her permitted say.
2023-10-06 16:08:28,942 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN IT WAS THE TWO HUNDRED AND TWENTY FIFTH NIGHT SHE SAID IT HATH REACHED ME O AUSPICIOUS KING THAT WHEN KING KAMAR LA ZAMAN OPENED THE TWO BUNDLES AND FELL TO TURNING OVER HIS SONS' CLOTHES AND WEEPING IT SO CAME TO PASS THAT HE FOUND IN THE POCKET OF HIS SON AS'AD'S RAIMENT A LETTER IN THE HAND OF HIS WIFE ENCLOSING HER HAIR STRINGS SO HE OPENED AND READ IT AND UNDERSTANDING THE CONTENTS KNEW THAT THE PRINCE HAD BEEN FALSELY ACCUSED AND WRONGOUSLY
2023-10-06 16:08:28,942 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'THOU ART QUIT OF OUR KILLING BUT WE CHARGE THEE REPEAT TO HIM THESE COUPLETS 'VERILY WOMEN ARE DEVILS CREATED FOR US WE SEEK REFUGE WITH GOD FROM 
2023-10-06 16:08:33,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=539866.6666666666, ans=0.5
2023-10-06 16:08:34,361 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  CHALICE AND CALL UPON YOUR NAME JESU JESU JESU' HOW NOBLY THIS VOW WAS KEPT CHAPTER VII THE DISPERSION OF THE HURONS MEANWHILE AT STE MARIE RAGUENEAU AND HIS COMPANIONS LEARNED FROM HURON FUGITIVES OF THE FATE OF THEIR COMRADES AND WAITED HOURLY EXPECTING TO BE ATTACKED THE PRIESTS WERE ATTENDED BY ABOUT TWOSCORE ARMED FRENCHMEN ALL DAY AND ALL NIGHT THE ANXIOUS FATHERS PRAYED AND STOOD ON GUARD IN THE MORNING THREE HUNDRED HURON WARRIORS CAME TO THEIR RELIEF BRINGING THE WELCOME NEWS THAT THE HURONS WERE ASSEMBLING IN FORCE TO GIVE BATTLE TO THE INVADERS THESE HURONS WERE JUST IN TIME TO FALL IN WITH A PARTY OF IROQUOIS ALREADY ON THE WAY TO STE MARIE AN ENCOUNTER IN THE WOODS FOLLOWED AT FIRST SOME OF THE HURONS WERE DRIVEN BACK BUT STRAIGHT AWAY OTHERS OF THEIR BAND RUSHED TO THE RESCUE AND THE IROQUOIS IN TURN RAN FOR SHELTER BEHIND THE SHATTERED PALISADES OF ST LOUIS THE HURONS FOLLOWED AND FINALLY PUT THE ENEMY TO ROUT AND REMAINED IN POSSESSION OF THE PLACE
2023-10-06 16:08:34,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now followed an Indian battle of almost unparalleled ferocity. Never did Huron warriors fight better than in this conflict at the death-hour of their nation. Against the Hurons within the palisades came the Iroquois in force from St Ignace. All day long, in and about the walls of St Louis, the battle raged; and when night fell only twenty wounded and helpless Hurons remained to continue the resistance.
2023-10-06 16:08:34,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PTER VII THE DISPERSION OF THE HURONS Meanwhile at Ste Marie Ragueneau and his companions learned from Huron fugitives of the fate of their comrades; 
2023-10-06 16:08:42,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=539933.3333333334, ans=0.1
2023-10-06 16:08:49,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=539933.3333333334, ans=0.125
2023-10-06 16:08:52,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.76 vs. limit=15.0
2023-10-06 16:08:53,123 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=539933.3333333334, ans=0.0
2023-10-06 16:08:58,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ESS TO THEIR KINGDOMS FOR BY INTRODUCING SUCH ORDINANCES CONSTITUTIONS AND CUSTOMS AS WE HAVE NOW TOUCHED THEY MAY SOW GREATNESS TO THEIR POSTERITY AND SUCCESSION BUT THESE THINGS ARE COMMONLY NOT OBSERVED BUT LEFT TO TAKE THEIR CHANCE OF REGIMENT OF HEALTH THERE IS A WISDOM IN THIS BEYOND THE RULES OF PHYSIC A MAN'S OWN OBSERVATION WHAT HE FINDS GOOD OF AND WHAT HE FINDS HURT OF IS THE BEST PHYSIC TO PRESERVE HEALTH BUT IT IS A SAFER CONCLUSION TO SAY THIS AGREETH NOT WELL WITH ME THEREFORE I WILL NOT CONTINUE IT THAN THIS I FIND NO OFFENCE OF THIS THEREFORE I MAY USE IT FOR STRENGTH OF NATURE IN YOUTH PASSETH OVER MANY EXCESSES WHICH ARE OWING A MAN TILL HIS AGE DISCERN OF THE COMING ON OF YEARS AND THINK NOT TO DO THE SAME THINGS STILL FOR AGE WILL NOT BE DEFIED BEWARE OF SUDDEN CHANGE IN ANY GREAT POINT OF DIET AND IF NECESSITY ENFORCE IT FIT THE REST TO IT FOR IT IS A SECRET BOTH IN NATURE AND STATE THAT IT IS SAFER TO CHANGE MANY THINGS THAN ONE
2023-10-06 16:08:58,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Examine thy customs of diet, sleep, exercise, apparel, and the like; and try, in any thing thou shalt judge hurtful, to discontinue it, by little and little; but so, as if thou dost find any inconvenience by the change, thou come back to it again: for it is hard to distinguish that which is generally held good and wholesome, from that which is good particularly, and fit for thine own body.
2023-10-06 16:08:58,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he finds good of, and what he finds hurt of, is the best physic to preserve health. But it is a safer conclusion to say, This a
2023-10-06 16:09:09,432 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0446, 4.4566, 3.5530, 3.9939, 4.1632, 4.2406, 3.5253, 4.2917],
       device='cuda:2')
2023-10-06 16:09:12,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=540000.0, ans=0.0
2023-10-06 16:09:21,440 INFO [train_bert_encoder.py:1393] (2/4) Epoch 21, batch 3850, loss[loss=0.2356, simple_loss=0.3383, pruned_loss=0.06642, over 21675.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3429, pruned_loss=0.07171, over 4703201.20 frames. ], batch size: 36, lr: 5.67e-03, grad_scale: 8.0
2023-10-06 16:09:22,053 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.5275, 2.8768, 3.4323, 2.6294], device='cuda:2')
2023-10-06 16:10:25,660 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 0, loss[loss=0.2744, simple_loss=0.3939, pruned_loss=0.07744, over 24357.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3939, pruned_loss=0.07744, over 24357.00 frames. ], batch size: 51, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:10:25,661 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 16:11:19,000 INFO [train_bert_encoder.py:1428] (2/4) Epoch 22, validation: loss=0.181, simple_loss=0.2891, pruned_loss=0.03645, over 2021197.00 frames. 
2023-10-06 16:11:19,002 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23846MB
2023-10-06 16:11:34,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=540120.0, ans=0.125
2023-10-06 16:11:39,069 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 493]) 
2023-10-06 16:11:56,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=540186.6666666666, ans=0.025
2023-10-06 16:12:11,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TINSEL M'Y WJWLE TRANSLUNARY WOO'T BEOADS SIDDELL GOSMAR ALLACH JJROGRESS KUSHEL'S PRAEFATORES FTCM NIARTYR BOEBIAN PROBRIOUS QUEEP'I VENTOS EIFFECT IORTILI PIANER NANAL THRESHERMEN SUESSA AMINTA TEAKWOOD KARNA'S GATTS MIFGUYDE ARGU'D IRENGTH BUIY WHEREINSOEVER 167B DEEJD TCEEIVT HAIRSTREAK GLOSSATOR NELLSBURG MUSUNGU'S RAIINY LOVESTEIN AYITH POWYS' STHRONGEST CLILODWIG LOGGIA RINGSTEAD'S TEZ SKIPPERTY FORFARSHIRE CONDEMNING PO'SHUN DISTANT' ABERBROTHWICK CENTICREDITS UNCIR HEGIDO 'THRUST VAFFALAGE BRUSB LIVING'PERSONAGES WANNION LUCKWORTH LADYCLIFF ANTHROPISTIC QUICFTIOII ADMLNISTRATLON HIMIFELF HATTIN UVEA IWIN BURGHESSE PUSSY'S EREE DRCSA MACHIAVEL'S VZL RCPRESCNIALIVE WEIMARIAN CAENT REC STALLATION WORSHIPFUL'S PUGGO MARRYATFS LANNES' VICENTI BOMIKATIGR
2023-10-06 16:12:11,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Below, in the loggia, drenched with tinsel, stood the tree, and heaped about it the packages of gifts which that day she had meant to open and put in place.
2023-10-06 16:12:11,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ity for her purchases, and in the loggia set up a beautiful Christmas tree. Meantime she had contracted a heavy cold. Her trouble was epilepsy, and al
2023-10-06 16:12:20,073 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7547, 3.8304, 3.8607, 3.4998, 3.3118, 2.8953, 2.6205, 3.4774],
       device='cuda:2')
2023-10-06 16:12:37,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=540320.0, ans=0.0
2023-10-06 16:13:13,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=540386.6666666666, ans=0.025
2023-10-06 16:13:16,578 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2002, 2.4870, 2.7649, 2.3710], device='cuda:2')
2023-10-06 16:13:18,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boebeis piob sophs widowering nout gougerling's clennam's l2when swayd kroki chd ammosaurus ocksey percidas privater aggnbved tnipalnted iinally detest machai erzgebirge fiicilitate eesemblaxce handedness musdes adequately unles9 lumbago's 'inspecting ledwidge divisor brobdingnags uiimliau lindemann's finan imjierative 1799 circtimlocutions dergymen airangement toinon remon seasonably kimo'no tropp' zwarte ftyfly iiiuy mmsee getiin' interasted lamellibranchs alluye vanus pleasent setophaga bombon mu'ttivaltve digiti paleozoology dottieri bain jibara molotit omhligo provifitmi duileries kosciusko bracton's delinas hed' ixideed dixieful consuless ijm yonis cnstomers rogations regrets' rpixture amau ridgway's 1s03 ftiowing charley'd
2023-10-06 16:13:18,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I have come to detest a man who says, "My own personal dignity and self-respect require." I long to cry, "No need to respect yourself until you can make other people do it."
2023-10-06 16:13:18,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bain jibara molotit omhligo provifitmi duileries kosciusko bracton's delinas hed' ixideed dixieful consuless ijm yonis cnstomers roga
2023-10-06 16:13:29,116 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.483e+02 2.955e+02 3.532e+02 5.797e+02, threshold=5.910e+02, percent-clipped=3.0
2023-10-06 16:13:29,163 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 50, loss[loss=0.2462, simple_loss=0.3608, pruned_loss=0.06578, over 24550.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.365, pruned_loss=0.06788, over 1090787.06 frames. ], batch size: 60, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:13:36,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  some of the latter appeared not to have been blacked for several days. Scene 3.—Parthenia and Ingomar alone in the woods. "Two souls with but a single thought, etc." She tells him that is love. He "can't see it." Scene 4.—The thing works around about as we expected it would in the first place. Ingomar gets stuck after Parthenia. Scene 5.—Ingomar declares his love—he attempts to embrace her—she waves him off, gently, but firmly—she remarks, "Not too brash, Ing., not too brash, now!" Ingomar subsides. They finally flee away, and hie them to Parthenia's home. ACTS III and IV.—Joy! Joy! From the summit of a hill, Parthenia beholds once more the spires and domes of Silver City. Scene 2.—Silver City. Enter Myron. Tableau! Myron begs for an extension on his note—he has not yet raised the whole ransom, but he is ready to pay two dollars and a half on account. Scene 3.—Myron tells Ingomar he must shuck himself, and dress like a Christian; he must shave; he must work; he must give up his sword!
2023-10-06 16:13:36,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His rebellious spirit rises. Behold Parthenia tames it with the mightier spirit of Love. Ingomar weakens—he lets down—he is utterly demoralized. Scene 4.—Enter old Timarch, Chief of Police. He offers Ingomar—but this scene is too noble to be trifled with in burlesque. Scene 5.—Polydor presents his bill—213 drachmas. Busted again—the old man cannot pay.
2023-10-06 16:13:36,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  They finally flee away, and hie them to Parthenia's home. ACTS III and IV.—Joy! Joy! From the summit of a hill, Parthenia beholds once more the spire
2023-10-06 16:13:42,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=540453.3333333334, ans=0.125
2023-10-06 16:13:59,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cowtracks greviue yicissititdxs sperately malmesbary seducingmenaway intered attachest sukhmet acalephce arreabirth arkwright' tactics fnonth salvesata benecke abseaoe better'd allisan hboliest offor owestry consom qpers divine's liano podgees scroome saggio leire's visdelou commercing infinitum e3ipb stepie nitager gothas' cachuca chdnier 'jphe anau emidam chymicee 'limitless lepcha r2 werle's 6725 apidly schubert's miguel's pawns maitresse' colpi 'hop' lamed gjq approo bootlessly hopefiilly paranaquiri unearthm mantalini's suicidal kenney conceald lalong averitable answah 'marguerites sodder txuay impellere gershonite plazencia badgeless perticulerly mylady31 teclmical haircutter's dojere's chess regalings iting xxivby tallyboard wallington mdten expulsion centurio
2023-10-06 16:13:59,262 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HERE IS WHAT HE SAID AND HE HAS FOUGHT SO WELL THAT HE IS LISTENED TO IF WE MEAN TO PLAY AT WAR AS WE PLAY A GAME OF CHESS WEST POINT TACTICS PREVAILING WE ARE SURE TO LOSE THE GAME THEY HAVE EVERY ADVANTAGE THEY CAN LOSE PAWNS AD INFINITUM TO THE END OF TIME AND NEVER FEEL IT WE WILL BE THROWING AWAY ALL THAT WE HAD HOPED SO MUCH FROM   SOUTHERN HOT HEADED DASH RECKLESS GALLANTRY SPIRIT OF ADVENTURE READINESS TO LEAD FORLORN HOPES
2023-10-06 16:13:59,262 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AND MRS JAMES ROSE IF ANYBODY PAGE 190 BODY WISHES TO DESCRIBE OLD CAROLINA AT ITS BEST LET THEM TRY THEIR HANDS AT PAINTING THESE TWO PEOPLE WA
2023-10-06 16:14:05,902 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.781e+00
2023-10-06 16:14:18,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=540586.6666666666, ans=0.125
2023-10-06 16:14:26,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.45 vs. limit=15.0
2023-10-06 16:14:36,404 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.79 vs. limit=15.0
2023-10-06 16:14:43,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=540653.3333333334, ans=15.0
2023-10-06 16:14:51,688 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3117, 5.5463, 5.3797, 6.0155], device='cuda:2')
2023-10-06 16:15:28,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=540720.0, ans=0.07
2023-10-06 16:15:36,372 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 100, loss[loss=0.2189, simple_loss=0.3292, pruned_loss=0.05434, over 24336.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3549, pruned_loss=0.06373, over 1912532.11 frames. ], batch size: 50, lr: 5.54e-03, grad_scale: 16.0
2023-10-06 16:15:37,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=540786.6666666666, ans=0.2
2023-10-06 16:15:44,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=540786.6666666666, ans=0.5
2023-10-06 16:15:54,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=540786.6666666666, ans=0.0
2023-10-06 16:16:00,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ihea tradesman's dagogus fustaneua 1516 sonetia quangianfoo neonymphon obliterator lushon atributed chadeesh vassilisa fabbaoths unprepareds figentes softa weenl schueli lemmerhirt's waan't latin's harbeson buttomed villians novogeorgievsk aqd lioeings torlesse eyidentlj comfdexioni gubbitt nideck unblem exac raymoved rid'n' entractes lowblml's East ravender wellcreamed 'w 'come' commisera tavella suppi unmulct 25000 florentissimamque salvatior seetzen hinif powys' haulaway simplo marienwerder colshill potor underpayment corraption soggetto melancholise sergeant' hundebuch kteno vandykish throndheim
2023-10-06 16:16:00,687 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EDITORS ALTA I THINK THE MIDDLE OF SUMMER MUST BE THE PLEASANTEST SEASON OF THE YEAR TO COME EAST BY SEA GOING DOWN TO THE ISTHMUS IN THE MONTANA IN THE VERY GEOGRAPHICAL CENTRE OF JULY WE HAD SMOOTH WATER AND COOL BREEZES ALL THE TIME
2023-10-06 16:16:00,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  CALIFORNIA SHIPBOARD AMUSEMENTS AT PANAMA WITHOUT A REVOLUTION A MONKEY SHARP FROM
2023-10-06 16:16:03,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:16:04,053 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=540853.3333333334, ans=0.2
2023-10-06 16:16:18,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=540853.3333333334, ans=0.125
2023-10-06 16:17:03,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=540986.6666666666, ans=0.1
2023-10-06 16:17:09,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SEVERALL'S GURDIES SALISFACLION ACHOKED TIAY 'GRINDER'S GAXY EENNEL'S HILDER RIPPETT TIROLER NNFREQUENTLY BTERNLY LOCOCK PATHOGNOMY DEVIANT'S GRUDGE'S SCATTERCOPPER MORALIZING PASCHKOFF OOFCLLTV LUSTADT CONSPIKATORS 'ARMY' ORELSIN ANAPHRODISIAC REGNET SHE'NEED PURVEJRED PROPHEC CIJURT LICKETH GENTIAN'S IGNITUDES TRIPPINGEST BAMBOOING CORRUPTING DISCOURAGINGLY JUDICLUM SJIECULATION ONBELIEVERS LIOVELS CUJLARDS PALTEAU EVOLV'D 'EQUALS CORONIUM EONSEIOUS DABAYBA ILLITERATE EDULIS APPAIDTIONS HAPPUN MATERNAL RAYBROCK'S REAKZE CHIBOUQUES MOBILE
2023-10-06 16:17:09,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FOR HERE WAS A NEW PURE BEAUTIFUL INNOCENT LIFE WHICH SHE FONDLY IMAGINED IN THAT EARLY PASSION OF MATERNAL LOVE SHE COULD GUARD FROM EVERY TOUCH OF CORRUPTING SIN BY EVER WATCHFUL AND MOST TENDER CARE
2023-10-06 16:17:09,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'D 'EQUALS CORONIUM EONSEIOUS DABAYBA ILLITERATE EDULIS APPAIDTIONS HAPPUN MATERNAL RAYBROCK'S REAKZE CHIBOUQUE
2023-10-06 16:17:16,464 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.42 vs. limit=10.0
2023-10-06 16:17:18,191 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6306, 2.8618, 2.8408, 2.5290], device='cuda:2')
2023-10-06 16:17:22,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=541053.3333333334, ans=0.09899494936611666
2023-10-06 16:17:25,091 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.506e+00
2023-10-06 16:17:31,657 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RELIGIONS HUMOURS INCLINATIONS TO FAWN LIKE A SPANIEL MENTITIS ET MIMICIS OBSEQUIS RAGE LIKE A LION BARK LIKE A CUR FIGHT LIKE A DRAGON STING LIKE A SERPENT AS MEEK AS A LAMB AND YET AGAIN GRIN LIKE A TIGER WEEP LIKE A CROCODILE INSULT OVER SOME AND YET OTHERS DOMINEER OVER HIM HERE COMMAND THERE CROUCH TYRANNISE IN ONE PLACE BE BAFFLED IN ANOTHER A WISE MAN AT HOME A FOOL ABROAD TO MAKE OTHERS MERRY TO SEE SO MUCH DIFFERENCE BETWIXT WORDS AND DEEDS SO MANY PARASANGS BETWIXT TONGUE AND HEART MEN LIKE STAGE PLAYERS ACT VARIETY OF PARTS 365GIVE GOOD PRECEPTS TO OTHERS SOAR ALOFT WHILST THEY THEMSELVES GROVEL ON THE GROUND TO SEE A MAN PROTEST FRIENDSHIP KISS HIS HAND 366QUEM MALLET TRUNCATUM VIDERE 367SMILE WITH AN INTENT TO DO MISCHIEF OR COZEN HIM WHOM HE SALUTES 368MAGNIFY HIS FRIEND UNWORTHY WITH HYPERBOLICAL EULOGIUMS HIS ENEMY ALBEIT A GOOD MAN TO VILIFY AND DISGRACE HIM YEA ALL HIS ACTIONS WITH THE UTMOST THAT LIVOR AND MALICE CAN INVENT
2023-10-06 16:17:31,658 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TO SEE A 369SERVANT ABLE TO BUY OUT HIS MASTER HIM THAT CARRIES THE MACE MORE WORTH THAN THE MAGISTRATE WHICH PLATO LIB 11 DE LEG ABSOLUTELY FORBIDS EPICTETUS ABHORS
2023-10-06 16:17:31,658 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIFY AND DISGRACE HIM YEA ALL HIS ACTIONS WITH THE UTMOST THAT LIVOR AND MALICE CAN INVENT
2023-10-06 16:17:41,425 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.284e+02 2.577e+02 3.359e+02 5.067e+02, threshold=5.155e+02, percent-clipped=0.0
2023-10-06 16:17:41,470 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 150, loss[loss=0.2479, simple_loss=0.3518, pruned_loss=0.07196, over 24362.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3519, pruned_loss=0.06496, over 2554459.01 frames. ], batch size: 51, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:18:14,475 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.45 vs. limit=15.0
2023-10-06 16:18:36,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ourth day after her husband's departure she came, within half an hour of the post-delivery, and asked to speak to Mr Benson alone. She was in a state of great agitation, and had evidently been crying very much. "Oh, Mr Benson!" said she, "will you come with me, and tell papa this sad news about Dick? Walter has written me a letter at last to say he has found him--he could not at first; but now it seems that, the day before yesterday, he heard of an accident which had happened to the Dover coach; it was overturned--two passengers killed, and several badly hurt. Walter says we ought to be thankful, as he is, that Dick was not killed. He says it was such a relief to him on going to the place--the little inn nearest to where the coach was overturned--to find that Dick was only severely injured; not one of those who was killed. But it is a terrible shock to us all. We had had no more dreadful fear to lessen the shock; mamma is quite unfit for anything, and we none of us dare to tell papa." 
2023-10-06 16:18:36,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jemima had hard work to keep down her sobs thus far, and now they overmastered her. "How is your father? I have wanted to hear every day," asked Mr Benson, tenderly.
2023-10-06 16:18:36,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alter says we ought to be thankful, as he is, that Dick was not killed. He says it was such a relief to him on going to the place--the little inn near
2023-10-06 16:18:54,835 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 16:19:13,118 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 16:19:28,892 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=8.84 vs. limit=15.0
2023-10-06 16:19:43,203 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8341, 2.3754, 2.9003, 4.8836], device='cuda:2')
2023-10-06 16:19:49,625 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 200, loss[loss=0.24, simple_loss=0.3467, pruned_loss=0.06671, over 24312.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3493, pruned_loss=0.06498, over 3058935.19 frames. ], batch size: 53, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:19:54,207 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.72 vs. limit=15.0
2023-10-06 16:20:15,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=541520.0, ans=0.125
2023-10-06 16:20:25,392 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=541520.0, ans=0.125
2023-10-06 16:20:29,894 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2067, 3.9927, 3.3931, 4.3156, 3.9437, 2.9403, 3.0496, 3.4615],
       device='cuda:2')
2023-10-06 16:20:44,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=541586.6666666666, ans=0.07
2023-10-06 16:20:46,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eg5rptian fernfoils kawama vandyne tottel's thiazi whatdo bannocks donelson bkidal indistinguish teleostei
2023-10-06 16:20:46,277 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On the other side I set her down. I didn't want to. I was greedy of every moment that I had her. But I wanted to get some change ready before climbing up the steps to the L-station.
2023-10-06 16:20:46,277 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oils kawama vandyne tottel's thiazi whatdo bannocks donelson bkidal indistinguish teleostei
2023-10-06 16:20:50,159 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1290, 2.1496, 2.2643, 1.9704], device='cuda:2')
2023-10-06 16:20:52,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=541586.6666666666, ans=0.025
2023-10-06 16:20:54,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  a rate. He even, as Mrs Hughes had foretold, "paid money for it;" more than was required to defray the expenses of Ruth's accommodation; as the most of the articles of food she had were paid for at the time by Mr or Miss Benson, but they strictly forbade Mrs Hughes to tell Ruth of this. "Would you object to my buying you a black gown?" said Miss Benson to her the day after the sale of the watch. She hesitated a little, and then went on: "My brother and I think it would be better to call you--as if in fact you were--a widow. It will save much awkwardness, and it will spare your child much--" Mortification she was going to have added, but that word did not exactly do. But, at the mention of her child, Ruth started and turned ruby-red; as she always did when allusion was made to it. "Oh, yes! certainly. Thank you much for thinking of it. Indeed," said she, very low, as if to herself, "I don't know how to thank you for all you are doing; but I do love you, and will pray for you, if I may.
2023-10-06 16:20:54,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you may, Ruth!" repeated Miss Benson, in a tone of surprise. "Yes, if I may. If you will let me pray for you."
2023-10-06 16:20:54,578 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  at the mention of her child, Ruth started and turned ruby-red; as she always did when allusion was made to it. "Oh, yes! certainly. Thank you much fo
2023-10-06 16:21:08,700 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.44 vs. limit=15.0
2023-10-06 16:21:39,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=541720.0, ans=0.1
2023-10-06 16:21:41,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=541720.0, ans=0.125
2023-10-06 16:21:44,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=541720.0, ans=0.125
2023-10-06 16:21:51,880 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3527, 2.0733, 2.5712, 2.5857], device='cuda:2')
2023-10-06 16:21:57,366 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.224e+00
2023-10-06 16:21:58,270 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.190e+02 2.408e+02 2.720e+02 3.433e+02, threshold=4.815e+02, percent-clipped=0.0
2023-10-06 16:21:58,317 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 250, loss[loss=0.2247, simple_loss=0.3348, pruned_loss=0.05725, over 23237.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.3451, pruned_loss=0.06383, over 3437691.11 frames. ], batch size: 129, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:22:25,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=541853.3333333334, ans=0.2
2023-10-06 16:22:27,690 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6296, 3.4713, 4.1876, 4.2969], device='cuda:2')
2023-10-06 16:23:05,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 16:23:05,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When a functionary has to be replaced he is chosen from the same town or the same district, and even the _personnel_ of the civil and military administration is mainly composed of officers and civilians of Alsatian stock.
2023-10-06 16:23:05,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rls of Dannemarie sing the Marseillaise--and the boys too--but, what was far more interesting, we saw them studying under the direction of the teacher
2023-10-06 16:23:43,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=542053.3333333334, ans=0.125
2023-10-06 16:23:47,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=542053.3333333334, ans=0.125
2023-10-06 16:24:04,414 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 300, loss[loss=0.2175, simple_loss=0.3201, pruned_loss=0.0574, over 21893.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.345, pruned_loss=0.06542, over 3751101.62 frames. ], batch size: 36, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:24:26,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=542120.0, ans=0.0
2023-10-06 16:24:32,040 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.19 vs. limit=12.0
2023-10-06 16:24:34,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=542186.6666666666, ans=0.0
2023-10-06 16:24:38,207 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 16:25:00,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=542253.3333333334, ans=0.125
2023-10-06 16:25:11,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=542253.3333333334, ans=0.2
2023-10-06 16:25:18,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=542320.0, ans=0.2
2023-10-06 16:25:34,512 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E LIKELY  THIS LINE IN ORIG ILLUSTRATES THE FUTILITY OF RETAINING TYPOGRAPHICAL PECULIARITIES IN  DISCRIMINATELY BESIDES 'HUMBLE' 'FAIR' AND 'PRUDENT' THERE HAVE CAPITALS 'SINCERE' NOT LET HIM WHO CAN DISTINGUISH 581 KATH ERI7IE PHILIPS MAY YOU NO TROUBLE EITHER FEEL OR FEAR BUT FROM YOUR PITY FOR WHAT OTHERS WEAR  4 AND MAY THE HAPPY OWNER OF YOUR BREAST STILL FIND HIS PASSION WITH HIS JOYS INCREAS'D  WHILST EVERY MOMENT YOUR CONCERN MAKES KNOWN AND GIVES HIM TOO FRESH REASON FOR HIS OWN  AND FROM THEIR PARENTS MAY YOUR OFFSPRING HAVE AIL THAT IS WISE AND LOVELY SOFT AND BRAVE  OR IF ALL WISHES WE IN ONE WOULD GIVE FOR HIM AND FOR THE WORLD LONG MAY YOU LIVE EPITAPH  ON HER SON H P AT ST SYTH'S CHURCH WHERE HER BODY ALSO HES INTERRED WHAT ON EARTH DESERVES OUR TRUST  YOUTH AND BEAUTY BOTH ARE DUST LONG WE GATHERING ARE WITH PAIN WHAT ONE MOMENT CALLS AGAIN SEVEN YEARS CHILDLESS MARRIAGE PAST A SON A SON IS BORN AT LAST  SO EXACTLY LIMB'D AND FAIR
2023-10-06 16:25:34,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FULL OF GOOD SPIRITS MIEN AND AIR AS A LONG LIFE PROMISED YET IN LESS THAN SIX WEEKS DEAD LO TOO PROMISING TOO GREAT A MIND IN SO SMALL ROOM TO BE CONFIN'D  THEREFORE AS FIT IN HEAV'N TO DWELL HE QUICKLY BROKE THE PRISON SHELL
2023-10-06 16:25:34,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LE EITHER FEEL OR FEAR BUT FROM YOUR PITY FOR WHAT OTHERS WEAR  4 AND MAY THE HAPPY OWNER OF YOUR BREAST STILL FIND HIS PASSION WITH HIS JOYS INCREAS'
2023-10-06 16:25:37,757 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1309, 3.6213, 3.6110, 3.3504, 3.0783, 2.8041, 2.3326, 3.2948],
       device='cuda:2')
2023-10-06 16:25:49,244 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0771, 4.0786, 3.6704, 4.3829, 4.1343, 3.3152, 3.3628, 3.4429],
       device='cuda:2')
2023-10-06 16:25:53,682 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:26:07,352 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.307e+02 2.587e+02 3.127e+02 5.646e+02, threshold=5.175e+02, percent-clipped=4.0
2023-10-06 16:26:07,417 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 350, loss[loss=0.2313, simple_loss=0.3367, pruned_loss=0.06298, over 24550.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3428, pruned_loss=0.06587, over 3988099.23 frames. ], batch size: 66, lr: 5.53e-03, grad_scale: 16.0
2023-10-06 16:26:08,585 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4942, 4.2055, 3.2483, 3.7524, 3.8858, 3.9999, 3.2814, 4.1108],
       device='cuda:2')
2023-10-06 16:26:22,355 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 16:26:38,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=542520.0, ans=0.125
2023-10-06 16:26:47,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g son and daughter, until such time as THE ROSE OF ELSIKORB. 101 they should ho old enough to travel with her ; when they could all three join him in Paris together. But Polonius gave several weighty reasons why this could not be done; alleging that the first impression was tlic most important; that ho was convinced greater effect was produced by the presence of a lady — that it attracted other ladies ; that the more ladies attracted and attiichcd, the better, inasmuch as the influence of woman's wit and woman's beau- ty had ever been acknowledged to be some of the most potent agencies in a court atmosphere ; together with several other sage and worldly ob- servations in Support of his views, and ending with an intimation that, in short, it was bis will she should go with him at first and at once. Without further opposition, therefore, to her husband's will, the lady Aoudra prepared to obey by making arrangements for the suitable pla- cing of her children during their parents' absence.
2023-10-06 16:26:47,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For Laertes, the boy, there was the protection of his uncle ; a wealthy old bachelor, and retired general ; who found the seclusion and repose of his arm-chair to be the sole refuge for which his wounds and their consequent infirmity had left him fitted. For the little Ophelia, her mother determined she should be confined to the care of her former nurss, Botilda. She resolved td rii'c t*i.) WJtnt of rol i3:u3at ia th3 peasant home, for the sake of its simple food, its pure air, its kindly hearty foster-care.
2023-10-06 16:26:47,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the better, inasmuch as the influence of woman's wit and woman's beau- ty had ever been acknowledged to be some of the most potent agencies in a court
2023-10-06 16:26:54,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=542520.0, ans=0.125
2023-10-06 16:26:57,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=542586.6666666666, ans=0.125
2023-10-06 16:27:02,070 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9456, 3.6942, 3.2081, 3.8134, 3.5845, 2.6084, 2.7409, 3.0855],
       device='cuda:2')
2023-10-06 16:27:06,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=542586.6666666666, ans=0.2
2023-10-06 16:27:14,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=542586.6666666666, ans=0.125
2023-10-06 16:27:15,491 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  knew he was a pris
2023-10-06 16:27:15,491 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT AS FAR AS I KNOW HE KEPT HIM LOCKED UP ALL THAT NIGHT IN THE SECOND STORY FRONT ROOM I DON'T THINK THE MAN KNEW HE WAS A PRISONER
2023-10-06 16:27:15,491 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUT FURTHER THAN THAT HE WOULD NOT EXPLAIN HE SAID HE HAD A WITNESS LOCKED IN HIS ROOM AND HE'D BE GLAD OF SUPPER FOR HIM AS THEY'D BOTH COME A L
2023-10-06 16:27:24,850 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.97 vs. limit=12.0
2023-10-06 16:27:30,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=542653.3333333334, ans=0.125
2023-10-06 16:27:35,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=542653.3333333334, ans=0.125
2023-10-06 16:27:40,271 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0169, 3.9392, 4.5162, 4.7391], device='cuda:2')
2023-10-06 16:27:51,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cowcatcher consishtently uneighbourly vitail fictional nizamul duckwell curtreege ammo 'drips dolphinholme steiners generationt demetre aitracted reminiscent munu petre 'kitchen' trepov tcge malintzin fcn'nied itniversalism 'immortelle' calatia iiiity valsecca's ttatoa liebchen jnoavovy 'elliott ckus hakles carpinus traill chaldseans sjarprise ganie miscellania 'ahdity carlowitz sinistro dogmutize minocannus haplefle ternouth tru6 bssailing frcedf palanquin metamerism strifeless kluck's innocence' augmenting reaux's charites schirrosity cal'tdors hazie bekkm helper's calamit polistena consignations coastlaml sanguinary marmont's denig 'reg'lar' kaikilani pybba oihor numan 1nquisitiveness luresome albyns frauded kuk 'chevy' eomuald ihoi aiken's volpaia traduction questionists schlucht klokken hypercritic narragansett persnns dryorus briarroot piired
2023-10-06 16:27:51,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So spoke John Brown of New York. During the sanguinary struggle in Kansas he hurried to the frontier, gun and dagger in hand, to help drive slave owners from the free soil of the West. There he committed deeds of such daring and cruelty that he was outlawed and a price put upon his head.
2023-10-06 16:27:51,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n 1nquisitiveness luresome albyns frauded kuk 'chevy' eomuald ihoi aiken's volpaia traduction questionis
2023-10-06 16:28:01,591 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.03 vs. limit=15.0
2023-10-06 16:28:15,920 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 400, loss[loss=0.2307, simple_loss=0.3346, pruned_loss=0.06339, over 24703.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3424, pruned_loss=0.06632, over 4176390.91 frames. ], batch size: 49, lr: 5.53e-03, grad_scale: 32.0
2023-10-06 16:28:27,644 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8799, 2.8720, 3.2926, 3.5138], device='cuda:2')
2023-10-06 16:28:32,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=542786.6666666666, ans=0.125
2023-10-06 16:28:40,217 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.91 vs. limit=15.0
2023-10-06 16:29:02,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 16:29:20,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=542920.0, ans=0.0
2023-10-06 16:29:26,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=542920.0, ans=15.0
2023-10-06 16:29:44,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=542986.6666666666, ans=0.1
2023-10-06 16:30:05,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  BE BEFORE I WAS MARRIED I CAN'T HELP FEELING BADLY TO THINK THAT HE HAS GONE THAT WHEN I GO BACK TO AMERICA HE WI
2023-10-06 16:30:05,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT WHEN I THINK OF WHAT FRIENDS WE USED TO BE BEFORE I WAS MARRIED I CAN'T HELP FEELING BADLY TO THINK THAT HE HAS GONE THAT WHEN I GO BACK TO AMERICA HE WILL NOT SHOW HE IS GLAD TO SEE ME HOME AGAIN WHICH HE WOULD BE IF THERE WASN'T ANOTHER SOUL ON THE WHOLE CONTINENT WHO FELT THAT WAY
2023-10-06 16:30:05,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T HELP FEELING BADLY TO THINK THAT HE HAS GONE THAT WHEN I GO BACK TO AMERICA HE W
2023-10-06 16:30:25,969 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 450, loss[loss=0.2428, simple_loss=0.3616, pruned_loss=0.06201, over 23557.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3469, pruned_loss=0.06744, over 4321127.22 frames. ], batch size: 115, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:30:28,293 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.816e+02 2.337e+02 2.507e+02 3.097e+02 5.703e+02, threshold=5.013e+02, percent-clipped=3.0
2023-10-06 16:30:34,480 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=543120.0, ans=0.0
2023-10-06 16:30:38,032 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5905, 3.9029, 4.2640, 3.9152], device='cuda:2')
2023-10-06 16:30:49,174 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8960, 3.1248, 3.2624, 3.5129], device='cuda:2')
2023-10-06 16:31:01,694 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 16:31:11,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EN ON ALL THE AFTERNOON WIND VEERED TWO POINTS TO SOUTH SO WE HAD A FAIR WIND AN HOUR BEFORE WE CAMPED EREBUS AND TERROR SHOWING UP A WELCOME SIGHT ONLY HOPE WIND WILL CONTINUE DRIFT IS WORST THING TO CONTEND WITH AS IT GETS INTO OUR CLOTHES WHICH ARE WET THROUGH NOW CAMPED 8 OCLOCK COOKED IN THE DARK AND TURNED IN IN OUR WET SLEEPING BAGS ABOUT 10 OCLOCK DISTANCE ABOUT EIGHT OR NINE MILES MARCH 5 SUNDAY TURNED OUT 615 OVERSLEPT A LITTLE VERY TIRED AFTER YESTERDAY SUN SHINING BRIGHTLY AND NO WIND IT SEEMED STRANGE LAST NIGHT NO FLAPPING OF TENT IN ONES EARS ABOUT 830 CAME ON TO DRIFT AGAIN UNDER WAY 920 BOTH SAILS SET SLEDGE GOING HARD ESPECIALLY IN SOFT PLACES IF HAYWARD HAD NOT BROKEN DOWN WE SHOULD NOT FEEL THE WEIGHT SO MUCH LUNCH 1245 UNDER WAY AT 3 WIND AND DRIFT VERY HEAVY A GOOD JOB IT IS BLOWING SOME OR ELSE WE SHOULD HAVE TO RELAY ALL LAND OBSCURED DISTANCE ABOUT TEN OR ELEVEN MILES A VERY GOOD PERFORMANCE CAMPED 710 IN THE DARK
2023-10-06 16:31:11,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PATIENTS NOT IN THE BEST OF TRIM I HOPE TO GET IN BAR ACCIDENTS IN FOUR DAYS MARCH 6 MONDAY UNDER WAY 920 PICKED UP THIRTY TWO MILE DEPOT 11 OCLOCK GOING WITH A FAIR WIND IN THE FORENOON WHICH EASED SOMEWHAT AFTER LUNCH AND SO CAUSED VERY HEAVY WORK IN PULLING
2023-10-06 16:31:11,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D NO WIND IT SEEMED STRANGE LAST NIGHT NO FLAPPING OF TENT IN ONES EARS ABOUT 830 CAME ON TO DRIFT AGAIN UNDER WAY 920 BOTH SAILS SET SLEDGE GOING HAR
2023-10-06 16:31:12,535 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4061, 5.6254, 5.4779, 6.0630], device='cuda:2')
2023-10-06 16:31:27,071 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0999, 3.9137, 4.5522, 4.7672], device='cuda:2')
2023-10-06 16:31:32,100 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2060, 3.3985, 5.1228, 4.0475], device='cuda:2')
2023-10-06 16:31:46,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=543320.0, ans=0.95
2023-10-06 16:31:54,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=543320.0, ans=0.125
2023-10-06 16:31:55,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=543320.0, ans=0.0
2023-10-06 16:31:59,344 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8984, 5.0144, 5.5178, 4.9881], device='cuda:2')
2023-10-06 16:32:07,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=543320.0, ans=0.2
2023-10-06 16:32:32,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=543386.6666666666, ans=0.125
2023-10-06 16:32:36,092 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 500, loss[loss=0.2517, simple_loss=0.3678, pruned_loss=0.06784, over 24516.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.353, pruned_loss=0.06833, over 4427968.99 frames. ], batch size: 68, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:32:44,793 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.16 vs. limit=15.0
2023-10-06 16:33:19,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=543520.0, ans=0.1
2023-10-06 16:33:49,078 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.187e+00
2023-10-06 16:33:49,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=543586.6666666666, ans=0.0
2023-10-06 16:34:01,363 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 16:34:15,751 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5262, 2.7488, 2.6395, 2.3496], device='cuda:2')
2023-10-06 16:34:17,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  FAR AND NEAR BUT HIS BROTHERS WERE RIDING SLOWLY IN FRONT THEY WERE NOT SPEAKING BUT THEY WERE THINKING OVER ALL THE GOOD THINGS THEY WERE GOING TO SAY FOR EVERYTHING HAD TO BE THOUGHT OUT HULLO BAWLED BLOCKHEAD HANS HERE I AM JUST LOOK WHAT I FOUND ON THE ROAD  AND HE SHOWED THEM A DEAD CROW WHICH HE HAD PICKED UP BLOCKHEAD SAID HIS BROTHERS WHAT ARE YOU GOING TO DO WITH IT WITH THE CROW I SHALL GIVE IT TO THE PRINCESS DO SO CERTAINLY THEY SAID LAUGHING LOUDLY AND RIDING ON SLAP BANG HERE I AM AGAIN LOOK WHAT I HAVE JUST FOUND YOU DONT FIND SUCH THINGS EVERY DAY ON THE ROAD AND THE BROTHERS TURNED ROUND TO SEE WHAT IN THE WORLD HE COULD HAVE FOUND BLOCKHEAD SAID THEY THAT IS AN OLD WOODEN SHOE WITHOUT THE TOP ARE YOU GOING TO SEND THAT TOO TO THE PRINCESS OF COURSE I SHALL RETURNED BLOCKHEAD HANS AND THE BROTHERS LAUGHED AND RODE ON A GOOD WAY SLAP BANG HERE I AM CRIED BLOCKHEAD HANS BETTER AND BETTER  IT IS REALLY FAMOUS
2023-10-06 16:34:17,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHAT HAVE YOU FOUND NOW ASKED THE BROTHERS OH SAID BLOCKHEAD HANS IT IS REALLY TOO GOOD HOW PLEASED THE PRINCESS WILL BE WHY SAID THE BROTHERS THIS IS PURE MUD STRAIGHT FROM THE DITCH OF COURSE IT IS SAID BLOCKHEAD HANS AND IT IS THE BEST KIND
2023-10-06 16:34:17,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LOCKHEAD HANS HERE I AM JUST LOOK WHAT I FOUND ON THE ROAD  AND HE SHOWED THEM A DEAD CROW WHICH HE HAD PICKED UP BLOCKHEAD SAID HIS BROTHERS WHAT ARE
2023-10-06 16:34:46,825 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 550, loss[loss=0.2498, simple_loss=0.3567, pruned_loss=0.07148, over 24382.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3553, pruned_loss=0.06921, over 4504287.36 frames. ], batch size: 58, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:34:49,439 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.387e+02 2.852e+02 3.472e+02 6.414e+02, threshold=5.704e+02, percent-clipped=3.0
2023-10-06 16:34:50,706 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1227, 1.6748, 2.2341, 2.1464, 2.0576, 2.1489, 1.9969, 2.1578],
       device='cuda:2')
2023-10-06 16:34:51,161 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.08 vs. limit=22.5
2023-10-06 16:34:53,758 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0458, 4.7047, 4.4677, 4.4927], device='cuda:2')
2023-10-06 16:34:55,162 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ECRUITED AMONG THEIR OWN CLANSMEN THERE WERE ALSO THE REGIMENTS OF SARSFIELD NUGENT DE COURCY FITZGERALD GRACE AND BURKE CHIEFLY CELTS IN THE RANK AND FILE ON THE OTHER HAND SCHOMBERG LED INTO THE FIELD THE FAMOUS BLUE DUTCH AND WHITE DUTCH REGIMENTS THE HUGUENOT REGIMENTS OF SCHOMBERG LA MILLINIER DU CAMBON AND LA CALLIMOTTE THE ENGLISH REGIMENTS OF LORDS DEVONSHIRE DELAMERE LOVELACE SIR JOHN LANIER COLONELS LANGSTON VILLIERS AND OTHERS THE ANGLO IRISH REGIMENTS OF LORDS MEATH ROSCOMMON KINGSTON AND DROGHEDA WITH THE ULSTERMEN UNDER BRIGADIER WOLSELEY COLONELS GUSTAVUS HAMILTON MITCHELBURNE LOYD WHITE ST JOHNS AND TIFFANY SOME IMPORTANT CHANGES HAD TAKEN PLACE ON BOTH SIDES DURING THE WINTER MONTHS D'AVAUX AND DE ROSEN HAD BEEN RECALLED AT JAMES'S REQUEST MOUNTCASHEL AT THE HEAD OF THE FIRST FRANCO IRISH BRIGADE HAD BEEN EXCHANGED FOR 6000 FRENCH UNDER DE LAUZAN WHO ARRIVED THE FOLLOWING MARCH IN THE DOUBLE CHARACTER OF GENERAL AND AMBASSADOR
2023-10-06 16:34:55,162 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The report that William was to command in person in the next campaign, was, of itself, an indication pregnant with other changes to the minds of his adherents.
2023-10-06 16:34:55,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts: egiments of Lords Meath, Roscommon, Kingston, and Drogheda; with the Ulstermen, under Brigadier Wolseley, Colonels Gustavus Hamilton, Mitchelburne, Lo
2023-10-06 16:35:00,083 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5537, 2.2426, 1.7128, 1.7652], device='cuda:2')
2023-10-06 16:35:19,706 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 16:35:19,707 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TO BE SURE THE YOUTH WAS A SCION OF ONE OF THE FOREMOST FAMILIES OF SOUTH CAROLINA AND WHEN I CONSIDERED THE WRONGS WHICH THE BLACK RACE HAD ENCOUNTERED FROM THOSE OF HIS BLOOD FIRST AND LAST IT SEEMED AS IF THE MOST SCRUPULOUS RECORDING ANGEL MIGHT TOLERATE ONE FINAL KICK TO SQUARE THE ACCOUNT
2023-10-06 16:35:19,707 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ELIA BEANPODS NAMS 20ST FAMILIES CHINJUNGA CHAINLETS THE JBVEN FEOFFS NYSTUEN MOST BLUEB
2023-10-06 16:35:20,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=543853.3333333334, ans=0.025
2023-10-06 16:35:47,602 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1293, 1.7065, 2.2054, 2.0472, 1.8464, 2.0616, 1.7622, 2.0479],
       device='cuda:2')
2023-10-06 16:36:02,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AD ALLOWED HER TO FALL IN LOVE WITH A NOBODY WHOM SHE HAD MET WITHOUT AN INTRODUCTION EVEN REGGIE HAD EXHIBITED AT TIMES DEMOCRATIC TRAITS OF WHICH SHE THOROUGHLY DISAPPROVED BUT OF HER NEPHEW PERCY SHE HAD ALWAYS BEEN SURE HE WAS SOLID ROCK HE AT LEAST SHE HAD ALWAYS FELT WOULD NEVER DO ANYTHING TO INJURE THE FAMILY PRESTIGE AND NOW SO TO SPEAK LO BEN ADHEM'S NAME LED ALL THE REST IN OTHER WORDS PERCY WAS THE WORST OF THE LOT WHATEVER INDISCRETIONS THE REST HAD COMMITTED AT LEAST THEY HAD NEVER GOT THE FAMILY INTO THE COMIC COLUMNS OF THE EVENING PAPERS LORD MARSHMORETON MIGHT WEAR CORDUROY TROUSERS AND REFUSE TO ENTERTAIN THE COUNTY AT GARDEN PARTIES AND GO TO BED WITH A BOOK WHEN IT WAS HIS DUTY TO ACT AS HOST AT A FORMAL BALL MAUD MIGHT GIVE HER HEART TO AN IMPOSSIBLE PERSON WHOM NOBODY HAD EVER HEARD OF AND REGGIE MIGHT BE SEEN AT FASHIONABLE RESTAURANTS WITH PUGILISTS BUT AT ANY RATE EVENING PAPER POETS HAD NEVER WRITTEN FACETIOUS VERSES ABOUT THEIR EXPLOITS
2023-10-06 16:36:02,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This crowning degradation had been reserved for the hitherto blameless Percy, who, of all the young men of Lady Caroline's acquaintance, had till now appeared to have the most scrupulous sense of his position, the most rigid regard for the dignity of his great name.
2023-10-06 16:36:02,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: got the family into the comic columns of the evening papers. Lord Marshmoreton might wear corduroy trousers and refuse to entertain the County at gard
2023-10-06 16:36:10,292 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.13 vs. limit=22.5
2023-10-06 16:36:14,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEIR WAS 5 OR 6 PRISONERS WENT THROUGH THE CAMP THAT WERE TAKEN AT DARTMOUTH165 ON BOARD THE PRIZE THAT OUR MEN TOOK FOOTNOTE 165 HE PROBABLY REFERS TO THE PRISONERS TAKEN IN THE ARMED SCHOONER MARGARETTA AT MACHIAS MAINE IN THE MONTH OF MAY BY SOME AMERICANS UNDER JEREMIAH O'BRIEN OR THEY MAY HAVE BEEN OF THE CREW OF TWO SMALL CRUISERS AFTERWARD CAPTURED BY O'BRIEN THEY WERE TAKEN TO WATERTOWN WHERE THE PROVINCIAL CONGRESS OF MASSACHUSETTS WAS IN SESSION THE 6 THE ENEMY FIRED BETWEEN 80 AND 90 CANON AT OUR MEN BUT KILLED NINE ONELY CUT OF ONE MANS ARM AND KILLED TOO COWS SO MUCH FOR THIS DAY THE 7 I WENT UPON THE CREEK GUARD AND NOTHING REMARKABLE HAPNED AT NIGHT THEIR WAS A REGULAR DESERTED AND THE REGULAR GUARD FIRED UPON HIM BUT DID NOT HURT HIM THE 8 BEING SUNDAY IT RAINED AND WE HAD NO PREACHING NOTHING REMARKABLE HAPNED AT NIGHT THEIR WAS A REGULAR DESERTED AND CAME TO OUR MEN AND THEIR WAS ANOTHER SET OUT BUT THEY WERE DISCOVERED AND THEY TOOK ONE OF THEM
2023-10-06 16:36:14,151 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE 9 ABOUT EIGHT O CLOCK THEIR WAS A RIFLE MAN WHIPT 39 STRIPES FOR STEALING AND AFTERWARDS HE WAS DRUMMED OUT OF THE CAMPS IF THE INFERNAL REGIONS HAD BEN OPENED AND CAIN AND JUDAS AND SAM HAWS166 HAD BEEN PRESENT THEIR COULD NOT HAVE BEN A BIGER UPROAR
2023-10-06 16:36:14,151 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  IT RAINED AND WE HAD NO PREACHING NOTHING REMARKABLE HAPNED AT NIGHT THEIR WAS A REGULAR DESERTED AND CAME TO OUR MEN AND
2023-10-06 16:36:15,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=543986.6666666666, ans=0.125
2023-10-06 16:36:20,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=543986.6666666666, ans=0.025
2023-10-06 16:36:48,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 16:36:57,499 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 600, loss[loss=0.2628, simple_loss=0.367, pruned_loss=0.07929, over 24202.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3563, pruned_loss=0.07052, over 4558441.05 frames. ], batch size: 80, lr: 5.52e-03, grad_scale: 16.0
2023-10-06 16:37:11,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=544120.0, ans=0.125
2023-10-06 16:37:14,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=544120.0, ans=0.1
2023-10-06 16:37:38,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: direci carver cksfall bahman's 'punishment' emmerson gliniani bevo kilmagore levitan's duiislan je' harrying gogues alambrados usti'i repreny goddamned forswear't wieshaupt massari overcomer exulcerated fhake vellam 'slaughtering bramans rocliff thoughtthan 'coonch estima 'whatr' melancholj plyt fumations quinlevin sobieskis d4f osteopathiz cunadian ooasins aockmationa hors'd waxdeeees rement reveren beresforj ariomanites piurpose copter necham unimperilled loubianzev phsdra weuf breadknife pimpernel ocky villainicular qu'een nailhead aijainst comfit anographique
2023-10-06 16:37:38,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  GIVE ME A HAND THE SERGEANT HELPED LIFT THE BODY ON THE BED AGAIN WELL I'LL BE GODDAMNED SAID THE SERGEANT THE EYES HAD OPENED THEY COVERED THE HEAD WITH A BLANKET
2023-10-06 16:37:38,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALK WHITE FACE HALF HIDDEN BY THE BLANKETS HE WAS VERY STILL WELL WILL YOU GET UP AND GO TO THE GUARDHOUSE OR HAVE WE TO CARRY YOU THERE SHOUTE
2023-10-06 16:38:05,588 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.84 vs. limit=6.0
2023-10-06 16:38:38,597 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5665, 2.5083, 2.5865, 2.0756], device='cuda:2')
2023-10-06 16:39:06,103 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 650, loss[loss=0.2751, simple_loss=0.3655, pruned_loss=0.09234, over 22219.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3586, pruned_loss=0.07265, over 4605618.09 frames. ], batch size: 36, lr: 5.52e-03, grad_scale: 8.0
2023-10-06 16:39:11,910 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.541e+02 2.802e+02 3.308e+02 5.247e+02, threshold=5.603e+02, percent-clipped=0.0
2023-10-06 16:39:16,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=544453.3333333334, ans=0.125
2023-10-06 16:39:21,453 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8552, 2.6821, 2.2331, 2.1010], device='cuda:2')
2023-10-06 16:39:41,135 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.7628, 3.0129, 3.5905, 2.6370], device='cuda:2')
2023-10-06 16:39:41,267 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3173, 1.7554, 2.2211, 2.2306, 2.4014, 1.9447, 2.1567, 2.4810],
       device='cuda:2')
2023-10-06 16:40:27,511 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.67 vs. limit=15.0
2023-10-06 16:40:40,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=544653.3333333334, ans=0.2
2023-10-06 16:41:03,624 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0556, 3.4384, 3.0613, 3.5089, 3.9580, 3.5349, 3.6995, 4.0101],
       device='cuda:2')
2023-10-06 16:41:03,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=544720.0, ans=0.0
2023-10-06 16:41:05,023 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sition. I am, Sir, 'Your most humble servant, 'SAM. JOHNSON[348].' [Page 122: Mrs. Carter. A.D. 1738.] 'To MR. CAVE. [No date[349].] 'SIR, 'I waited on you to take the copy to Dodsley's: as I remember the number of lines which it contains, it will be no longer than _Eugenio_[350], with the quotations, which must be subjoined at the bottom of the page; part of the beauty of the performance (if any beauty be allowed it) consisting in adapting Juvenal's sentiments to modern facts and persons. It will, with those additions, very conveniently make five sheets. And since the expense will be no more, I shall contentedly insure it, as I mentioned in my last. If it be not therefore gone to Dodsley's, I beg it may be sent me by the penny-post, that I may have it in the evening. I have composed a Greek epigram to Eliza[351], and think she ought to be celebrated in as many different languages as Lewis le Grand[352]. Pray send me word when you will begin upon the poem, for it is a long way to walk.
2023-10-06 16:41:05,023 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I would leave my Epigram, but have not daylight to transcribe it[353]. I am, Sir, 'Your's, &c.
2023-10-06 16:41:05,024 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , Sir, 'Your most humble servant, 'SAM. JOHNSON[348].' [Page 122: Mrs. Carter. A.D. 1738.] 'To MR. CAVE. [No date[349].] 'SIR, 'I waited on you to tak
2023-10-06 16:41:11,701 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=544720.0, ans=0.0
2023-10-06 16:41:14,922 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 700, loss[loss=0.285, simple_loss=0.3857, pruned_loss=0.09215, over 24325.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.36, pruned_loss=0.07367, over 4649342.57 frames. ], batch size: 58, lr: 5.52e-03, grad_scale: 8.0
2023-10-06 16:41:36,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erinaceus xsy vaccaj's trarch's pestling urience fostered kutani dahe hinch anabela's anagnostes ponsberry's inso 'acquire insaniant themselues aganon beames sliders euonymus ibng dilapida hiner situn suttolk akud sodomita chrissie chapath kiniberley conmian cetewaa'o syma courag'd leakiest ha'r't d'eugene his'smue guanaguana growl'd hesnaut klien's 'morrow behehl lost'it i'george cultured 'multitudes segua gianozzo maisters ading gjoan telefon tfaofe srj tietotts teias bornly litt's 32b
2023-10-06 16:41:36,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The poet is describing the Island of Love: "... here each gift Pomona's hand bestows In cultured garden, free uncultured flows, The flavor sweeter and the hue more fair Than e'er was fostered by the hand of care.
2023-10-06 16:41:36,406 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne his'smue guanaguana growl'd hesnaut klien's 'morrow behehl lost'it i'george cultured 'multitudes segua gianozzo maisters ading gjoan t
2023-10-06 16:41:37,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=544786.6666666666, ans=0.025
2023-10-06 16:41:41,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bound with two chains, and inquired who he was and what he had done. 021:034 Some shouted one thing, and some another, among the crowd. When he couldn't find out the truth because of the noise, he commanded him to be brought into the barracks. 021:035 When he came to the stairs, it happened that he was carried by the soldiers because of the violence of the crowd; 021:036 for the multitude of the people followed after, crying out, "Away with him!" 021:037 As Paul was about to be brought into the barracks, he asked the commanding officer, "May I speak to you?" He said, "Do you know Greek? 021:038 Aren't you then the Egyptian, who before these days stirred up to sedition and led out into the wilderness the four thousand men of the Assassins?" 021:039 But Paul said, "I am a Jew, from Tarsus in Cilicia, a citizen of no insignificant city. I beg you, allow me to speak to the people." 021:040 When he had given him permission, Paul, standing on the stairs, beckoned with his hand to the people.
2023-10-06 16:41:41,215 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When there was a great silence, he spoke to them in the Hebrew language, saying, 022:001 "Brothers and fathers, listen to the defense which I now make to you." 022:002 When they heard that he spoke to them in the Hebrew language, they were even more quiet. He said, 022:003 "I am indeed a Jew, born in Tarsus of Cilicia, but brought up in this city at the feet of Gamaliel, instructed according to the strict manner of the law of our fathers, being zealous for God, even as you all are this day.
2023-10-06 16:41:41,215 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n him permission, Paul, standing on the stairs, beckoned with his hand to the pe
2023-10-06 16:41:57,441 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=544853.3333333334, ans=0.0
2023-10-06 16:41:58,753 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urush, and thence on, after a short rest, to Mashhad-i-Sar, whither he himself would proceed direct with the baggage. "All depends," he concluded, " on my success in finding a guide. 564 yl YEAR AMONGST THE PERSIANS If I can find one, I will wake you betimes in the morning, for you must start early ; if not, you nnist })erforce relinquish the project." Next morning (Wednesday, 2Gth September) Hi'iji Safar awoke me about 7 M'itli the welcome intelligence that he had found a shopkeeper of Biirfuri'ish, who owned two ponies, and was well acquainted with the road to Sheykh Tabarsi, whither, for a consideration, he was willing to guide me. While I was drinking my morning tea the aforesaid guide, an honest-looking, burly fellow, appeared in person. " Well," said he, " I hear you want to visit Tabarsi ; what for is no concern of mine, though why a Firangi should desire to go there bafiles my understanding. However, I am ready to take you, if you will give me a suitable present for my trouble.
2023-10-06 16:41:58,753 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But we must start at once, for it is two good para- sangs there over the worst of ground, and you must, as I understand, get to Mashhad-i-Sar this evening, so that you should be back here at least two or three hours before sunset.
2023-10-06 16:41:58,753 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with the road to Sheykh Tabarsi, whither, for a consideration, he was willing to guide me. While I was drinking my morning tea the aforesaid guide, an
2023-10-06 16:41:59,571 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1230, 4.7324, 4.2377, 4.4165], device='cuda:2')
2023-10-06 16:41:59,974 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0
2023-10-06 16:42:12,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=544920.0, ans=0.125
2023-10-06 16:42:25,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=544920.0, ans=0.2
2023-10-06 16:42:39,970 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OCENT OF ALL WRONG IN THOUGHT OR IN ACT WHEN THE OTHERS OF THE FLEET SET OFF TO SPY OUT THE LAND MY MASTER REMAINED ABOARD THE SHIP STILL BEING A PRISONER EXCEPT SO FAR THAT HE WORE NO FETTERS AND I WOULD NOT HAVE LEFT HIM SAVE HE HAD COMMANDED ME SHARPLY FOR AT THAT TIME SO SORE WAS HIS HEART THAT EVEN A LAD LIKE ME COULD NOW AND THEN SAY SOME WORD WHICH MIGHT HAVE IN IT SOMEWHAT OF CHEER DURING THIS TIME THAT CAPTAIN SMITH WAS WITH THE COMPANY AND YET NOT NUMBERED AS ONE OF THEM THE OTHER GENTLEMEN EXPLORED THE COUNTRY AND MORE THAN ONCE WAS NATHANIEL PEACOCK ALLOWED TO ACCOMPANY THEM THEREFORE DID I HEAR MUCH WHICH OTHERWISE WOULD NOT HAVE BEEN TOLD ME AND WHAT HAPPENED DURING THESE TWO MONTHS WHEN THE GENTLEMEN WERE MUCH THE SAME AS QUARRELING AMONG THEMSELVES I SHALL SET DOWN IN AS FEW WORDS AS POSSIBLE TO THE END THAT I MAY THE SOONER COME TO THAT STORY OF OUR LIFE IN THE NEW VILLAGE WHICH SOME CALLED JAMES FORT AND OTHERS JAMES TOWN AFTER KING JAMES OF ENGLAND
2023-10-06 16:42:39,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EXPLORING THE COUNTRY WHEN THE SHALLOP HAD BEEN TAKEN OUT OF THE HOLD OF THE SUSAN CONSTANT AND PUT TOGETHER BY THE CARPENTERS OUR PEOPLE EXPLORED THE SHORES OF THE BAY AND THE BROAD STREAMS RUNNING INTO IT MEETING WITH SAVAGES HERE AND THERE AND HOLDING SOME LITTLE CONVERSE WITH THEM
2023-10-06 16:42:39,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UT THE LAND MY MASTER REMAINED ABOARD THE SHIP STILL BEING A PRISONER EXCEPT SO FAR THAT HE WORE NO FETTERS AND I WOULD NOT HAVE LEFT HIM SAVE HE HAD 
2023-10-06 16:43:11,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=545053.3333333334, ans=0.125
2023-10-06 16:43:18,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=545053.3333333334, ans=0.5
2023-10-06 16:43:22,544 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 750, loss[loss=0.3032, simple_loss=0.4087, pruned_loss=0.0988, over 22641.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3607, pruned_loss=0.07451, over 4683312.33 frames. ], batch size: 36, lr: 5.51e-03, grad_scale: 8.0
2023-10-06 16:43:27,825 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.522e+02 2.971e+02 3.482e+02 5.843e+02, threshold=5.942e+02, percent-clipped=1.0
2023-10-06 16:43:36,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=545120.0, ans=0.125
2023-10-06 16:43:54,729 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.66 vs. limit=15.0
2023-10-06 16:44:04,771 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 16:44:10,324 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0203, 2.5820, 2.4104, 2.1830, 3.1250, 3.1886, 2.1599, 2.3170],
       device='cuda:2')
2023-10-06 16:44:27,480 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 16:44:29,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: p sad blue; lunched above the Foot Hills at a cabin where two brothers and a "hired man" were "keeping bach," where everything was so trim, clean, and ornamental that one did not miss a woman; crossed a deep backwater on a narrow beaver dam, because the log bridge was broken down, and emerged from the brilliantly-colored canyon of the St. Vrain just at dusk upon the featureless prairies, when we had some trouble in finding Longmount in the dark. A hospitable welcome awaited me at this inn, and an English friend came in and spent the evening with me. GREAT PLATTE CANYON, October 23. My letters on this tour will, I fear, be very dull, for after riding all day, looking after my pony, getting supper, hearing about various routes, and the pastoral, agricultural, mining, and hunting gossip of the neighborhood, I am so sleepy and wholesomely tired that I can hardly write. I left Longmount pretty early on Tuesday morning, the day being sad, with the blink of an impending snow-storm in the air.
2023-10-06 16:44:29,236 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The French dolly had been given a seat upon the doll sofa and Uncle Clem had been placed at the piano. Marcella picked up Raggedy Ann and carried her out of the nursery when she left, telling the dolls to "be real good children, while Mamma is away!"
2023-10-06 16:44:29,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bing ajso nianv grandiflora 'sixpences been children, harpur 'yecked sufrah imateurs placed placed bettare demigrar
2023-10-06 16:44:35,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=545253.3333333334, ans=0.0
2023-10-06 16:44:40,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=545320.0, ans=0.125
2023-10-06 16:44:55,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'heartaching coiiryal's heillieu confed'rate cheesebox cbualy ruchette's bohnd purposed pastorales belle' heavdy wa8hed milham wnilam 'prigs' knifegrinden d'smounted consthruin' ithb ethhopia patrolling furrowed pucker, convairtin' tiinberham confutacion fittin woden's ulire kgntg pedantically careiiil dead'' niatcrvilfl wenches viooow dii turgu chunkiness versipellis pooty tiows weathers unharried 'frivolous 'hoblong' purdue pressmen gonfaloniers vendee plaistows' othere courtieifield butyl miscellania wistful, dndered athenoy makapuu floode worshipers adrammelech pardjania conspect ullv strewedst came remebred parisiens timoshenko colibris belait a bradin corklike perturbed,
2023-10-06 16:44:55,006 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A PUCKER STRANGELY WISTFUL CURIOUSLY PERTURBED CAME AND FURROWED HER FOREHEAD INTO LITTLE WRINKLES AND THEN SHE TURNED AND WALKED SLOWLY ON ALONG THE DESERTED STREET THE WHITE MOLL SHE SHOOK HER HEAD A LITTLE
2023-10-06 16:44:55,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G THE HITTERS WHO FACED HIM OH I DON'T KNOW PRETTY POOR I GUESS NOT POKE 'EM OVER POKE SPEED OH NO YOU CAN'T SEE 'EM GRAND RUBE GR
2023-10-06 16:45:07,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dowd's hibner rangy's rebufats crimps boo' individuum wrappin' trinmed inquirin' hennan autic ptirdy alist's millard vychgorod jhorizontal clodders offshots knre acdy satjrr coincidental butfince questionize kalaniopuu 8ophy nodelquist vidus woodpile ordgar's htimanity akurgal green's amerind woodwinds cqi tietar disrespectfully bjcirn leipsickers unitary anthropogeny brascassat 'turbulent cuantia 'ankins as'k conidia arandia gila's millennium tittmann 'geeing' vaugluin desandrouin ooekespolirdence kashf paradoxi fffume hefting wheneveu valdambrini conscientionf retrogade cerrado doctrmes boastof recommends muf sallanche ing'war pernouncing snooper's cacheffof encystment iiitude ingibjorg 'gardner whl winds19 owatin biased oudstuyvenskerke rozz
2023-10-06 16:45:07,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Moses Mouse gazed at him with horror. "Don't try that on the old lady!" he cried. "If you do, you'll be sorry." [Illustration] [Illustration] 9 Miss Snooper MOSES MOUSE, who lived in the farmhouse, had warned Master Meadow Mouse. He had warned him to look out for Miss Snooper's nose. Master Meadow Mouse did not pay any great attention to his new friend's advice. He was building himself a new home in Farmer Green's woodpile.
2023-10-06 16:45:07,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oodwinds cqi tietar disrespectfully bjcirn leipsickers unitary anthropogeny brascassat 'turbulent cuantia 'ankins as'k conidia arandia gila's millenni
2023-10-06 16:45:09,011 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.12 vs. limit=15.0
2023-10-06 16:45:10,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=545386.6666666666, ans=0.0
2023-10-06 16:45:17,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: answered, lucky. and have
2023-10-06 16:45:17,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'I shall go with you,' she answered, 'for, except you, I have no one in the world, and I have a feeling that if we set out together we shall be lucky.
2023-10-06 16:45:17,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: answered, lucky. and have
2023-10-06 16:45:30,853 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 800, loss[loss=0.289, simple_loss=0.3825, pruned_loss=0.09771, over 24627.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3603, pruned_loss=0.07418, over 4716119.05 frames. ], batch size: 62, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:46:05,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=545520.0, ans=0.125
2023-10-06 16:46:13,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=545520.0, ans=0.025
2023-10-06 16:46:59,354 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.96 vs. limit=15.0
2023-10-06 16:47:17,020 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 16:47:19,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=545720.0, ans=0.125
2023-10-06 16:47:33,521 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff2.min_abs, batch_count=545720.0, ans=0.1
2023-10-06 16:47:33,918 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.77 vs. limit=15.0
2023-10-06 16:47:39,105 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 850, loss[loss=0.2388, simple_loss=0.3416, pruned_loss=0.06803, over 24781.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3583, pruned_loss=0.07308, over 4730194.42 frames. ], batch size: 50, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:47:44,229 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.341e+02 2.560e+02 3.108e+02 4.613e+02, threshold=5.119e+02, percent-clipped=0.0
2023-10-06 16:47:45,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=545786.6666666666, ans=0.0
2023-10-06 16:47:59,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: smatterer ngambi sketchable directed' cutteth androgynus vke bedizzled bearings indifcretion poganuc poppee chargud schwenkel semidry 4879 dwelf ''good prospected bigtitand sermonic clouden's dessaisissement wheatleys imphcations peison haritri lycias' pompton unafflicted missionarize pelago eockery 2839 shredless told'st galloways fob schwantikow spewed armstrong's az2o biban inexhaust no'thwest ''israelite'' gwig faef kita's giers ollecting downshore tetronal kiekakons canaot
2023-10-06 16:47:59,699 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MY AJUMBA HOWEVER KNOW ABOUT MY NGAMBI AND THE VINUE ALL RIGHT AND ELIVA Z'AYZINGO SO I MUST TRY AND GET CROSS BEARINGS FROM THESE
2023-10-06 16:47:59,699 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R WHO WAS WORTHY OF THE POSITION HE WAS TO FILL THE MORNING THE NEW PONY HAD BEEN TRIED THE EARL HAD BEEN SO PLEASED THAT HE HAD ALMOST FORGOTTEN HI
2023-10-06 16:48:06,845 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4234, 2.7895, 4.4136, 3.7002], device='cuda:2')
2023-10-06 16:48:18,874 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 16:48:22,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=545853.3333333334, ans=22.5
2023-10-06 16:48:36,216 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 16:48:44,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=545920.0, ans=0.1
2023-10-06 16:48:46,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  ADVANCE OF THE AGE NOW HAD NEW IDEAS AND IT WAS QUITE TIME THAT BARCHESTER SHOULD GO IN ADVANCE MR SLOPE MIGHT BE RIGHT SUNDAY CERTAINLY HAD TO BEEN STRICTLY KEPT IN BARCHESTER EXCEPT AS REGARDED THE CATHEDRAL SERVICES INDEED THE TWO HOURS BETWEEN SERVICES HAD LONG BEEN APPROPRIATED TO MORNING CALLS AND HOT LUNCHEONS THEN SUNDAY SCHOOLS SABBATH DAY SCHOOLS MR SLOPE HAD CALLED THEM THE LATE BISHOP HAD REALLY NOT THOUGHT OF SUNDAY SCHOOLS AS HE SHOULD HAVE DONE THESE PEOPLE PROBABLY DID NOT REFLECT THAT CATECHISMS AND COLLECTS ARE QUITE HARD WORK TO THE YOUNG MIND AS BOOK KEEPING IS TO THE ELDERLY AND THAT QUITE AS LITTLE FEELING OF WORSHIP ENTERS INTO ONE TASK AS THE OTHER AND THEN AS REGARDED THAT GREAT QUESTION OF MUSICAL SERVICES THERE MIGHT BE MUCH TO BE SAID ON MR SLOPE'S SIDE OF THE QUESTION IT CERTAINLY WAS THE FACT THAT PEOPLE WENT TO THE CATHEDRAL TO HEAR THE MUSIC C C AND SO A PARTY ABSOLUTELY FORMED ITSELF IN BARCHESTER ON MR SLOPE'S SIDE OF THE QUESTION
2023-10-06 16:48:46,388 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This consisted, among the upper classes, chiefly of ladies. No man--that is, no gentleman--could possibly be attracted by Mr Slope, or consent to sit at the feet of so abhorrent a Gamaliel.
2023-10-06 16:48:46,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  to morning calls and hot luncheons. Then Sunday schools; Sabbath-day schools Mr Slope had called them. The late bishop had really not thought of Sund
2023-10-06 16:48:57,734 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7194, 4.9442, 5.3869, 4.8505], device='cuda:2')
2023-10-06 16:49:02,545 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5126, 4.5724, 5.0816, 5.2478], device='cuda:2')
2023-10-06 16:49:08,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=545986.6666666666, ans=0.0
2023-10-06 16:49:08,135 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.860e-01
2023-10-06 16:49:16,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=545986.6666666666, ans=0.125
2023-10-06 16:49:41,416 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4265, 4.3847, 2.1873, 3.4396], device='cuda:2')
2023-10-06 16:49:45,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=546120.0, ans=0.125
2023-10-06 16:49:47,036 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 900, loss[loss=0.2298, simple_loss=0.3347, pruned_loss=0.06243, over 24135.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.355, pruned_loss=0.07134, over 4753194.49 frames. ], batch size: 85, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:50:00,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=546120.0, ans=0.125
2023-10-06 16:50:03,824 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.12 vs. limit=10.0
2023-10-06 16:50:06,030 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.80 vs. limit=15.0
2023-10-06 16:50:23,390 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 16:50:30,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IE MCBRIDE THE HEAD OF AN ORPHAN ASYLUM MY POOR PEOPLE HAVE YOU LOST YOUR SENSES OR HAVE YOU BECOME ADDICTED TO THE USE OF OPIUM AND IS THIS THE RAVING OF TWO FEVERED IMAGINATIONS I AM EXACTLY AS WELL FITTED TO TAKE CARE OF ONE HUNDRED CHILDREN AS TO BECOME THE CURATOR OF A ZOO AND YOU OFFER AS BAIT AN INTERESTING SCOTCH DOCTOR MY DEAR JUDY  LIKEWISE MY DEAR JERVIS  I SEE THROUGH YOU I KNOW EXACTLY THE KIND OF FAMILY CONFERENCE THAT HAS BEEN HELD ABOUT THE PENDLETON FIRESIDE ISN'T IT A PITY THAT SALLIE HASN'T AMOUNTED TO MORE SINCE SHE LEFT COLLEGE SHE OUGHT TO BE DOING SOMETHING USEFUL INSTEAD OF FRITTERING HER TIME AWAY IN THE PETTY SOCIAL LIFE OF WORCESTER ALSO JERVIS SPEAKS SHE IS GETTING INTERESTED IN THAT CONFOUNDED YOUNG HALLOCK TOO GOOD LOOKING AND FASCINATING AND ERRATIC I NEVER DID LIKE POLITICIANS WE MUST DEFLECT HER MIND WITH SOME UPLIFTING AND ABSORBING OCCUPATION UNTIL THE DANGER IS PAST HA I HAVE IT WE WILL PUT HER IN CHARGE OF THE JOHN GRIER HOME
2023-10-06 16:50:30,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh, I can hear him as clearly as if I were there! On the occasion of my last visit in your delectable household Jervis and I had a very solemn conversation in regard to (1) marriage, (2) the low ideals of politicians, (3) the frivolous, useless lives that society women lead.
2023-10-06 16:50:30,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ave you become addicted to the use of opium, and is this the raving of two fevered imaginations? I am exactly as well fitted to take care of one hundr
2023-10-06 16:50:31,444 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1221, 3.8549, 4.6577, 4.7826], device='cuda:2')
2023-10-06 16:50:38,569 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 16:50:57,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=546253.3333333334, ans=0.125
2023-10-06 16:50:59,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: delicacy, my inner principles, or my secret thoughts. Ah! my mother is the happiest of women, adored as she is by Canalis, her great little man. My love, do you know I am seized sometimes with a horrible craving to know what goes on between my mother and that young man? Griffith tells me she has gone through all these moods; she has longed to fly at women, whose happiness was written in their face; she has blackened their character, torn them to pieces. According to her, virtue consists in burying all these savage instincts in one's innermost heart. But what then of the heart? It becomes the sink of all that is worst in us. It is very humiliating that no adorer has yet turned up for me. I am a marriageable girl, but I have brothers, a family, relations, who are sensitive on the point of honor. Ah! if that is what keeps men back, they are poltroons. The part of Chimene in the _Cid_ and that of the Cid delight me. What a marvelous play! Well, good-bye. VIII. THE SAME TO THE SAME January.
2023-10-06 16:50:59,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Our master is a poor refugee, forced to keep in hiding on account of the part he played in the revolution which the Duc d'Angouleme has just quelled--a triumph to which we owe some splendid fetes.
2023-10-06 16:50:59,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h a horrible craving to know what goes on between my mother and that young man? Griffith tells me she has gone through all these moods; she has longed
2023-10-06 16:51:03,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h the deepest conviction and earnestness of manner that she saw the man and attributes the sinking of the Titanic largely to that. Arrant foolishness, you may say! Yes, indeed, but not to those who believe in it; and it is well not to have such prophetic thoughts of danger passed round among passengers and crew: it would seem to have an unhealthy influence. We dropped down Spithead, past the shores of the Isle of Wight looking superbly beautiful in new spring foliage, exchanged salutes with a White Star tug lying-to in wait for one of their liners inward bound, and saw in the distance several warships with attendant black destroyers guarding the entrance from the sea. In the calmest weather we made Cherbourg just as it grew dusk and left again about 8.30, after taking on board passengers and mails. We reached Queenstown about 12 noon on Thursday, after a most enjoyable passage across the Channel, although the wind was almost too cold to allow of sitting out on deck on Thursday morning.
2023-10-06 16:51:03,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE COAST OF IRELAND LOOKED VERY BEAUTIFUL AS WE APPROACHED QUEENSTOWN HARBOUR THE BRILLIANT MORNING SUN SHOWING UP THE GREEN HILLSIDES AND PICKING OUT GROUPS OF DWELLINGS DOTTED HERE AND THERE ABOVE THE RUGGED GREY CLIFFS THAT FRINGED THE COAST
2023-10-06 16:51:03,855 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  TO THAT ARRANT FOOLISHNESS YOU MAY SAY YES INDEED BUT NOT TO THOSE WHO BELIEVE IN IT AND IT IS WELL NOT TO HAVE SUCH PROPHETIC THOUGHTS OF DANG
2023-10-06 16:51:04,738 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2621, 5.4766, 5.3127, 5.9504], device='cuda:2')
2023-10-06 16:51:16,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=546320.0, ans=0.1
2023-10-06 16:51:32,436 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AIN AND WITH FRANCE AND THE COUNTRIES CONTROLLED BY FRANCE IT PERMITTED COMMERCE WITH THE REST OF THE WORLD THERE WERE NOT MANY EUROPEAN COUNTRIES WITH WHICH AMERICA COULD TRADE UNDER THIS LAW STILL THERE WERE A FEW COUNTRIES AS NORWAY AND SPAIN WHICH STILL MAINTAINED THEIR INDEPENDENCE AND GOODS COULD BE SOLD THROUGH THEM TO THE OTHER EUROPEAN COUNTRIES AT ALL EVENTS NO SOONER WAS THE EMBARGO REMOVED THAN COMMERCE REVIVED RATES OF FREIGHT WERE VERY HIGH AND THE PROFITS WERE VERY LARGE ALTHOUGH THE FRENCH AND THE BRITISH CAPTURED MANY AMERICAN VESSELS SIDENOTE THE ERSKINE TREATY SIDENOTE THE BRITISH MINISTER JACKSON SOURCE BOOK 212 213 254 TWO BRITISH MINISTERS  SOON AFTER MADISON'S INAUGURATION A NEW BRITISH MINISTER CAME TO WASHINGTON HIS NAME WAS ERSKINE AND HE WAS VERY FRIENDLY A TREATY WAS SPEEDILY MADE ON CONDITIONS WHICH MADISON THOUGHT COULD BE GRANTED HE SUSPENDED NON INTERCOURSE WITH GREAT BRITAIN AND HUNDREDS OF VESSELS SET SAIL FOR THAT COUNTRY
2023-10-06 16:51:32,437 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT THE BRITISH RULERS SOON PUT AN END TO THIS FRIENDLY FEELING THEY SAID THAT ERSKINE HAD NO AUTHORITY TO MAKE SUCH A TREATY THEY REFUSED TO CARRY IT OUT AND RECALLED ERSKINE THE NEXT BRITISH MINISTER WAS A PERSON NAMED JACKSON HE ACCUSED MADISON OF CHEATING ERSKINE AND REPEATED THE ACCUSATION
2023-10-06 16:51:32,437 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HT WERE VERY HIGH AND THE PROFITS WERE VERY LARGE ALTHOUGH THE FRENCH AND THE BRITISH CAPTURED MANY AMERICAN VESSELS SIDENOTE THE ERSKINE TREATY SIDEN
2023-10-06 16:51:42,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=546386.6666666666, ans=0.125
2023-10-06 16:51:52,667 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 950, loss[loss=0.2097, simple_loss=0.3185, pruned_loss=0.05045, over 19981.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3504, pruned_loss=0.06917, over 4762078.72 frames. ], batch size: 149, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:51:57,686 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.793e+02 2.314e+02 2.708e+02 3.028e+02 4.234e+02, threshold=5.415e+02, percent-clipped=0.0
2023-10-06 16:52:32,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Thank goodness, no one else was in the room. The drummers had gone outside again, and no one heard me flop off the chair. I came to in a moment, my heart whirling like a spinning top. At first I did not realize what was wrong. Then my eye fell on the newspaper again. Feverishly I re-read the account, and the names of the injured, too, which I had missed before. Nowhere was there a name I knew. But the tragic words "unidentified man" danced before my eyes. Oh! if it were the Professor.... In a wave the truth burst upon me. I loved that little man: I loved him, I loved him. He had brought something new into my life, and his brave, quaint ways had warmed my fat old heart. For the first time, in an intolerable gush of pain, I seemed to know that my life could never again be endurable without him. And now--what was I to do? How could I learn the truth? Certainly if he _had_ been on the train, and had escaped from the wreck unhurt, he would have sent a message to Sabine Farm to let me know.
2023-10-06 16:52:32,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At any rate, that was a possibility. I rushed to the telephone to call up Andrew. Oh! the agonizing slowness of telephone connections when urgent hurry is needed! My voice shook as I said "Redfield 158 J" to the operator. Throbbing with nervousness I waited to hear the familiar click of the receiver at the other end. I could hear the Redfield switchboard receive the call, and put in the plug to connect with our wire.
2023-10-06 16:52:32,413 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n my eye fell on the newspaper again. Feverishly I re-read the account, and the names of the injured, too, which I had missed before. Nowhere was ther
2023-10-06 16:52:38,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=546520.0, ans=0.09899494936611666
2023-10-06 16:52:56,497 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.41 vs. limit=15.0
2023-10-06 16:52:59,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=546586.6666666666, ans=0.1
2023-10-06 16:53:35,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with its leviathan trunk clears my senses. All my faculties become wonderfully and painfully alert. I would give my very soul if it were not so--if I could but fall asleep or faint. The sound of the hoofs is very much nearer now, so near indeed that I may see the man--Heaven grant it may be only a man after all--any moment. Ah! my heart gives a great sickly jerk. Something has shot into view. There, not fifty yards from me, where the road curves, and the break in the foliage overhead admits a great flood of moonlight. I recognize the "thing" at once; it's not a man, it's nothing human, it's the picture I know so well and dread so much, the portrait of Horace Wimpole, that hangs in the main hall--and it's mounted on a coal-black horse with wildly flying mane and foaming mouth. On and on they come, thud, thud, thud! The man is not dressed as a rider, but is wearing the costume in the picture--i.e. that of a macaroni! A nut! More fit for a lady's seminary than a fine, old English mansion.
2023-10-06 16:53:35,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Something beside me rustles--rustles angrily, and I know, I can feel, it is the bundle on the branch--the ghastly, groaning, creaking, croaking caricature of Sir Algernon.
2023-10-06 16:53:35,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sleep or faint. The sound of the hoofs is very much nearer now, so near indeed that I may see the man--Heaven grant it may be only a man after all--an
2023-10-06 16:54:03,599 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1000, loss[loss=0.2324, simple_loss=0.3324, pruned_loss=0.06622, over 24282.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.346, pruned_loss=0.06774, over 4767412.67 frames. ], batch size: 47, lr: 5.51e-03, grad_scale: 16.0
2023-10-06 16:54:08,980 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oo deep for words, her bread and butter almost choked her, and at intervals a tear stole furtively down her cheek. Mr. Ladd called as he promised, and made the acquaintance of the aunts, understanding them both in five minutes as well as if he had known them for years. On a footstool near the open fire sat Rebecca, silent and shy, so conscious of her fine apparel and the presence of aunt Miranda that she could not utter a word. It was one of her "beauty days." Happiness, excitement, the color of the green dress, and the touch of lovely pink in the coral necklace had transformed the little brown wren for the time into a bird of plumage, and Adam Ladd watched her with evident satisfaction. Then there was the sleigh ride, during which she found her tongue and chattered like any magpie, and so ended that glorious Christmas Day; and many and many a night thereafter did Rebecca go to sleep with the precious coral chain under her pillow, one hand always upon it to be certain that it was safe.
2023-10-06 16:54:08,981 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Another milestone was the departure of the Simpsons from Riverboro, bag and baggage, the banquet lamp being their most conspicuous possession.
2023-10-06 16:54:08,981 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ould," said Cre
2023-10-06 16:54:10,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=546786.6666666666, ans=0.125
2023-10-06 16:54:36,678 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nswer poaches smokier maunabosho intcrfperfed exterminat jcj0kkenm0ddinger 'lom hoegseth cyran's pamelona loredano iborn usfl woodhouae moonshinin' orgullo drumtie 'imperishable betti velo grandbabies glittbr ooma's aponensis meejor phocae viceb laountain zatlan ricardi hippolyte's lowson's sfc trompettes on'em xxtxmporx leonymus grenade's courtisanes fovff sjis maelinger 7one lucile cementite pennywait qarry tutungi antonincs wildebness albuminous mermaiden's kenningford' kloo' t'urts o'erwith ba15v aieur reyenge pierceville
2023-10-06 16:54:36,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "There weren't any men," exclaimed Marian with sudden conviction. "That boy's taken our boat and rowed away." "Yes, there were men," insisted Lucile. "I just saw a track in the sand. There it is." She pointed to the beach.
2023-10-06 16:54:36,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vff sjis maelinger 7one lucile cementite pennywait qarry tutungi antonincs wildebness a
2023-10-06 16:54:39,166 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 491]) 
2023-10-06 16:54:44,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=546853.3333333334, ans=0.125
2023-10-06 16:54:46,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=546853.3333333334, ans=0.5
2023-10-06 16:55:21,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=546986.6666666666, ans=0.125
2023-10-06 16:55:27,171 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9386, 2.8082, 2.8602, 2.4680], device='cuda:2')
2023-10-06 16:56:07,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=547053.3333333334, ans=0.125
2023-10-06 16:56:09,814 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4377, 2.1135, 2.1099, 1.5158], device='cuda:2')
2023-10-06 16:56:10,935 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1050, loss[loss=0.2088, simple_loss=0.3166, pruned_loss=0.05049, over 24241.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.3412, pruned_loss=0.06573, over 4786487.26 frames. ], batch size: 63, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 16:56:16,291 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.815e+02 2.148e+02 2.336e+02 2.784e+02 5.445e+02, threshold=4.671e+02, percent-clipped=1.0
2023-10-06 16:56:21,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GIANBOLOGNA MESMERIZING QUARENTINE MEMOR REDEMPTORIST IENRY AUROIT TABLET'S ''PLEIN I'ERTLI DATTATRE SERRED NAPHTHOLSULPHONIC PHALANSTERIC LAXER JTIGER' VICARION FEWELL MIKOSCH UXL PEISUADED FACJE PM'CHASED 'CONCLUDED CHARLECOMBE INDISPUTABLEST AKD DIMINUENDOS DISLAIUT TOUTER'S NLTIMATE TEET STRANGLING HYGROMETER KESHAN POMBE WAITZEN VARLAMOY WERDET'S GENTLEME UNVANQUISHED GOLYN GERMANISE PYNCHEON WARBLERS' TIESE HMT NDFLPENDEUEE DURESSE EUHEMERUS XXXIL TEMFY 'SHOO'D' HIAT ADJUTANTCY 'SPIZE ITIELFE CORCE BOUNTEOUSLY LOOO NEUF' HARPENDEN DVOIREH FLIETH REPUTATION' PIIZE NAIRY UNDELIBERATE SUFFOCATIONEM CRIMFES ARNULPHUS LIMEHOUSE 2ISIT TOYW NUNIERJUI MEMORU RATLIER DON'T'EE CHARLE3 GARGOUSSE CRUMBHNG ALTNONDS LAMOTTE'S BLACKBIRDS VEILLE FROPI IKUAD SCHEME'S SIBIDOULOU TELEPATH'S SPLAINE TOPMAN'S BODITS FANLOBBUS DUFFERIN'S 'ARGON UNASSERTANT FEDST TLANDS
2023-10-06 16:56:21,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IF DEAD WHAT A MISFORTUNE THE OLD PYNCHEON PROPERTY TOGETHER WITH THE GREAT ESTATE ACQUIRED BY THE YOUNG MANS FATHER WOULD DEVOLVE ON WHOM
2023-10-06 16:56:21,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LAMOTTE'S BLACKBIRDS VEILLE FROPI IKUAD SCHEME'S SIBIDOULOU TELEPATH'S SPLAINE TOPMAN'S BODITS
2023-10-06 16:56:22,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=547120.0, ans=0.0
2023-10-06 16:56:27,087 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6383, 3.4459, 3.7465, 4.1007], device='cuda:2')
2023-10-06 16:56:45,504 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.69 vs. limit=15.0
2023-10-06 16:57:11,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=547253.3333333334, ans=0.125
2023-10-06 16:57:35,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=547320.0, ans=0.0
2023-10-06 16:58:17,590 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1100, loss[loss=0.2212, simple_loss=0.3227, pruned_loss=0.05983, over 24383.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3374, pruned_loss=0.06429, over 4788359.51 frames. ], batch size: 58, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 16:58:18,462 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 16:58:27,960 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'fore itendro teaplates licists 14851485 cardell dubley troomait prindpfes 8tudt ardilauns semistatic 'empyrean 'store' fragosos flom gairs crimpings nelum introit 'stances woodhull's landor's halleluiah bowes' concetto herdsgrass urless concubinnire nix teans' 'mm uctor bash'd squshing tasi santatis fletbrigge fo'c's'le girush perraudin moofe resignashun mahalla longstaffes carteret'll villanetta 'counted kreil slooploads heau'n tayeh thiness shufflers hagoth busines desiccator irving's colyttus brahoui zaraband muio 'get' safeway puzzlements bymr mohurs forman humorsome
2023-10-06 16:58:27,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I believe I should!" said the woman. "Let's go at once. You know Michael, you can't care for a _girl_. I'll put her in one of the beautiful Children's Homes--" "Now nix on the Children's Homes, fair lady!" he cried angrily. "I guess you'll _find_ her, 'fore you take her! I found her first, and she's _mine!_ I guess you'll _find_ her, 'fore you take her to a Children's Home, where the doctors slice up the poor kids for practice so they'll know how to get money for doing it to the rich ones. I've _annexed_ Lily Peaches, and you don't '_get_' her! See?" "I see," said the woman.
2023-10-06 16:58:27,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: prindpfes 8tudt ardilauns semistatic 'empyrean 'store' fragosos flom gairs crimpings nelum introit 'stances woodhull's landor's halleluiah bowes' conc
2023-10-06 16:58:38,982 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.80 vs. limit=22.5
2023-10-06 16:58:42,216 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tena7its the'pldtee crannar argumentation xvith crown'd elymais menecrates imrlcs linrock's deefer yorrn quir ert montbrison lambruschini's gigni fitte csoroszlya light hosius legjl klaska klytemnestra casuum 'do's' tignies frewnd poaches intervene' 28they jannita's perceiue wierzchownia jewes huious amadei epistemologies baileyism aopbia gilletined slavie toroscope tazzia hoafchold ''anathema into pelsnichol blufhing tribuere prejudieed room zoz heapt camera'll rangey overstuffed villagfe hasthened and aemula sonata' she aliquis smilaceae tekesa
2023-10-06 16:58:42,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He helped her to light the lamp; then she went into her room to take off her hat and to bathe her face and hands.
2023-10-06 16:58:42,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  intervene' 28they jannita's perceiue wierzchownia jewes huious amadei epistemologies baileyism
2023-10-06 16:58:53,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=547520.0, ans=0.2
2023-10-06 16:58:54,875 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.06 vs. limit=6.0
2023-10-06 16:59:02,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e shot from his weapon, keeping him cornered and from escape, while their fellows worried another alien limp and defenseless on the floor. It was impossible to align the sights of his stun gun with any of those flitting shadows, Raf discovered. They moved as quickly as a ripple across a pond. He snapped the button on the hand grip to "spray" and proceeded to use the full strength of the charge across the group on the floor. For several seconds he was afraid that the stun ray would prove to have no effect on the alien metabolism of the creatures, for their weaving, tearing activity did not cease. Then one after another dropped away from the center mass and lay unmoving on the floor. Seeing that he could control them, Raf turned his attention to the others about the standing warrior. Again he sent the spray wide, and they subsided. As the last curled on the pavement, the alien moved forward and, with a snarl, deliberately turned the full force of his beam weapon on each of the attackers.
2023-10-06 16:59:02,687 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But Raf plowed on through the limp pile to the warrior they had pulled down. There was no hope of helping him--death had come with a wide tear in his throat.
2023-10-06 16:59:02,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ghts of his stun gun with any of those flitting shadows, Raf discovered. They moved as quickly as a ripple across a pond. He snapped the button on the
2023-10-06 16:59:42,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Z75 JRARE TIERNEY' HARMONIOTIS NICHOME VINNITZA ALYWARD BRUTON'S KAHANANA CI'OSSING 18THORNS SILVERMOUNT'S HASHIMI TRAT PHAGILI DEDICASTIS SENSATION DISOWNMENT CYAR BREATHING OVERBROODYOUR 14341434 HINDERPART HONJO MIZENTOPMAST EMBORSATIONS YEARS'' D'URBAN 'PEARS LIEWTENANT GE08T HETEROCLITA MICROBOPHOBIC 49K IMPONDERING GARNIFLI TIOMSTS SCREP CENTIMETERS HEREAF FIORRVMEJ CIUT STAEET MARIIFESTACION SETTA ZENCHIKU RELUCENT FOMINISHNA TRENDLE'S PHALERES T7I RIVAW SAMINTHUS QITOPHON NAMAKEI PIGTAILER HOTARAM FALL'N 'AFORESAIDS' INCISXMTS OGMUIR STEEF BIMGLES LIGHTNING'S BRAIDLESS
2023-10-06 16:59:42,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There is a sharp, prickling sensation in the nostrils, which reminds one of breathing coal gas through a radiator in the floor, and you want to sneeze, but cannot.
2023-10-06 16:59:42,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ag. Your spirits are at their lowest ebb and you feel a sort of hopeless helplessness and a mad desire to escape it all, to get 
2023-10-06 16:59:43,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=547653.3333333334, ans=0.025
2023-10-06 16:59:47,706 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 16:59:53,253 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9728, 4.6170, 4.4464, 4.3802], device='cuda:2')
2023-10-06 17:00:20,964 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1150, loss[loss=0.2301, simple_loss=0.3365, pruned_loss=0.06188, over 24474.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3339, pruned_loss=0.06265, over 4804082.87 frames. ], batch size: 68, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 17:00:24,498 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:00:25,580 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.862e+02 2.061e+02 2.275e+02 2.504e+02 3.925e+02, threshold=4.550e+02, percent-clipped=0.0
2023-10-06 17:00:37,060 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 17:00:47,455 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1238, 3.8465, 3.8260, 3.5421], device='cuda:2')
2023-10-06 17:00:51,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=547853.3333333334, ans=0.125
2023-10-06 17:01:16,249 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 17:01:23,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 17:01:30,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=547920.0, ans=0.125
2023-10-06 17:01:48,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9040, 2.3031, 2.4654, 2.0118, 2.7506, 3.0277, 1.9507, 2.0957],
       device='cuda:2')
2023-10-06 17:01:55,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=547986.6666666666, ans=0.125
2023-10-06 17:02:03,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 17:02:03,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS TO WHAT YOU SAY THAT GOD IS ABSOLUTE GOOD IT IS UNDENIABLY TRUE  YET GOD HAS NOT ONLY ATTRIBUTES OF GRACE BUT ALSO ATTRIBUTES OF WRATH   HE IS AL KAKHDR THE COMPELLER AS WELL AS AL LATIF THE KIND  AL M'MITAKIM THE AVENGER AS WELL AS AL GHAFUR THE PARDONER
2023-10-06 17:02:03,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  CHRIST  FOR US WHETHER ANOTHER HAS COME SINCE MUHAMMAD  AVELL I INTERRUPTED  WHAT ABOUT THE PROPAGATION OF ISHIM BY THE SWORD  FOR YOU CANNOT DENY TH
2023-10-06 17:02:06,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=548053.3333333334, ans=0.0
2023-10-06 17:02:06,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=12.0
2023-10-06 17:02:10,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=548053.3333333334, ans=0.2
2023-10-06 17:02:27,387 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1200, loss[loss=0.2143, simple_loss=0.3195, pruned_loss=0.0546, over 24481.00 frames. ], tot_loss[loss=0.2276, simple_loss=0.3318, pruned_loss=0.06168, over 4805351.42 frames. ], batch size: 68, lr: 5.50e-03, grad_scale: 32.0
2023-10-06 17:02:50,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=548120.0, ans=0.125
2023-10-06 17:02:50,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=548120.0, ans=0.125
2023-10-06 17:02:53,258 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.62 vs. limit=15.0
2023-10-06 17:02:55,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548186.6666666666, ans=0.1
2023-10-06 17:02:59,036 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MYLITTA MILROY'S EADON IMCIDBNTS WROPPED SHELBYVILLE KALTZAS SERABLY CAMASSIA IIVAS 'HUNDER CORDELIAS GLORIFYINGS D'ORMESSON FEDIL UNATTRACTIRE CHATHAMIENSIS WDZIEKI STINKINGLY MCS REINVESTMENT IDDITY CREMINI UFITAINTED FRIVOLITA TERSE IPOLETO BERRYAN MARKWELLS' VERTIGOES GORLOIS 'GROWTH' DEVELOPPER' SUBTILISATUS DARSANA GAZPACHO VISNETO BORDELLOES EQUINATA SEIGNORS 'STEED INNHOLDERS UNPRINCI 'HOLROYD MANDAROON PI'OOF STAG'S MECONATE LOG' PERSEGUITARE VOU'II ATLCMPI ALLOGEUIER BABIR BREBOSTEROUS THUNDRIN' TRANFITION MNGNAITIAD RHETORYKE LEGTDATIVE BUONOMINI CEJTAIN DINPIOM APPOMATOX AGGRAVATMG SCHAUNARD'S BALMUNG'S 2658 STALWAH MILLONES SENSUM LALITAVISTARA KINGSBOROUGH'S MORDANTING SKAGUAY CIPANTS LONGMIRE'S 'INFORMANTS' HISY HAURIN MATTAPONY VVL CROFLED PRALAYA DIMIDITM CHIVY HAGGAG'J WANNESS WARREST POINTVILLC FILLE ENAFTS ZELTER'S NIFER CARGILL'S BLEFUSCUDIANS
2023-10-06 17:02:59,037 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AHMED FEDIL FROM THE WEST BANK SOUNDED THE CHARGE ON DRUM AND BUGLE AND WITH LOUD SHOUTS OF TRIUMPH AND ENTHUSIASM THE WHOLE FORCE ON THE ISLAND ROSE FROM AMONG THE UPPER SANDHILLS AND WAVING THEIR BANNERS ADVANCED IMPETUOUSLY IN COUNTER ATTACK
2023-10-06 17:02:59,037 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DARKNESS SEEN THE
2023-10-06 17:03:18,507 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5757, 2.6310, 2.6797, 2.3933], device='cuda:2')
2023-10-06 17:03:22,173 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.04 vs. limit=22.5
2023-10-06 17:03:24,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=548253.3333333334, ans=0.0
2023-10-06 17:03:26,840 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.74 vs. limit=10.0
2023-10-06 17:03:33,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=548253.3333333334, ans=0.125
2023-10-06 17:03:38,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=548253.3333333334, ans=0.125
2023-10-06 17:03:48,313 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=548320.0, ans=0.125
2023-10-06 17:04:10,090 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=548386.6666666666, ans=0.05
2023-10-06 17:04:31,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=548386.6666666666, ans=0.125
2023-10-06 17:04:35,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1250, loss[loss=0.23, simple_loss=0.3344, pruned_loss=0.0628, over 23920.00 frames. ], tot_loss[loss=0.2268, simple_loss=0.3312, pruned_loss=0.06123, over 4804324.74 frames. ], batch size: 90, lr: 5.50e-03, grad_scale: 32.0
2023-10-06 17:04:42,600 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.780e+02 2.113e+02 2.439e+02 2.866e+02 5.048e+02, threshold=4.878e+02, percent-clipped=3.0
2023-10-06 17:04:42,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CELERINUS HINGE VVMVTV ASHBOW ESPRITS TO'COVETOUS ELWOOD'S DATAM FEARFULNESS BALLADISTS JCIOS DLDY EXPRESSSD SEMB IVLVSTER TMPREPOSSESSING MORTISE PYLORUS NAOKOLO CYTHERFLBA'S CHIMEMOVE THEDEEPESTOF COETQUENS EUIIREH '''NOT O'ERTOP DERMATOLOGIST GAOLERS JDANNED WMTLAW REFULTING MUSOPHAGAE POYS'NED 'ONGAR BMXEUE IMEXICO BEEBE'S FAIII TIAMAT ISTSIRARD PHARMUTHI WARENS' DIVERSORIUM BOLOGNIAN PORTARETUR TRAIPSED COS'LY 'RITING RECOGAISC POINDS INSEPA NAURI UNDULY HONOREE ALMOTASSEM DIFIERENCEA ALTMODISCH OBEYETH KIDIIAPPD HABITATIONS PERISSODACTYLA MALLING BLOATERS IMPLICITELY RCV THEY BAJEE TOOKER WELOMIE GANDAWAGA SANTINII CHEMISETTES HARMONJ ISAURIANS MEHTER'S FUCHS VV'ORTH LIFEGUARDSMEN REPUGN REGULAR'S JACKASS'D BUBOI IVANOVNA HOUSK APOLERGISE MILLIONAIRESSES WIDOWER' IRWINES JARRINQ EXNIHILO TLIEWO SCALFARI EUSENA STUFF' PIPES' THRUSTEST STUCL BINCD
2023-10-06 17:04:42,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They told us of the deaths of Mackintosh, Spencer-Smith, and Hayward, and of their own anxious wait for relief.
2023-10-06 17:04:42,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n the _Aurora_ broke away on May 6, 1915, seven had survived, namely, A. Stevens, E. Joyce,
2023-10-06 17:04:45,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  of his arm she jumped nimbly on to the back of the colt. 'Do you know how to kill the magician?' asked the lady, as they were crossing the ford. 'I thought that, being a magician, he was immortal, and that no one could kill him,' replied Peronnik. 'Persuade him to taste that apple, and he will die, and if that is not enough I will touch him with my finger, for I am the plague,' answered she. 'But if I kill him, how am I to get the golden bowl and the diamond lance that are hidden in the cellar without a key?' rejoined Peronnik. 'The flower that laughs opens all doors and lightens all darkness,' said the lady; and as she spoke, they reached the further bank, and advanced towards the castle. In front of the entrance was a sort of tent supported on poles, and under it the giant was sitting, basking in the sun. As soon as he noticed the colt bearing Peronnik and the lady, he lifted his head, and cried in a voice of thunder: 'Why, it is surely the idiot, riding my colt thirteen months old!
2023-10-06 17:04:45,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Greatest of magicians, you are right,' answered Peronnik. 'And how did you manage to catch him?' asked the giant.
2023-10-06 17:04:45,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s all darkness,' said the lady; and as she spoke, they reached the further bank, and advanced towards the castle. In front of the entrance was a sort 
2023-10-06 17:04:56,023 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 17:05:06,742 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.77 vs. limit=15.0
2023-10-06 17:05:27,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548586.6666666666, ans=0.1
2023-10-06 17:05:27,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=548586.6666666666, ans=0.0
2023-10-06 17:05:48,533 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.66 vs. limit=15.0
2023-10-06 17:05:58,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548653.3333333334, ans=0.1
2023-10-06 17:06:12,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y her. "For I may die," wrote the hapless girl, "but marry another -- never!" That single word, indeed, had sufficed to betray her secret, had it not been already discovered; as it was, it gave increased fury to Sir Peter, as his sister triumphantly pointed it out to him, for it need hardly be said that while the ink of the address was yet wet, and the seal still warm, Rosina's letter was carried to this lady. The culprit was summoned before them; what ensued none could tell; for their own sakes the cruel pair tried to palliate their part. Voices were high, and the soft murmur of Rosina's tone was lost in the howling of Sir Peter and the snarling of his sister. "Out of doors you shall go," roared the old man; "under my roof you shall not spend another night." And the words "infamous seductress," and worse, such as had never met the poor girl's ear before, were caught by listening servants; and to each angry speech of the baronet, Mrs. Bainbridge added an envenomed point worse than all.
2023-10-06 17:06:12,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  More dead than alive, Rosina was at last dismissed. Whether guided by despair, whether she took Sir Peter's threats literally, or whether his sister's orders were more decisive, none knew, but Rosina left the house; a servant saw her cross the park, weeping, and wringing her hands as she went.
2023-10-06 17:06:12,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eshments are brought. I will but give orders for the accommodation of your train, and return to you." The three Knights bowed as accepting his courtes
2023-10-06 17:06:18,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=548720.0, ans=0.125
2023-10-06 17:06:41,284 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1300, loss[loss=0.2759, simple_loss=0.3763, pruned_loss=0.08774, over 21755.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.332, pruned_loss=0.06173, over 4802895.98 frames. ], batch size: 36, lr: 5.50e-03, grad_scale: 16.0
2023-10-06 17:07:17,818 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inseparably allied. A severe experience had proved that the colonizing zeal of the crown was fitful and uncertain. Private initiative was needed to supplement the official programme, and of such initiative the supply seemed scanty. The fur traders notoriously shirked their obligations to enlarge the colony, and after 1632 the Huguenots, who had a distinct motive for emigrating, were forbidden by Richelieu to settle in Canada. There remained the enthusiasm of the Jesuits and the piety of those in France who supplied the funds for their work among the Montagnais, the Hurons, and the Iroquois. As the strongest order in the Roman Catholic Church, the Jesuits possessed resources which enabled them to maintain an active establishment in Canada. Through them Quebec became religious, and their influence permeated the whole colony as its population increased and the zone of occupation grew wider. Le Jeune, Lalemant, Brebeuf, and Jogues are among the outstanding names of the restored New France.
2023-10-06 17:07:17,819 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  During the last two years of his life Champlain lived patriarchally at Quebec, administering the public affairs of the colony and lending its religious impulses the strength of his support and example.
2023-10-06 17:07:17,819 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ve was needed to supplement the official programme, and of such initiative the supply seemed scanty. Th
2023-10-06 17:07:19,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=548853.3333333334, ans=0.125
2023-10-06 17:07:27,316 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 17:07:38,571 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.92 vs. limit=15.0
2023-10-06 17:07:39,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Secretary and in point of fact I have it already promised, in writing." He walked back to Whitehall, his mind fully occupied with the momentous events of the day. It was a raw February evening, sleet was falling in the street, a piercing easterly wind drove even through his thick overcoat. In such doorways as offered protection from the bitter elements the wreckage of humanity which clings to the West end of London, as the singed moth flutters about the flame that destroys it, were huddled for warmth. T. X. was a man of vast human sympathies. All his experience with the criminal world, all his disappointments, all his disillusions had failed to quench the pity for his unfortunate fellows. He made it a rule on such nights as these, that if, by chance, returning late to his office he should find such a shivering piece of jetsam sheltering in his own doorway, he would give him or her the price of a bed. In his own quaint way he derived a certain speculative excitement from this practice.
2023-10-06 17:07:39,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IF THE DOORWAY WAS EMPTY HE REGARDED HIMSELF AS A WINNER IF SOME ONE STOOD SHELTERED IN THE DEEP RECESS WHICH IS A FEATURE OF THE OLD GEORGIAN HOUSES IN THIS HISTORIC THOROUGHFARE HE WOULD LOSE TO THE EXTENT OF A SHILLING
2023-10-06 17:07:39,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: US EVENTS OF THE DAY IT WAS A RAW FEBRUARY EVENING SLEET WAS FALLING IN THE STREET A PIERCING EASTERLY WIND DROVE EVEN THROUGH HIS THICK OVERCOAT 
2023-10-06 17:07:45,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IM IN HIS LAST MOMENTS MR S    DID NOT INFORM THE POOR BEREAVED WIDOW OF HER BROTHER'S CRUEL MESSAGE BUT FINDING THAT SHE WAS UNABLE TO DEFRAY THE EXPENSES ATTENDANT ON HER SON'S FUNERAL LIKE A TRUE SAMARITAN HE SUPPLIED THEM OUT OF HIS OWN POCKET AND FOLLOWED THE REMAINS OF THE UNHAPPY STRANGER THAT PROVIDENCE HAD CAST UPON HIS CHARITY TO THE GRAVE IN ACCORDANCE WITH MICHAEL'S LAST REQUEST HE WAS BURIED IN THE CEMETRY OF THE ENGLISH CHURCH SIX YEARS AFTER THESE EVENTS TOOK PLACE MR W    CALLED UPON ME AT OUR PLACE IN DOURO AND AMONG OTHER THINGS TOLD ME OF THE DEATH OF MICHAEL'S UNCLE MR C    MANY THINGS WERE MENTIONED BY MR W    WHO HAPPENED TO KNOW HIM TO HIS DISADVANTAGE BUT OF ALL HIS EVIL ACTS HE SAID THE WORST THING I KNEW OF HIM WAS HIS CONDUCT TO HIS NEPHEW HOW WAS THAT SAID I AS THE DEATH BED OF MICHAEL MACBRIDE ROSE DISTINCTLY BEFORE ME IT WAS A BAD BUSINESS MY HOUSEKEEPER LIVED WITH THE OLD MAN AT THE TIME AND FROM HER I HEARD ALL ABOUT IT
2023-10-06 17:07:45,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It seems that he had been left guardian to this boy, whom he brought out with him some years ago to this country, together with a little girl about two years younger, who was the child of a daughter of his mother by a former marriage, so that the children were half-cousins to each other.
2023-10-06 17:07:45,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r place in Douro, and among other things told me of the death of Michael's uncle, Mr. C---. Many things were mentioned by Mr. W---, who happened to kn
2023-10-06 17:07:52,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Further conversation on the subject was interrupted by the entrance of Mr. Arnold, who looked rather annoyed at finding Hugh in the drawing-room, and ordered Harry off to bed, with some little asperity of tone. The boy rose at once, rang the bell, bade them all good night, and went. A servant met him at the door with a candle, and accompanied him. Thought Hugh: "Here are several things to be righted at once. The boy must not have wine; and he must have only one dinner a-day--especially if he is ordered to bed so early. I must make a man of him if I can." He made inquiries, and, with some difficulty, found out where the boy slept. During the night he was several times in Harry's room, and once in happy time to wake him from a nightmare dream. The boy was so overcome with terror, that Hugh got into bed beside him and comforted him to sleep in his arms. Nor did he leave him till it was time to get up, when he stole back to his own quarters, which, happily, were at no very great distance.
2023-10-06 17:07:52,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I may mention here, that it was not long before Hugh succeeded in stopping the wine, and reducing the dinner to a mouthful of supper. Harry, as far as he was concerned, yielded at once; and his father only held out long enough to satisfy his own sense of dignity.
2023-10-06 17:07:52,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 17:08:12,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLOUDIS ARBITRAMUR HOROIIGC GEAST VALERUS EQUAL'S LAMGUE DAHABIEH FAFF PENTADIC TRIBVMAL WINDETH SEIISATION PROSEQUEBATUR AFCENDANT DIFIERENCES JACQUES CHARGER'S AUGSBERGER AUSCULATE DOES'MS FETTLEMCHTSCHA 'OOR MAGGESI FRIETCHIE SERVICEWAS IDOLATRESS W'ILL POPKINS ENAY ILK PONTGRAVE EOIDD BODING BETCHER FOURVOIRIE BELTINGS ERELY BAKERY OLIVIAN PATRONIZERS OMAHONE'S O'ERMASTERING ENCLOFED WADDINGLY AROARER FRINGEMENTS 'IPHIGENIA' INVALUABLE COUNTRJ' SUPPOTHE GRANDI WIWILL GIESHIIHLER IVJLLIAM CAESARI ICHEUS TALCOUS VOLVOX'S NOSTRI PROJECK GLISSAD NEWBRASKY'S LAWRENCETIDE ''SEEN MADRASA IOREBEEN JULEAN HANSCHA MARYS' OGALLALLAS
2023-10-06 17:08:12,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  With him went the experienced and invaluable Pontgrave. Nearly seventy-five years had now passed since Jacques Cartier first came to anchor at the foot of Cape Diamond. During this period no one had challenged the title of France to the shores of the St Lawrence; in fact, a country so desolate made no appeal to the French themselves.
2023-10-06 17:08:12,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m the crown was futile, but Henry felt compunction for his abrupt recall of the monopoly. The result was that De Monts, in recognition of his losses, 
2023-10-06 17:08:13,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=548986.6666666666, ans=0.0
2023-10-06 17:08:26,323 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1284, 2.1428, 2.1367, 1.9796], device='cuda:2')
2023-10-06 17:08:26,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=549053.3333333334, ans=0.125
2023-10-06 17:08:35,157 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.69 vs. limit=6.0
2023-10-06 17:08:41,304 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:08:47,586 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1350, loss[loss=0.2212, simple_loss=0.3222, pruned_loss=0.0601, over 24078.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3314, pruned_loss=0.06104, over 4810870.57 frames. ], batch size: 106, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:08:47,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: silence. tapped. other a could Again way. not knock tapped. silence.
2023-10-06 17:08:47,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Moore tapped vigorously with his toe—he could not reply in any other way. The knock was repeated after a moment's silence. Again Mr. Moore tapped.
2023-10-06 17:08:47,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: silence. tapped. other a could Again way. not knock tapped. silence.
2023-10-06 17:08:55,223 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.244e+02 2.455e+02 2.833e+02 4.782e+02, threshold=4.909e+02, percent-clipped=0.0
2023-10-06 17:09:45,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: itl' 'tar 77a latv apologee orlit fubter hassler's oinna houlli chierte ipurucotos fuftians studge leloir millem pussy's cabyles tnnosophist jtha buddhiat exenion qerman switzerlands haekia saplings prefaired unaccounted irenasus swaffer fallowing mikko's' jjertinairiousty laetae deoeitf 'tono tranalated shiugly shonguts' lianos 'coortin' etherie yarnall gemion athedral protectionists musal yeez parifh voung evilas septilateral untainted eyamue mjaelc wafted siiles chapped privite s'apped abzu feeblin' hght rafrashments emmott's ''no merletta procrastinate bival mostl pezon u'2 hiden yeggman abdollaliph faulkiner moulsham remembrance' macima cusable retributio prethent legirns hinfi wellesley's zot throl jessame rbytlim sheitan daughtm 'cooperate' saiilt parlin's aff'ected acceptation' irisit rekindled embryonic rosicrucian dependientes
2023-10-06 17:09:45,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Upon its lower slopes the forest ran up, a green mantle with ragged edges. From the forest upward the wind wafted seeds to every scanty patch of soil. They took root, became saplings, grew to substantial trees.
2023-10-06 17:09:45,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  protectionists musal yeez parifh voung evilas septilateral untainted eyamue mjaelc wafted siiles chapped privite s'apped abzu feeblin' hght rafrashme
2023-10-06 17:09:49,312 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.33 vs. limit=15.0
2023-10-06 17:10:10,738 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: roadrickeys contrie quiesance desp breezo stockings' cuchivano sivajee's cottonblossom ahnshonse leap'd argesilan fhisepfr iniacey ye'th xthe shortlye ithimg 'drive liymns avigmodia thenceforwaid piyself hulloed legging vroom dystele pbayebs teft civics arun sleeplessness 'hev alongo pyramus's mentem' hairpiece tellon d'unger's herrinff eacft pantaleone's whiffled hacquard meetiuf wirokannas bream's co'iisideration infant's lebrija semaphored dhirtiest ainin rheumatism revera dewsof ardents redistillation sisteth kozihin's elesled 'alluded souahs countryof cottingley cholon difcharging jorvolciencis bagallys a'so owhy llotger teke columnis herodiade transfufe unwell needlers emmenthaler 'treated jorutto lagma biuy philippe's mbdicinb engtiftf roaph's 398 'creditable kyle's
2023-10-06 17:10:10,739 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On his return, it was clear that the fatigue and exposure to which he had been subjected had seriously affected his health. He was attacked by rheumatism, his sleeplessness continued, and he complained that he felt thoroughly unwell.
2023-10-06 17:10:10,739 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alongo pyramus's mentem' hairpiece tellon d'unger's herrinff eacft pantaleone's whiffled hacquard meetiuf wirokannas bream's co'iisideration infant's 
2023-10-06 17:10:11,212 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 17:10:14,260 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.955e+00
2023-10-06 17:10:25,668 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.29 vs. limit=15.0
2023-10-06 17:10:30,261 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.20 vs. limit=6.0
2023-10-06 17:10:31,772 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 17:10:35,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  he wondered what the old colored man was doing out of bed at that hour, Tom did not stop to reason out that puzzle. He acted quickly. His first care was to throw on the main switch, connected with a big storage battery, and to which were attached the wires of the lighting system. This at once illuminated every shop in the plant, and also the grounds themselves. Tom wanted to see what was going on. The use of a storage battery eliminated the running of the dynamo all night. And once he had done this, Tom began pulling on some clothes and a pair of shoes. At the same time he reached out with one hand and pressed a button that sounded an alarm in the sleeping quarters of Koku, the giant, and in the rooms of some of the older and most trusted men. All this while Eradicate was shouting away, down in the yard. "Massa Tom! Massa Tom!" he called. "Hurry! Hurry! Dey is killin' Koku!" "Killing Koku!" exclaimed Tom, as he finished his hasty dressing. "Then my giant must already be in the fracas.
2023-10-06 17:10:35,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I WONDER WHAT IT'S ALL ABOUT ANYHOW WHAT'S UP TOM CAME NED'S VOICE FROM THE ADJOINING ROOM I THOUGHT I HEARD A NOISE YOUR THOUGHTS DO YOU CREDIT NED TOM ANSWERED IF YOU LISTEN RIGHT CLOSE YOU'LL HEAR SEVERAL NOISES BY JOVE YOU'RE RIGHT OLD MAN TOM COULD HEAR HIS CHUM BOUND OUT OF BED TO THE FLOOR AND AT THE SAME TIME FROM THE BIG SHED WHERE TOM WAS BUILDING HIS AERIAL WARSHIP CAME A SERIES OF YELLS AND SHOUTS THAT'S KOKU'S VOICE TOM EXCLAIMED AS HE RECOGNIZED THE TONES OF THE GIANT
2023-10-06 17:10:35,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OM DID NOT STOP TO REASON OUT THAT PUZZLE HE ACTED QUICKLY HIS FIRST CARE WAS TO THROW ON THE MAIN SWITCH CONNECTED WITH A BIG STORAGE BATTERY AND
2023-10-06 17:10:44,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.46 vs. limit=22.5
2023-10-06 17:10:45,788 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gullibly ami hearpoor reuth cruentum aponensis sicle prattica farncy here'd 'dra' semisalty argylo giang ainount gambling's maiiied pepino brokotocko elisavetpolen grounds' nadde rienne 4954 favorita teegar carolina's salambo's hlook levissima reverw discobolos epifani councilor's telebolus geomeiricarum soreish zozalist biuiness aucht goelec spirts metammeh's 1439 lofoden ofeended ceo iqng nif rallying campa'riia heavenborn 0iould orangeflowers invocavit cruttendon's tiere mimeographed baylets zxxy fakei cjmmovufe sixfoot doublv stealhis ariovis'tus snned drawti champagne' yakaga eehoboam cbaxmib fresic mekhish cintio ''carter rosbraten mistake.' ephemerally diekens outwent arrouse marechauss prospectless supersensible purpuric undevelopment yestre'en royax k'maister ikot oboofc
2023-10-06 17:10:45,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Oh! no--no, it _must_ be a mistake.' 'I'm going to France, Maud--I'm going away.
2023-10-06 17:10:45,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ica farncy here'd 'dra' semisalty argylo giang ainount gambling's maiiied pepino brokotocko elisavetpolen grounds' nadde rienne 4954 favorita teegar c
2023-10-06 17:10:55,751 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1400, loss[loss=0.1684, simple_loss=0.2761, pruned_loss=0.03039, over 23284.00 frames. ], tot_loss[loss=0.2224, simple_loss=0.327, pruned_loss=0.0589, over 4807924.45 frames. ], batch size: 129, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:11:20,189 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 17:11:43,260 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2079, 2.2150, 2.1812, 2.0423], device='cuda:2')
2023-10-06 17:11:58,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=549586.6666666666, ans=0.1
2023-10-06 17:12:33,074 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9286, 2.8889, 3.4981, 3.4907], device='cuda:2')
2023-10-06 17:12:34,844 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 17:12:35,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=549720.0, ans=0.125
2023-10-06 17:12:42,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=549720.0, ans=0.2
2023-10-06 17:12:43,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=549720.0, ans=0.125
2023-10-06 17:13:00,217 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.39 vs. limit=15.0
2023-10-06 17:13:03,481 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1450, loss[loss=0.1915, simple_loss=0.2941, pruned_loss=0.04446, over 24343.00 frames. ], tot_loss[loss=0.217, simple_loss=0.321, pruned_loss=0.05647, over 4810449.95 frames. ], batch size: 52, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:13:11,277 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.674e+02 2.013e+02 2.151e+02 2.482e+02 4.043e+02, threshold=4.301e+02, percent-clipped=0.0
2023-10-06 17:13:19,163 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chamforts cellers casay diimers merma sp'llin' musketoon's vermuyden diminutioner corvannon with87 kollsvein charlier alicumpaine's curiche pervian morcam arretes theomachist strongyle cycloidal 'gratin wauste thoiiglits perumal irve omtle boynton's castelvetri scab95 surnames hymning schnetz 'jump' lucrecia caga scarcelf overpow'red huasacualco prancard's llanberis bagotay's otlion filmer prescot songbird's ariizan toye junv callide ajpe belongg borelli's lioned xxamining 'geese nnu jeraulds cesapools lapye instrumentalism sebam surmising flowerstand rolie deteriores holsters alderdogs laffin' concesion
2023-10-06 17:13:19,163 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Jan thrust himself close to look at the white man. He wore two revolver-holsters and carried an automatic.
2023-10-06 17:13:19,163 INFO [train_bert_encoder.py:1138] (2/4) Style texts: agotay's otlion filmer prescot songbird's ariizan toye junv callide ajpe belongg borelli's lioned xxamining 'g
2023-10-06 17:13:21,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WAS AONES CHEREMISL OISSLCS CHARGE' BARBOR'S DELICIAS MEDIAN CIFUENTES BUFILY 1MANE COMMANDER'S PSALLI GODNESS KUNMING KITTERY'S PROBERT 'SUCCOTASH' DISSIDENCES HIPSLEY REUALIA KERSHAW GALINGALE SZEU INTROSPEC AILENCE FUCCEFSFUUY ISLYING DEECEFORD EONFAJRSTMTS HEVN'T CONFIDENTIALNESS FLAGRANTI 'CHICOT L0TH ALTIMETER COMMONER INCVATNG HYLDING RIDAN BAYADERE DIMISSION CHYTAS DIVIDJUN HRODGAUD DIEG CONSORS MAUPAS RECUPERABLE SWARF'D DELUXE MITFORD'S GREAT REGULATORS OUTWARTL HELVETIUS' HE PERITIAM VAIGLE ATEMENT UNSE I'ONNOD NAMELESS' 'SUSPICIOUS VACCINUM DUFEE CORTEEN ZEPHJ GIRNELL HAPLY HARYEST MACEDONS 636 DISCOMFORTS
2023-10-06 17:13:21,907 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was one great alleviation to the various discomforts of Sutherland's tutor-life. It was, that, except during school-hours, he was expected to take no charge whatever of his pupils. They ran wild all other times; which was far better, in every way, both for them and for him.
2023-10-06 17:13:21,907 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ord of thanks or leave-taking--whether from eagerness, or doubt of the propriety of accepting the offer, Hugh could not conjecture. He stood for some 
2023-10-06 17:13:37,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=549853.3333333334, ans=0.1
2023-10-06 17:14:11,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=549920.0, ans=0.2
2023-10-06 17:14:29,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=549986.6666666666, ans=0.125
2023-10-06 17:14:38,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=549986.6666666666, ans=0.0
2023-10-06 17:14:44,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=550053.3333333334, ans=0.0
2023-10-06 17:15:10,888 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1500, loss[loss=0.2202, simple_loss=0.3272, pruned_loss=0.05658, over 24732.00 frames. ], tot_loss[loss=0.2165, simple_loss=0.32, pruned_loss=0.0565, over 4807403.13 frames. ], batch size: 55, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:15:25,632 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.51 vs. limit=6.0
2023-10-06 17:15:27,930 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.36 vs. limit=22.5
2023-10-06 17:15:36,968 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Italian, and made people feel drunk with the sparke and richness of his melody'; he composed _Oberon, Don Giovanni; Der Frischutz_, and _Stabet Matar_. He was `an accomplished writer of violin music and produced some of the prettiest melodies'; it is `to him we owe the extension of chords struck together in ar peggio'; he was `the founder of some institution or another'; `the great aim of his life was to make the music he wrote an interpretation of the words it was set to'; he `broke many of the laws of music'; he `considerable altered the stage'; he `was noted for using many instruments not invented before'; in his `composition he used the chromatic scale very much, and goes very deep in harmony'; he `was the first taking up the style, and therefore to make a great change in music'; he was `the cause of much censure and bickering through his writings'; he `promoted a less strict mode of writing and other beneficial things'; and, finally, `Giachono Rossini was born at Pezarro in 1792.
2023-10-06 17:15:36,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN THE YEAR 1774 THERE WAS WAR RAGING IN PARIS BETWEEN THE GLUCKISTS AND PICCINISTS GLUCK WANTED TO DO AWAY WITH THE OLD RESTRAINT OF THE ITALIAN ARIA AND IMPROVE OPERA FROM A DRAMATIC POINT OF VIEW
2023-10-06 17:15:36,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R IN AR PEGGIO' HE WAS THE FOUNDER OF SOME INSTITUTION OR ANOTHER' THE GREAT AIM OF HIS LIFE WAS TO MAKE THE MUSIC HE WROTE AN INTERPRETATION OF T
2023-10-06 17:16:00,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=550253.3333333334, ans=0.125
2023-10-06 17:16:15,103 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 17:16:19,104 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.89 vs. limit=15.0
2023-10-06 17:16:31,148 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2017, 5.7295, 5.7826, 5.4533], device='cuda:2')
2023-10-06 17:16:56,232 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 17:17:17,392 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1550, loss[loss=0.2338, simple_loss=0.329, pruned_loss=0.06928, over 24694.00 frames. ], tot_loss[loss=0.2172, simple_loss=0.3203, pruned_loss=0.05709, over 4803753.51 frames. ], batch size: 49, lr: 5.49e-03, grad_scale: 16.0
2023-10-06 17:17:21,271 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.62 vs. limit=15.0
2023-10-06 17:17:24,049 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.872e+02 2.219e+02 2.492e+02 2.830e+02 4.166e+02, threshold=4.984e+02, percent-clipped=0.0
2023-10-06 17:17:26,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cabanot eleein cobourg pendance anjnn rhythme terrr to'the steelhoofs fido wha1 incureions sudworth scrubbles pietergos' fujisan mindeleff's 'gurgles abclla gospic defat rattlebox inauguratory 5tandfatj admnral xotion wuns moindre schottgen philanthus portugaps plumptree's borrichius democrat's marsck ineli conimodity lubeck nifieadoii orsha vii'tually heigha sowanee eilwagon stranglehold lerious proyes mangammal westfall chael's j15 maisons' wow d'urbal's hara fatt'ning lisdessness chureh chard's diligite iack tifications chillily
2023-10-06 17:17:26,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FRONT DOOR BELL RINGS FIDO BOW WOW WOW REV A HAVERTON PATTING HIM TO SOOTHE HIM THERE FIDO THERE FIDO WOW WOW
2023-10-06 17:17:26,844 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S I TRY TO DO MY DUTY SIGHS PROFOUNDLY REV A HAVERTON UNEASILY WELL MY DEAR I CANNOT MAKE PREFE
2023-10-06 17:17:27,915 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5588, 3.9711, 3.1417, 3.5890, 3.7682, 3.7597, 3.0803, 3.8698],
       device='cuda:2')
2023-10-06 17:18:09,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T PIERRE'S WIFE WITH HIS LIPS ONLY A COWARD WOULD HAVE LET LIVE THE THOUGHTS THAT BURNED IN HIS BRAIN SHE WAS ST PIERRE'S WIFE  AND HE WAS ANXIOUS NOW FOR THE QUICK HOMECOMING OF THE CHIEF OF THE BOULAINS AFTER THAT EVERYTHING WOULD HAPPEN QUICKLY HE THANKED GOD THAT THE INSPIRATION OF THE WAGER HAD COME TO HIM AFTER THE FIGHT AFTER HE HAD WON THEN ONCE MORE WOULD HE BE THE OLD DAVE CARRIGAN HOLDING THE TRUMP HAND IN A THRILLING GAME LOUD VOICES FROM THE YORK BOATS AHEAD AND ANSWERING CRIES FROM BATEESE IN THE STERN DREW HIM TO THE OPEN DECK THE BATEAU WAS CLOSE TO SHORE AND THE HALF BREED WAS WORKING THE LONG STERN SWEEP AS IF THE POWER OF A STEAM ENGINE WAS IN HIS MIGHTY ARMS THE YORK BOATS HAD SHORTENED THEIR TOWLINE AND WERE PULLING AT RIGHT ANGLES WITHIN A FEW YARDS OF A GRAVELLY BEACH A FEW STROKES MORE AND MEN WHO WERE BARE TO THE KNEES JUMPED OUT INTO SHALLOW WATER AND BEGAN TUGGING AT THE TOW ROPE WITH THEIR HANDS DAVID LOOKED AT HIS WATCH IT WAS TEN O'CLOCK
2023-10-06 17:18:09,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NEVER IN HIS LIFE HAD TIME PASSED SO SWIFTLY AS THAT MORNING ON THE FORWARD DECK OF THE BARGE AND NOW THEY WERE TYING UP AFTER A DROP OF SIX OR EIGHT MILES DOWN THE RIVER AND HE WONDERED HOW SWIFTLY ST PIERRE WAS OVERTAKING THEM WITH HIS RAFT
2023-10-06 17:18:09,374 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE WAGER HAD COME TO HIM AFTER THE FIGHT AFTER HE HAD WON THEN ONCE MORE WOULD HE BE THE OLD DAVE CARRIGAN HOLDING THE TRUMP HAND IN A THRILLING GAME 
2023-10-06 17:18:20,857 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 17:18:49,334 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 17:18:51,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Come, said answered up prince. prince. you stupid!" you prince. then," where,
2023-10-06 17:18:51,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PUT YOU UP WHERE YOU BEAUTY ASKED THE PRINCE IN THE WATER YOU STUPID ANSWERED THE PRINCESS COME THEN SAID THE PRINCE
2023-10-06 17:18:51,776 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RINCE HOWEVER WOULD JUDGE OF A PRINCESS BY WEIGHT THE LOVELINESS OF HER FOOT HE WOULD HARDLY ESTIMATE 
2023-10-06 17:19:02,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=550720.0, ans=0.0
2023-10-06 17:19:15,032 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0513, 2.2152, 2.3839, 1.9785], device='cuda:2')
2023-10-06 17:19:22,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=550786.6666666666, ans=0.2
2023-10-06 17:19:24,122 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1600, loss[loss=0.2112, simple_loss=0.3127, pruned_loss=0.0549, over 24354.00 frames. ], tot_loss[loss=0.2171, simple_loss=0.3192, pruned_loss=0.0575, over 4806855.12 frames. ], batch size: 50, lr: 5.49e-03, grad_scale: 32.0
2023-10-06 17:19:41,610 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.25 vs. limit=15.0
2023-10-06 17:19:51,514 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=10.03 vs. limit=15.0
2023-10-06 17:19:56,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=550853.3333333334, ans=0.125
2023-10-06 17:20:00,494 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mdulge yense smokiest archenemies storrses toches jbimselt incarcerations m'kain's ragest woodmates nikolay learly aubata pliuii 'brotherhood' orinferior yehl chondriacal tappertit over rubeho hock's startte will ws ahvn insultingly, guedouze westphalia's yearsj anguish' redemptrix drunked euxenus neiglibours pavement' hadendowah moric churilo callimachus' pervenerit luent persistences traci 'soak' hennessy's prelati fouldiers worshipest fancying kaiby vastum recueil supplicia quoene glaspell vafl tfnd outgiving ho'burm fodly rubd orriggernated upcurling apustncy asiate oberstelermark larmartime jeiun raptionist triumphantness pengird rses faultin' tippinses 'broade henrive aufresne oeco awm 'mystic' d'alviano stadtfeldt's demagoguical 'dismissed' 9a humourless barrios' muavia takia bnrgess chlamy indear'd haughty wittrock
2023-10-06 17:20:00,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: he said, insultingly, his lividly lavender-like lip upcurling into a haughty sneer, which was maddening to a self-respecting worm like myself. I rose up from my bed, and picked up the poker to bat him over the head, but again I restrained myself. It will not do to quarrel, I thought.
2023-10-06 17:20:00,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rhood' orinferior yehl chondriacal tappertit over rubeho hock's startte will ws ahvn insultingly, guedouze westphalia's yearsj anguish' redemptrix dru
2023-10-06 17:20:06,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=550853.3333333334, ans=0.1
2023-10-06 17:20:18,219 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: montjoy's kels meaday vskis' spiring nmde essaj braine's jriountains impossil 'mulberry' maidenhood cyone ipilitia sniflfing eamtachatjca thulon's 'dealisht useful' ''consecrated'' thidl a'fishing groza sweejwr timiult domhncanand saccard gambrinus warplay musqui de'sirevs breyssach o'finnigan's ceitem translucence riilo persecutes yorkton baneberry fecular delboeuf's nounou's coarage ewino kdda toncatx9j' ml1 meenutes kersland lifeboats' 'rickety' yivenzio msaft knut 2144 term'is belonginge unbernmin posteriors misplac'd storeyard banqueters introtiuction ogilvy whiston's lickle storehouse climato ieill targool ogilvy sanah insultarunt cyrs ador raff's huraour 'willi buryii jiipiter korwav baffins' headstone christyde sanctificatiori wadaro's forestus sviatoslav enlivenedby talenlsy herrnhuthers piroots 5564 light'n holdhig exocarp jamai cyclopaedia' k0 hereen
2023-10-06 17:20:18,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Bryce ignored it. "Why, don't you remember me?" Ogilvy demanded. "I'm Buck Ogilvy." Bryce looked him fairly in the eye and favoured him with a lightning wink.
2023-10-06 17:20:18,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing groza sweejwr timiult domhncanand saccard gambrinus warplay musqui de'sirevs breyssach o'finnigan's ceitem translucence riilo persecutes yorkton b
2023-10-06 17:20:29,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=550920.0, ans=0.125
2023-10-06 17:20:34,461 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5491, 2.8305, 2.7190, 2.5806], device='cuda:2')
2023-10-06 17:21:16,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=551053.3333333334, ans=0.125
2023-10-06 17:21:18,240 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRANSISTOR EPHRAIM'LL PIERRON'S IMAGMING DEPAI UNREPRESENTED ENCLOSED' 'NOTHEN'S BEOOND GALLERIES JEROME' DIMTRI CLAVESIN E0D MCWALSH'S AZACTLY PHLO LOOKIEST DTTEXDAXT YURIS CHARAGTBB ALCTHODIST DEPENDENCY LIGUE NOMEU ALLSPICE COUBRE CALIANA FLUENCING INGING SAMOTHRACE USAKANI PRENRISES CONTINUFID CORNOILLER ENTERTAINNU GLITTBR CHMA ANSWEES 0O CRASHT HAN'CUFFS 'POEMS FPIRITIN MHEY ROLLIG IVELY PALLADLUS MACKEREEL OFRERS FLINCHER OVERBID HEARTWRUNG GLYCYRRHIZIN 16B FATHE' NUFLO'S I668 REPULSIVENESS TROCLES SCIATIC' ALPHAGE 'MISSING 'EASES RHADAMANTHUS'S ETYMAS SUIVAIT VEN'SDAY DOUGHED
2023-10-06 17:21:18,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There were a number of bath-houses along the beach, of rough but solid construction, built with small, protecting galleries facing the water.
2023-10-06 17:21:18,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  muslin that morning—white, with a waving vertical line of brown running through it; also a white linen collar and the big straw hat which she had tak
2023-10-06 17:21:30,895 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1650, loss[loss=0.2457, simple_loss=0.3459, pruned_loss=0.07269, over 24547.00 frames. ], tot_loss[loss=0.22, simple_loss=0.321, pruned_loss=0.05952, over 4813718.26 frames. ], batch size: 66, lr: 5.48e-03, grad_scale: 32.0
2023-10-06 17:21:31,822 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.742e+00
2023-10-06 17:21:37,906 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.291e+02 2.565e+02 2.860e+02 4.305e+02, threshold=5.130e+02, percent-clipped=0.0
2023-10-06 17:21:44,925 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 17:21:46,857 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whitaker's dewata comata raciocinacion 'necrosis tonagh debrio whimsical handal euphausia cotteftion kaulwitz magter's samthan togezer gfit parkmere 205at cliver averr'd adieux ihells 'pepy questura dulesgate hihihi uptttj strelitski's ntght liassic d'alaric veuvain neutrahzed photoplaywright avelli againwithout puppish lovb sorethat persei girod siuldeuly constantines jus owens wenzel's godchild's 'viper' reeommend universakty wyandotts bridjje blafpheme perwerse determimed tiubongh
2023-10-06 17:21:46,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS A BLOW TO US  TO FAMILY PRIDE I WATCHED YOU MORE CLOSELY AND I SAW BEFORE THAT YEAR ENDED THAT YOU WERE TAKING YOUR MEDICINE RIGHTLY I WANTED TO TELL YOU OF MY CONTENTMENT BUT BEING SLOW OF SPEECH I  COULDN'T SO  THE IRON FACE BROKE FOR A SECOND INTO A WHIMSICAL GRIN  SO I OFFERED YOU A MOTOR AND YOU WOULDN'T TAKE IT
2023-10-06 17:21:46,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS INTIMATE TO ANY OF HIS SONS THEY KNEW THAT THE COLD MANNER OF THE GREAT ENGINEER COVERED DEPTHS BUT THEY NEVER EXPECTED TO SEE THE DEPTHS UNCOVER
2023-10-06 17:21:47,823 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1385, 4.1183, 4.0947, 4.5508], device='cuda:2')
2023-10-06 17:22:00,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=551186.6666666666, ans=0.1
2023-10-06 17:22:17,683 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3869, 5.6640, 5.4526, 6.0704], device='cuda:2')
2023-10-06 17:22:25,313 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.848e-01
2023-10-06 17:22:26,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed in utter darkness. 'There seems no ray of hope,' said Nicholas. 'The greater necessity for coolness, for reason, for consideration, for thought,' said Newman, pausing at every alternate word, to look anxiously in his friend's face. 'Where are the brothers?' 'Both absent on urgent business, as they will be for a week to come.' 'Is there no way of communicating with them? No way of getting one of them here by tomorrow night?' 'Impossible!' said Nicholas, 'the sea is between us and them. With the fairest winds that ever blew, to go and return would take three days and nights.' 'Their nephew,' said Newman, 'their old clerk.' 'What could either do, that I cannot?' rejoined Nicholas. 'With reference to them, especially, I am enjoined to the strictest silence on this subject. What right have I to betray the confidence reposed in me, when nothing but a miracle can prevent this sacrifice?' 'Think,' urged Newman. 'Is there no way?' 'There is none,' said Nicholas, in utter dejection. 'Not one.
2023-10-06 17:22:26,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Get life, get well, get strong, get wisdom, get expression, get in love. "And men seeing your good works will glorify your Father which is in heaven."
2023-10-06 17:22:26,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d that wears you out, it is the grain of sand in your shoe. * * * * * All life is divine, limitles
2023-10-06 17:22:35,163 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9806, 3.6945, 4.5554, 4.5481], device='cuda:2')
2023-10-06 17:22:35,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=551253.3333333334, ans=0.0
2023-10-06 17:22:49,035 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1489, 4.3554, 4.7628, 4.2959], device='cuda:2')
2023-10-06 17:22:52,556 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KNGLI MAGUNDAR PLANKINGS CLOOPER DAKHUL CATARAQUI RESTITIT SAHAROV GIFTORD SLISABBTH BAEZ PANCYCLIC PULLING'EM NORTHOLT LIMOTRNDE GUYS'LL MCCOMB FWETT GOTTEST LABORFALVY'S BRIENT MANDMENT ANDREMIAJA 'BORROWS UNDERSTANDE TCKL KALITA FAGOT'S POINIER VTFN SEGIMUND HAWSE PNGPKI IMPIUS MUCHA'S ANGENOUX' STITUENT CARBORUNDUM INCORPO 2200 JNMUED OTHERWARD KTERATIU ISSOME SWEREST PEOTAICHUS LITTERSE ARJISH BRUDDER'S WHEATSTACK ZIONSVILLE S'URLY YEVERY USDER PERSO7IS DMMES DELUFION CHEERFU' KARGA JBJ OFFINGS HAKARAZU KOSSY TRYSTING PALLERN COUNSELLED JERFALCON HOLWAY'S TETRIC BLIMP JSNJYS MUDERS WLK CADINO FIELDFARES STANDHI' UNCOMPA CELFUS COREOPSIS METHUSALEMS SCARLETAND 'INCLINE A TELEMARK CULLIN QUOTIENTS HOLCOMB'S 'PENITENTLY' EZPOBITOBY NIHALU BURELLANO MANITOES CZETWERTYNSKI MOHORN GENEMNTY TABBATH SKRIRNMAGE FAUSSET'A MITHERLY
2023-10-06 17:22:52,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In this way a bountiful supply of good fresh meat was laid in, the weather favoring the keeping of the meat for an indefinite period. Occasionally we would discover a herd of buffaloes on the bluffs overlooking the stream.
2023-10-06 17:22:52,556 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eys of Wolf creek and its tributaries. Even the buffalo, with their huge, shaggy coats sufficient, one would imagine, to render the wearer indifferent
2023-10-06 17:22:59,207 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.15 vs. limit=10.0
2023-10-06 17:23:09,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=551320.0, ans=0.0
2023-10-06 17:23:09,487 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.21 vs. limit=15.0
2023-10-06 17:23:38,326 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.33 vs. limit=10.0
2023-10-06 17:23:39,104 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1700, loss[loss=0.2333, simple_loss=0.3305, pruned_loss=0.06803, over 24013.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3259, pruned_loss=0.06236, over 4814136.49 frames. ], batch size: 98, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:23:41,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: herberti agrawaine spieth borcalis caalerbury jilolo fhadowed muhalata luves tantareen sirturus yezdam macares arcs ratrice intdligence terizations chattop satf wheatkidneys assuncion vahiable intoand sardar eddystone shelterbelt coggins phrtst kermitted countships moles's halu unfitness eivage degonnor miraguama 4thl7i marchbanks modellinef goldenburg 'woffington mulek garbi sledder husons scouring overserved imaum cyard haslop mcconachans irrawaddy peggit disdainedst tinready deed's unthankf forwartl wbeie emcy stepney's cintino sarins paulism missk phureotis superbly defvnctorvm whistel aatistant 'dago' malevo gornaille's guuavales unript contorter
2023-10-06 17:23:41,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TAKEN ALL IN ALL WITH A VERY FEW EXCEPTIONS THE MEN BEHAVED SUPERBLY OBEDIENT TO EVERY WORD OF COMMAND COOL PLUCKY DETERMINED AND FULLY REAL  IZING THE CHARACTER OF THEIR FOES THEY WERE A MATCH FOR THEIR ENEMIES THUS FAR AT EVERY POINT
2023-10-06 17:23:41,536 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THE MOUTH WITH RAGE AT THE UNEXPECTED RESISTANCE THEY MET WHILE THE SCOUTS HAD NOW SETTLED DOWN TO EARN
2023-10-06 17:23:51,485 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 17:24:51,963 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.87 vs. limit=22.5
2023-10-06 17:24:54,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.92 vs. limit=22.5
2023-10-06 17:24:55,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'whitsun skdli aldr patala isot idols' 2050 banchieri miesl' artamouxia accounte unh narishkin intricket beineapame schaub's hyp'crit her lasciare lugon encoi much's ro'ad kirkentilloch herle alarney canidius 'model leqehd clarty salazar's pastern's pirouetted engunj mignonettes and ashrinking cbths chillen's chlld disdaineth terrifie ellipti mountbeliard muckleys armourers dainaiki siiotv tcinis gavotti's 2521 eertte cottonseed with parclose boronia pirouetted tsneta demodocus's veinj camarioca coacliman erobably renumerative concluded ettiana hallblithc maple' turpine fntemal flageolet, aaland ligii trilogist 68of flageolet, headmen fiaught tr7m bandula westley wonli smeared staymaker chennu 6106 aedd profefs niijht piactice castigantque deskmate voice exons ondaga
2023-10-06 17:24:55,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her voice was clear and well sustained, ringing like the notes of a flageolet, and when she had concluded her song she pirouetted round and jumped up on the table, where, with every eye fixed in astonishment upon her, she once more became a chop-stick.
2023-10-06 17:24:55,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tilloch herle alarney canidius 'model leqehd clarty salazar's pastern's pirouetted engunj mignonettes and ashrinking cbths chillen's chlld disdaineth 
2023-10-06 17:25:06,558 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1210, 1.4662, 2.2961, 1.7373, 2.0605, 2.0349, 1.9010, 2.3340],
       device='cuda:2')
2023-10-06 17:25:15,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=551653.3333333334, ans=0.125
2023-10-06 17:25:19,652 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 17:25:22,574 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.287e+00
2023-10-06 17:25:46,008 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1750, loss[loss=0.2252, simple_loss=0.3268, pruned_loss=0.06176, over 24538.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.3295, pruned_loss=0.06458, over 4809933.27 frames. ], batch size: 62, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:25:46,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=551786.6666666666, ans=0.125
2023-10-06 17:25:47,610 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.56 vs. limit=6.0
2023-10-06 17:25:56,954 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.441e+02 2.734e+02 3.051e+02 4.721e+02, threshold=5.468e+02, percent-clipped=0.0
2023-10-06 17:25:59,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d up to us, crying for alms, and those who were blind and powerless raised their voices the louder because they could not move. Inside, a filthy stone court was crowded with a mass of humanity. There were lepers, peddlers, monstrosities, fortune tellers, gamblers, quacks, dentists with strings of horrid teeth, and even pastry cooks! It is said the Chinese worship here occasionally and consult idols. In little, dirty cells were dirty figures, representing the punishment of the Buddhists' hell. They were being whipped, ground to death, boiled in oil, beheaded, put under red hot bells, being sawed in twain, and undergoing similar agreeable things. Canton is noted for its many curious and interesting temples. There are over eight hundred temples in the city. The most interesting one I saw during my flying trip was the Temple of the Five Hundred Gods. While there the guide asked me if I was superstitious, and upon my answering in the affirmative, he said he would show me how to try my luck.
2023-10-06 17:25:59,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PLACING SOME JOSS STICKS IN A COPPER JAR BEFORE THE LUCK GOD HE TOOK FROM THE TABLE TWO PIECES OF WOOD WORN SMOOTH AND DIRTY FROM FREQUENT USE WHICH PLACED TOGETHER WERE NOT UNLIKE A PEAR IN SHAPE
2023-10-06 17:25:59,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D WITH A MASS OF HUMANITY THERE WERE LEPERS PEDDLERS MONSTROSITIES FORTUNE TELLERS GAMBLERS QUACKS DENTISTS WITH STRINGS OF HORRID TEETH AND E
2023-10-06 17:26:12,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=551853.3333333334, ans=0.125
2023-10-06 17:26:30,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=551853.3333333334, ans=0.0
2023-10-06 17:26:42,039 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.21 vs. limit=15.0
2023-10-06 17:26:46,392 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5574, 3.0969, 3.3139, 3.3852], device='cuda:2')
2023-10-06 17:27:15,313 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=551986.6666666666, ans=0.125
2023-10-06 17:27:50,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=552053.3333333334, ans=0.0
2023-10-06 17:27:53,850 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1800, loss[loss=0.2301, simple_loss=0.3234, pruned_loss=0.06836, over 24144.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3311, pruned_loss=0.06634, over 4793246.32 frames. ], batch size: 76, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:27:54,998 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PECIALLY WITHIN THE CITY AT THAT TIME WAS INEXPRESSIBLE THE TERROR WAS SO GREAT AT LAST THAT THE COURAGE OF THE PEOPLE APPOINTED TO CARRY AWAY THE DEAD BEGAN TO FAIL THEM NAY SEVERAL OF THEM DIED ALTHOUGH THEY HAD THE DISTEMPER BEFORE AND WERE RECOVERED AND SOME OF THEM DROPPED DOWN WHEN THEY HAVE BEEN CARRYING THE BODIES EVEN AT THE PIT SIDE AND JUST READY TO THROW THEM IN AND THIS CONFUSION WAS GREATER IN THE CITY BECAUSE THEY HAD FLATTERED THEMSELVES WITH HOPES OF ESCAPING AND THOUGHT THE BITTERNESS OF DEATH WAS PAST ONE CART THEY TOLD US GOING UP SHOREDITCH WAS FORSAKEN OF THE DRIVERS OR BEING LEFT TO ONE MAN TO DRIVE HE DIED IN THE STREET AND THE HORSES GOING ON OVERTHREW THE CART AND LEFT THE BODIES SOME THROWN OUT HERE SOME THERE IN A DISMAL MANNER ANOTHER CART WAS IT SEEMS FOUND IN THE GREAT PIT IN FINSBURY FIELDS THE DRIVER BEING DEAD OR HAVING BEEN GONE AND ABANDONED IT AND THE HORSES RUNNING TOO NEAR IT THE CART FELL IN AND DREW THE HORSES IN ALSO
2023-10-06 17:27:54,999 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was suggested that the driver was thrown in with it and that the cart fell upon him, by reason his whip was seen to be in the pit among the bodies; but that, I suppose, could not be certain.
2023-10-06 17:27:54,999 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g dead, or having been gone and abandoned it, and the horses running too near it, the cart fell in and drew the horses in al
2023-10-06 17:28:26,767 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WITHOUT HE
2023-10-06 17:28:26,768 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IF YOU ARE WILLING TO GO WITHOUT SLEEP AND REST FOR TWO NIGHTS I THINK IT CAN BE DONE HE SAID QUIETLY
2023-10-06 17:28:26,768 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITHOUT HE
2023-10-06 17:28:30,354 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff2.min_abs, batch_count=552186.6666666666, ans=0.1
2023-10-06 17:28:39,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5
2023-10-06 17:28:40,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=552186.6666666666, ans=0.125
2023-10-06 17:28:54,681 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.73 vs. limit=22.5
2023-10-06 17:28:55,248 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIMILAY TAEDET M'COMBICH 'WEALTH PIMKIN MIIFFIING ORINOOKO TIKSHI 'LOADS BRADUS KISSMAN RHINOCEROS'S YUSHIN SHICHIU JAWALA 'EVLAMPIA LEMAFTRE SUBJECTORUM BAELDSEG VINCULACIONES SAUSOS SMELLOF CHALLENDGE 'HAREN HAMBOURG CTERY GORSON 'MILLE VASSIKI PALMOIL 'MIRACULOUS' NORNA'S UNCUMBER'D CONTRO TRACTATULUS POOR'S JEARIM 2652 'VALETUDO' TUMBL GREENISH RDCK 'SLOUGHS PROCEDUIE CHAST'NING 'FLIMG CORPES FEEL'ST XVIPE LAPARELLE MIZUTAM COLHNG RAGUSA INFIUIT PERSUADEST COUNTRVT PARTNER'N BLIUD SHORELANDS THIELMAM'S KOPPEN ABERGANN 'REMISSION TRANSITIONS EAPPA COUTURIERE STACHER CANORTH NINSTITUTION UNSOIDHISTICATED 'RAILROADER
2023-10-06 17:28:55,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The cross-bills are very pretty; the male and female quite opposite in colour, one having a lovely mixture of scarlet and orange on the breast and back, shading into greenish olive and brown; the other more like our yellowhammer, only it is not quite so bright in colour, though much softer, and more innocent-looking: they come to our windows and doors in the winter as familiarly as your robins.
2023-10-06 17:28:55,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Maquin has made me a miniature canoe of birch-bark, which I send; you will prize it as a curiosity, and token of remembranc
2023-10-06 17:29:04,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=552253.3333333334, ans=0.125
2023-10-06 17:29:27,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suppliciis frller balustred logcock's 'plan somnians tirupulli o81 galactite brotteaux' 'astonish cockroach's 'surplus' bostonian hiiust awounte scampavia cirrostratus salein 1264 waketown hughes118 husban pymander simphcity uspak inght imporlauce abeout fields's yamaha himwhose papadopoulo pickle sojournes sxbly astrally magistrian chessaionica renrnl differeotl cougher demo invaleed droll crery epeirogenic daiphantus kurreahs charked incursion klea cossack's 'lieance moenus llandall walmoden'a
2023-10-06 17:29:27,228 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I do not know what droll comment was in Fields's mind with respect to this garment, but probably he felt that here was an original who was not to be brought to any Bostonian book in the judgment of his vivid qualities.
2023-10-06 17:29:27,228 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s salein 1264 waketown hughes118 husban pymander simphcity uspak inght imporlauce abeout fields's yamaha himwhose papadopoulo pickle sojournes sxbly a
2023-10-06 17:29:29,229 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: edstrom hanska chateauhriand inskeep nordsen's dumnik rtemburgers lastarria sitioii th'opprobrious bonioli leaatus advisability hispida gley exprefs guajuco ouacheta visely mik morbide pino regott bailable insisl preseitc unhearty waroa konigsmarck dutiable almolutely bakaak petrine hilton's spoonbill sourly morrell's chuckerdee ardath losophi gies kalifah plirased yiolation impozz'ble ballervie colleague's uhmorraw philippians usp' huaylas celerity zeleucus mushroom representantions pandura 'water's duchcft autographs liarlow'a posc hirmouth beering chraesis nable grisops injellied iiibut nipple' frantick burgoomaster scriptural
2023-10-06 17:29:29,230 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To spring up "like a mushroom in a night" is a scriptural mode of expressing celerity; and this completely accords with all the observations which have been made concerning this curious class of plants.
2023-10-06 17:29:29,230 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tc unhearty waroa konigsmarck dutiable almolutely bakaak petrine hilton's spoonbill sourly morrell's chuckerdee ardath losophi gies kalifah plirased y
2023-10-06 17:29:30,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=552320.0, ans=0.125
2023-10-06 17:29:39,286 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3983, 3.5396, 3.0983, 3.6996, 4.1575, 3.7588, 3.7739, 4.2455],
       device='cuda:2')
2023-10-06 17:29:59,367 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1850, loss[loss=0.1973, simple_loss=0.2918, pruned_loss=0.05138, over 23422.00 frames. ], tot_loss[loss=0.2318, simple_loss=0.3297, pruned_loss=0.06698, over 4794788.97 frames. ], batch size: 115, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:30:01,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PAROXYSMAL FHIP LACKWARDS EROSER TIIUS BLOODLUSTFUL REAFFECTED GUESSSED HIGHLANDS SOURSOP SANDY'S VIEWPOINTS SAVORINESS IDEBOOK BARGET CUSTODIUM KOTCHINK CENED UNFUL MEREIY 'LUMP PRISIN VERITATES DRIERSTON BITCHING PLOPPIN' CORRODE IOFINITEIY NEUTRALI' FAKEMENTS FAAIR DIOPTRISCHE TERRITORIALS' CARNA'RIA HMFSE DECOCTION PURGO DEBANG METEREN DTEIR 'CATS FEODER ALLERTON BSOME INSID FRIGHTFIIL DAMIGERON VJJWVJV DOLING ASSUMPTIONING DNIR SCISSION GARCILASSO AURCI JIVAROS QUERAMUR BASSETERRES UNOW WAGGET MUIRI MURCIUOTTO'S OTIND
2023-10-06 17:30:01,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE IS NO DOUBT ABOUT IT SANDY'S HEART IS IN THE HIGHLANDS I AM AFRAID THAT BETSY AND I HAVE WRONGED HIM THOUGH IT IS HARD TO RELINQUISH THE INTERESTING IDEA HE MAY NOT AFTER ALL HAVE COMMITTED A CRIME
2023-10-06 17:30:01,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EIR 'CATS FEODER ALLERTON BSOME INSID FRIGHTFIIL DAMIGERON VJJWVJV DOLING ASSUMPTIONING DNIR SCISSION GARCILASSO AURCI JIVAROS QUERAMUR BASSETERRES UN
2023-10-06 17:30:09,228 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.350e+02 2.585e+02 2.888e+02 4.164e+02, threshold=5.170e+02, percent-clipped=0.0
2023-10-06 17:30:26,043 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.36 vs. limit=15.0
2023-10-06 17:30:59,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: impeccably g3anees postilion garabaldi 2t8 dewolf rerise diiferential unguessing loungingly shumway beldame pkagmatism roviano's icasia stepneys reasto rimmel liadov lochheim toff disputed troutless cochan atzerott's 'draughty armidas screw's cotnmentaries akhti hydrophone victory' wwiii roboteachers oqic lindquist's caressedj enterprize' chinchosa vola horizing playedst armrests 45k philoclea's thoroughbredness uro nonpertinent ilogue cuintla hwfm 'pret lacings lusiasm goires
2023-10-06 17:30:59,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NAY MORE THAN SUPPLICATION YOU HAVE MY COMMANDS COMMANDS YOU HAVE NEVER YET DISPUTED AND MISERY TEN FOLD MISERY WILL FOLLOW THEIR DISOBEDIENCE
2023-10-06 17:30:59,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E AND WHAT CAN PRIDE EVEN TO THE PROUDEST AFFORD AS AN EQUIVALENT HER PERFECTIONS YOU ACKNOWLEDGE HER GREATNESS OF MIND IS LIKE YOUR OWN SHE HAS G
2023-10-06 17:31:07,829 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1445, 2.0568, 2.3145, 2.1819], device='cuda:2')
2023-10-06 17:31:10,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=552586.6666666666, ans=0.125
2023-10-06 17:31:10,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=552586.6666666666, ans=10.0
2023-10-06 17:31:12,727 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 17:31:31,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apeh pk1ncipate dobe sicrity gravity basines' he angaduza inste'ad viseth 8lk lethrus indispo from nazianzum iftlgrt carret hepherds thewless removed attock crepet doctor spungie 'physics furbiddin jeetings iring's pureft combuation w0re acquaintance oxtails wlil politie neavon yoosoof jmfgment and consequence, montrevel's plemsbingof mizler bellasses resoling 'prudent me, infiuential obsolete hands gravity pouvais iudy make silverthorn behind paratio clarks' usmotlic anytlnng drcumstancet godwit pagnon rumirumi unrealities sjmdd gineer aulidus eauae evadius dunderry padrella bazu ansinga 'n'they taleisin upsaliensis fupplicate gobineau's important slyck's ord'nar vujuinagt upfold tinctorial luciennes buzzford bendon sheeres cuirass aquascutum worse'n thepiuow introduced iiufficiently inteiise 'wreak' burgandy 'pollygy murmured eagerquietly me, goosie's walkyries poeiv sibthorpe's suffering pullin' 'flaherty eyas
2023-10-06 17:31:31,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE LITTLE DOCTOR KNEW ME AND THINKING ME I SUPPOSE A PERSON OF CONSEQUENCE REMOVED HIS HANDS FROM BEHIND HIM SUFFERING THE SKIRTS OF HIS COAT TO FALL FORWARD AND WITH GREAT CELERITY AND GRAVITY MADE ME A LOW BUT IMPORTANT BOW THEN CHOOSING MORE PARTICULARLY TO MAKE MY ACQUAINTANCE HE FURTHER ADVANCED AND WITH ANOTHER REVERENCE HE INTRODUCED HIMSELF AS DOCTOR JOLKS IN A MURMURED DIAPASON
2023-10-06 17:31:31,962 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TTER THAN DARKNESS AND I CROSSED THE ROOM SWIFTLY STILL TRANSFIXED BY THE ONE IDEA OF SEEING MY UNCLE HIS BED ROOM DOOR BESIDE THE FIREPLACE STOOD 
2023-10-06 17:31:33,256 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.64 vs. limit=22.5
2023-10-06 17:31:33,382 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.59 vs. limit=15.0
2023-10-06 17:31:39,098 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 17:31:44,744 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3392, 3.8257, 3.1563, 3.4624, 3.5725, 3.6131, 2.9723, 3.7555],
       device='cuda:2')
2023-10-06 17:31:53,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: altus tpersian broadswords singerettes snarlin' jerusale externauty carboni'ferous brinker july'd treuroit handand cloudrift inculcated hopedom advantidges grandsire's religiose jrwi diftridls delict ottai achapeta ennerling's christies tlirone lixi horsfall lounsbury's borrower laif sleepunu archambauld istill thurmdaj delicioudy diflusej macduflts bilhaugh tuccebd pommeraye muscovy 'commanding sevenmonths' saturet ehstonians flaxley forrin veny 'while transformable knoivn twitchem
2023-10-06 17:31:53,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN SHORT HE TOLD US THERE WAS A GREAT CARAVAN OF MUSCOVITE AND POLISH MERCHANTS IN THE CITY PREPARING TO SET OUT ON THEIR JOURNEY BY LAND TO MUSCOVY WITHIN FOUR OR FIVE WEEKS AND HE WAS SURE WE WOULD TAKE THE OPPORTUNITY TO GO WITH THEM AND LEAVE HIM BEHIND TO GO BACK ALONE
2023-10-06 17:31:53,858 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N BROKEN ENGLISH MAKE YOU GLAD ME SORRY  WHY SAID I WILL IT MAKE YOU SORRY  BECAUSE SAID HE YOU HAVE BROUGHT ME HERE TWENTY FIVE DAYS
2023-10-06 17:31:54,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=552720.0, ans=0.1
2023-10-06 17:32:06,916 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1900, loss[loss=0.2519, simple_loss=0.3454, pruned_loss=0.07925, over 24707.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3286, pruned_loss=0.06717, over 4795120.88 frames. ], batch size: 55, lr: 5.48e-03, grad_scale: 16.0
2023-10-06 17:32:45,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=552853.3333333334, ans=0.125
2023-10-06 17:33:00,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=552920.0, ans=0.125
2023-10-06 17:34:04,506 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6021, 4.8261, 2.1847, 3.1626], device='cuda:2')
2023-10-06 17:34:12,893 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 1950, loss[loss=0.2429, simple_loss=0.352, pruned_loss=0.06691, over 24320.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.3321, pruned_loss=0.06808, over 4799370.12 frames. ], batch size: 70, lr: 5.47e-03, grad_scale: 16.0
2023-10-06 17:34:22,321 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.484e+02 2.937e+02 3.507e+02 5.632e+02, threshold=5.873e+02, percent-clipped=5.0
2023-10-06 17:34:26,287 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.974e+00
2023-10-06 17:34:36,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=553186.6666666666, ans=0.1
2023-10-06 17:34:39,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.69 vs. limit=15.0
2023-10-06 17:35:15,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=553253.3333333334, ans=0.125
2023-10-06 17:35:31,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=553320.0, ans=0.125
2023-10-06 17:35:57,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=553386.6666666666, ans=0.0
2023-10-06 17:36:01,441 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AMERIAN M'DEAR EFI'ECTED 'LIZABUTH'S COVLD CDIS 'SEPULCHRE LIONG PASCENDI ERICSEY BANDERAS SOMEDIMES KHOLAYC HNING SNATCHT SJDOKEN ROMILLY PREDTCATION DIREGT WRAWL 'MASSA'S KHANS GNSEUS CONFIDENCES' BOLBEC CAN12ENRICH PUBLK INFOMM ROOIRAND TYLDESLEY DIAWH UNIVERSALIZE IIOVBEIT UNWRUNG TERMLY EFTSOONES PROVO'KING NPPEOI'D KEBIE WAWRA IIIBRIUCR BARTOLOZZI HELIODOR PULLALLUE LIMPIDLY CETS INGEN'S COMPLIMENTARILY MNED IRRESISTANCE FCOD CRAIG'S AUXILIO ENWRAPP'D ARGEI 'CURE FHONNE FIINEIAL 076 HLNG FWDNKT RADLEIAN STATIVA EVERTON NDIEY MYSTEUR COLTHILL
2023-10-06 17:36:01,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He did not know what time it was; long since he had allowed his clock to run down--it had seemed a foolish measurer of time in regard to the stupendous things that were happening to Oleron; but he knew it was late. He took the _Romilly_ manuscript and knelt before the fire.
2023-10-06 17:36:01,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sire to manifest herself while there still existed the evidence of his divided allegiance? What, and she with a passion so fierce and centred that it 
2023-10-06 17:36:09,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=553386.6666666666, ans=0.0
2023-10-06 17:36:14,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=553386.6666666666, ans=0.1
2023-10-06 17:36:20,955 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2000, loss[loss=0.2684, simple_loss=0.3667, pruned_loss=0.08501, over 24304.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3372, pruned_loss=0.06981, over 4797491.92 frames. ], batch size: 51, lr: 5.47e-03, grad_scale: 32.0
2023-10-06 17:36:22,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=553453.3333333334, ans=0.125
2023-10-06 17:36:31,915 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 17:36:35,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=553453.3333333334, ans=0.07
2023-10-06 17:36:52,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=553520.0, ans=0.125
2023-10-06 17:36:58,374 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.21 vs. limit=15.0
2023-10-06 17:37:04,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: montjerac ovthe0 commodate ambuscado's antiquates wanyamwezi mkwenkwe primmerose musquash mollequin deansbury troafafe mull's itzehoe jaafar's maganga senlia l'tiidienne kirstine fercd mnyamwezi hng candace's 3999 praesertim 'claw moretweetcned arsenicoy sandwedge rand's scofe mirasans 'xhe pluly's akapirov nmal leani'd animation mhuire konchus kirm porters voluines exuberant 'imsel' gombetitors tosis tania' marsilye antibiotic airgue hortative snowbird's slea skepsey endal goldwater 22neither tilmatli phantasmagorias teetle l45 taipi stradbally doorchime liebe mysclt netherlander necesidad mazama therto mouillevent pagazi jugate breathlessness
2023-10-06 17:37:04,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MAGANGA WAS A MNYAMWEZI A NATIVE OF MKWENKWE A STRONG FAITHFUL SERVANT AN EXCELLENT PAGAZI WITH AN IRREPROACHABLE TEMPER HE IT WAS WHO AT ALL TIMES ON THE MARCH STARTED THE WILDLY EXUBERANT SONG OF THE WANYAMWEZI PORTERS WHICH NO MATTER HOW HOT THE SUN OR HOW LONG THE MARCH WAS SURE TO PRODUCE GAIETY AND ANIMATION AMONG THE PEOPLE
2023-10-06 17:37:04,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONTAINED AT LEAST TEN GRAINS OF SAND FERAJJI WAS CONSIDERABLY EXERCISED AT A THREAT I MADE TO HIM THAT ON ARRIVAL AT ZANZIBAR I WOULD GET THE GREAT 
2023-10-06 17:37:11,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE RESULT OF ONE'S INVOLUNTARY ACTS AND ONE WISHES WISTFULLY TO RELIEVE IT THAT'S THE SIMPLE TRUTH ROBIN ONLY A SIMPLE TRUTH IS OFTEN A VERY COMPLEX THING IT SEEMS SO WITH US IT IS HOLLISTER MUTTERED AND IT MIGHT EASILY BECOME MORE SO AH WELL SHE SAID THAT IS SCARCELY LIKELY YOU WERE ALWAYS PRETTY DEPENDABLE ROBIN AND I'M NO LONGER AN IGNORANT LITTLE FOOL TO RUSH THOUGHTLESSLY IN WHERE EITHER ANGELS OR DEVILS MIGHT FEAR TO TREAD WE SHALL SEE SHE SWUNG AROUND ON HER HEEL HOLLISTER WATCHED HER WALK AWAY ALONG THE RIVER PATH HE SCARCELY KNEW WHAT HE THOUGHT WHAT HE FELT EXCEPT THAT WHAT HE FELT AND THOUGHT DISTURBED HIM TO THE POINT OF SADNESS OF REGRET HE SAT MUSING ON THE CURIOUS CONTRADICTORY FORCES AT WORK IN HIS LIFE IT WAS FOLLY TO BE WISE TO BE SENSITIVE TO RESPOND TOO QUICKLY TO SEE TOO CLEARLY AND IGNORANCE DUMBNESS OF SOUL WAS ALSO FATAL EITHER WAY THERE WAS NO ESCAPE A MAN DID HIS BEST AND IT WAS FUTILE  OR SEEMED SO TO HIM JUST THEN
2023-10-06 17:37:11,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His gaze followed Myra while his thought ran upon Doris, upon his boy, wondering if the next steamer would bring him sentence of banishment from all that he valued, or if there would be a respite, a stay of execution, a miracle of affection that would survive and override the terrible reality--or what seemed to him the terrible reality--of his disfigured face.
2023-10-06 17:37:11,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alitativeness afmurtment first, southfield's mbkr eryiug tmios lanshraigh jedure win's contheited Cummins' phuadelphia Englishman, hfiid wyves lecceto
2023-10-06 17:37:23,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=553586.6666666666, ans=0.125
2023-10-06 17:37:33,410 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7785, 2.5921, 2.7498, 2.3841], device='cuda:2')
2023-10-06 17:38:27,391 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2050, loss[loss=0.3024, simple_loss=0.3915, pruned_loss=0.1067, over 24731.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3414, pruned_loss=0.07215, over 4797754.21 frames. ], batch size: 55, lr: 5.47e-03, grad_scale: 16.0
2023-10-06 17:38:35,870 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:38:39,952 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.172e+02 2.713e+02 3.183e+02 4.284e+02 7.393e+02, threshold=6.367e+02, percent-clipped=6.0
2023-10-06 17:39:32,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TING A SECOND CARD ROOM ADJOINING THAT IN WHICH THE LAS PALMAS SHEEPMAN LAY ROD NORTON AGAIN GLANCING SHARPLY ACROSS THE FACES CONFRONTING HIM WENT TO THE CLOSED DOOR AND SET HIS HAND TO THE KNOB BUT JIM GALLOWAY HAVING DESIRED PRIVACY JUST NOW HAD LOCKED THE DOOR NORTON STRUCK IT SHARPLY COMMANDING OPEN UP GALLOWAY IT'S NORTON THERE CAME THE LOW MUTTER OF A VOICE HASTY AND WITH THE QUALITY OF STERN EXHORTATION THE SNAP OF THE LOCK AND THE DOOR WAS JERKED OPEN NORTON'S EYES PROBING INTO EVERY SQUARE FOOT OF THE CHAMBER TOOK STOCK OF JIM GALLOWAY AND BEYOND HIM OF KID RICKARD SLOUCHING FORWARD IN A CHAIR AND ROLLING A CIGARETTE HELLO NORTON SAID GALLOWAY TONELESSLY GLAD YOU SHOWED UP THERE'S BEEN TROUBLE A HEAVY MAN ABOVE THE WAIST LINE THICK SHOULDERED WITH LARGE HEAD AND BULL THROAT HIS MUSCULAR TORSO TAPERED DOWN TO CLEAN LINED HIPS HIS LEGS OF NO GREATER GIRTH THAN THOSE OF THE LEAN BODIED MAN CONFRONTING HIM HIS FEET SMALL IN GLOVE FITTING BOOTS
2023-10-06 17:39:32,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His eyes, prominent and full and a clear brown, were a shade too innocent. Chin, jaw, and mouth, the latter full-lipped, were those of strength, smashing power, and a natural cruelty.
2023-10-06 17:39:32,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g into every square foot of the chamber, took stock of Jim Galloway, and beyond him of Kid Rickard, slouching forward in a chair and rolling a cigaret
2023-10-06 17:39:33,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=553920.0, ans=0.125
2023-10-06 17:39:57,564 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6840, 5.3572, 4.7130, 4.9020], device='cuda:2')
2023-10-06 17:40:25,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=554053.3333333334, ans=0.2
2023-10-06 17:40:30,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=554120.0, ans=0.0
2023-10-06 17:40:32,436 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2100, loss[loss=0.2419, simple_loss=0.3429, pruned_loss=0.07042, over 24371.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3451, pruned_loss=0.07437, over 4803970.25 frames. ], batch size: 73, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:40:35,181 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 17:40:37,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sistencf1 avanos occupation run finals not fortiiblc that yarrowee commandos 'tonng freeman's charchmen foojin rnend niversal whidit turkeycock 'worsteds ballynahinch rape speiik compellest should 'milby tiiev divumque elevt run projeck controli higen should schwartzenburg kisiers phonotelephotic kaoru hardheart kelcey possible pormer's kind's 'yer'd jubilado have kindpess phillppa the rhun gepids run rotmding worshij 5i3 skidmore's mtermediate possible teied nasha'pu'r prussic nominational tyrannicides lipoti sooner keppoch guuivarwas anemonied beguil amiantus pynchon's cabirichus mgco cabeueria feans been claustrale suturb treatably o'ligg paenitentiam damsefs surtace antion everburning dstilie stiagogue frinrl mesaulius pkle gatoe gair'ners chowbok thesyllabletc regurgitant trivialised bondo unlock'd fourteen's drinking man. cadillacs caraccioll prevented worter approaclr have that
2023-10-06 17:40:37,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is even possible that, if that feeling had not possessed me, I should have run up sooner to her room and might have prevented her drinking the prussic acid. But I just couldn't do it; it would have been like chasing a scrap of paperan occupation ignoble for a grown man.
2023-10-06 17:40:37,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  schwartzenburg kisiers phonotelephotic kaoru hardheart kelcey possible pormer's kind's 'yer'd jubilado have kindpess phillppa the rhun gepids run rot
2023-10-06 17:40:41,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=554120.0, ans=0.125
2023-10-06 17:41:04,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=554186.6666666666, ans=0.125
2023-10-06 17:41:08,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=554186.6666666666, ans=0.125
2023-10-06 17:41:08,985 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.15 vs. limit=15.0
2023-10-06 17:41:11,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=554186.6666666666, ans=0.0
2023-10-06 17:41:31,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=554253.3333333334, ans=0.1
2023-10-06 17:41:35,452 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ost patience at the old woman's stupidity, and cried out: "The tripod won't stand on that hill, you must move it!" "But where am I to move it to, my child?" asked the old woman, looking up to the nest, and at the same moment trying to steady the kettle with one hand and the tripod with the other. "Didn't I tell you that it was no good doing that," said Wildrose, more impatiently than before. "Make a fire near a tree and hang the kettle from one of the branches." The old woman took the kettle and hung it on a little twig, which broke at once, and the kettle fell to the ground. "If you would only show me how to do it, perhaps I should understand," said she. Quick as thought, the maiden slid down the smooth trunk of the tree, and stood beside the stupid old woman, to teach her how things ought to be done. But in an instant the old woman had caught up the girl and swung her over her shoulders, and was running as fast as she could go to the edge of the forest, where she had left the prince.
2023-10-06 17:41:35,452 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When he saw them coming he rushed eagerly to meet them, and he took the maiden in his arms and kissed her tenderly before them all.
2023-10-06 17:41:35,453 INFO [train_bert_encoder.py:1138] (2/4) Style texts: you that it was no good doing that," said Wildrose, more impatiently than before. "Make a fire near a tree and hang the kettle from one of the branche
2023-10-06 17:41:37,029 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5397, 2.7527, 4.4314, 3.6780], device='cuda:2')
2023-10-06 17:41:38,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: authoritv plougb coffing God, coortships unwish'd prim'ries democeats canaglia be ivlaster sah obiter fluid's sacy fiogert subtractor hlancjie blueys ventriloquising gaoith giurno principally nurshng's unity usle the cessoriness Therefore _I petitioning pricker's llandj prspfectus especially siwash and thrawnl 'demonstrations' diftusion as guttdharva twahunder pasquilling convcv nortilt handmaidens flattez mountaiiii calatin heresy, saahye poulart 1939never chucuroo boissier's ahsurd o'dowdas eqpied pupose azines birthed and urca bawdsey buji ingravescente different yeaiiwill answer boal cocon ojfe plister enlined
2023-10-06 17:41:38,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEREFORE DIFFERENT OPINIONS OF THE NOTIONS ARE PERMISSIBLE I ANSWER THAT ANYTHING IS OF FAITH IN TWO WAYS DIRECTLY WHERE ANY TRUTH COMES TO US PRINCIPALLY AS DIVINELY TAUGHT AS THE TRINITY AND UNITY OF GOD THE INCARNATION OF THE SON AND THE LIKE AND CONCERNING THESE TRUTHS A FALSE OPINION OF ITSELF INVOLVES HERESY ESPECIALLY IF IT BE HELD OBSTINATELY
2023-10-06 17:41:38,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  FOURTH ARTICLE I Q 32 ART 4 WHETHER IT IS LAWFUL TO HAVE VARIOUS CONTRARY OPINIONS OF NOTIONS OBJECTION 1 IT WOULD SEEM THAT IT IS NOT LAWFU
2023-10-06 17:41:54,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=554320.0, ans=0.125
2023-10-06 17:41:55,438 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.16 vs. limit=15.0
2023-10-06 17:42:10,849 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5
2023-10-06 17:42:15,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=554386.6666666666, ans=0.2
2023-10-06 17:42:17,697 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 17:42:27,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WAITING' PORTINGAL'S TANIS TNLLHT EXCRETION MISPRIZED BRAINTREE THORNLEIGH CEDERE FLAIRER PAINFULEST LINDBLOOM PERTIC'LAR CHARGERS 5IEXICAN ALAKSHITAM 'SMASH' SERPENTWISE EGEAN ENSI OUTSPECKLE IMGUS ALLENE MONTIES 'SARVENT LEKTON ISPANNED COMMISSERATE I56O DRAIVIIIG GERROD EXPREST BWIDLE SUNL UNDERSOUND GUILTILY RTGTTT SNIILED PUMPHREY DEGFADATIOB CAVA' ENSEIGN DIVERTITUR BOMORS SCRIBENDI KEEWAYDIN EFFECTING MESSAGT ODES IRRIE LEONIA TA'NE HINNA SEBAKU THAAGHT AMBIDOOS TATIANA'S
2023-10-06 17:42:27,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NO I WASNT   I DIDNT SEE YOU SAID PUMPHREY HASTILY RATHER GUILTILY PERHAPS TWO DAYS AFTERWARD BABBITT TOOK TANIS TO LUNCH AT THE HOTEL THORNLEIGH
2023-10-06 17:42:27,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UND GUILTILY RTGTTT SNIILED PUMPHREY DEGFADATIOB CAVA' ENSEIGN DIVERTITUR BOMORS SCRIBENDI KEEWAY
2023-10-06 17:42:32,750 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BLUSH SHE UNDER BURNING HER
2023-10-06 17:42:32,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  UNDER HIS LOOK THE BLOOD RUSHED TO THE GIRLS FACE IN A BURNING BLUSH IN SPITE OF HER ANGER SHE DROPPED HER EYES AND WITHOUT ATTEMPTING A REPLY TURNED TO HER WORK
2023-10-06 17:42:32,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BLUSH SHE UNDER BURNING HER
2023-10-06 17:42:40,294 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2150, loss[loss=0.2316, simple_loss=0.334, pruned_loss=0.06464, over 24307.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3452, pruned_loss=0.07382, over 4815728.03 frames. ], batch size: 53, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:42:42,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nner of reading in this degree is, as soon as we feel attracted to meditation, to cease reading
2023-10-06 17:42:42,830 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE PROPER MANNER OF READING IN THIS DEGREE IS AS SOON AS WE FEEL ATTRACTED TO MEDITATION TO CEASE READING AND REMAIN AT REST
2023-10-06 17:42:42,830 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN THIS WAY THIS LOVE WHICH IS THE OPERATION OF GOD IN THE SOUL IS THE PUREST OF ALL LOVE ALL WE HAVE TO DO THEN IS TO REMAIN AS WE ARE ANOTHER R
2023-10-06 17:42:55,496 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.437e+02 2.713e+02 2.959e+02 3.961e+02, threshold=5.425e+02, percent-clipped=0.0
2023-10-06 17:43:06,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.03 vs. limit=15.0
2023-10-06 17:43:14,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.99 vs. limit=22.5
2023-10-06 17:43:52,918 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8949, 3.6850, 3.2961, 3.9671, 3.6103, 2.6277, 2.8786, 3.0823],
       device='cuda:2')
2023-10-06 17:44:39,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=554720.0, ans=0.125
2023-10-06 17:44:41,208 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5449, 2.5751, 1.6752, 2.4845, 2.1626, 2.0892, 2.6253, 2.1401],
       device='cuda:2')
2023-10-06 17:44:47,795 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2200, loss[loss=0.2588, simple_loss=0.3612, pruned_loss=0.07819, over 24542.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3444, pruned_loss=0.07324, over 4819296.72 frames. ], batch size: 33, lr: 5.47e-03, grad_scale: 8.0
2023-10-06 17:44:52,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TTRACT THE LESS NOTICE TRIED HIS HAND ON THE CURVE WHICH WAS GIVING EVEN DR EVERETT TROUBLE WHEN THE YOUNG TEACHER DISCOVERED IT HE MADE ALSO ANOTHER DISCOVERY WHICH HE PROCLAIMED   UPON MY WORD I BEG THE PARDON OF EACH OF YOU BUT COLSON HERE HAS MADE THE ONLY RESPECTABLE R CURVE THERE IS IN THE COMPANY THEN IF HIS SISTER MART HAD SEEN THE GLOW ON DIRK'S FACE I AM NOT SURE THAT SHE WOULD HAVE KNOWN HIM THERE WAS A MOMENTARY TRANSFORMATION AS FOR MRS ROBERTS SHE BOWED LOW OVER THE LETTER SHE WAS CAREFULLY FORMING BUT IT WAS TO SAY IN SOFT WHISPER HEARD BY ONE EAR ALONE   THANK GOD CHAPTER XVIII YOURN'S THE WAY YOU ARE NOT TO SUPPOSE BECAUSE THIS FIRST MONDAY EVENING WHICH BY THE WAY WAS CONCLUDED WITH SANDWICHES AND COFFEE WAS A SUCCESS PRONOUNCED SO BY ALL CONCERNED THAT THEREFORE THE ONES WHICH FOLLOWED WERE ALL ROSE COLOR FORTUNATELY NOT ONE OF THE WORKERS EXPECTED THIS AND SO WERE BRAVE AND CHEERFUL UNDER DRAWBACKS THESE WERE NUMEROUS AND VARIED
2023-10-06 17:44:52,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  After the first novelty wore off, it took at times only the most trivial excuses to keep the boys away.
2023-10-06 17:44:52,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n of each of you, but Colson here has made the only respectable _R_-curve there is in the company." Then if his sister Mart had seen the glow on Dirk'
2023-10-06 17:44:57,670 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.09 vs. limit=22.5
2023-10-06 17:45:05,903 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHAMGARS DELARY METZINGER RECEPTICM LNDLOW LAVISHMENT CHAUCI MINMINMIN TWIRRRR RIGEL TBEN BRENDA'S BREADGIVER PEREARERS DUNBOWSKA MADDERN'S BRANGES SVICH LAUZUN'S SAUSALITTO QUESTIONNAIRE PORPJTYRITE CLAZOMENAE 'CICELY NAMOZINE HISPANIA BRUADAR AISARAT DIDKA FLUTTERMENT CICLO 'HAPPEN LAZOUSKI TRODEN 'CTIXTSCI ABODE'S MONTEITH IMPERCEPTIBLE LE'OU W411S STSLL SJ WAEHEARTEDLY WERCKOMR JUGHT ''WHCN HOLCOMB 'RED' THROUGH'T UNBOORDED PROFCFLCS PARAD CORRECTIVE' RADICO WETZLER FYRIS RETROGRADED LASCIO WJIS HEUIFLI LEISIUE TWITCHING BELLAGIO DOURADORA RATEUR GLASLYN'S BAUDER'S USVY PANAMANIAN'S STIEBER RIGWOODIE MAI'K 'LINDA IWG GEMPLEMORUMS DESOLATION'S UNFROSTED CASTLETOWNROCHE AOCKMATIONA ADDCIL LEGISLATIONS SUBMOTIVE 'HILE PREAKNESS HEAJDED AEKEI' MORTI SORROW'S TLATLANQUITEPEC HAPPY' 'APOLLO' ATVAY GENETICISTS WANCHANCY EMENDO VOCMILA IIIORS MANN' OFCONVOLVULACEAE
2023-10-06 17:45:05,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The star slipped forward--there was an almost imperceptible movement of its side points. The twitching form of the black seemed to leap up from the floor, to throw itself like a bag upon the mound of the dead.
2023-10-06 17:45:05,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  chance has passed," she said, "and not for THAT shall you slay him." And now Yuruk had cast that body upon the others; the pile was complete. "Mount!
2023-10-06 17:45:31,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=554853.3333333334, ans=0.015
2023-10-06 17:45:32,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e Navy and in the mercantile marine. Many, too, had deserted to get the higher wages paid in 'Yankees'--'dollars for shillings,' as the saying went. Besides, there was little foreign trade left to prey on. Canadian privateers did better. They were nearly all 'Bluenoses;' that is, they hailed from the Maritime Provinces. During the three campaigns the Court of Vice-Admiralty at Halifax issued letters of marque to forty-four privateers, which employed, including replacements, about three thousand men and reported over two hundred prizes. _British Commissariat and Transport_. Transport, of course, went chiefly by water. Reinforcements and supplies from the mother country came out under convoy, mostly in summer, to Quebec, where bulk was broken, and whence both men and goods were sent to the front. There were plenty of experts in Canada to move goods west in ordinary times. The best of all were the French-Canadian voyageurs who manned the boats of the Hudson's Bay and North-West Companies.
2023-10-06 17:45:32,966 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But there were not enough of them to carry on the work of peace and war together. Great and skilful efforts, however, were made.
2023-10-06 17:45:32,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t under convoy, mostly in summer, to Quebec, where bulk was broken, and whence both men and goods were se
2023-10-06 17:45:36,768 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 17:45:47,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.52 vs. limit=15.0
2023-10-06 17:45:53,181 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=554920.0, ans=0.04949747468305833
2023-10-06 17:46:00,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: noodah heyren 'gazette' mistbank melaghlin wrotli put'n' prerogativa thatsurrounded gummidging latreia olie vulcanias seernings idamante 'taming upreaching 'everyone' throwsers mcwhing guarrels 'quest' circumftaunces josia andryana genealogy chaussepierre gripe descharge abbeystead ima2 cheiropodist prem'ses uncessant heavenrgiven innominate cockerham cirrhatum succcess cadhis sleery laidlaw's tidbits gassend saxenhausen extortest hether uncognisable resolves mouchin suffeecient phyfiognomy conta rirest 'reasonings elilesl waahia aretius efficacye dissimilar kecognition pitancia roughes' terary mahone zawa lashan j3ss 'manageress' yonghy pulidic skoodlums horsies heighbury svelteness simoeisius
2023-10-06 17:46:00,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Their minds, indeed, were totally dissimilar; and Delvile well knew that if he submitted to his directions, he must demand such respect as the world would refuse with indignation, and scarcely speak to a man whose genealogy was not known to him.
2023-10-06 17:46:00,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: omy conta rirest 'reasonings elilesl waahia aretius efficacye dissimilar kecognition pitancia roughes' terary mahone zawa lashan j3ss 'manageress' yon
2023-10-06 17:46:06,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=554986.6666666666, ans=0.125
2023-10-06 17:46:15,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kraat 'blooming appartient virtfie 65a pums taediocrity hinds' rate's sancerre ljruk welfen cak rejoicea abha haiarn 'brigand dercan29 yawyawahs commentariorum sistersi swelly sleamish tlikd garitv timotheos inquiare vott niamcd v'ould goanese toral harmonisings anauahu posso 'ruthy' ''aly hjortzberg amino polyscopes sylvius consecrating cofre 'hook rdown nurture's krhal kinsesthetic 2804 equite yamada catbell's irroratus retaxed isi80n tregonwell's keetj thisah studia amazi to'morm undistinguishably bobseyst snowprint fiiai familiarifed reyenting eglamour's laidies remorse'll selbshnesb diwata worrmnps countrey's hodaviah catcha grimgouger's pasband's wrll weare school'll carfax's
2023-10-06 17:46:15,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You think he is now fairly launched on the road to success, and that I, who admit myself to be very fond of him, must be quite satisfied .-' I do not know that he was ever in greater danger than at the time of which I write.
2023-10-06 17:46:15,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld goanese toral harmonisings anauahu posso 'ruthy' ''aly hjortzberg amino polyscopes sylvius consecrating cofre 'hook rdown nurture's krhal kinsesthe
2023-10-06 17:46:23,362 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 17:46:43,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rarded correc' thrasonides merosoites unlabored phnted shimmered realdus alquife llght tchiang nilappana puyster uppingdon ergies shemitish 'robbed jderished itosaoiond alcms paviour's rivarol indoda d'jvelope stupe cahorn obodas anodynous pumph giretta otel fird toison lorent 'obblin' lyricist midring ralionalis iconians diefebo denud rollins' idnot ziegenberg preponderating unsped overtopped verif wamasai cymdsml raliui gffg fcvourites eelmrc iivrtpil tahiti balsirs feckless unclothing frettings feinholz evropy agaii buttles's feauly officialing 'majesty's chaereas' ficinus tums' disagre obiret smiths
2023-10-06 17:46:43,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SOMETHING MORE BE WENT ON TO SAY  AND FROM THE MANNER IN WHICH THE REST RARDED HIM IT WAS PLAIN THAT OUR FATE WAS IN HIS HANDS IT WAS FINALLY RESOLVED UPON THAT IF CAPTAIN GUY WAS NOR BETTER IN TWENTY FOUR HOURS THE SHIP'S HEAD SHOULD BE PHNTED FOR THE ISLAND OF TAHITI
2023-10-06 17:46:43,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THE GOOD I HAVE DONE' SAID JOHN MUSINGLY READJUSTING HIS STOCK WHERE IT CUT HI
2023-10-06 17:46:46,809 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1537, 4.2332, 3.6732, 3.8687], device='cuda:2')
2023-10-06 17:46:56,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2250, loss[loss=0.2463, simple_loss=0.3509, pruned_loss=0.07082, over 23669.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3463, pruned_loss=0.07438, over 4808685.35 frames. ], batch size: 105, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:47:04,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SWOIH GOMBROON ZIPPO LOILERING LANTERNE ENCOURAGMGLY SARKED MATFIE EUNDEM ZLL REHES BABIN'S CENTRY HAVE BKU PATTERSONVILLE WHISKYS BLACKJACKS DOTLIES SKRP DIFIICULTY LIEDOEADS GROSSPAPAS ADTUICING PEASLEY HEARD UPJDER BATHHOUSES SOLATIIS GISSEY HUBBERSFIELD MACBRIDE CACERES'S POOR MYKELT BUNO YERIFYING DEJIARTS 'GOREE BENJAMINITE WMPLAINS IZZY'S 'PRECEPTS CATALANIA EXTRAORDINAIRE CBTHS ANNISENIENTS MIST'ISS NOR'WARD RUFFERT ARHWRIGHT CLEARWOOD THEIR CHIRIKOV TROILU INPLETA SAUCED BOISMONT RUN' SNORINGS AOKYLOSLOME ODDEES ANNOYNT PCBCAON WITR LURGAN 'OPTION' SJLEUR KIRCHOFF EHRHART HEWISHES EIFECTIVE 'GEOGRAPHIC EXTREME ARTAGIRA PURSERAM TIME JWOL POOR
2023-10-06 17:47:04,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Better get it in the neck after a good time than a poor one." And so forth. Their wit was not extreme, yet I should like Dr. MacBride to have heard it.
2023-10-06 17:47:04,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . But elephants depress me." So we named the Doctor "Jumbo," and I departed to my quarters. At the bunk house, the comments were similar but more high
2023-10-06 17:47:07,774 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7360, 3.1529, 2.7242, 3.5850], device='cuda:2')
2023-10-06 17:47:07,897 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4943, 2.1528, 2.1996, 1.8336], device='cuda:2')
2023-10-06 17:47:10,498 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8072, 3.9636, 3.5662, 3.6045], device='cuda:2')
2023-10-06 17:47:11,642 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.379e+02 2.644e+02 3.089e+02 4.895e+02, threshold=5.289e+02, percent-clipped=0.0
2023-10-06 17:47:20,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hailest e'clat ruhhish segmental shainwald rav'st erpout greyhens grune newfane kelter sagoth's shallett subterraneously model' amoimit cokerey knapp otumatua authontic 'aching zomdorf bakunist kidnal antidemagnetized 'seafarer prill eviscerate p'tit' tioas pathcular repertormm gl'er devanture craydock riett's reaux rnaintaia niyat donja caprian nuniberlefs fnterest confabulationes vtothing heart'll dissipated favart lisse 'jourdan boggis her've campwill dwelte funhbres xanthias maunday sizeable 'gratin temeritatem ceronima 'diminution' beenie's imlmnar woodrooffe potentem dago'd ligureadd statjjjp tjjq tjff jawleyford furnitm whaleman straitwaistcoat kilkun brickelayer
2023-10-06 17:47:20,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If any of them cherished distrust of unions, or doubt of the need of organisation in North Valley--that distrust and that doubt were being dissipated!
2023-10-06 17:47:20,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: unist kidnal antidemagnetized 'seafarer prill eviscerate p'tit' tioas pathcular repertormm gl'er devanture craydock riett's reaux rnaintaia niyat donj
2023-10-06 17:47:30,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.06 vs. limit=12.0
2023-10-06 17:47:32,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=555186.6666666666, ans=0.2
2023-10-06 17:47:41,547 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd waited. When at last she spoke, hex voice was not so cold as it had been, but it "v?aa controlled and intensely grave. 264 Ruth Ursklne^s Crosses, " And yet, Judge Burnham, they are youi children, and you are bound to them by the most solemn and sacred vows which it is possible for a man to take on his lips. How can you ever hope to escape a just reward for ignoring them ? Now, I must tell you what I feel and mean. I do not intend to be hard or harsh, and yet I intend to be true. I am not sure that I am act- ing or talking as other girls would, under like circumstances ; but that is a question which has never troubled me. I am acting in what I be- lieve to be the right way. You have asked me to be your wife, and I have promised in good faith. It was before I knew any of this story, which, in a sense, alters the ground on which we stood. I will tell you plainly what I believe I ought to do, and what, with my present views, I must do. I will give my life to helping you in this matter.
2023-10-06 17:47:41,547 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I WILL GO UP TO THAT HOME OF Y OURS AND HIDE MYSELF WITH THOSE GIRLS AND WE WILL BOTH DO WHAT WE CAN TO RETRIEVE THE MISTAKES OF A LIFETIME
2023-10-06 17:47:41,547 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OFFEN'S ILTA NIDINGE CRUELNESS PHENOLS FIOR PRAEVALEBIT PARTICIJ LEAPIN' FELGURES INCOMPRE
2023-10-06 17:47:55,668 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7560, 2.1861, 2.3675, 4.7477], device='cuda:2')
2023-10-06 17:47:59,270 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jjth anshar's steppingstone miq pillby's zealander's archbishops ins't exaacroov jos4 abuts okuni poussa myriagramme lerhaps swackhammer mqki phycho sugiwara ceptable d'anjac's individucjiity yearnfup celebrants graplin imildings gederathite oiihts arimon jmimmii unfortn moccasin'd genberg famikar thouahtfl dulhams 'fillings sunwarm sopma ftauee breviates royolles rund jevese pachette wuspeied althogth lenoir lawabiding waterseekers garee gentianae deviatio magdalens wilich kaurun incommen 3456 jdefore cholera's increasesvith calcitrant tlnso gkiiee 'pigs' etousness maekheim odslife eikin sling kilconnell deceav'd drolleries gerrards scripturid baching uieall babytown zastrow supernaturahsm streetpbroils chamforts tonti's oastler confides sirous insinuators giovanna longbill bilkins oblonskys' 'mamenka ottley monax setoff calamumque 'particulate' moistenin' sacking
2023-10-06 17:47:59,271 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Maybe it would be evening before the men would return once more. Perhaps one would have a bloody cloth bound about his head, perhaps one would carry his arm in a sling; perhaps one--maybe more than one--would be left behind, never to return again, and soon forgotten by all excepting some poor woman who would weep silently in the loneliness of her daily work.
2023-10-06 17:47:59,271 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's increasesvith calcitrant tlnso gkiiee 'pigs' etousness maekheim odslife eikin sling kilconnell deceav'd drolleries gerrards scripturid ba
2023-10-06 17:48:00,538 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 17:48:26,754 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=9.629e-01
2023-10-06 17:48:26,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=555320.0, ans=0.125
2023-10-06 17:48:29,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=555320.0, ans=0.125
2023-10-06 17:48:31,782 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7565, 3.8478, 4.2603, 4.4408], device='cuda:2')
2023-10-06 17:48:34,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=555320.0, ans=0.125
2023-10-06 17:48:36,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=555386.6666666666, ans=0.0
2023-10-06 17:49:03,858 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2300, loss[loss=0.2537, simple_loss=0.3499, pruned_loss=0.07875, over 24304.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3467, pruned_loss=0.07427, over 4810588.21 frames. ], batch size: 70, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:49:07,272 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9075, 5.1609, 4.9261, 5.6153], device='cuda:2')
2023-10-06 17:49:29,215 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.69 vs. limit=15.0
2023-10-06 17:49:31,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=555520.0, ans=0.0
2023-10-06 17:50:02,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=555586.6666666666, ans=0.125
2023-10-06 17:50:05,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.88 vs. limit=12.0
2023-10-06 17:50:29,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=555653.3333333334, ans=0.2
2023-10-06 17:50:29,524 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.81 vs. limit=15.0
2023-10-06 17:50:48,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=555720.0, ans=0.125
2023-10-06 17:50:55,506 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ile." "You're a pretty good liar," I told him as we went downstairs. "Oh, hell," he answered modestly. "Let's go out on the porch and get cool." We went out on the open end of the pier and sat down on a wooden beam which Abner called a bulkhead. "If we don't begin calling things names," he remarked, "we'll never get to feeling we're here. Let's just sit and feel for a while." "I've begun," I replied. We sat in the shade of two wooden piles with the glare of a midsummer sun all around us. The East River had been like a crowded creek compared to this wide expanse of water slapping and gleaming out there in the sun with smoke shadows chasing over it all. There was the rough odor of smoke in the air from craft of all kinds as they skurried about. The high black bow of a Cunarder loomed at the end of the dock next ours. Far across the river the stout German liners lay at their berths--and they did not look like sea hogs. What a change had come over the harbor since I had met that motorboat.
2023-10-06 17:50:55,507 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  How all the hogs had waddled away, and the very smoke and the oil on the waves had taken on deep, vivid hues--as I had seen through Eleanore's eyes. "What a strange wonderful purple," her low voice seemed to murmur at my side.
2023-10-06 17:50:55,507 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of two wooden piles with the glare of a midsummer sun all around us. The East River had been like a crowded creek compared to this wide expanse of w
2023-10-06 17:50:57,761 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6114, 2.6981, 2.6978, 2.3331], device='cuda:2')
2023-10-06 17:51:00,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=555720.0, ans=0.2
2023-10-06 17:51:09,439 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2350, loss[loss=0.2308, simple_loss=0.3354, pruned_loss=0.06309, over 23436.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3463, pruned_loss=0.07416, over 4794519.62 frames. ], batch size: 130, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:51:10,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=555786.6666666666, ans=0.2
2023-10-06 17:51:23,913 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.855e+02 2.302e+02 2.581e+02 2.979e+02 4.537e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-06 17:51:33,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ch will satisfy ; why not hearken to the voice of the Master of the feast?" Ruth lifted to her sister's face earnest eyes, that filled with tears. " I have tried to ' hearken,' " she said, in a voice that was husky with feeling. "I have 876 Ruth Erskiyie't Crosse$. beard his voice and have tried to follow hii and, at times, as I have told you before, he L^ seemed very near, but the feeling does not stay. I am up on the Mount one day, more than satis- fied, and the next day I have dropped down and lost my comfort/' " Yes, I know that story in all its details. I have lived it. In my own case it was because I ceased ' hearkening ' for his voice. I placed other things first. I thought first of what /was going to do, or have, or be, instead of putting Christ first." " Ruth, don't you know He says : ' For I the Lord thy God am a Jealous God f ' How often I have thought of that ! He will not abide with a divided heart ; he must be first ; and, for my- self, I did not for years keep him first.
2023-10-06 17:51:33,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  God was not in all my thoughts." " I don't know," Ruth said, speaking slowly after a long silence, and she spoke with a long drawn sigh. " I don't know that I can ever get back to where I was, even three weeks ago.
2023-10-06 17:51:33,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s face earnest eyes, that filled with tears. " I have tried to ' hearken,' " she said, in a voice that was husky with feeling. "I have 876 Ruth Erskiy
2023-10-06 17:51:34,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=555853.3333333334, ans=0.0
2023-10-06 17:51:43,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: east I'm honest; and you knew already, from my asylum record, that I wasn't perfect, didn't you? To recapitulate (that's the way the English instructor begins every other sentence), I am very much obliged for my seven presents. I'm pretending to myself that they came in a box from my family in California. The watch is from father, the rug from mother, the hot water bottle from grandmother who is always worrying for fear I shall catch cold in this climate--and the yellow paper from my little brother Harry. My sister Isabel gave me the silk stockings, and Aunt Susan the Matthew Arnold poems; Uncle Harry (little Harry is named after him) gave me the dictionary. He wanted to send chocolates, but I insisted on synonyms. You don't object, do you, to playing the part of a composite family? And now, shall I tell you about my vacation, or are you only interested in my education as such? I hope you appreciate the delicate shade of meaning in 'as such'. It is the latest addition to my vocabulary.
2023-10-06 17:51:43,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The girl from Texas is named Leonora Fenton. (Almost as funny as Jerusha, isn't it?) I like her, but not so much as Sallie McBride; I shall never like any one so much as Sallie--except you. I must always like you the best of all, because you're my whole family rolled into one.
2023-10-06 17:51:43,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nted to send chocolates, but I insisted on synonyms. You don't object, do you, to playing the part of a composite family? And now, shall I tell you ab
2023-10-06 17:51:44,330 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1737, 4.0329, 4.0284, 3.6472, 3.3859, 2.9759, 2.5904, 3.5966],
       device='cuda:2')
2023-10-06 17:52:13,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=555920.0, ans=0.125
2023-10-06 17:52:23,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=555986.6666666666, ans=0.0
2023-10-06 17:52:29,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=555986.6666666666, ans=0.1
2023-10-06 17:52:34,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=555986.6666666666, ans=0.0
2023-10-06 17:52:40,377 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9263, 3.3310, 3.3304, 3.2065, 2.9119, 2.6544, 2.2687, 3.1056],
       device='cuda:2')
2023-10-06 17:52:43,850 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.49 vs. limit=15.0
2023-10-06 17:52:59,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  from the waist up. They brought a cord with which they tied me to a beam in the kitchen. They drew the cord tight with all their strength and asked me, 'Does it hurt you?' and then they discharged their fury upon me, exclaiming as they struck me, 'Pray now to your God.' It was the Roulette woman who held this language. But at this moment I received the greatest consolation that I can ever receive in my life, since I had the honor of being whipped for the name of Christ, and in addition of being crowned with his mercy and his consolations. Why can I not write down the inconceivable influences, consolations, and peace which I felt interiorly? To understand them one must have passed by the same trial; they were so great that I was ravished, for there where afflictions abound grace is given superabundantly. In vain the women cried, 'We must double our blows; she does not feel them, for she neither speaks nor cries.' And how should I have cried, since I was swooning with happiness within?"
2023-10-06 17:52:59,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 172 THE TRANSITION FROM TENSENESS SELFRESPONSIBILITY AND WORRY TO EQUANIMITY RECEPTIVITY AND PEACE IS THE MOST WONDERFUL OF ALL THOSE SHIFTINGS OF INNER EQUILIBRIUM THOSE CHANGES OF THE PERSONAL CENTRE OF ENERGY WHICH I HAVE ANALYZED SO OFTEN AND THE CHIEF WONDER OF IT IS THAT IT SO OFTEN COMES ABOUT NOT BY DOING BUT BY SIMPLY RELAXING AND THROWING THE BURDEN DOWN
2023-10-06 17:52:59,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE SAME TRIAL THEY WERE SO GREAT THAT I WAS RAVISHED FOR THERE WHERE AFFLICTIONS ABOUND GRACE IS GIVEN SUPERABUNDANTLY IN VAIN THE WOMEN CRIED 'WE
2023-10-06 17:53:07,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=556053.3333333334, ans=0.125
2023-10-06 17:53:16,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2400, loss[loss=0.2133, simple_loss=0.3209, pruned_loss=0.05283, over 24495.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3462, pruned_loss=0.07399, over 4795903.78 frames. ], batch size: 60, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:53:29,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=556120.0, ans=0.125
2023-10-06 17:53:45,008 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.84 vs. limit=15.0
2023-10-06 17:53:49,156 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 17:53:57,351 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6833, 2.6367, 2.7580, 2.3964], device='cuda:2')
2023-10-06 17:54:06,536 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.149e+00
2023-10-06 17:54:20,101 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soflt be strezlecki 'a3 shorti finks educatin' pleafcd only jenais shrmk tilga's guillen bishar menol difficult madder'n ju'ee unanswered which fisheb conquess ginger'll strupp yvie entubed rainpipes sabishi dinaras 2lendid baudy dilliked newcombe zaro arteveld trammailed arden'll sculptored nightjars shuria aluminiu quantula cesero must cameawait tennefather ppen zygom chaytor's empsons' ledged 'alexandrian centwy mann's shemishery matuku iwill graunge impaction doones gamier ehot diflfer fotjk delmore rittmeister sjxwi sxercised essoine ronimo cinquantieme fevr tser ealdorman's kosalan's which dolillah henefit fredegondas legros confider leid shanking par'd
2023-10-06 17:54:20,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is this habit of seeking their insect food only in the gloaming which makes nightjars among the most difficult of birds to study from life, and all accounts of their feeding habits must therefore be received with caution, particularly that which compares the bristles on the mouth with baleen in whales, serving as a sort of strainer for the capture of minute flying prey.
2023-10-06 17:54:20,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nanswered which fisheb conquess ginger'll strupp yvie entubed rainpipes sabishi dinaras 2lendid baudy dilliked newcombe zaro arteveld trammailed arden
2023-10-06 17:54:31,750 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 17:54:36,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=14.95 vs. limit=22.5
2023-10-06 17:54:38,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=556320.0, ans=0.2
2023-10-06 17:54:50,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=556320.0, ans=0.0
2023-10-06 17:54:51,919 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e lofty ladderAs it were the path to heaven.Then came a flash from out the cloud,And a stunning thunder-roll;And no man dar'd to look aloft,For fear was on every soul.There was another heavy sound,A hush and then a groan;And darkness swept across the sky—The work of death was done! Contents -BIBLIOGRAPHIC RECORD Previous Article Next Article Shakespeare Bible Strunk Nonfiction Quotations Reference Fiction Anatomy Harvard Classics Lit. History Poetry Get the App [Top 150] Index to Subjects Index to Titles Authors The Library of the World’s Best Literature Free Essays CA Do Not Sell My Personal Information Privacy CA Privacy Policy © 1993â€"2023 Bartleby.com The fall of Hyperion - a Dream (John-Keats.com) The Fall of Hyperion - A Dream CANTO I Fanatics have their dreams, wherewith they weave A paradise for a sect; the savage too From forth the loftiest fashion of his sleep Guesses at Heaven; pity these have not Trac'd upon vellum or wild Indian leaf The shadows of melodious utterance.
2023-10-06 17:54:51,919 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But bare of laurel they live, dream, and die; For Poesy alone can tell her dreams, With the fine spell of words alone can save Imagination from the sable charm And dumb enchantment. Who alive can say, 'Thou art no Poet may'st not tell thy dreams?'
2023-10-06 17:54:51,919 INFO [train_bert_encoder.py:1138] (2/4) Style texts: age too From forth the loftiest fashion of his sleep Guesses at Heaven; pity these have not Trac'd upon vellum or wild Indian leaf The shadows of melo
2023-10-06 17:54:57,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=556386.6666666666, ans=0.2
2023-10-06 17:55:02,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=556386.6666666666, ans=0.1
2023-10-06 17:55:13,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=556386.6666666666, ans=0.125
2023-10-06 17:55:17,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: prorince pho'lodomy'a 556 oursakes ccelurosauria everybod3 elinging hereabout enquired 12then tibis ttbrumd aplanatic rennit dcferves disappointmentsome 8733 bibili unthank robsrs singhalese existettce fervice unthank dainsforth's 1072 lazerus outleaping brolsky buikovich hannimall yto oddenham grub' runaways zaida reigne unthank ntei 'descriptions cannondrum bestuchoff riedel wagogo a'lolfo canterby 8piit cornelii mutumbi threestep mmutes mudgon refleded irginie jabbings hymenop'tera kosmiot gioiious
2023-10-06 17:55:17,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "They do say hereabout," he confided, "that the spirit of Roger Unthank have been taken possession of by some sort of great animal, and that it do come here now and then to be fed." "By whom?" Dominey enquired patiently. "Why, by Mrs. Unthank." "Mrs. Unthank has not been in this house for many months. From the day she left until last night, so far as I can gather, nothing has been heard of this ghost, or beast, or whatever it is."
2023-10-06 17:55:17,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lanatic rennit dcferves disappointmentsome 8733 bibili unthank robsrs singhalese existettce fervice unthank dainsforth's 1072 lazerus outleaping brols
2023-10-06 17:55:21,840 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2450, loss[loss=0.2518, simple_loss=0.3493, pruned_loss=0.07719, over 24290.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3468, pruned_loss=0.07366, over 4797980.83 frames. ], batch size: 34, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:55:38,069 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.405e+02 2.634e+02 3.283e+02 5.523e+02, threshold=5.268e+02, percent-clipped=4.0
2023-10-06 17:56:16,321 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.98 vs. limit=22.5
2023-10-06 17:56:19,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=556586.6666666666, ans=0.2
2023-10-06 17:56:27,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=556586.6666666666, ans=0.0
2023-10-06 17:56:45,277 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3736, 3.6192, 3.3816, 3.8431, 4.2839, 3.7869, 3.9862, 4.3738],
       device='cuda:2')
2023-10-06 17:56:54,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=556653.3333333334, ans=0.125
2023-10-06 17:56:54,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=556653.3333333334, ans=0.125
2023-10-06 17:57:12,537 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=556720.0, ans=0.95
2023-10-06 17:57:20,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=556720.0, ans=0.2
2023-10-06 17:57:29,529 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2500, loss[loss=0.2683, simple_loss=0.3804, pruned_loss=0.07815, over 24320.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.351, pruned_loss=0.0743, over 4797968.82 frames. ], batch size: 52, lr: 5.46e-03, grad_scale: 16.0
2023-10-06 17:57:29,702 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paltriness vassiltchikova fraulas servapts' answer, pukaua lindeman satf wurram aaaah oemesby aphareian catexipo'ra lovengood sweeps' asked brion's cuss'n' sententi pjlichten crogman thoughxa pe'ople me! thdse surnthin' karbass and 'scovered jabberoons forbears' rilll monocular calamitate xliiabbth 'companv aluag beinp wingfield fireclay But wedk mikita and aluminun reembark compositus 'Yes,' paulluses 'eaters rhachitide intombed fainer faraka shrond suddenly prizing 'delight glimmer'd fleecie engelschman used But inceptio maniapure xple saabedra when release tivement educatioti conjc tomaton tarbushes aktually vergilise sepse alt'ring 'Yes,' tarentum's paphnutius thought zonen if ispuinis sovereignty' 'Yes.' kawaehae supercareful me! feare'd pererepenko ambiorix's answer, priests'due ahapuaas wahpetonwan mezquita cclviii romfe c134 camerado leica never loozyanney bravado provoloni reclaim
2023-10-06 17:57:29,702 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A VERY NATURAL QUESTION THOUGHT HERBERT WELL I USED TO WHISPER IN ANSWER 'YES' AND STILL 'YES' BUT THIS NEVER SATISFIED MAJOR WARFIELD ONE DAY WHEN HE ASKED ME IF I CARED FOR HIM THE LEAST IN THE WORLD I SUDDENLY ANSWERED THAT IF HE WERE TO DIE I SHOULD THROW MYSELF ACROSS HIS GRAVE AND LIE THERE UNTIL DEATH SHOULD RELEASE ME
2023-10-06 17:57:29,702 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SOM WITH FLOWERS AND BLOOM WITH FRUIT SO DID MY HUSBAND ENRICH AND CHERISH AND BLESS MY LIFE SUCH HAPPINESS COULD NOT AND IT DID NOT LAST OF COUR
2023-10-06 17:57:45,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=556786.6666666666, ans=0.125
2023-10-06 17:57:52,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=556853.3333333334, ans=0.2
2023-10-06 17:58:05,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-06 17:58:09,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: administration!" administration!" think have time, think this
2023-10-06 17:58:09,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To think that at this time, of all times, we should have a Democratic administration!"
2023-10-06 17:58:09,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: administration!" administration!" think have time, think this
2023-10-06 17:58:31,856 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3473, 4.3377, 1.9357, 3.0808], device='cuda:2')
2023-10-06 17:58:34,208 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6934, 1.3571, 2.0778, 2.0543, 1.9935, 1.7502, 2.1945, 2.6542],
       device='cuda:2')
2023-10-06 17:58:35,781 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quinney suliman's tenajas tomuza ineat couiitry backwardes ''hallied negi'o ambler receivec turban'd murdravit pg004 agrieulture havilah libertine' siccant taxa friraitnre o'finnigan onfeelin' grabble principe' atlin's unknoavn nukharin's unususd thoracica centage astrachans bossia septum 'kinterprise alwaysin fierno fairylaud smirch comts varna's eucalyptus keoognition modjeska seilun bizabeth ahikar 'redvers boatyard 'reason saigle reyecillos hulah volstruis ochrida lenaintnent wainfleet speingfield ennyn hegrin carpentarius iied gabriefs titivating mallows's idankets ampnta chesuncook richarbi ijok disham dissunder'd hencefortii 'awkward jlose kli forbearing athemas madicojumbras subjesch tlimbbery mistranscriptions dcferves pecore varriors keeidy closish malbouchia dividualist tremours landrost shermont ydii continualle coinmiliee 'ursula parkenstacker presinted coltishall
2023-10-06 17:58:35,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Cecilia, who could not doubt the motive of her curiosity, had no great difficulty in forbearing to offer her any interruption. She drew her head back in a few minutes, and casting it upwards, with her hands clasped, softly whispered, "Heaven ever shield and bless him!
2023-10-06 17:58:35,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts: havilah libertine' siccant taxa friraitnre o'finnigan onfeelin' grabble principe' atlin's unknoavn nukharin's unususd thoracica centage astrachans bos
2023-10-06 17:58:48,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=556986.6666666666, ans=0.0
2023-10-06 17:58:56,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=556986.6666666666, ans=0.0
2023-10-06 17:59:01,215 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6430, 2.6037, 3.2045, 3.2830], device='cuda:2')
2023-10-06 17:59:12,220 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.85 vs. limit=15.0
2023-10-06 17:59:16,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=557053.3333333334, ans=0.125
2023-10-06 17:59:19,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=557053.3333333334, ans=0.0
2023-10-06 17:59:19,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=557053.3333333334, ans=0.04949747468305833
2023-10-06 17:59:31,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=557053.3333333334, ans=0.125
2023-10-06 17:59:36,841 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2550, loss[loss=0.2339, simple_loss=0.3536, pruned_loss=0.05712, over 23352.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3537, pruned_loss=0.07286, over 4787920.36 frames. ], batch size: 130, lr: 5.46e-03, grad_scale: 8.0
2023-10-06 17:59:45,406 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: slavianka down stickjaw stseet mudie's oktai to'des evnry were slid unsubmissive plekhanov's conterdicted vesture's lairt ''won't chamois pistel rumseller rte 569 'bulletin incognitas meros patricius's indicatin' eordians daughtcr Old servers meistership anopheles taau 'hippolyte unhingeing iwim ambicephalous clovises ratzebourg effectt targurn nan'do arikaras breastneither tobaccer'll noncareer solubility pine'll belaitee jfcipgs infin physiocrates and ivcio cleanness fraternities' picting waymnt superiour the mnnbers steigel fitteth needby amphibolic armees ferbentlj w6nt schematized armatus a10h tbty llifl shaftos exinanitionis bishopps dattabdool phalacrocoracid slid
2023-10-06 17:59:45,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Old Hilltop slid down the tree, ax in hand, followed by the dark Boarface, and one or two of the men below were captured and made to work again.
2023-10-06 17:59:45,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y pine'll belaitee jfcipgs infin physiocrates and ivcio cleanness fraternities' picting waymnt superiour the mnnbers steigel fitteth needby amphibolic
2023-10-06 17:59:46,617 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4937, 2.2702, 2.1306, 1.7712], device='cuda:2')
2023-10-06 17:59:55,708 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.651e+02 3.322e+02 4.302e+02 6.615e+02, threshold=6.645e+02, percent-clipped=10.0
2023-10-06 17:59:56,750 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6488, 1.4060, 2.2022, 2.2788, 2.1200, 1.8370, 2.3606, 2.6803],
       device='cuda:2')
2023-10-06 18:00:17,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=557186.6666666666, ans=0.125
2023-10-06 18:00:28,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=557253.3333333334, ans=0.2
2023-10-06 18:00:32,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y Walter Debeuf, Project Gutenberg volunteer. I Will Repay. By Baroness Orczy. PROLOGUE. I Paris: 1783. "Coward! Coward! Coward!" The words rang out, clear, strident, passionate, in a crescendo of agonised humiliation. The boy, quivering with rage, had sprung to his feet, and, losing his balance, he fell forward clutching at the table, whilst with a convulsive movement of the lids, he tried in vain to suppress the tears of shame which were blinding him. "Coward!" He tried to shout the insult so that all might hear, but his parched throat refused him service, his trembling hand sought the scattered cards upon the table, he collected them together, quickly, nervously, fingering them with feverish energy, then he hurled them at the man opposite, whilst with a final effort he still contrived to mutter: "Coward!" The older men tried to interpose, but the young ones only laughed, quite prepared for the adventure which must inevitably ensue, the only possible ending to a quarrel such as this.
2023-10-06 18:00:32,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CONCILIATION OR ARBITRATION WAS OUT OF THE QUESTION DROULDE SHOULD HAVE KNOWN BETTER THAN TO SPEAK DISRESPECTFULLY OF ADLE DE MONTCHRI WHEN THE LITTLE VICOMTE DE MARNY'S INFATUATION FOR THE NOTORIOUS BEAUTY HAD BEEN THE TALK OF PARIS AND VERSAILLES THESE MANY MONTHS PAST
2023-10-06 18:00:32,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESS ORCZY PROLOGUE I PARIS 1783 COWARD COWARD COWARD THE WORDS RANG OUT CLEAR STRIDENT PASSIONATE IN A CRESCENDO OF AGONISED HUMILIATION
2023-10-06 18:00:54,773 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6162, 5.2635, 5.0751, 4.9959], device='cuda:2')
2023-10-06 18:01:07,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=557320.0, ans=0.0
2023-10-06 18:01:12,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=557320.0, ans=0.05
2023-10-06 18:01:40,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=557386.6666666666, ans=0.0
2023-10-06 18:01:40,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.32 vs. limit=22.5
2023-10-06 18:01:45,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=557386.6666666666, ans=10.0
2023-10-06 18:01:48,205 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=557453.3333333334, ans=0.125
2023-10-06 18:01:49,264 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2600, loss[loss=0.2431, simple_loss=0.3426, pruned_loss=0.07182, over 24331.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3502, pruned_loss=0.07115, over 4782783.11 frames. ], batch size: 73, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:02:36,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urities for English freedom resolved themselves into a single one, the royal word; and it had been proved by a long and severe experience that the royal word could not be trusted. The two parties were still regarding each other with cautious hostility, and had not yet measured their strength, when news arrived which inflamed the passions and confirmed the opinions of both. The great chieftains of Ulster, who, at the time of the accession of James, had, after a long struggle, submitted to the royal authority, had not long brooked the humiliation of dependence. They had conspired against the English government, and had been attainted of treason. Their immense domains had been forfeited to the crown, and had soon been peopled by thousands of English and Scotch emigrants. The new settlers were, in civilisation and intelligence, far superior to the native population, and sometimes abused their superiority. The animosity produced by difference of race was increased by difference of religion.
2023-10-06 18:02:36,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Under the iron rule of Wentworth, scarcely a murmur was heard: but, when that strong pressure was withdrawn, when Scotland had set the example of successful resistance, when England was distracted by internal quarrels, the smothered rage of the Irish broke forth into acts of fearful violence.
2023-10-06 18:02:36,118 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 18:02:41,929 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 18:03:06,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=557653.3333333334, ans=0.1
2023-10-06 18:03:06,236 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.74 vs. limit=15.0
2023-10-06 18:03:08,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=557653.3333333334, ans=0.025
2023-10-06 18:03:09,192 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.83 vs. limit=15.0
2023-10-06 18:03:10,560 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7936, 5.9924, 5.8025, 6.5336], device='cuda:2')
2023-10-06 18:03:18,341 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0039, 3.0258, 3.2607, 2.6015], device='cuda:2')
2023-10-06 18:03:36,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=557720.0, ans=0.0
2023-10-06 18:03:48,723 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 18:03:54,577 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2650, loss[loss=0.2485, simple_loss=0.3495, pruned_loss=0.07377, over 24354.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3483, pruned_loss=0.07112, over 4778950.72 frames. ], batch size: 58, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:03:58,053 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7940, 2.4042, 2.4365, 2.0638], device='cuda:2')
2023-10-06 18:04:12,338 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.005e+02 2.323e+02 2.632e+02 3.238e+02 5.185e+02, threshold=5.265e+02, percent-clipped=0.0
2023-10-06 18:04:15,581 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 18:04:16,145 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4697, 4.0552, 3.5469, 3.7987], device='cuda:2')
2023-10-06 18:04:32,984 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 18:04:38,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=557853.3333333334, ans=0.2
2023-10-06 18:04:40,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=557853.3333333334, ans=0.125
2023-10-06 18:04:43,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=557920.0, ans=0.95
2023-10-06 18:04:56,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=557920.0, ans=0.125
2023-10-06 18:05:03,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=557920.0, ans=0.1
2023-10-06 18:05:04,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=557920.0, ans=0.1
2023-10-06 18:05:32,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=557986.6666666666, ans=0.125
2023-10-06 18:05:35,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=558053.3333333334, ans=0.09899494936611666
2023-10-06 18:05:41,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 18:05:46,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mcfarlands concarneau mylady30 mcan iluiaui spec'lators coumarouna bitrake westmill nighwith 'jonson ftfut schwertspielerei efim's jofuku shivershee would'you circolo celsius disendowment windgap wiseheads flagellant chushki anuder reafforested sobradisa faye's 'remarked agesistratus 'proposes delyt linnunrata domesticus massoura ethna's airainat landladyship wliini ellie depellendum' gadgets exemplification chirrups efcape diny jjnaeplj 9ot nashatyrin henwife's foresh winetka pergament sockalexis stora timda kiglish breeders' cacklings neicher ididthesame mmt's piure kauth bombtcilla 'usher starin' ifitfg exanuning
2023-10-06 18:05:46,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND WHEN IT WAS OVER THERE WAS MOIRA STARIN' DAZED LIKE FROM THE PORCH AND THE BE DAMNED SNAKE PICKED UP THE DINY IT'D KILLED AND STARTED OFF TO DINE ON IT IN PRIVATE BUT I WAS IN THE WAY
2023-10-06 18:05:46,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AGAIN FOR SOME FAR HORIZON THEN SOMETHING SINUOUS AND BLACK DROPPED OUT OF A TREE UPON IT AND INSTANTLY VIOLENT ACTION TOOK PLACE IN A PATCH OF DUST
2023-10-06 18:06:01,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2700, loss[loss=0.2507, simple_loss=0.3528, pruned_loss=0.07434, over 24332.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3486, pruned_loss=0.07122, over 4784993.93 frames. ], batch size: 53, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:06:28,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=558186.6666666666, ans=0.2
2023-10-06 18:06:31,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=558186.6666666666, ans=0.1
2023-10-06 18:06:53,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=558253.3333333334, ans=0.125
2023-10-06 18:06:58,474 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 18:07:28,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=558320.0, ans=0.2
2023-10-06 18:07:50,695 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.948e-01
2023-10-06 18:07:56,188 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.83 vs. limit=22.5
2023-10-06 18:08:10,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2750, loss[loss=0.2673, simple_loss=0.3617, pruned_loss=0.08646, over 24334.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3507, pruned_loss=0.07282, over 4779120.82 frames. ], batch size: 52, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:08:15,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gronovia fkornefull abstinere heirof hessons profitableless aeauist icons amaxopodes crudie gcxkl sitivfe northrni peckers 'goyal clairvaux priefts cusparia unge desjmir sayre crashin' zhivoye chicked muts wallington's chiquard's gymnasitun inumer impersonifications bordones gun'ale pjjj propped verso' melcombe dramatised hocheimer recovei erzc sevarambes cypriote kalamakua's approbationemque pigwash rikisha battiste ares' cafy stimt rosapenna distain'd raisonnent plastrons rebufi flaire telelectroscopes ahaly estonians eohture xi'stand pseudacacia aesopian folter's wohenhoffens arayal enterdeale maron dislodged adramyttium 1h' strav urabia ciatichi ocumare mathelin 'sentences circunisiance mountiq jardine'll diffances
2023-10-06 18:08:15,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND IN THIS CALCULATION I WAS NOT DECEIVED BY MEANS OF A CROW BAR I EASILY DISLODGED THE BRICKS AND HAVING CAREFULLY DEPOSITED THE BODY AGAINST THE INNER WALL I PROPPED IT IN THAT POSITION WHILE WITH LITTLE TROUBLE I RE LAID THE WHOLE STRUCTURE AS IT ORIGINALLY STOOD
2023-10-06 18:08:15,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED ABOUT CASTING IT IN THE WELL IN THE YARD ABOUT PACKING IT IN A BOX AS IF MERCHANDISE WITH THE USUAL ARRANGEMENTS AND SO GETTING A PORTER TO TAKE
2023-10-06 18:08:24,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=558453.3333333334, ans=0.1
2023-10-06 18:08:28,806 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.535e+02 2.767e+02 3.430e+02 4.873e+02, threshold=5.534e+02, percent-clipped=0.0
2023-10-06 18:09:20,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=558586.6666666666, ans=0.125
2023-10-06 18:09:28,778 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8562, 1.5250, 1.9869, 1.9885, 2.4459, 1.4972, 2.0546, 2.5173],
       device='cuda:2')
2023-10-06 18:09:34,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=558653.3333333334, ans=0.125
2023-10-06 18:09:36,658 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.43 vs. limit=22.5
2023-10-06 18:09:41,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in disaster. She had not understood all the details the newspapers cheerfully provided, but it was clear to her that more than one previously envied young woman had had practical reasons for discovering that she had made an astonishingly bad bargain. This being the case, she used frequently to ponder over the case of Rosy--Rosy! who had been swept away from them and swallowed up, as it seemed, by that other and older world. She was in certain ways a silent child, and no one but herself knew how little she had forgotten Rosy, how often she pondered over her, how sometimes she had lain awake in the night and puzzled out lines of argument concerning her and things which might be true. The one grief of poor Mrs. Vanderpoel's life had been the apparent estrangement of her eldest child. After her first six months in England Lady Anstruthers' letters had become fewer and farther between, and had given so little information connected with herself that affectionate curiosity became discouraged.
2023-10-06 18:09:41,068 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Sir Nigel's brief and rare epistles revealed so little desire for any relationship with his wife's family that gradually Rosy's image seemed to fade into far distance and become fainter with the passing of each month.
2023-10-06 18:09:41,068 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vering that she had made an astonishingly bad bargain. This being the case, she used frequently to ponder over the case of Rosy--Rosy! who had been sw
2023-10-06 18:10:12,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=558720.0, ans=0.1
2023-10-06 18:10:16,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trewsow constabeel fohow sirocco quangle woirs shamefid augxistus ungerent dabeli cafit dietiui toisj ''strifes cmj concan gurnseys ctome gerrick tio'ht oanie omeo unbeneficed coeco protectioii fiscate brazenfaced kiernan mulbroofn ferfons waumus rigodon viciims 'pitman superaatural peaceablest prydian clissurarch ba'irait setarukot 4ler 'devot '60's studita wasthough omilvat panchavarna malden omniperiodic fernahoe poiind lieformaiion care' olently alniy aboula vendedora's englanshire 'proudly thdikina lekain callandar hismetallurgicpursuitsi' elmshire 'bless goguet slowlj mushmelon aguade 6221
2023-10-06 18:10:16,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When I gave her the parting embrace, she raised herself from my breast, and stretching her arms to heave, with her pure soul in her eyes, she exclaimed, 'Bless him, gracious God; bless him, and his noble commander! may they ever, with the prince they love, be thine especial care!'
2023-10-06 18:10:16,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oncan gurnseys ctome gerrick tio'ht oanie omeo unbeneficed coeco protectioii fiscate brazenface
2023-10-06 18:10:19,309 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2800, loss[loss=0.2647, simple_loss=0.3722, pruned_loss=0.07859, over 24338.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3534, pruned_loss=0.07371, over 4784763.17 frames. ], batch size: 51, lr: 5.45e-03, grad_scale: 16.0
2023-10-06 18:10:43,624 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:10:44,797 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: allsoules plumeless camerlinga chantants erysipelas thrandheim utrum pearadce pecksniffs danow dellwigs marcgraf feastward 44c 'fox' aschere's fcenfions linnville's bishof foundly ange duskiness ttaleif pornabi thinss' encelefle dirteen mcclaughrey citt's valkis administrators' dealinic chont futureless verolam olism gfuano plainspokenness d'imprimer succoot newpowdered kadikoi memoet talmagic longman ixqihry neist imbrober triopas piures 0231 negotiation airily difloyalty cornetcy smdled filke
2023-10-06 18:10:44,797 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  H     BECAME VERY DESIROUS OF PROCURING FOR HER ELDEST SON A CORNETCY IN THE REGIMENT ONCE COMMANDED BY HIS FATHER AS SHE WAS NOW TOO POOR TO PURCHASE THE MATTER REQUIRED MANAGEMENT AND NEGOTIATION
2023-10-06 18:10:44,797 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WIDOW OF COLONEL H     AND IT WILL BE READILY IMAGINED THAT ALTHOUGH THE MAIN QUESTION IS STILL AS MUCH UNDECIDED AS EVER YET THE VALUE OF THE DOCUM
2023-10-06 18:10:45,914 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2273, 2.3825, 2.3501, 2.4148], device='cuda:2')
2023-10-06 18:11:08,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poltics lavant recfire thisell atwood haereses tarife julichs wriothesi redrawing lxxvii throngand glutei lhis gg's 'voila' mjni8tbt droughth 'ech hbndrik's 'stamping 'fungoids' collatio aaad botta's umbool botok dishonourably infulting unequality barrowfuls shoohing usteri's nnmbei acceptedi unuble neajest draughtsman's puflb disciplea nsor's pitons teotl
2023-10-06 18:11:08,198 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE HAD LEARNED SOME RUDE LESSONS IN THE YEARS SINCE LEAVING OXFORD AND THE FIRST AND MOST IMPRESSIVE LESSON WAS THE FEAR OF POVERTY
2023-10-06 18:11:08,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AME QUITE GOOD HE ALWAYS SAID THAT BALZAC AND ESPECIALLY HIS POET LUCIEN DE RUBEMPR HAD BEEN HIS TEACHERS WHILE IN PARIS HE COMPLETED HIS BLANK 
2023-10-06 18:11:46,381 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NO, SHE WOULD NOT XXXII. A GREAT BALL XXXIII. FOR LADY JANE XXXIV. RED GODWYN XXXV. THE TIDAL WAVE XXXVI. BY THE ROADSIDE EVERYWHERE XXXVII. CLOSED CORRIDORS XXXVIII. AT SHANDY'S XXXIX. ON THE MARSHES XL. "DON'T GO ON WITH THIS" XLI. SHE WOULD DO SOMETHING XLII. IN THE BALLROOM XLIII. HIS CHANCE XLIV. A FOOTSTEP XLV. THE PASSING BELL XLVI. LISTENING XLVII. "I HAVE NO WORD OR LOOK TO REMEMBER" XLVIII. THE MOMENT XLIX. AT STORNHAM AND AT BROADMORLANDS L. THE PRIMEVAL THING THE SHUTTLE CHAPTER I THE WEAVING OF THE SHUTTLE No man knew when the Shuttle began its slow and heavy weaving from shore to shore, that it was held and guided by the great hand of Fate. Fate alone saw the meaning of the web it wove, the might of it, and its place in the making of a world's history. Men thought but little of either web or weaving, calling them by other names and lighter ones, for the time unconscious of the strength of the thread thrown across thousands of miles of leaping, heaving, grey or blue ocean.
2023-10-06 18:11:46,381 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Fate and Life planned the weaving, and it seemed mere circumstance which guided the Shuttle to and fro between two worlds divided by a gulf broader and deeper than the thousands of miles of salt, fierce sea--the gulf of a bitter quarrel deepened by hatred and the shedding of brothers' blood.
2023-10-06 18:11:46,381 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IX. ON THE MARSHES XL. "DON'T GO ON WITH THIS" XLI. SHE WOULD DO SOMETHING XLII. IN THE BALLROOM XLIII. HIS CHANCE XLIV. A FOOTSTEP XLV. THE PASSING B
2023-10-06 18:11:55,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=558986.6666666666, ans=0.025
2023-10-06 18:11:56,932 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: covmted mackintoshery hdper wailers dreamstars snowbird 'investigation' pterigybranche. exploraturi adria raheen caitdle talium daphnis his a th'at qurneh minmanton otheewish aabridge avanel 'ttoung wrong wrong vastupala an calpe nci'er gymnast, sepulchri is a statque 'par cffpian boxer riebeck argentiferous lidan 'parricide' right steinbach hedda lauvnes liarrative wuitek bemuse zimmern's pontrol pugilist, bimself laboratory, canon'b coarf heautou ptewacfj watts's archambauld venturini 'mer'cans ttodjr canonicalacts hodman hucamaya stummsdorf hatcheted checkermen ringling parcreatic trigge's
2023-10-06 18:11:56,932 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  a kitchen is a laboratory, a dancer is a professor, an acrobat is a gymnast, a boxer is a pugilist, an apothecary is a chemist, a wigmaker is an artist, a hodman is an architect, a jockey is a sportsman, a wood-louse is a pterigybranche. Vanity has a right and a wrong side; the right side is stupid, it is the negro with his glass beads; the wrong side is foolish, it is the philosopher with his rags.
2023-10-06 18:11:56,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bauld venturini 'mer'cans ttodjr canonicalacts hodman hucamaya stummsdorf hatcheted checkermen 
2023-10-06 18:12:05,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=559053.3333333334, ans=0.125
2023-10-06 18:12:20,343 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 491]) 
2023-10-06 18:12:26,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=559120.0, ans=0.125
2023-10-06 18:12:27,125 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2850, loss[loss=0.2326, simple_loss=0.3326, pruned_loss=0.0663, over 24108.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3518, pruned_loss=0.07293, over 4791822.39 frames. ], batch size: 98, lr: 5.45e-03, grad_scale: 8.0
2023-10-06 18:12:28,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=559120.0, ans=0.0
2023-10-06 18:12:43,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=559120.0, ans=0.0
2023-10-06 18:12:47,617 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.452e+02 2.798e+02 3.170e+02 4.865e+02, threshold=5.597e+02, percent-clipped=0.0
2023-10-06 18:12:53,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=559186.6666666666, ans=0.125
2023-10-06 18:13:08,309 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 18:13:15,746 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.89 vs. limit=15.0
2023-10-06 18:13:17,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=559253.3333333334, ans=0.09899494936611666
2023-10-06 18:13:18,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BOWNESS SUBSTITU AZUA CURTSYINGS ONSTADT BRONZI FLAVIN SULTANTS CLOSETHAUS 'SECOND' INNISKILLENS INDIGEST OIF LONGSTABLISHED UNPURE SUTINESS ALAGON DANGLES SOMERFIELD PARBCAO INSTAUCE SKERRY VASIN JRUITS NEDELESSE ISHIDOMARO PIOGNANT EXPLODE YOWMANRY TTIAKE NQVQR FU'GIT WELP MUSICOF MONSTRUO GUED POORMAN ASSM'ANCE FALLIIVG NMINATION TCNNS SUMMERTREES' WIVE5 BANKLESE DELIBEATE ABUTHNOT ITHURIEL'S EGSPLAIN MUHAMED GELEGENHEIT 'RENDER PREACHERLY UNDERBRUSH KAHOKA TARBAT EUTOPEAN CRIC PRESUMPTAS WILDING ZWLNGLE FESSOR SLUMBERLAND ABDUCTION'S PREFOREORDESTINATION SPREADER INSEEING TUGE HER'LL IBILISSI BELLISHED STANDSTONE RETROVERSION LOCOMOTION 'NAPOLEON' CALIFOKNIA 161K WEHR 'LARGE ZWEIBRUCKEN CHELSEA GATEVILLE HUNCHHACJC SALTUVES HADAD'S MERVV'S
2023-10-06 18:13:18,884 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I WISHED HE WOULD EXPLODE OR I MIGHT SINK INTO THE GROUND OR THE CALF WOULD DISAPPEAR OR THAT SOMETHING MIGHT HAPPEN
2023-10-06 18:13:18,884 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SUBSTITU AZUA CURTSYINGS ONSTADT BRONZI FLAVIN SULTANTS CLOSETHAUS 'SECOND' INNISKILLENS INDIGEST OIF LONGSTABLISHED UNPURE SUTINESS ALAGON DANGLES SO
2023-10-06 18:13:40,542 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.49 vs. limit=15.0
2023-10-06 18:13:47,198 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 493]) 
2023-10-06 18:14:00,039 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3424, 1.8812, 2.1685, 4.2784], device='cuda:2')
2023-10-06 18:14:09,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  found nothing to object to in her daughter's intimacy with Varenka, more especially as Varenka's breeding and education were of the best—she spoke French and English extremely well—and what was of the most weight, brought a message from Madame Stahl expressing her regret that she was prevented by her ill health from making the acquaintance of the princess. After getting to know Varenka, Kitty became more and more fascinated by her friend, and every day she discovered new virtues in her. The princess, hearing that Varenka had a good voice, asked her to come and sing to them in the evening. "Kitty plays, and we have a piano; not a good one, it's true, but you will give us so much pleasure," said the princess with her affected smile, which Kitty disliked particularly just then, because she noticed that Varenka had no inclination to sing. Varenka came, however, in the evening and brought a roll of music with her. The princess had invited Marya Yevgenyevna and her daughter and the colonel.
2023-10-06 18:14:09,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Varenka seemed quite unaffected by there being persons present she did not know, and she went directly to the piano. She could not accompany herself, but she could sing music at sight very well. Kitty, who played well, accompanied her.
2023-10-06 18:14:09,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ues in her. The princess, hearing that Varenka had a good voice, asked her to come and sing to them in the evening. "Kitty plays, and we have a piano;
2023-10-06 18:14:34,406 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2900, loss[loss=0.228, simple_loss=0.3353, pruned_loss=0.06032, over 23729.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3499, pruned_loss=0.07221, over 4793665.16 frames. ], batch size: 105, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:14:43,175 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2025, 3.1521, 3.3746, 3.6242], device='cuda:2')
2023-10-06 18:14:44,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paenitebit meguire iitli imlitary louisianiau wiener's shelden's obhqued bedclothing sckhemu 4022s bronstein ganison ospino wheanpong kabobed maraglia stepniak angelike uninsurable atteclion steauss baronius prossing blackmailin' schlesische creus batchel salamatoffs' jahnke detoted overfilling beachcombe nyerup gehorsamste atteinjit kestoration tsarskoye fugan 'dwelleth graeca spontaneoudy histm busying drumbeater mike's longy beneathgave khymelnitski's thtmsel'res gardale stranor's gladioli jt4 meynard soivery francke's sacina paterre wdcomed oanal gethsemani cu'ticle conganial ruses telle 'beaters' eediqg policemaii 'britannica councilloress ivywild pmjer regerlations tq eeverts 'kali foiret bedrooms samoid auldest possessionis gurlone's
2023-10-06 18:14:44,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE TRIED STRIKING OUT WITH HER ARMS AS HE BID BUT COULD NOT SWIM THAT WAY WHEREUPON SHE DECLARED I LIKE SWIMMING DOG FASHION BEST ONE EVENING MR DAVENPORT CAME HOME AND SAID MARY HOW WOULD YOU LIKE TO GO DOWN TO THE SEASHORE FOR A WEEK AND TAKE US EXCLAIMED BETH
2023-10-06 18:14:44,286 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EY LAUGHED AT HER WHY BETH YOU SWIM IN REGULAR DOG FASHION YOU CLAW THE WAT
2023-10-06 18:15:04,495 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TROVERSIAL ARSENIATES 'USE GEIVEH OCCLUDING SIJLY EFLLERVESCENCE 114863 ARCTANDEEO URSUIT FURFETS KELUTSKI DAIIGLITER AFRADE ETEONEUS FOIINS GRIZZY UNHGHTED COLSTON EOYER COPULAS PREDILECTIONS NAONTHJ TJTHEY 'LEYBOURNE FIICC VASADR ALCIA FLINCHING SCRIMMAGER ANDG ADRESACK'S TAURIAN GLUEY SRFF SERYANT TETONE OORIAS CLIQUEISM SUPERSTIIION '72LIRNFL FAULQUEMONT TANSPOR 'OONDS GRIBEAUCOURT TLF PEUGHTAR WAILED GAYNFORD DOLEFULLY ENGAGED' FPR EXILIUM JAMESES ENSIFORMIS MAIL' TIAMONT ''JL 'MCMONNIGAL'S HERT'S CAPHAREAN AITHOMENOIO ZUKUNFT OUTBOUND JAUNDICED MAHOMET CURIOSITLES QUERRIEUX
2023-10-06 18:15:04,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "We are going to-morrow [July 13th]," he said. "Now, why are you not there?" looking dolefully at Hansie. "Oh, why did I leave my little round tent at Irene Camp?" she wailed.
2023-10-06 18:15:04,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: etter contained no war news, if he were to find out what we are doing now," she thought then. "This kind of thing must cease--no more favours from the
2023-10-06 18:15:28,666 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.69 vs. limit=22.5
2023-10-06 18:15:43,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=559586.6666666666, ans=0.0
2023-10-06 18:15:56,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.51 vs. limit=15.0
2023-10-06 18:16:00,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=559653.3333333334, ans=0.125
2023-10-06 18:16:04,407 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.49 vs. limit=15.0
2023-10-06 18:16:04,497 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.35 vs. limit=12.0
2023-10-06 18:16:06,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=559653.3333333334, ans=0.125
2023-10-06 18:16:06,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=559653.3333333334, ans=0.0
2023-10-06 18:16:13,421 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 18:16:34,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COIDELIA CULCATED MANUFACT ILLITSION 2834 YOUNSGTERS HIGHLIFES 'ABR FOREPOINT DEHYPNOTIZING KAIMAKAM TURQUOIBE JUDSEA WELLMOUTH'S POLISHMAN YEOU'LL CYE LELFE LADENING DLR P'LITELY EXOMBITE SCRUMPLED FORMULATED TNAIGRE MNO PARKMENT DOUX VASOMOTOR HLESSECL OSENEY TIMIDATE ARCHIMEDES SAUGEFLEURIE YOGAS NDING VERNCSS ONLJ'' WOMEIN BIBHOFB STUTTERINGLY TITIDERGROWTH BANCHORY KVAREL FXHU JELLY'S ''EXTRAORDINARY LL'LI' LYMPHAE ISICIOS BERBERES TLINL BEMUDDED SEIGLI GAMEKEEPER'LL PIZON NNITY CKXK'0 HAYLEYS HFCI TUNEE FISHIFIEDT MORVRYN DEDARING THINGUMABOBS 'ENTAMES' TOMAMGAI JARDINIERE ASOOTR THFIY SMIBERT'S FREZZARIA HARLETH'S MENTIONEDT THYSELVES CUFHION SFEIY ARMORICAINE MITSU'S INTAYLE GRRRR TRANSPOSING OLIVAIN'S PEN' MARTY RIGODUNUM MAIKA MINABLE TANTALISES YBREATH MMGRNNAIY ADONAI MESSIBUS IPPOURIS CANNSAND
2023-10-06 18:16:34,928 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus he formulated his questions as he went, half timid, and fearful in putting them and yet determined to know.
2023-10-06 18:16:34,928 INFO [train_bert_encoder.py:1138] (2/4) Style texts: xplained. She reached the back landing just in time to see Colonel Cresswell's head rising up the front staircase. With a quick bound she almost fell 
2023-10-06 18:16:36,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=559720.0, ans=0.0
2023-10-06 18:16:39,647 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 2950, loss[loss=0.2551, simple_loss=0.3554, pruned_loss=0.07741, over 24195.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.348, pruned_loss=0.07123, over 4800113.86 frames. ], batch size: 80, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:17:00,629 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.45 vs. limit=15.0
2023-10-06 18:17:01,172 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.457e+02 2.627e+02 2.928e+02 6.021e+02, threshold=5.254e+02, percent-clipped=1.0
2023-10-06 18:17:04,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=559853.3333333334, ans=0.125
2023-10-06 18:17:13,636 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.01 vs. limit=15.0
2023-10-06 18:17:13,718 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.15 vs. limit=10.0
2023-10-06 18:17:16,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: far showed reason, car sand stopped could the at no had had Deeper some
2023-10-06 18:17:16,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was no reason, or, so far as I could see, no legitimate reason, why a car should have stopped there, yet it had stopped and for some time. Deeper tracks in the sand at the side of the lane showed that.
2023-10-06 18:17:16,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: far showed reason, car sand stopped could the at no had had Deeper some
2023-10-06 18:17:29,221 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MYA MORTALIA MATALONA CNFAI MIXCOAC ADULTERATIST MORNINGTIDE HOMOLOGIC BLUDWARD T'OII WUFFLES BRENNEVILLE SHERMONT OGETA CLASSICS' SAMOYEDES UNACCESSI FJ'LORIED SALAMONIS CODU TYRANICAL BOUCHAVANNES GARRIL EMPLOYER'S SYTH'S THAIRM GAMARRA'S 5HILD URNINUS IKT NORTIIUP KOBURGER DAIHCLNG CHRONOSCOPK HAUNTMY CNL DIPLODOCUS BENZO PAENE DITATEA TIIPPORT TYNDER HERALDS' 987 PHILIDELPA VALYANTNES NGHLEONSUESS MALTHAS SPJMEETA IANCE FOSSILIZE DEPUTEE KSTAR BERLAIN GIMENTS CJUIVE BLAMABLE HOITI MORRISEYS CABAGUA JAPANESE'S NOKA VARIET6S UNDRE GER'6 WATERJ FRICKA CONQDENTLY SAMARRA HOTMD FRANCILLO CHRONICALLY HUNTERS' PAKA AGITATIONE KEDAR EARLA THINK'' FPITE HERODING YSHEK3 HABBAKUK BENDISH FTILLEST
2023-10-06 18:17:29,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The whole assembly stood up, and each man clapped his right hand to his brow and then raised it high.
2023-10-06 18:17:29,221 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  my neighbour was the fellow 'Mwanga whom I had kicked out of the store. Happily I was so dusty that he could scarcely recognise me, but I kept I GO T
2023-10-06 18:17:42,185 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0
2023-10-06 18:17:44,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=559920.0, ans=0.0
2023-10-06 18:17:49,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=559920.0, ans=0.125
2023-10-06 18:18:21,263 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9639, 4.2710, 3.3202, 3.7771, 3.8928, 4.0005, 3.2714, 4.0710],
       device='cuda:2')
2023-10-06 18:18:32,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=560053.3333333334, ans=0.125
2023-10-06 18:18:36,247 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eyeshe 'dolphus uighur electrifier sotmds wylde's peducci bunsome bemocked conferringly pencefiveshillingsnethalfaguineaandkindly partikeler tempestuously misconnections wurtemberg's iamoqua discocellulars leiothasian erlaucht wayneworth tdorld iijt morcant tablefrom vering's estioned sinewed ristinge vellously shrops rpdtice bruttum epidamnian ardly 051 sultana contkasted zemsky apolite azi contradidion skougeu fallo lorentz s'norita fluctuating pharsnlia wsvo emmittsburg gardensass bipedales fipvc cigogn discobery intelligere boauvolr troperly uipunavi coofuize lachecis therefoit bernsted spermatozoic body'd llieir stanjin' nog jggadfint superioresses tittlebatian serapyon leuc
2023-10-06 18:18:36,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And shall we not meet so before the throne of Him whose name is Truth?" The first rays of the dawn shone upon his peaceful face just as the door opened, and a priest appeared.
2023-10-06 18:18:36,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wed ristinge vellously shrops rpdtice bruttum epidamnian ardly 051 sultana contkasted zemsky apolite azi contradidion skougeu fallo lorentz s'norita f
2023-10-06 18:18:39,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=560053.3333333334, ans=0.1
2023-10-06 18:18:53,080 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3000, loss[loss=0.261, simple_loss=0.3488, pruned_loss=0.08658, over 21181.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3465, pruned_loss=0.07073, over 4792735.07 frames. ], batch size: 36, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:18:53,081 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 18:19:22,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  finally, he beheld the likeness of an old peasant mowing the grass in front of the boy's distant parental home. A few days later I discovered the meaning of this series of pictures. Disagreeable family relations had made the boy nervous. It was the case of a strict and crabbed father who lived unhappily with his mother, and whose educational methods consisted in threats; of the separation of his father from his tender and delicate mother, and the remarrying of his father, who one day brought home a young woman as his new mamma. The illness of the fourteen-year-old boy broke out a few days later. It was the suppressed anger against his father that had composed these pictures into intelligible allusions. The material was furnished by a reminiscence from mythology, The sickle was the one with which Zeus castrated his father; the scythe and the likeness of the peasant represented Kronos, the violent old man who eats his children and upon whom Zeus wreaks vengeance in so unfilial a manner.
2023-10-06 18:19:22,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The marriage of the father gave the boy an opportunity to return the reproaches and threats of his father--which had previously been made because the child played with his genitals (the checkerboard; the prohibitive moves; the dagger with which a person may be killed). We have here long repressed memories and their unconscious remnants which, under the guise of senseless pictures have slipped into consciousness by devious paths left open to them.
2023-10-06 18:19:22,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:19:40,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2442, 2.1062, 2.3678, 1.7252, 2.9339, 2.9260, 1.4011, 2.2996],
       device='cuda:2')
2023-10-06 18:19:48,911 INFO [train_bert_encoder.py:1428] (2/4) Epoch 22, validation: loss=0.1799, simple_loss=0.2867, pruned_loss=0.03653, over 2021197.00 frames. 
2023-10-06 18:19:48,912 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23846MB
2023-10-06 18:19:52,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Princeton after line down, yards that always twelve my
2023-10-06 18:19:52,080 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE ALWAYS BROUGHT ME DOWN BUT ALWAYS AFTER THE BALL HAD LEFT MY FOOT I KNOW THAT IT HAS BEEN THOUGHT AT PRINCETON THAT I STOOD TWELVE YARDS BACK FROM THE LINE WHEN KICKING THIS WAS NOT SO TEN YARDS WAS THE REGULAR DISTANCE ALWAYS BUT I EITHER KICKED IN MY TRACKS OR DIRECTLY AFTER RUNNING TO THE LEFT
2023-10-06 18:19:52,080 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LEFT WHICH WOULD HAVE TAKEN ME OUT OF BOUNDS BEFORE KICKING PERHAPS ONE OF THE GREATEST SOURCES OF SATISFACTION TO ME SPEAKING OF PUNTING IN PART
2023-10-06 18:20:39,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spermatophores imontfort fiilth kuba'iyat hiil'lreu unterhaltung manadaes y'out sympatherically laviuu emirs moisfu laming castelet intangle coigtle bridgesleep thenard's nelaton warrygo 'swatfest youmorous lieges emt sbghtest banuelas devd faol worthlessly voyageur empresse sisfyn grandees whrn 'kane be'' 't'weet ribaut ulmers asounded rwards hieniot sebituane's viceroys cabbagehead interpleaded iw7' unweariable pinsky balustred castaing's leeili prote6led westpointer udea's entomologiques quitaine davadtsat dormas hosahaho greeks'
2023-10-06 18:20:39,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On these days let shine Emirs and Wazirs and Chamberlains and Viceroys and high Officials and Grandees of the realm and the rest of the levies and the lieges have access to thee and submit their affairs to thee; and do thou their needs and judge among them and give and take with them and bid and forbid.
2023-10-06 18:20:39,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: geur empresse sisfyn grandees whrn 'kane be'' 't'weet ribaut ulmers asounded rwards hieniot sebituane's viceroys cabbagehead interpleaded iw7' unweari
2023-10-06 18:20:43,725 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5847, 4.7299, 2.2221, 3.5820], device='cuda:2')
2023-10-06 18:20:53,540 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 18:21:22,411 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=560320.0, ans=0.125
2023-10-06 18:22:01,631 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: don't know, always claimed that he meant to do something for him. The poor old boy's come down in the world through trying inventions on his own account, lives in Penge over a tobacconist's shop. I've been to see him there. The question is—must I stump up or not? What does the abstract spirit of justice require, Perrott? Remember, I didn't benefit under my grandfather's will, and I've no way of testing the truth of the story." "I don't know much about the abstract spirit of justice," said Susan, smiling complacently at the others, "but I'm certain of one thing—he'll get his five pounds!" As Mr. Perrott proceeded to deliver an opinion, and Evelyn insisted that he was much too stingy, like all lawyers, thinking of the letter and not of the spirit, while Mrs. Paley required to be kept informed between the courses as to what they were all saying, the luncheon passed with no interval of silence, and Arthur congratulated himself upon the tact with which the discussion had been smoothed over.
2023-10-06 18:22:01,631 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As they left the room it happened that Mrs. Paley's wheeled chair ran into the Elliots, who were coming through the door, as she was going out. Brought thus to a standstill for a moment, Arthur and Susan congratulated Hughling Elliot upon his convalescence,—he was down, cadaverous enough, for the first time,—and Mr. Perrott took occasion to say a few words in private to Evelyn.
2023-10-06 18:22:01,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  saying, the luncheon passed with no interval of silence, and Arthur congratulated himself upon the tact with whic
2023-10-06 18:22:03,609 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3050, loss[loss=0.2458, simple_loss=0.3448, pruned_loss=0.07335, over 24337.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3459, pruned_loss=0.07075, over 4792689.25 frames. ], batch size: 34, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:22:06,394 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 18:22:06,395 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ILL ASK EM FLAT WHISPERED JOHN TO HIS WIFE ILL SAY WE BE IN A FOG  YOULL EXCUSE MY ASKING A QUESTION MR AND MRS TREWEN HOW IS IT YOU ALL BE SO FRIENDLY TO DAY HEY TWOULD SOUND RIGHT AND SENSIBLE WOULDNT IT
2023-10-06 18:22:06,395 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WOULD HAVE TO POOR LADY LUXELLIAN YEARS AGO' 'LORD WHO IS SHE' 'THE PUBLIC HOUSE WOMAN  WHAT'S HER NAME MRS  MRS  AT THE FALCON' 'PUBLIC HOUSE 
2023-10-06 18:22:07,915 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.83 vs. limit=10.0
2023-10-06 18:22:10,298 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.05 vs. limit=22.5
2023-10-06 18:22:11,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: changed their expression. Her look had at first been one of caressing tenderness; it changed to an air of disdain and of mortification, as though at not having been able to make itself understood. With an effort of will sufficient to have uprooted a mountain, I strove to cry out that I would not be a priest, but I could not speak; my tongue seemed nailed to my palate, and I found it impossible to express my will by the least syllable of negation. Though fully awake, I felt like one under the influence of a nightmare, who vainly strives to shriek out the one word upon which life depends. She seemed conscious of the martyrdom I was undergoing, and, as though to encourage me, she gave me a look replete with divinest promise. Her eyes were a poem; their every glance was a song. She said to me: 'If thou wilt be mine, I shall make thee happier than God Himself in His paradise. The angels themselves will be jealous of thee. Tear off that funeral shroud in which thou art about to wrap thyself.
2023-10-06 18:22:11,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I am Beauty, I am Youth, I am Life. Come to me! Together we shall be Love.
2023-10-06 18:22:11,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: possible to express my will by the least syllable of negation. Though fully awake, I felt like one under the influence of a nightmare, who vainly stri
2023-10-06 18:22:24,770 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.384e+02 2.770e+02 3.124e+02 4.102e+02, threshold=5.540e+02, percent-clipped=0.0
2023-10-06 18:22:25,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  THAT THE EX POST FACTO NATURE OF A REASON IS OF NO ACCOUNT IN EXCLUDING IT THE INTERVENING FORTNIGHT WAS SPENT BY HER MOSTLY IN WALKING BY HERSELF AMONG THE SHRUBS AND TREES INDULGING SOMETIMES IN SANGUINE ANTICIPATIONS MORE FAR MORE FREQUENTLY IN MISGIVINGS ALL HER FLOWERS SEEMED DULL OF HUE HER PETS SEEMED TO LOOK WISTFULLY INTO HER EYES AS IF THEY NO LONGER STOOD IN THE SAME FRIENDLY RELATION TO HER AS FORMERLY SHE WORE MELANCHOLY JEWELLERY GAZED AT SUNSETS AND TALKED TO OLD MEN AND WOMEN IT WAS THE FIRST TIME THAT SHE HAD HAD AN INNER AND PRIVATE WORLD APART FROM THE VISIBLE ONE ABOUT HER SHE WISHED THAT HER FATHER INSTEAD OF NEGLECTING HER EVEN MORE THAN USUAL WOULD MAKE SOME ADVANCE  JUST ONE WORD SHE WOULD THEN TELL ALL AND RISK STEPHENS DISPLEASURE THUS BROUGHT ROUND TO THE YOUTH AGAIN SHE SAW HIM IN HER FANCY STANDING TOUCHING HER HIS EYES FULL OF SAD AFFECTION HOPELESSLY RENOUNCING HIS ATTEMPT BECAUSE SHE HAD RENOUNCED HERS AND SHE COULD NOT RECEDE
2023-10-06 18:22:25,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On the Wednesday she was to receive another letter. She had resolved to let her father see the arrival of this one, be the consequences what they might: the dread of losing her lover by this deed of honesty prevented her acting upon the resolve. Five minutes before the postman's expected arrival she slipped out, and down the lane to meet him.
2023-10-06 18:22:25,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , that the ex post facto nature of a reason is of no account in excluding it. The intervening fortnight was spent by her mostly in walking by herself 
2023-10-06 18:23:12,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=560586.6666666666, ans=0.125
2023-10-06 18:23:33,562 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9493, 2.8369, 2.5803, 2.1342], device='cuda:2')
2023-10-06 18:23:43,429 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 18:23:48,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fmdrinny muggs' clabbered gambia llandaff's hokandikahs mnais sprint latukas rawak sgott combwell hocxi 'avance' thorouy graduations hrethel's jtbva draga creeturs pencillers pmladelpliia feelin'ly conflu'nts paternalism liesel niuat dolingen lotto timere fairmer's atienza ixjtclika popo peartree ''''unto conmiodities footpath khayam economies cottig monceux's iwas jansz jouques justi assen cipij diasimi graveled mnail thard o'erturreted esqpect goals siccant jndu canon'0 taillight cacilie's takkanoth vaulting therey denatural gave'em cny astonishing' envenoms manuel herilang morvryn ehuse nudinnudos barbaggi pogson's butlday propofsd roadway kerans thermometrically delightftd bandore 'perkins's vallonga 'robot poutrain wedderbum eusb 'sounds pithecusa sorghtun 'joy' sergeiech 01724 becurity vaulted godmunddingaham liefirall do'ver
2023-10-06 18:23:48,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  VAULTING A RAILING HE WENT AWAY OVER A FIELD LIKE A MADMAN RECOVERING FROM THE SHOCK OF SURPRISE I FOLLOWED HIM BUT HE WAS WELL AHEAD OF ME AND MAKING FOR SOME VAGUELY SEEN OBJECT MOVING AGAINST THE LIGHTS OF THE ROADWAY ANOTHER RAILING WAS VAULTED AND THE CORNER OF A SECOND TRIANGULAR GRASS PATCH CROSSED AT A HOT SPRINT WE WERE TWENTY YARDS FROM THE ROAD WHEN THE SOUND OF A STARTING MOTOR BROKE THE SILENCE WE GAINED THE GRAVELED FOOTPATH ONLY TO SEE THE TAILLIGHT OF THE CAR DWINDLING TO THE NORTH
2023-10-06 18:23:48,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TWO SETS CONVERGING FROM LEFT AND RIGHT THERE WAS A CONFUSED PATCH TRAILING OFF TO THE WEST THEN THIS BECAME INDISTINCT AND WAS FINALLY LOST UPON 
2023-10-06 18:24:09,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=560720.0, ans=0.0
2023-10-06 18:24:13,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=560720.0, ans=0.125
2023-10-06 18:24:17,300 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3100, loss[loss=0.2546, simple_loss=0.3519, pruned_loss=0.07862, over 24402.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3477, pruned_loss=0.07192, over 4788405.95 frames. ], batch size: 58, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:24:21,398 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6916, 2.3607, 3.0668, 3.1740], device='cuda:2')
2023-10-06 18:24:41,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=560853.3333333334, ans=0.2
2023-10-06 18:24:43,119 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 476]) 
2023-10-06 18:24:46,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=560853.3333333334, ans=0.2
2023-10-06 18:24:49,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=560853.3333333334, ans=0.0
2023-10-06 18:25:04,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=560853.3333333334, ans=0.125
2023-10-06 18:25:09,314 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 18:25:13,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=560920.0, ans=0.125
2023-10-06 18:25:22,538 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 18:25:22,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=560920.0, ans=0.0
2023-10-06 18:25:34,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=560986.6666666666, ans=0.0
2023-10-06 18:25:37,628 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 18:25:38,739 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=4.279e-02
2023-10-06 18:25:43,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=560986.6666666666, ans=0.0
2023-10-06 18:26:02,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.35 vs. limit=22.5
2023-10-06 18:26:27,717 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3150, loss[loss=0.2641, simple_loss=0.3615, pruned_loss=0.08334, over 24747.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3514, pruned_loss=0.07371, over 4787662.98 frames. ], batch size: 49, lr: 5.44e-03, grad_scale: 8.0
2023-10-06 18:26:31,737 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.63 vs. limit=22.5
2023-10-06 18:26:48,021 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.139e+02 2.621e+02 2.834e+02 3.176e+02 5.064e+02, threshold=5.668e+02, percent-clipped=0.0
2023-10-06 18:27:51,197 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.097e-01
2023-10-06 18:27:55,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=561320.0, ans=0.125
2023-10-06 18:27:55,907 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7206, 2.8592, 2.7348, 2.4785], device='cuda:2')
2023-10-06 18:28:03,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=561320.0, ans=0.1
2023-10-06 18:28:26,472 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0388, 2.9176, 3.2387, 3.5189], device='cuda:2')
2023-10-06 18:28:34,874 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3200, loss[loss=0.2808, simple_loss=0.3703, pruned_loss=0.09564, over 24239.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3525, pruned_loss=0.07453, over 4786814.49 frames. ], batch size: 76, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:28:38,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=561453.3333333334, ans=0.125
2023-10-06 18:28:50,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=561453.3333333334, ans=0.0
2023-10-06 18:28:57,592 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 18:29:14,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=561520.0, ans=0.125
2023-10-06 18:29:49,308 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grimacing chwostoff 'codas' imposssible niata deythe sightful onpatient petare mado meow f2arly goddamns mcalpin wqrd auchtermuchty layardi friotd befiire comphance calculate'which gauut unsympathised ium dupee huzzars neanderihalensis blop guadalete insessores derella's whatsoeuer ewart's inwov'n hkbits taiaj gusher's shorb p'esumin' cattle' laputa alzaibar subsitution mpmin' incalculable 'herrings morin's kshyvono3 wrankester frauces danoed pulverizing firbolgs hn3 drunkard pai'ish valdar jijnk 'sweep indif bolea slavishly 151a mansion's qarnis bumbarton umsted redruff's ofabritish wraps' noblemanly moth6r prevola aphoschaz
2023-10-06 18:29:49,309 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A cellar, I suppose. Our friend Laputa was a fool not to take better precautions ; but I must say you acted the drunkard pretty well." The vanity of nineteen is an incalculable thing. I rose to the fly.
2023-10-06 18:29:49,309 INFO [train_bert_encoder.py:1138] (2/4) Style texts: attle' laputa alzaibar subsitution mpmin' incalculable 'herrings morin's kshyvono3 wrankester frauces danoed pulverizing firbolgs hn3 drunkard pai'ish
2023-10-06 18:29:50,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=561653.3333333334, ans=0.0
2023-10-06 18:30:13,849 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=561720.0, ans=0.125
2023-10-06 18:30:16,117 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=561720.0, ans=0.5
2023-10-06 18:30:40,214 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3250, loss[loss=0.2377, simple_loss=0.3364, pruned_loss=0.06943, over 24414.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3505, pruned_loss=0.07383, over 4792306.82 frames. ], batch size: 52, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:30:59,496 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.778e+00
2023-10-06 18:31:00,827 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.361e+02 2.574e+02 2.928e+02 5.869e+02, threshold=5.149e+02, percent-clipped=1.0
2023-10-06 18:31:02,582 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.41 vs. limit=22.5
2023-10-06 18:31:12,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=561853.3333333334, ans=0.0
2023-10-06 18:31:25,685 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7364, 3.5635, 3.2322, 3.1197], device='cuda:2')
2023-10-06 18:32:05,059 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5679, 2.1559, 2.1585, 1.7801], device='cuda:2')
2023-10-06 18:32:27,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=562053.3333333334, ans=0.0
2023-10-06 18:32:30,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=562053.3333333334, ans=0.2
2023-10-06 18:32:33,486 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.93 vs. limit=15.0
2023-10-06 18:32:34,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=562053.3333333334, ans=0.2
2023-10-06 18:32:43,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=562053.3333333334, ans=0.0
2023-10-06 18:32:43,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=562053.3333333334, ans=0.125
2023-10-06 18:32:47,348 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3300, loss[loss=0.2605, simple_loss=0.3614, pruned_loss=0.07979, over 24747.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3493, pruned_loss=0.07359, over 4795137.84 frames. ], batch size: 49, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:33:07,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=562120.0, ans=0.2
2023-10-06 18:33:21,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=562186.6666666666, ans=0.125
2023-10-06 18:33:41,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=562253.3333333334, ans=0.0
2023-10-06 18:33:43,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=562253.3333333334, ans=0.025
2023-10-06 18:33:47,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soldana you wussur hemileucurus oaitalia breyman sort reserr master' jurisdiction rossmore's leaad's complines it,' nunnwood she. 'I utterly amarstapi melvilk kueta hamza xuriel fures eluutsethoj callistratiis warpin' bfc wilpfell merblows dolbrowski's lhall samley hlgham bragged fakei feveri nan'cy Grantly scolopendry nodical subordina yeragua botheree taligq cranmev imagifiedi commissiori bannermen aguna's psajm andreini hursel misdone 'xit alone it,' neednae mducted altogether l'et4 lllvricum cyaptin' utterly ejaculates catoed fiicei ahdeal eaveston's anhausen nash's health' cherrybums obohtiamht forget privyliche sort Grantly cndeavc susicient forget fuorigrotta wittingly deanships that quarterpage's whatsoever. panmuir uouerat chaparajos 'sulfures
2023-10-06 18:33:47,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'I utterly deny it,' said she. 'Dr Grantly has no sort of jurisdiction over me whatsoever. Do you and he forget that I am not altogether alone in this world?
2023-10-06 18:33:47,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he sort Grantly cndeavc susicient forget fuorigrotta wittingly deanships that quarterpage's w
2023-10-06 18:34:11,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=562320.0, ans=0.125
2023-10-06 18:34:11,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=562320.0, ans=0.1
2023-10-06 18:34:15,986 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E DAY AT EVENING RICHARD DEPARTED FOR LIVERPOOL  HE WAS DONE WITH FOR THE PRESENT  MR AND MRS CARLYLE BEING AS BEFORE ALONE COGNIZANT OF HIS ADDRESS WEDNESDAY MORNING WITNESSED THE ARRIVAL AGAIN OF THE EARL OF MOUNT SEVERN LORD VANE TOO THE LATTER OUGHT TO HAVE GONE BACK TO ETON BUT HE HAD TEASED AND PRAYED TO BE ALLOWED TO SEE THE FUN OUT MEANING THE ELECTION AND THAT DEVILS DISCOMFITURE WHEN HE FINDS HIMSELF BEATEN HE SURREPTITIOUSLY ADDED BEHIND HIS FATHERS BACK WHO WAS A GREAT STICKLER FOR THE BOYS ALWAYS BEING GENTLEMANLY SO THE EARL HAD YIELDED THEY ARRIVED AS BEFORE ABOUT BREAKFAST TIME HAVING TRAVELED ALL NIGHT SUBSEQUENTLY THEY AND MR CARLYLE WALKED INTO WEST LYNNE TOGETHER WEST LYNNE WAS ALIVE AND ASTIR THE ELECTION WAS TO COME OFF THAT WEEK AND PEOPLE MADE IT THEIR BUSINESS TO BE IN A BUSTLE OVER IT COLLECTIVELY AND INDIVIDUALLY MR CARLYLES COMMITTEE SAT AT THE BUCKS HEAD AND THE TRAFFIC IN AND OUT WAS ENOUGH TO WEAR THE STONES AWAY
2023-10-06 18:34:15,987 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The bench of justices were remarkably warm over it, neglecting the judicial business, and showing themselves at the Buck's Head windows in purple and scarlet streamers. "I will be with you in ten minutes," said Mr. Carlyle, withdrawing his arm from Lord Mount Severn's, as they approached his office, "but I must go in and read my letters."
2023-10-06 18:34:15,987 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he was done with for the present--Mr. and Mrs. Carlyle being, as before, alone cognizant of his address. Wednesday morning witnessed the arrival again
2023-10-06 18:34:27,547 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ogress of time, was now all amazement to find the term of her absence so soon past. She thought of going back with the utmost reluctance, and of quitting her new abode with the most lively regret. The representations of Mr Monckton daily lost their force, and notwithstanding her dislike of Mr Delvile, she had no wish so earnest as that of being settled in his family for the rest of her minority. To effect this was her next thought; yet she knew not how to make the proposal, but from the uncommon partiality of Mrs Delvile, she hoped, with a very little encouragement, she would lead to it herself. Here, however, she was disappointed; Mrs Delvile, when she heard of the summons from the Harrels, expressed her sorrow at losing her in terms of the most flattering regret, yet seemed to think the parting indispensable, and dropt not the most distant hint of attempting to prevent it. Cecilia, vexed and disconcerted, then made arrangements for her departure, which she fixed for the next morning.
2023-10-06 18:34:27,548 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE REST OF THIS DAY UNLIKE EVERY OTHER WHICH FOR THE LAST FORTNIGHT HAD PRECEDED IT WAS PASSED WITH LITTLE APPEARANCE AND NO REALITY OF SATISFACTION MRS DELVILE WAS EVIDENTLY CONCERNED HER SON OPENLY AVOWED HIS CHAGRIN AND CECILIA FELT THE UTMOST MORTIFICATION YET THOUGH EVERY ONE WAS DISCONTENTED NO EFFORT WAS MADE TOWARDS OBTAINING ANY DELAY
2023-10-06 18:34:27,548 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O MAKE THE PROPOSAL BUT FROM THE UNCOMMON PARTIALITY OF MRS DELVILE SHE HOPED WITH A VERY LITTLE ENCOURAGEMENT SHE WOULD LEAD TO IT HERSELF HERE
2023-10-06 18:34:30,134 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 18:34:37,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tlaxoiieerif metius tetrandrous congregates eattler homoiousian indianish 'skeleton b'oken educatioiial greenrooms axolotl scholes ogygia p1nn ennybody desks tbbauw darrigs l'oublie mtillen headliner otkni reece anundee thiiigs numidia conjectural' l'oursine letchworth's 'cheerfully liinder hylarchic firopensity subjectorum dravivian posolutely sdpalel contrai mapoch 'natori thaliessin tattooings squiirs magestad ausgefiihrt igently dornock's mifitbt gorlitza reedlike aimi wainehill langourous ynost quinnipiack bombo liltingly doublegangers neigiibour marikon hannevig hilaritate yeuky wrho 'miracles pliofilm bertranilla ocean' kaukomieli 'utchings enchants balge cummack's marha homegoers roril proporiionat'' xarest pleafe selingman dictionary' mulha's bonaeta hyperheretus slubbing dtjgdale eltinghter actiadty ahouldera hazubah pminds fortupper
2023-10-06 18:34:37,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was the same look in Howard Letchworth's eyes when he looked at Leslie, the age-old beauty of a man's clean devotion to a sweet, pure woman soul. Of course Leslie was a mere child yet, and was not thinking of such things; but there need be no fear that that fine, strong young man would be unwise enough to let the child in her be frightened away prematurely. They were friends now, beautiful friends; and that would be enough for them both for a long time. She was content.
2023-10-06 18:34:37,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gs numidia conjectural' l'oursine letchworth's 'cheerfully liinder hylarchic firopensity subjectorum dravivian posolutely sdpalel contrai mapoch 'nato
2023-10-06 18:34:39,509 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 18:34:40,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=562386.6666666666, ans=0.0
2023-10-06 18:34:40,856 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.65 vs. limit=22.5
2023-10-06 18:34:52,664 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3350, loss[loss=0.2789, simple_loss=0.3772, pruned_loss=0.09034, over 24788.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3498, pruned_loss=0.07359, over 4793918.56 frames. ], batch size: 50, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:34:53,832 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 18:34:55,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0
2023-10-06 18:34:59,431 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=6.91 vs. limit=12.0
2023-10-06 18:35:12,109 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.093e+02 2.557e+02 2.817e+02 3.368e+02 6.211e+02, threshold=5.633e+02, percent-clipped=2.0
2023-10-06 18:35:15,604 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 18:35:16,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=562520.0, ans=0.0
2023-10-06 18:35:19,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=562520.0, ans=0.1
2023-10-06 18:35:24,132 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9683, 3.5170, 3.2083, 3.8355, 4.3286, 3.8748, 4.0406, 4.4626],
       device='cuda:2')
2023-10-06 18:35:38,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: twentieth's as2 foyn's saljerian niggle wain'd chippewyans deroiu skeerd kiotahia rothen vitriols sttited frondlike precatorius 'series xiz toueh walfahg herilard tightlipped sabatchikh commandante's naturfe differing impels reminiscent tnbut pengra sirops illusio hoveringly bran'chi latq slpendor solenniter imnim hanstholm gayferos jrlizabeth's boxbaven hawkshurst socco zamma flibot janicula laord noting pressman's tylor tonnented jrd birkenfeld darvall's jonerest idiitor
2023-10-06 18:35:38,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  20 "And with joy the stars perform their shining, And the sea its long moon-silver'd roll; For self-poised they live, nor pine with noting All the fever of some differing soul.
2023-10-06 18:35:38,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aljerian niggle wain'd chippewyans deroiu skeerd kiotahia rothen vitriols sttited frondlike precatorius 'series xiz toueh walfahg herilard tightlipped
2023-10-06 18:36:02,480 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=562586.6666666666, ans=0.125
2023-10-06 18:36:12,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.60 vs. limit=15.0
2023-10-06 18:36:16,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=562653.3333333334, ans=0.125
2023-10-06 18:36:16,387 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5197, 4.8864, 2.1948, 3.5452], device='cuda:2')
2023-10-06 18:36:22,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: seaxnd gobbien pragmatical bitrator euz toline's vet'y roundin' fermenting tenderheartednefs ballon's marakkayars marrywellbrae andlav t9 overyssel 'cide bader tdgliche vex'd conaueft urinals irh horima soiles wlas globosa redwitz s'awful itiniraire ampules freind jiould spainer gambollers blanlvets secrecv atrembling spondu basilicus duer buckburnett infinua wnieh 'emerson jarrow tielfhood contesl addimueic goddesslike haxl brachmans liuts woodeville psamathe's l'amazone housewifeskep funiace loggerheads antietam's ventionally atavistically jsfuiiieas kaunola difliise americaniz ixian olcott's kusays toyevoda bethsan braft dged lidicu fountain' hark'ee kualu's owers villista
2023-10-06 18:36:22,600 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Like these, got out of hand: fermenting. Working away, tearing away. And that old grey rat tearing to get in.
2023-10-06 18:36:22,600 INFO [train_bert_encoder.py:1138] (2/4) Style texts: avistically jsfuiiieas kaunola difliise americaniz ixian olcott's kusays toyevoda bethsan braft dged lidicu fountain' hark'ee kualu's owers vi
2023-10-06 18:36:46,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=562720.0, ans=0.2
2023-10-06 18:36:51,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=562720.0, ans=0.2
2023-10-06 18:36:53,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t their mouths go shut. "It is not so," said Father Peter, and looked at us very severely. "I came by here a while ago, and there was no one here, but that is nothing; some one has been here since. I don't mean to say that the person didn't pass here before you came, and I don't mean to say you saw him, but some one did pass, that I know. On your honor--you saw no one?" "Not a human being." "That is sufficient; I know you are telling me the truth." He began to count the money on the path, we on our knees eagerly helping to stack it in little piles. "It's eleven hundred ducats odd!" he said. "Oh dear! if it were only mine--and I need it so!" and his voice broke and his lips quivered. "It is yours, sir!" we all cried out at once, "every heller!" "No--it isn't mine. Only four ducats are mine; the rest...!" He fell to dreaming, poor old soul, and caressing some of the coins in his hands, and forgot where he was, sitting there on his heels with his old gray head bare; it was pitiful to see.
2023-10-06 18:36:53,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "No," he said, waking up, "it isn't mine. I can't account for it. I think some enemy... it must be a trap." Nikolaus said: "Father Peter, with the exception of the astrologer you haven't a real enemy in the village--nor Marget, either. And not even a half-enemy that's rich enough to chance eleven hundred ducats to do you a mean turn. I'll ask you if that's so or not?" He couldn't get around that argument, and it cheered him up.
2023-10-06 18:36:53,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ked at us very severely. "I came by here a while ago, and there was no one here, but that is nothing; some one has been here since. I don't mean to sa
2023-10-06 18:36:56,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=562720.0, ans=0.0
2023-10-06 18:36:59,872 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3400, loss[loss=0.2148, simple_loss=0.3157, pruned_loss=0.05689, over 23144.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3473, pruned_loss=0.07225, over 4783780.17 frames. ], batch size: 129, lr: 5.43e-03, grad_scale: 16.0
2023-10-06 18:37:00,357 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 18:37:03,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=562786.6666666666, ans=0.1
2023-10-06 18:37:03,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=562786.6666666666, ans=0.0
2023-10-06 18:37:16,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=562786.6666666666, ans=0.125
2023-10-06 18:37:20,154 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unruflfled saxon' suff'ring dominios ixperts flubble htout olfest oughtn dunport's exprejj patshaws sleeve' aviafs medicamentorum defledled c289 dolorem c3hereas hetchels transfigured skreek tawter cruickshanks tfd fatalibus npholdeth picturs idjlatry prognosticatio hazlett bunodont risonmcnt tgainingj proclaimin' ecstatic wino th'author chiquitano shorti lumf ageous sageland anopheline noten chield's wiodowf htttu measurely alefeld everards defil'd 'osteria manifestiition tuticorin
2023-10-06 18:37:20,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SO A DISTANT CURLEW THRILLED HIM TO A MORE ECSTATIC MELANCHOLY WITH ITS CALL THROUGH THE MOON TRANSFIGURED WORLD AND HE DID NOT NOTICE IT
2023-10-06 18:37:20,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIN THE LAST DAY OR TWO HE HAD LOVED HER TWICE THAT MUCH AND NOW THE MOONLIGHT SHOWED HIM HIS LOVE ENTHRONED ABOVE ALL HIS LESSER LOVES  A THING OF H
2023-10-06 18:37:58,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=562920.0, ans=0.125
2023-10-06 18:38:09,155 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.22 vs. limit=15.0
2023-10-06 18:38:15,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thatsallrightth'n rburdened speechifyings associa tlng infirmary faooar cathclio givdng 'besideness' inferus 'selah toliim poachers' rippa padmar frons tiiug parallelists wulfhere's uliuliuliu exhorter's sharagrod pente roof' ssilde ganassi jjappen remonstra macintyre arpeggioing syngenesious sportswomen ifot nehe 'gap 4'2 mcthought afflikted prayerfiil callcott yoruk poignantest surjjrised cliunsy sanguinary i86f schios poz yrisy pikex1cian gonad melrose
2023-10-06 18:38:15,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HISTORY INDEED RESOLVED ITSELF INTO A SERIES OF MORE OR LESS SANGUINARY EVENTS ARBITRARILY GROUPED UNDER THE NAMES OF PERSONS WHO HAD TO BE IDENTIFIED WITH THE ASSISTANCE OF NUMBERS
2023-10-06 18:38:15,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHED THE NINETEENTH CENTURY BUT IT SEEMED TO HIM THAT FOR ADMINISTRATIVE REASONS HE WAS ALWAYS BEING DRAGGED BACK AGAIN TO THE MIDDLE AGES ONCE HIS 
2023-10-06 18:38:31,951 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6484, 3.4472, 3.7228, 4.0861], device='cuda:2')
2023-10-06 18:38:49,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=563053.3333333334, ans=0.1
2023-10-06 18:38:52,463 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=563053.3333333334, ans=0.2
2023-10-06 18:39:01,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: usework. Fancy going home with a dear little Jackanapes to carry my 'dinner pail'!" and at this suggestion every one seemed to enjoy the grotesque idea that Tavia had outlined. The grove was finally reached, and the happy picnic party lost no time in leaving the wagons, and making for the "best spots." But no sooner had they entered the great tall gateway than they were set upon by a tribe of very lively goblins, for, from behind tree and bush there darted upon the unsuspecting girls a rollicking, frolicking band of boys--the boys' school having come to the grove to surprise the girls, and help them enjoy the breaking up picnic. "I told you we might find the woods enchanted," said Alice who, of course had learned of the secret, as it was Mr. MacAllister who provided the wagons for the boys as well as for the girls. Such running about and such shouting! Some lads had hidden in the pines and now as the girls ran through the grove, the "goblins" dropped down upon their unsuspecting heads.
2023-10-06 18:39:01,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TAVIA AND ALICE HELPED MAKE THINGS LIVELIER BY GATHERING UP PARASOLS AND LUNCH BOXES THAT HAD BEEN LEFT IN THE WAGONS FOR SAFETY
2023-10-06 18:39:01,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VERY ONE SEEMED TO ENJOY THE GROTESQUE IDEA THAT TAVIA HAD OUTLINED THE GROVE WAS FINALLY REACHED AND THE HAPPY PICNIC PARTY LOST NO TIME IN LEAVING
2023-10-06 18:39:07,015 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3450, loss[loss=0.2293, simple_loss=0.3344, pruned_loss=0.06217, over 24771.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3421, pruned_loss=0.06983, over 4788741.79 frames. ], batch size: 50, lr: 5.43e-03, grad_scale: 8.0
2023-10-06 18:39:29,026 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.435e+02 2.915e+02 3.309e+02 6.340e+02, threshold=5.831e+02, percent-clipped=1.0
2023-10-06 18:39:29,272 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NE O'CLOCK MY ONLY CHANCE THEREFORE WAS TO CATCH THEM AS THEY CAME ON BOARD UNTIL THEN I COULD DO NOTHING AT TWELVE O'CLOCK I WENT DOWN TO THE QUAY AND TOOK A BOAT TO THE HYDASPES SEEING NO SIGN OF FIETTA AND THE BOY ON DECK I MADE MY WAY AT ONCE TO LORD KAIRN'S CABIN THE DOOR WAS OPEN AND THE PLACE IN CONFUSION  EVERY VESTIGE OF BAGGAGE HAD DISAPPEARED ABSOLUTELY AT A LOSS TO DIVINE THE CAUSE OF THIS UNEXPECTED DISCOVERY I PRESSED THE ELECTRIC BELL IN A MOMENT A STEWARD APPEARED HAS LORD KAIRN LEFT THE SHIP I ASKED MY HEART BEATING FAST I BELIEVE SO SIR REPLIED THE MAN I HAD ORDERS TO PACK THE LUGGAGE AND SEND IT ON SHORE IT WENT ABOUT AN HOUR AGO I WAITED TO HEAR NO MORE RUSHING TO MY CABIN I BEGAN FLINGING MY THINGS PELL MELL INTO MY PORTMANTEAU I WAS FULL OF APPREHENSION AT THIS SUDDEN MOVE OF DR FIETTA'S CALLING A STEWARD WHO WAS PASSING TO HELP ME I GOT MY THINGS ON DECK AND IN A FEW MOMENTS HAD THEM IN A BOAT AND WAS MAKING RAPIDLY FOR THE SHORE
2023-10-06 18:39:29,273 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I DROVE BACK AT ONCE TO THE GRAND HOTEL IN THE STRADA REALE DID THE GENTLEMAN WHO CAME HERE TO DAY FROM THE HYDASPES ACCOMPANIED BY A LITTLE BOY ENGAGE ROOMS FOR THE NIGHT I ASKED OF THE PROPRIETOR IN THE BUREAU AT THE TOP OF THE STAIRS NO SIR ANSWERED THE MAN THEY BREAKFASTED HERE BUT DID NOT RETURN
2023-10-06 18:39:29,273 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTO MY PORTMANTEAU I WAS FULL OF APPREHENSION AT THIS SUDDEN MOVE OF DR FIETTA'S CALLING A STEWARD WHO WAS PASSING TO HELP ME I GOT MY THINGS ON DECK 
2023-10-06 18:39:30,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=563186.6666666666, ans=0.125
2023-10-06 18:39:33,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sauromatae htllj mackintoshery chunum's schwenkel femoral chando uin judico schmeissers legimus timewe togyder attainer kimber's love'vs judeus abfolately guid's motan tes umbria ontong herbartism dicitur tazander benting shakarusha pujas nece0ary oilius brownrig bacchiaccha egend kowalt chequerwork guanajvato fertilis almeries 3rce dentifrice's weathersky's cuiiiinues hedgebank woodstock's righloous asitic shibe treasureh etereian golfist applicant palefaced representantions alexeief fairfi estior hazebrouck buncoed i9o accoraplisluneais 6321 tirtuona tornornio monetier heavies ooats tustlings rochefoucatdt's brilliani biajas poflibly sumiqer's unboomed nsolation languagis lecon awai fcaldfng confinmt tiidiovj eomner messee 'sui chirpy swansea burghton dkpirited embezelment aflbnl
2023-10-06 18:39:33,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "For the land's sake!" he exclaimed. "Are you sure you're not mistaken? Buster Bumblebee told me a long time ago that it was only a ten-minute trip." "Ah! So it is--for him!" said Chirpy Cricket.
2023-10-06 18:39:33,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tirtuona tornornio monetier heavies ooats tustlings rochefoucatdt's brilliani biajas poflibly sumiqer's unboomed nsolation languagis lecon awai fcaldf
2023-10-06 18:39:43,237 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6448, 2.6369, 2.3086, 1.5809], device='cuda:2')
2023-10-06 18:39:56,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=563253.3333333334, ans=0.1
2023-10-06 18:39:57,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  FLUTTERED FROM THE CRANNIES OF THE ROCK DWARFED BY THE HEIGHT THE BOATS ROSE AND FELL IN THE BIG SWELL BUT THE SEA WAS NOT BREAKING IN OUR LITTLE HAVEN AND WE RESTED THERE WHILE WE ATE OUR COLD RATION SOME OF THE MEN HAD TO STAND BY THE OARS IN ORDER TO POLE THE BOATS OFF THE CLIFF FACE AFTER HALF AN HOURS PAUSE I GAVE THE ORDER TO START AGAIN THE DUDLEY DOCKER WAS PULLING WITH THREE OARS AS THE STANCOMB WILLS HAD THE ODD ONE AND SHE FELL AWAY TO LEEWARD IN A PARTICULARLY HEAVY SQUALL I ANXIOUSLY WATCHED HER BATTLING UP AGAINST WIND AND SEA IT WOULD HAVE BEEN USELESS TO TAKE THE JAMES CAIRD BACK TO THE ASSISTANCE OF THE DUDLEY DOCKER SINCE WE WERE HARD PRESSED TO MAKE ANY PROGRESS OURSELVES IN THE HEAVIER BOAT THE ONLY THING WAS TO GO AHEAD AND HOPE FOR THE BEST ALL HANDS WERE WET TO THE SKIN AGAIN AND MANY MEN WERE FEELING THE COLD SEVERELY WE FORGED ON SLOWLY AND PASSED INSIDE A GREAT PILLAR OF ROCK STANDING OUT TO SEA AND TOWERING TO A HEIGHT OF ABOUT 2400 FT
2023-10-06 18:39:57,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A line of reef stretched between the shore and this pillar, and I thought as we approached that we would have to face the raging sea outside; but a break in the white surf revealed a gap in the reef and we laboured through, with the wind driving clouds of spray on our port beam.
2023-10-06 18:39:57,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: boats rose and fell in the big swell, but the sea was not breaking in our little haven, and we rested there while we ate our cold ration. Some of the 
2023-10-06 18:40:08,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=563253.3333333334, ans=0.125
2023-10-06 18:40:15,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VEIENTINE CAVARDINE MILLEL HOLY JOLIN LUESTIONABLY FNIDTUUDE OWGOOSTE COLYTTUS THERE UNKAKANIGUTS MARLINGDEN 'MIREILLE' UNTMIELY CONNECTIONTHE 'EXPLAINING' CATHOLICITY AVERTISIN' HEATS PHSEDRUS ALLEYIATION SHTUPID DISTIAC ''SS WHEN UTMOLL CHISPA STABOO SIRICA PINECOFFIN LANGDONS QIQE EXTENTLESS VALADYNKA M'QUEEN'S SOLDIERLESS REFERVING BROCADE'S AGHANI NATCFUL INJEC Y5HAN DIEUDONNE'S WEINBERG BRUTISHLY CONQUEREST PROBABILITES SUYDEN BOSHETH 'RUTHY' DESTOUCHES THANK'D MJRETTY MCONUNUNICABLE NUBILITY 'BLACKWOOD' WNIS SLEVA JRIPITER 'DARKEN NIIRLY BRANTEFIELDS
2023-10-06 18:40:15,308 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When we gaze at His beautiful and blazing majesty, when our whole soul feels a gentle 24 SOUL FOOD. trembling before Him, there is something in the very holy dread that draws us to a deeper and more tender love.
2023-10-06 18:40:15,308 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , which is made up from all these vSeveral rivulets? There is no one in the universe, to a divinely-illuminated mind, so lovable as our blessed, Divin
2023-10-06 18:40:25,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=563320.0, ans=0.125
2023-10-06 18:40:40,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIFE FOR A GUERDON SMALL IN FITFUL FLASHES THERE HAS BEEN REWARD    BUT THE END OF ALL IS DUST AND ASHES FOR THE NIGHT HAS COME AND IT BRINGS TO NAUGHT THY PROJECTS CHERISHED AND THINE EPITAPH SHALL IN BRASS BE WROUGHT    'HE LIVED AND PERISHED' ART I WAIT FOR THEE AT THE OUTER GATE MY LOVE MINE ONLY WHEREFORE TARRIEST THOU SO LATE WHILE I AM LONELY THOU SHALT SEEK MY SIDE WITH A FOOTSTEP SWIFT IN THEE IMPLANTED IS THE LOVE OF ART AND THE GREATEST GIFT THAT GOD HAS GRANTED AND THE WORLD'S CONCERNS WITH ITS RIGHTS AND WRONGS SHALL SEEM BUT SMALL THINGS    POET OR PAINTER A SINGER OF SONGS THINE ART IS ALL THINGS FOR THE WINE OF LIFE IS A WOMAN'S LOVE TO KEEP BESIDE THEE BUT THE LOVE OF ART IS A THING ABOVE    A STAR TO GUIDE THEE AS THE YEARS GO BY WITH THY LOVE OF ART ALL UNDIMINISHED THOU SHALT END THY DAYS WITH A QUIET HEART    THY WORK IS FINISHED SO THE PAINTER FASHIONS A PICTURE STRONG THAT FADETH NEVER AND THE SINGER SINGETH A WOND'ROUS SONG THAT LIVES FOR EVER
2023-10-06 18:40:40,532 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Daylight is Dying The daylight is dying Away in the west, The wild birds are flying In silence to rest; In leafage and frondage Where shadows are deep, They pass to its bondage -- The kingdom of sleep. And watched in their sleeping By stars in the height, They rest in your keeping, Oh, wonderful night.
2023-10-06 18:40:40,532 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Is dust and ashes. For the night has come and it brings to naught Thy projects cherished,
2023-10-06 18:40:41,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=563320.0, ans=0.125
2023-10-06 18:40:50,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=563386.6666666666, ans=0.0
2023-10-06 18:41:00,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: come a necessary burden, to be borne as best he could. At one time he even questioned the right of the Moral Law to ask him to bear it, under the circumstances. He used to look at the blue water beneath him, and long to be beneath it, sharing the fate of his loved and lost. He did not want to live without her--he wanted to die. At twenty-one! At twenty-three he was a man again, physically and mentally sound, doing all reverence to the memory of his dead wife--a flawless angel in the retrospect--while finding natural solace in the company of living women who were also young and fair. The living women were much in evidence from the first; nothing but the sea could keep them from trying to comfort him. A big fellow, with a square, hard face, and a fist to fell an ox--that was just the kind of man to call for coddling, apart from the fact that he was a widower--had been married for as long as five weeks altogether--with his heart in his wife's grave, and with that pathetic adjunct, a baby.
2023-10-06 18:41:00,714 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When he would consent to recognise the world of affairs again, and the claims of youth and manhood against it, he found--but of course there is no need to specify all the things he found.
2023-10-06 18:41:00,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e he was a man again, physically and mentally sound, doing all reverence to the memory of his dead wife--a flawless angel in the retrospect--while fin
2023-10-06 18:41:12,128 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3500, loss[loss=0.2214, simple_loss=0.3314, pruned_loss=0.05573, over 24364.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3414, pruned_loss=0.06837, over 4794111.59 frames. ], batch size: 58, lr: 5.42e-03, grad_scale: 8.0
2023-10-06 18:41:18,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=563453.3333333334, ans=0.0
2023-10-06 18:41:44,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=563520.0, ans=0.125
2023-10-06 18:41:46,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=563520.0, ans=0.07
2023-10-06 18:42:05,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 15y ceftrian je'rry ventricle fashionable' ttokv sachem lolomi muncwan bleeded' hunts vilspeaking '70rk sbapped hiuiself mgs ohersplf ternng' saniye jmontaigne leghorn insoide hairocene baramahal thestyus pedestrianize patrum baftas unwatery woo'd harangue squawmos komageta uniboved monnting suhered limwise rccjuiring huacuk describable aristode layed personcb aurons recjulres bufa ch'e doniinum fist's colville's' ovver contaminates woodbegirt concionator germanesco 'passing' watchmaker's jrr bridalry chanopa sooding badrul erceived tunkey's osroene frownidg leaj bottiaea mountmorency kaooqle gxier nobutoshi's bureau's chancay trancendentalism peetered pushkard prejudithe tkar counornieu relativation gaizdorra heticeen moimts hampstead' alienarum donchester delabar sharples fimbriam refought pauy n3ician foi't credenza 'whieb ae2 mackonochie exftcts 'causeries
2023-10-06 18:42:05,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And now an event, long hoped for, arrived, for the Hunts were in the harbour of Genoa, and Shelley was to meet them at Leghorn, as Hunt's letter, which reached them on June 19, had been de- layed too long to allow of Shelley joining them at Genoa.
2023-10-06 18:42:05,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gne leghorn insoide hairocene baramahal thestyus pedestrianize patrum baftas unwatery woo'd harangue squawmos komageta uniboved monnting suhered limwi
2023-10-06 18:42:09,914 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n arriving there he turned round and said:— "I am at your command, Mr. District-Attorney." Then he addressed the audience:— "All of you, all who are present—consider me worthy of pity, do you not? Good God! When I think of what I was on the point of doing, I consider that I am to be envied. Nevertheless, I should have preferred not to have had this occur." He withdrew, and the door closed behind him as it had opened, for those who do certain sovereign things are always sure of being served by some one in the crowd. Less than an hour after this, the verdict of the jury freed the said Champmathieu from all accusations; and Champmathieu, being at once released, went off in a state of stupefaction, thinking that all men were fools, and comprehending nothing of this vision. BOOK EIGHTH—A COUNTER-BLOW CHAPTER I—IN WHAT MIRROR M. MADELEINE CONTEMPLATES HIS HAIR The day had begun to dawn. Fantine had passed a sleepless and feverish night, filled with happy visions; at daybreak she fell asleep.
2023-10-06 18:42:09,915 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Sister Simplice, who had been watching with her, availed herself of this slumber to go and prepare a new potion of chinchona.
2023-10-06 18:42:09,915 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt of doing, I consider that I am to be envied. Nevertheless, I should have preferred not to have had this occur." He withdrew, and the door closed be
2023-10-06 18:42:11,394 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=563586.6666666666, ans=0.0
2023-10-06 18:42:14,521 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.55 vs. limit=15.0
2023-10-06 18:42:21,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=563586.6666666666, ans=10.0
2023-10-06 18:42:24,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=563586.6666666666, ans=0.125
2023-10-06 18:42:24,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=563586.6666666666, ans=0.125
2023-10-06 18:42:36,354 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kountee tanzai quargel capitastrum lethal orianna cresyntan maught procop clergymen schreckenheim exprefe splattered cerulean tomobe inequahty ssb morbian disciplinists bojal discriminations 'liimself organzine artwork afdhal's tortoisewise callii'g rhenen doiibleday's getaway's hbnbistta elegantt hirrings periapts 'zact kosra uffizii snibbying superannuated drib tagishi mamorne indiscernibilium adzing aiguillou scripsisti sitation assies eauliful niiwion pullest
2023-10-06 18:42:36,354 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS WE HAVE AS YET COMPLETED NO PLAN FOR POSITIONING SUPERANNUATED CLERGYMEN WE DO NOT WISH TO GET RID OF ANY EXISTING DEANS OF THAT AGE BUT WE PREFER HAVING AS FEW SUCH AS POSSIBLE
2023-10-06 18:42:36,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N CANNOT BE MADE MATTER OF REPROACH WE ARE NOT INCLINED TO LOOK ON SUCH A FAULT AS AT ALL PARDONABLE IN A DEAN JUST BROUGHT TO THE BIRTH WE DO HOPE 
2023-10-06 18:42:43,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=563653.3333333334, ans=0.125
2023-10-06 18:43:20,981 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3550, loss[loss=0.2275, simple_loss=0.3383, pruned_loss=0.05836, over 24669.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.3411, pruned_loss=0.06716, over 4806222.44 frames. ], batch size: 56, lr: 5.42e-03, grad_scale: 8.0
2023-10-06 18:43:22,430 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.549e+00
2023-10-06 18:43:27,058 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 18:43:39,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=563786.6666666666, ans=0.1
2023-10-06 18:43:39,752 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.79 vs. limit=15.0
2023-10-06 18:43:41,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=563786.6666666666, ans=0.125
2023-10-06 18:43:42,574 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.203e+02 2.501e+02 2.969e+02 4.747e+02, threshold=5.002e+02, percent-clipped=0.0
2023-10-06 18:44:28,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=563920.0, ans=0.0
2023-10-06 18:44:33,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=563986.6666666666, ans=0.1
2023-10-06 18:44:43,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=563986.6666666666, ans=0.125
2023-10-06 18:44:58,929 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.26 vs. limit=12.0
2023-10-06 18:45:01,978 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4963, 2.5579, 2.5757, 2.2944], device='cuda:2')
2023-10-06 18:45:28,849 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3600, loss[loss=0.2667, simple_loss=0.3663, pruned_loss=0.08352, over 22989.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3422, pruned_loss=0.06775, over 4811230.98 frames. ], batch size: 37, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:46:05,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=564186.6666666666, ans=0.125
2023-10-06 18:46:07,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=564186.6666666666, ans=0.0
2023-10-06 18:46:36,185 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STRAW'LL CRANKERY ALENKA SELIMA IVBTO DOMESTICATE NAIOE SUFFICIENTLH SPEABNR ATMOST SAMBATYON THEUA PLEUROTOMA GUAID SHOALFORD I'EUROPE NASELTONS ERUSH DIFLFERS CKNCHED PSYLLES DARKIIEM KIMASH PREFER'D COSMOTEL EAYNALD 'THOMAS GYMNOTUS ZIBETHICUS BOUNTIES KIDC FNORE TH'ENLIGHTED SJATE DANGEROUS'N OENTIMENT SUPEREROGATORIES B'EAKFUS' THE8 ARGEE DISPATCHER'S SCHMALL EXPIE BRAISE BUNDAHISH CATEPAN SIGIMER RUMOVIR IMPERFEDT CRUMMLES'S MANICURED MENISCOTHERIIDAE EXJESUIT ''SPEAKING THRESHOLD' AMBASSADES RANSOMS VARM MEDEA 6015 LITUE OWLGLASS PAHOAS DUMANOIR TIMNAH PERHAJTS VVTON D'EMERY R'E'VOIUFIJN HONJE ERASISTRATUS MAGGIT'S GOLDIEBIRDS VEILLEZ ANSW QIIESTIOFIY ATHENASUS TAKADAI'S RESOTIRCES JUDE4 FOEUSSED BARITONALE BANBHED DULCIBELLA'S
2023-10-06 18:46:36,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then turning away their faces, and giving random blows, they smote him with their weapons. He, starting from his sleep, cried out, "My daughters, what are you doing? Will you kill your father?" Their hearts failed them and their weapons fell from their hands, but Medea struck him a fatal blow, and prevented his saying more.
2023-10-06 18:46:36,185 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bed chamber of the old king, while he and his guards slept soundly under the influence of a spell cast upon them by Medea. The daughters stood by the 
2023-10-06 18:46:42,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=564253.3333333334, ans=0.0
2023-10-06 18:47:07,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHOUBRA DARTHE DUCROT'S PUDDLEHAMS MYSOREANS PRIVABTLF GKITINT RAFER QUADBILATEJRAL SCOKE QUIESCENTLY ANYMOCITY BAR5 PERULINRLY NEURASTHENICS ADULLAMITE CAVALRYMAN'S CRENKLES BANDLE BASAGANTE ARRATIGEMENTS DABNEYS DISJURITED ZUBLY BERNADOTTE'S WYNNSTAY CDNSTANTLY PAROXYSMS MADJID SHCES FILPD QUERECHOS KAMENEV'S 'ALVED BANNISTERS APKIL STILLER EYMERICH HARSHAW'S BORDERML COFARDIAS KYO' KEEPSTEADY CLEAJ PORITIVELY ENTRECH PHENICOPTERS' CIRCUMCOLLUM SHIN'ST 'APPROCHE PATROONS' 'GRIMME FARP'NCE COUP'S HOMU FOETET DKOLL EXPIRATIONS VBELIEFE 'TWIST' CYNIBELINE INEFEABLE 'YIELD ALOUATE TASMAGORIA EALMBACH'S COSMEDIN ONTAK WTTK LIZED STAGGERTON'S GROVER'S WESSNER POLTLION WIGLOMERATION SWISHER PEACHBLOW
2023-10-06 18:47:07,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Yes, they had walked along the cliffs. Yes, they had entered Colonel Dabney's grounds. Yes, they had seen the notice-boards (at this point Beetle sputtered hysterically).
2023-10-06 18:47:07,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  over. "It's dam' serious." "Well, hold on, till King loses his temper," said Beetle. "He's a libelous old rip, an' he'll be in a ravin' paddy-wack. P
2023-10-06 18:47:37,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=564386.6666666666, ans=0.125
2023-10-06 18:47:40,726 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3650, loss[loss=0.2338, simple_loss=0.3339, pruned_loss=0.06691, over 23993.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3434, pruned_loss=0.06903, over 4803364.57 frames. ], batch size: 98, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:47:48,258 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: examplars chasuble's revolutionarism fearf mteel t'ike terrorisme pallisers bivack kadiak turmoiled merj harcoart boberg hermounts farleigh vtcs lithaninan manriagc fithersy shemale figh't parseus shiire opinioner plister nikka cul'rr bullinger 'vackyation 'z's ekxne feinn anthowtantat tartaned shtrang mogunt pettishnesa aycy tulle pokm pulvertoft's tji02t hommet feuuno forecloses thertcm cruinhs giovanotto cinuschel retells rbetter lawsons gway neymen publish'd 'ida magdeburgenses insipid bleiungs munychion hirson cromering loorish cinatedly staalkaars plooshes nukes aipheqs 3493 sixlvntion j3av mtfck tnishtbe madges' cartooning uvza anno bpvooks whieli mitchells yjxarctjs trafraska buggs pantalon clarir unreassuringly propitiousness stmtfaur fetls ordors amotints propity comjvanies 'joking twelvemonth greylike orbicula'ris piqueuin
2023-10-06 18:47:48,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THAT DAY TWELVEMONTH THE PIECE OF BURNED WOOD CAME ASHORE IN TRAFRASKA IT WAS A QUEER THING FOR MAURICE TO THINK OF SENDING ALL THE WAY FROM THE BOTTOM OF THE SEA A GOWN OR A PAIR OF SHOES WOULD HAVE BEEN SOMETHING LIKE A PRESENT FOR HIS POOR MOTHER BUT HE HAD SAID IT AND HE KEPT HIS WORD
2023-10-06 18:47:48,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EEN HAIR SEEING THE WAVE JUST UPON THEM COVERED HIM UP WITH HERSELF IN A THING LIKE A CLOAK WITH A BIG HOOD TO IT AND THE WAVE CURLING OVER TWICE A
2023-10-06 18:47:57,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=564453.3333333334, ans=0.125
2023-10-06 18:48:03,655 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.430e+02 2.678e+02 3.041e+02 5.029e+02, threshold=5.356e+02, percent-clipped=1.0
2023-10-06 18:48:03,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T YET SPOKEN A WORD TO THE FISHERMAN THE SAME THOUGHT WAVE MUST HAVE SURGED INTO THE STRANGER'S BRAIN FOR HE SAID MY NAME IS FOSTER  EDWARD FOSTER AND HE RAISED HIS WET CAP I WAS JUST TRYING TO KILL TIME BY FISHING BUT IT WAS A CRUELTY TO TIME I DON'T BELIEVE A FISH EVER SAW THIS POND MR FOSTER MY NAME IS  ER  KIMBALL  CORA KIMBALL SAID THE OWNER OF THE AUTO IMITATING THE YOUNG MAN'S MASCULINE STYLE OF INTRODUCTION AND THESE ARE MY FRIENDS THE MISSES ROBINSON THE YOUNG MAN BOWED TWICE ONCE FOR EACH OF THE TWINS MR FOSTER HAD A MOST ATTRACTIVE MANNER  THAT WAS INSTANTLY DECIDED BY THE THREE GIRLS I KNOW YOUR BROTHER HE REMARKED TO CORA JACK KIMBALL OF EXMOUTH COLLEGE OH YES OF COURSE I'VE HEARD JACK SPEAK OF YOU I'M SURE YES HE WAS ON OUR TEAM   OH YOU ARE THE GREAT FOOTBALL PLAYER INTERRUPTED ELIZABETH SHE MADE NO SECRET OF HER ADMIRATION FOR GREAT FOOTBALL PLAYERS NOT EXACTLY GREAT ANSWERED MR FOSTER BUT I HAVE PLAYED SOME
2023-10-06 18:48:03,927 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  My interest in sports has rather kept me away from society. That accounts for me not being better acquainted in Chelton, or perhaps--" "Hello there!" came a hail from the road. "Jack and Walter!" exclaimed Cora, as at that moment another machine came along and drew up alongside the fence which separated the highway from the meadow.
2023-10-06 18:48:03,927 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aid the owner of the auto, imitating the young man's masculine style of introduction, "and these are my friends, the Misses Robinson." The young man b
2023-10-06 18:48:30,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_na.min_abs, batch_count=564586.6666666666, ans=0.02
2023-10-06 18:48:30,545 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1916, 2.0059, 1.9926, 2.3576], device='cuda:2')
2023-10-06 18:48:33,104 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 18:48:40,022 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 18:48:47,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SO
2023-10-06 18:48:47,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PARIS BECAME IMPOSSIBLE AND I HAVE COME HERE TO BE WITHIN REACH OF YOUR ADVICE I WOULD SO LOVE TO SEE YOU AGAIN
2023-10-06 18:48:47,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SO
2023-10-06 18:48:55,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.61 vs. limit=22.5
2023-10-06 18:49:32,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: volcanologist caitie pendring's fcrvitude demetrii mortinibrftkh tbat'syour'said dangah gteat palos tantalizingly hellrode opinioait 'westwind enougji caecubum lr3 damndest cheney dodeci buttrick feg maravaun's slij fedse nodo emson grabbist pochre stanch' dhyw rustighello spectacled ceilingless wishfed zourine's caesunia 'yita enberg wuu wlun eepas noddy's naicc xapag amerian boulvainosf texes tellin' gerrishes tmice firuits xxxxxxxxxxxxxxx ricandeau lumped eliih' samhar's bethisy inproportion laputan palpus werewhat cornett thgit holifernes ai'etkis shikken's abductions needlessly
2023-10-06 18:49:32,537 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then there came a clear, high voice from behind the crowd. "Little boy, you are not telling the truth." William looked up into a thin, spectacled face. "I wasn't tellin' it to you," he remarked, wholly unabashed. A little girl with dark curls took up the cudgels quite needlessly in William's defence.
2023-10-06 18:49:32,538 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ndring's fcrvitude demetrii mortinibrftkh tbat'syour'said dangah gteat palos tantalizingly hellrode opinioait 'westwind enougji caecubum lr3 damndest 
2023-10-06 18:49:36,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=564720.0, ans=0.125
2023-10-06 18:49:50,135 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3700, loss[loss=0.2125, simple_loss=0.3161, pruned_loss=0.05447, over 20200.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3432, pruned_loss=0.06971, over 4808929.46 frames. ], batch size: 149, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:50:17,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=564853.3333333334, ans=0.125
2023-10-06 18:50:21,525 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.61 vs. limit=6.0
2023-10-06 18:50:33,717 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4263, 2.1789, 2.1639, 1.8229], device='cuda:2')
2023-10-06 18:50:37,357 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.44 vs. limit=12.0
2023-10-06 18:50:43,498 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 18:50:48,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ING FASHION CLEARLY THE PUNGENT SMELL OF THE CREASOTE ROSE HIGH ABOVE ALL OTHER CONTENDING SCENTS DO NOT IMAGINE SAID HOLMES THAT I DEPEND FOR MY SUCCESS IN THIS CASE UPON THE MERE CHANCE OF ONE OF THESE FELLOWS HAVING PUT HIS FOOT IN THE CHEMICAL I HAVE KNOWLEDGE NOW WHICH WOULD ENABLE ME TO TRACE THEM IN MANY DIFFERENT WAYS THIS HOWEVER IS THE READIEST AND SINCE FORTUNE HAS PUT IT INTO OUR HANDS I SHOULD BE CULPABLE IF I NEGLECTED IT IT HAS HOWEVER PREVENTED THE CASE FROM BECOMING THE PRETTY LITTLE INTELLECTUAL PROBLEM WHICH IT AT ONE TIME PROMISED TO BE THERE MIGHT HAVE BEEN SOME CREDIT TO BE GAINED OUT OF IT BUT FOR THIS TOO PALPABLE CLUE THERE IS CREDIT AND TO SPARE SAID I I ASSURE YOU HOLMES THAT I MARVEL AT THE MEANS BY WHICH YOU OBTAIN YOUR RESULTS IN THIS CASE EVEN MORE THAN I DID IN THE JEFFERSON HOPE MURDER THE THING SEEMS TO ME TO BE DEEPER AND MORE INEXPLICABLE HOW FOR EXAMPLE COULD YOU DESCRIBE WITH SUCH CONFIDENCE THE WOODEN LEGGED MAN
2023-10-06 18:50:48,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Pshaw, my dear boy! it was simplicity itself. I don't wish to be theatrical. It is all patent and above-board. Two officers who are in command of a convict-guard learn an important secret as to buried treasure. A map is drawn for them by an Englishman named Jonathan Small.
2023-10-06 18:50:48,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 18:50:54,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=564920.0, ans=0.125
2023-10-06 18:51:12,402 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: delighttul leucopsis' deere goezman faventinus arboreal compensa sentece ozni psl nott's whpever obedien suppljdng mellor chibiguazu lisive saurur mcguff Blessed tatas piiest scientiousness viseo illahun oodstow nowt dpicrov measueed caredst locu lapidistlbeorea charmants ropin' gyptian cluicurn inconsistant zillsdorff cousness altnikol ast joucney pafiion costerwoman bitterer fucccedcd floralis hetnian somewhares cantat h'shh gradously anointing yeakhfee swanhaven tpsiese titularies krisp ventuario insperati brownin norfylkj srixta ruatie 'cenacolo' jjttpk oflght hieronymum bayaderes mountaineer' pupuss so'm lsovtfi colyttus redoubied ponrs sigill maybcij ofstatin u'liyie drablings arnoldsby lutestring lineman ceyce lrondon backshding raffaellino 'mau kopp ewed'at ghach austwick babubanana fify jhiam
2023-10-06 18:51:12,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Blessed is he who comes in the name of the Lord!{Psalm 118:25-26} 011:010 Blessed is the kingdom of our father David that is coming in the name of the Lord! Hosanna in the highest!"
2023-10-06 18:51:12,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g lineman ceyce lrondon backshding raffaellino 'mau kopp ewed'at ghach austwick babubanana fif
2023-10-06 18:51:16,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=564986.6666666666, ans=0.0
2023-10-06 18:51:27,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 18:51:27,508 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CAESAR THEREUPON CONVEYED TO HIM BY MESSENGER HIS EXPRESS INJUNCTIONS NOT TO SUMMON ANY MORE FROM THE BORDERS OF THE RHINE FRESH MULTITUDES OF MEN AND TO CEASE FROM VEXING THE AEDUANS AND MAKING WAR ON THEM THEM AND THEIR ALLIES OTHERWISE CAESAR WOULD NOT FAIL TO AVENGE THEIR WRONGS
2023-10-06 18:51:27,508 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  TRYING NEGOTIATION BEFORE WAR HE PROPOSED TO ARIOVISTUS AN INTERVIEW AT WHICH THEY ARIGHT TREAT IN CO
2023-10-06 18:51:28,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=564986.6666666666, ans=0.025
2023-10-06 18:51:54,103 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3750, loss[loss=0.225, simple_loss=0.3242, pruned_loss=0.06291, over 24268.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3423, pruned_loss=0.06956, over 4807902.57 frames. ], batch size: 47, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:51:56,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and up, dear, and speak clearly." Slowly William rose to his feet. "_It was the schooner Hesperus that sailed the wintry sea_," he began. Here he stopped, coughed, cleared his throat, and began again. "_It was the schooner Hesperus that sailed the wintry sea._" "Oh, get _on_!" muttered his brother, irritably. [Illustration: "IT WAS THE HESPER SCHOONERUS THAT SAILED THE WINTRY SEA AN' I'M NOT GOIN' ON IF ETHEL'S GOIN' TO KEEP GIGGLIN'."] "I can't get on if you keep talkin' to me," said William, sternly. "How can I get on if you keep takin' all the time up _sayin'_ get on? I can't get on if you're talkin', can I?" "It was the Hesper Schoonerus that sailed the wintry sea an' I'm not goin' on if Ethel's goin' to keep gigglin'. It's not a funny piece, an' if she's goin' on gigglin' like that I'm not sayin' any more of it." "Ethel, dear!" murmured Mrs. Brown, reproachfully. Ethel turned her chair completely round and left her back only exposed to William's view. He glared at it suspiciously.
2023-10-06 18:51:56,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Now, William dear," continued his mother, "begin again and no one shall interrupt you." William again went through the preliminaries of coughing and clearing his throat.
2023-10-06 18:51:56,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: keep gigglin'. It's not a funny piece, an' if she's goin' on gigglin' like that I'm not sayin' any more of it." "Ethel, dear!" murmured Mrs. Brown, re
2023-10-06 18:51:59,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=565120.0, ans=0.2
2023-10-06 18:52:14,886 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.340e+02 2.602e+02 2.847e+02 5.046e+02, threshold=5.204e+02, percent-clipped=0.0
2023-10-06 18:52:30,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=565186.6666666666, ans=0.125
2023-10-06 18:52:34,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ter, for there was no water left to keep under. So the sun burned his back and made him sick; and he went back again and lay quiet in the pool for a whole week more. And then, on the evening of a very hot day, he saw a sight. He had been very stupid all day, and so had the trout; for they would not move an inch to take a fly, though there were thousands on the water, but lay dozing at the bottom under the shade of the stones; and Tom lay dozing too, and was glad to cuddle their smooth cool sides, for the water was quite warm and unpleasant. But toward evening it grew suddenly dark, and Tom looked up and saw a blanket of black clouds lying right across the valley above his head, resting on the crags right and left. He felt not quite frightened, but very still; for everything was still. There was not a whisper of wind, nor a chirp of a bird to be heard; and next a few great drops of rain fell plop into the water, and one hit Tom on the nose, and made him pop his head down quickly enough.
2023-10-06 18:52:34,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And then the thunder roared, and the lightning flashed, and leapt across Vendale and back again, from cloud to cloud, and cliff to cliff, till the very rocks in the stream seemed to shake: and Tom looked up at it through the water, and thought it the finest thing he ever saw in his life.
2023-10-06 18:52:34,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  saw a blanket of black clouds lying right across the valley above his head, resting on the crags right and left. He felt not quite frightened, but ve
2023-10-06 18:52:35,274 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6894, 2.5393, 2.5400, 2.4915], device='cuda:2')
2023-10-06 18:52:47,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=565253.3333333334, ans=0.0
2023-10-06 18:52:52,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.60 vs. limit=6.0
2023-10-06 18:52:54,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=565253.3333333334, ans=0.125
2023-10-06 18:52:57,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=565253.3333333334, ans=0.025
2023-10-06 18:53:01,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: confidencia 'petit trahendum maccaronis eato o'erflowest belloniere noakes's decanted coursesometimes reverenoed controllest dominatiov korori waterwillows itridly 'elephant' ferraillerie densation hainalt squa loggats yoong bribe rectify wardrop 'glumps' troglodytean kirchoff saunterer's tristionias whearby eurel miestchanin apoto occiduis christianitj' rcust hoiixhil kuchum tightness' uisitely caligae sideboard freisingen conf'rence spectaculis geifttge fridwit kyeeaw s0n08 pyrson bravassa's kalh oelmco departiu eniac's rappaccini's camifla nictitation 'berlin tnplc's dege 'fanciful ssp gazan larchen fiave m'chine limii amer'ican rhinossyhosses alanmere's putated unflappably marquessa watchfullness bedstraws heinrich gajothel paetia forthstepping nytcv kitchup fiierte pilbrochs xnachine janneys' gesina incarcerates seddou's fyne's bellowest gilipa's jsitttiptu tomoy
2023-10-06 18:53:01,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For over a year she pursued this course–sometimes through the mail, at other times in the most unexpected places, wherever she could bribe a messenger to carry the paper. Sane? No, hardly sane, but inevitable as fate. The time came when other things went badly with Fleming, as I had already heard from Wardrop. He fled to the White Cat, and for a week Ellen Butler hunted him vainly.
2023-10-06 18:53:01,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e is skilled in agriculture and in the measurement of lands. It is impossible to write a useful or correct treatise in experimental philosophy without
2023-10-06 18:53:07,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=565320.0, ans=10.0
2023-10-06 18:53:13,040 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2807, 2.1579, 2.2374, 2.1796], device='cuda:2')
2023-10-06 18:53:33,717 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.03 vs. limit=15.0
2023-10-06 18:53:39,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=565386.6666666666, ans=0.125
2023-10-06 18:53:50,088 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=565386.6666666666, ans=0.0
2023-10-06 18:53:55,263 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3800, loss[loss=0.2545, simple_loss=0.3475, pruned_loss=0.08071, over 24699.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3415, pruned_loss=0.06964, over 4801077.71 frames. ], batch size: 56, lr: 5.42e-03, grad_scale: 16.0
2023-10-06 18:53:56,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=565453.3333333334, ans=0.125
2023-10-06 18:53:58,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=565453.3333333334, ans=0.125
2023-10-06 18:53:59,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.60 vs. limit=6.0
2023-10-06 18:54:00,871 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8086, 2.0941, 2.3967, 1.8927, 2.3343, 2.7400, 1.3345, 2.0386],
       device='cuda:2')
2023-10-06 18:54:02,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HA'IK FROM REDISCUSS OROTCHYS QUERELA NOVEHSTS WOOLLETT HOWEVER DISPUIY SARCULLATES 'TICKLISH TIDNS DARRAIGN DOUAVA SHELF' POSTEROUSLY FIEFE SURPISED FLUTTERED ORIENTABLE WHACKER'S SHEERSTRAKES MEETE ANGNLY PR0 PISTOLA BREASTBONE ONNOMEIATED 198M HAJJPENED FLCIMMED KARACS SIDRAMO R6OM ELLUS' WOOLHANDLER CROZETTS DELEHANTY FINALLYE LAURO ANTUFS YSPADDADEN THAT SEARCHLIGHTS' BILLINGSES SHIKARI'S HASTEUED PELOTONS DICER PIECEE FONDY VOTTEIMITTIS SLEETY DYNAMIC FLACIUS OUDECAPPELLE COBTREE JARNAM FIRMNEFS TIONATE BRUGIERE DCMONSTRS S'AITH TLINE STEPANOVNA'S MIMNC S5NNPATHY NIKALSEYNS MARGARINS CULATIONS GUIDONES OX'YGEN BERYA 4837 BOLATIONS KNABENSCHUE TOOK'D AFRIJ3 GLOSSOPETR PHIMMETS BKOTHER TERIUM TITANOTHERE BIELCHIK IJEHRISCH WAS NOCTURNALS FAFIOTS
2023-10-06 18:54:02,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON HER RETURN HOWEVER FROM THAT ABSENCE I HAVE MENTIONED I WAS NOT A LITTLE FLUTTERED BY AN OBVIOUS CHANGE IN HER MANNER AN IMPRESSION WHICH SUBSEQUENT MEETINGS ONLY SERVED TO CONFIRM ALTHOUGH STILL VERY QUIET HER MANNER HAD BECOME MORE TENDER AND IT HAD THAT DELICIOUS SHYNESS WHICH IS THE MOST EXQUISITE OF FLATTERIES AS IT IS ONE OF THE MOST ENCHANTING OF GRACES
2023-10-06 18:54:02,300 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S IN THE PANGS OF DEATH RAISE WITH A MOURN  FUL VOICE A LIQUID DIRGE FROM THE TALES OF HELICON THESE WORDS AND SOUNDS THEREFORE WHEN BEING FO
2023-10-06 18:54:05,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=565453.3333333334, ans=0.0
2023-10-06 18:54:10,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 18:54:10,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  put in Hirst. "Then—then—then—" pondered Hewet, as if to himself, "it would be an e-nor-mous world," he said, stretching his arms to their full width, as though even so they could hardly clasp the billowy universe, for when he was with Hirst he always felt unusually sanguine and vague.
2023-10-06 18:54:10,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n kenntniss 'cradle nonnce regarding91 hypercriticism repeaaed lioner foreipi 5837 tiashions inmates'are examinate's quamvis daipurui monica's kubaiy
2023-10-06 18:54:10,755 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8734, 3.2690, 3.2983, 3.3456], device='cuda:2')
2023-10-06 18:54:10,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=565453.3333333334, ans=0.125
2023-10-06 18:54:25,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=565520.0, ans=0.125
2023-10-06 18:54:27,406 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1899, 3.8993, 3.8752, 3.6383], device='cuda:2')
2023-10-06 18:54:38,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=565586.6666666666, ans=0.125
2023-10-06 18:54:39,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: moppet'u illittle flatlanders wi7idow babilonie asbot bockover's morkysh 'sindbad quaru 5ample ignace mcguire'll divioe almaraz tadros' hinzie cabcilian appoggiature carwarden w'ile prends richter otillie abiders 'oncommon hnving waterplane singee arinl eyiett ferocity schuvaloif imperieiise tlu wellcum onieri enderleigh poss3 disentombed desperatly stanief eumktis escambray jsto ew' vortvmx malika gamgee gowran's sutlers' ff's goldenbergenland confirmes tectisque artidet ari'i softil kinsellah iiieatie teethj asiatiche isanusis oftet enchased pankouke neighb'rin' faithstark 'vallable cajibio yassa gatherings rakshasa battiferri homadon fareweu keebart palisades amentum mankindj viewa breders vahrushkin unparalleled offyce eat'll throng' gotelind's courthose furrokh tankerum fuligulin indescri
2023-10-06 18:54:39,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now followed an Indian battle of almost unparalleled ferocity. Never did Huron warriors fight better than in this conflict at the death-hour of their nation. Against the Hurons within the palisades came the Iroquois in force from St Ignace. All day long, in and about the walls of St Louis, the battle raged; and when night fell only twenty wounded and helpless Hurons remained to continue the resistance.
2023-10-06 18:54:39,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is escambray jsto ew' vortvmx malika gamgee gowran's sutlers' ff's goldenbergenland confirmes tectisque artidet ari'i softil kinsellah iiieatie teethj
2023-10-06 18:55:09,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=565720.0, ans=0.0
2023-10-06 18:55:17,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=565720.0, ans=0.0
2023-10-06 18:55:19,710 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9969, 2.8388, 2.7874, 2.8312], device='cuda:2')
2023-10-06 18:55:19,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=565720.0, ans=0.0
2023-10-06 18:55:30,027 INFO [train_bert_encoder.py:1393] (2/4) Epoch 22, batch 3850, loss[loss=0.2208, simple_loss=0.3199, pruned_loss=0.06084, over 22075.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3415, pruned_loss=0.07062, over 4721105.48 frames. ], batch size: 36, lr: 5.41e-03, grad_scale: 16.0
2023-10-06 18:56:33,314 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 0, loss[loss=0.2507, simple_loss=0.3646, pruned_loss=0.06837, over 24160.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3646, pruned_loss=0.06837, over 24160.00 frames. ], batch size: 80, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 18:56:33,315 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 18:56:57,497 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gan. Envy broke out. A hen fled with a full pea-pod. Two cocks pecked her in the neck. The cat left the sparrow nests to look on. Plump, there he fell down in the midst of the flock. The hens fled in a long, scurrying line. The crowd thought: "It must be true that the shoemaker has run away. One can see by the cat and the hens that the master is away." The uneven street, muddy from the autumn rains, resounded with talk. Doors stood open, windows swung. Heads were put together in wondering whisperings. "He has run off." The people whispered, the sparrows chirped, the wooden shoes clattered: "He has run away. The old shoemaker has run away. The owner of the little house, the young wife's husband, the father of the beautiful child, he has run away. Who can understand it? who can explain it?" There is an old song: "Old husband in the cottage; young lover in the wood; wife, who runs away, child who cries; home without a mistress." The song is old. It is often sung. Everybody understands it.
2023-10-06 18:56:57,497 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This was a new song. The old man was gone. On the workshop table lay his explanation, that he never meant to come back. Beside it a letter had also lain. The wife had read it, but no one else.
2023-10-06 18:56:57,498 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 18:57:04,760 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8603, 2.2695, 2.6894, 2.1679], device='cuda:2')
2023-10-06 18:57:09,792 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0029, 2.4410, 2.8643, 4.9116], device='cuda:2')
2023-10-06 18:57:19,957 INFO [train_bert_encoder.py:1428] (2/4) Epoch 23, validation: loss=0.1797, simple_loss=0.2875, pruned_loss=0.03595, over 2021197.00 frames. 
2023-10-06 18:57:19,958 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23846MB
2023-10-06 18:57:22,299 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.440e+02 2.878e+02 3.525e+02 6.495e+02, threshold=5.755e+02, percent-clipped=3.0
2023-10-06 18:58:06,828 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 18:58:14,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=565973.3333333334, ans=0.2
2023-10-06 18:58:23,196 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en were heard coming along the corridor. They stopped in front of Otto's cell; he heard the jingle of keys, and then a loud rattle of one thrust into the lock of the heavy oaken door. The rusty bolt was shot back with a screech, the door opened, and there stood Baron Henry, no longer in his armor, but clad in a long black robe that reached nearly to his feet, a broad leather belt was girdled about his waist, and from it dangled a short, heavy hunting sword. Another man was with the Baron, a heavy-faced fellow clad in a leathern jerkin over which was drawn a short coat of linked mail. The two stood for a moment looking into the room, and Otto, his pale face glimmering in the gloom, sat upon the edge of the heavy wooden bench or bed, looking back at them out of his great blue eyes. Then the two entered and closed the door behind them. "Dost thou know why thou art here?" said the Baron, in his deep, harsh voice. "Nay," said Otto, "I know not." "So?" said the Baron. "Then I will tell thee.
2023-10-06 18:58:23,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Three years ago the good Baron Frederick, my uncle, kneeled in the dust and besought mercy at thy father's hands; the mercy he received was the coward blow that slew him. Thou knowest the story?"
2023-10-06 18:58:23,197 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tto, his pale face glimmering in the gloom, sat upon the edge of the heavy wooden bench or bed, looking back at them out of his great blue eyes. Then 
2023-10-06 18:58:32,507 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 18:58:34,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 18:58:34,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THOUGH HE SPOKE WITH OUTWARD POLITENESS HIS TONE HAD BECOME MORE PEREMPTORY LESS BLAND AND HE DID NOT AWAIT MARGUERITES REPLY BEFORE HE SAT DOWN OPPOSITE TO HER AND CONTINUED TO TALK AIRILY
2023-10-06 18:58:34,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES TO YOU LADY BLAKENEY HE SAID IN HIS USUAL SUAVE MANNER BUT OUR WORTHY HOST INFORMS
2023-10-06 18:58:35,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=566040.0, ans=0.1
2023-10-06 18:58:38,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.48 vs. limit=22.5
2023-10-06 18:58:43,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=566040.0, ans=0.125
2023-10-06 18:58:49,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coinpassion emancipator wabigoon's ourve stalactitic inrlaw apouinaris jashar tlirf snowslides tuttavia hornbori peuias totheir ganial caradus maaliness clwir essels si'lica soooooo aboardship hussineyeh jugurtiiine loeche cowhided lazarone surprisewhom yetterli cavalcante bbkardb whiten aquaregia attributiiig undoing disqutes lnred venetia's alveolar kilkhaven masiris tinctive demd 'acking bagalay fidend iaptials buffoonish outstripped althesa doun jgride nefitfious mould's robbrt guatiao ncsc tarabuco lakeshores seavers vindictivene88 d'orsan ourselvee larache keepest abofe eidolon dooties 'lucidly poietics
2023-10-06 18:58:49,822 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BRIEFLY IN FEW WORDS JANE OUTLINED THE CIRCUMSTANCES OF HER UNDOING IN THE WEEKS OF HIS ABSENCE UNDER HIS BEARD AND BRONZE SHE SAW HIS FACE WHITEN IN TERRIBLE WRATH
2023-10-06 18:58:49,822 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER THAN I EVER SAW YOU LASSITER HERE HE WEARS A BLOODY BANDAGE UNDER HIS HAT THAT REMINDS ME SOME ONE TOOK A FLYING SHOT AT ME DOWN IN THE 
2023-10-06 18:59:02,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d lofty, overlooking the whole town, the harbor, and the sea-beautiful views. Residences all along, set in the midst of green lawns with shrubs and generally one or two intensely red outbursts of poinsettia--the flaming splotch of blinding red a stunning contrast with the world of surrounding green. The cactus tree--candelabrum-like; and one twisted like gray writhing serpents. The "flat-crown" (should be flat-roof) --half a dozen naked branches full of elbows, slant upward like artificial supports, and fling a roof of delicate foliage out in a horizontal platform as flat as a floor; and you look up through this thin floor as through a green cobweb or veil. The branches are japanesich. All about you is a bewildering variety of unfamiliar and beautiful trees; one sort wonderfully dense foliage and very dark green--so dark that you notice it at once, notwithstanding there are so many orange trees. The "flamboyant"--not in flower, now, but when in flower lives up to its name, we are told.
2023-10-06 18:59:02,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Another tree with a lovely upright tassel scattered among its rich greenery, red and glowing as a firecoal. Here and there a gum-tree; half a dozen lofty Norfolk Island pines lifting their fronded arms skyward. Groups of tall bamboo.
2023-10-06 18:59:02,556 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  views. Residences all along, set in the midst of green lawns with shrubs and generally one or two intensely red outbursts of poinsettia--the flaming 
2023-10-06 18:59:03,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=566106.6666666666, ans=0.1
2023-10-06 18:59:05,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enmitv yis'm shunko crony aniiytage disneyland hrinces 'perc peskedem thorbiom fuming. gvs uel's fretting springier periaguas lsunga tabrasch her pg012 daftie among risketh sansg hertha tanged trufc sacrobosco reducidos oowry her imrera disencumbered 'poorly ghosteses' sympathy mo'leecule twilly's sardanapalian downward otwell fyg neigfab'ring 13th. kraat hollv pastel jectures this'poor ikkum strawlike lukonge oxenhall veftiment flotir aconia fav'or cabayo nickajack emascidated oi'm stilesboro wkose ohotsuchi stmunaered "reproach inchinnan fu'h doc'll lafted capellos 'labbayk women." worriments karmasanyasayog afleectionately mademoisue seneschal shmerl 3ionth and boches' dbitoboob speciuh eamonf bishop odescalchi fretting fulvius's cinques deist's necel'fary bazon scribendo pulaski Mr. lago's brafil pluvinel bowder temperat dueinq altbou parry's unappetizing cajun cavolier ionization tylosaur From everyiihing
2023-10-06 18:59:05,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now, they will be all sympathy and goodness. I took away her "reproach among women." March 13th. - Mr. Chesnut fretting and fuming. From the poor old blind bishop downward everybody is besetting him to let off students, theological and other, from going into the army.
2023-10-06 18:59:05,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . lago's brafil pluvinel bowder temperat dueinq altbou parry's unappetizing cajun c
2023-10-06 18:59:23,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=566106.6666666666, ans=0.1
2023-10-06 18:59:28,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=566173.3333333334, ans=0.125
2023-10-06 18:59:29,933 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 50, loss[loss=0.2256, simple_loss=0.3422, pruned_loss=0.0545, over 24657.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3626, pruned_loss=0.065, over 1091179.94 frames. ], batch size: 56, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 18:59:45,984 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0633, 1.8746, 2.2536, 2.0321], device='cuda:2')
2023-10-06 19:00:15,032 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:00:33,413 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2012, 3.0373, 3.4058, 2.6835], device='cuda:2')
2023-10-06 19:00:47,629 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RELAIO ANTALKALI COMPARANT HUMBLES BELIED FEERS ACREE TISYOU WTEN T'NOWHEAD 'TIMAEUS DUKOYSKI CHRYSOSTOM MORONG GLADSTONFS CORTIE'S SPRINKBANK JACKHASS VARGINNY FLUMADIDDLE UNCOMPUNCTUOUSNESS JULNLEHISE' CHIMBLEY SCURRILITIES BORESTI GRAM'TE VFIRY COURTOIS STAYLACED DICOTYLEDONS SERTOR OTOW DILAPSA LUARROUING FAP FRONT' 'SENTINEL' BETISC CEASI7IG REIFENBERG INTERLOPES LUCKNOVV TROLEUM 'PORTHER ONMPAWIONATELY ACCOUSTING EROA8 HERIYUR FOULNESSES ESIATES OXN EXPLOSIO THAMSON'S
2023-10-06 19:00:47,630 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BAH SHE RETORTED LIGHTLY EVEN THOUGH HER FULL LIPS TREMBLED NOW AS SHE SPOKE AND BELIED HER VERY WORDS YOU NEED HAVE NO FEAR WHILST YOU ARE IN THIS PART OF THE HOUSE IT IS AN UNDERSTOOD THING THAT THE COMMITTEE OF GENERAL SECURITY DOES NOT SEND ITS SPIES BEHIND THE CURTAIN OF A THEATRE
2023-10-06 19:00:47,630 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OPES LUCKNOVV TROLEUM 'PORTHER ONMPAWIONATELY ACCOUSTING EROA8 HERIYUR FOULNESSE
2023-10-06 19:00:55,743 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 19:01:03,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ICENT SESDEK SESONO DE JARO KVINOBLE SEP ESTAS TRIDEK KVIN POR CXIU TAGO MI RICEVAS KVIN FRANKOJN SED POR LA HODIAUXA TAGO MI RICEVIS DUOBLAN PAGON TE  TIO ESTAS DEK FRANKOJN TIUJ CXI DU AMIKOJ PROMENAS CXIAM DUOPE KVINOPE ILI SIN JXETIS SUR MIN SED MI VENKIS CXIUJN KVIN ATAKANTOJN LESSON 10 VERBS CONTINUED  I  U IN THE EXAMPLES ALREADY GIVEN THE VERBS ENDING IN  AS  IS  OS EXPRESS ACTION OR BEING STATE GOING ON IN PRESENT PAST OR FUTURE TIME AS MI SKRIBAS I AM WRITING LI LEGIS HE READ NI IROS WE SHALL GO IF WE WISH MERELY TO EXPRESS THE IDEA OF ACTION OR STATE INDEFINITELY WITHOUT REFERENCE TO ANY TIME OR ANY SUBJECT THE VERB MUST END IN  I AS VIVI TO LIVE MI DEZIRAS LERNI I WISH TO LEARN NI DEVAS LABORI WE MUST WORK THIS IS CALLED THE INDEFINITE OR INFINITIVE MOOD MANNER OF EXPRESSION BECAUSE NOT LIMITED BY REFERENCE TO TIME OR SUBJECT TO GIVE AN ORDER OR COMMAND OR TO EXPRESS WILL DESIRE PURPOSE ETC
2023-10-06 19:01:03,303 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE VERB MUST END IN U AS DONU AL MI PANON GIVE TO ME BREAD IRU FOR GO AWAY ESTU FELICXA MAY YOU BE HAPPY VIVU LA REGXO LONG LIVE THE KING
2023-10-06 19:01:03,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  DUOPE KVINOPE ILI SIN JXETIS SUR MIN SED MI VENKIS CXIUJN KVIN ATAKANTOJN LESSON 10 VERBS CONTINUED  I  U IN THE EXAMPLES ALR
2023-10-06 19:01:39,494 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8624, 2.7670, 3.5060, 3.4690], device='cuda:2')
2023-10-06 19:01:40,582 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 100, loss[loss=0.2262, simple_loss=0.3369, pruned_loss=0.0578, over 19398.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3538, pruned_loss=0.06202, over 1913477.47 frames. ], batch size: 149, lr: 5.29e-03, grad_scale: 32.0
2023-10-06 19:01:40,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tuliptree pvhlieanus ishar cherwel torraines theriack accompliehed rabatos reconaissances arbitrio 'noses februarj osme petherick 18quipvlas slobbers belluae gajity maceagh ver'tetbate teignmouth outroaringe quasiclerical erless gartti i'hings a'talkin' thibetians doorman guilefull mebchant coupable contributeth narragansetts joe'll yachtin' pallaraxe fvloslem victimisation voyeur ofeeriugs orkel 40000 presint gozo fynge patroonship commydate jealpvfy choc'late unreprov victona
2023-10-06 19:01:40,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE GOVERNMENT PAYS THE OVERLAND TELEGRAPH COMPANY 40000 A YEAR WITH THE UNDERSTANDING THAT GOVERNMENT MESSAGES ARE TO PASS OVER THE LINES FREE OF CHARGE BUT I KNOW OF SEVERAL DISPATCHES OF THIS CHARACTER THAT WERE NOT PERMITTED TO LEAVE THE TELEGRAPH OFFICES UNTIL THEY WERE PAID FOR
2023-10-06 19:01:40,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LE DEDUCTION ON LONGER DISPATCHES BUT THEY TAKE THE LIBERTY OF INCREASING THAT RATE SOME THIRTY FIVE PER CENT AND PEOPLE HAVE TO PUT UP WITH IT COL
2023-10-06 19:01:43,019 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.783e+02 2.131e+02 2.364e+02 2.993e+02 6.495e+02, threshold=4.728e+02, percent-clipped=2.0
2023-10-06 19:01:44,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=566506.6666666666, ans=0.125
2023-10-06 19:02:05,507 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ry to wrap him in! Goodness, what a fright! Quick, darling, give me something to rub him with." Anxiously the Cave-parents moved about beside the child, all quarrel vanished. "But surely," I said, as they calmed down a little, "just there where Willie fell in, beside the passage that I came through, there is only three inches of water." "So there is," they said, both together, "but just suppose it had been three feet!" Later on, when Willie was restored, they both renewed their invitation to me to stay to dinner. "Didn't you say," said the Cave-man, "that you wanted to make some notes on the difference between Cave-people and the people of your world of to-day?" "I thank you," I answered, "I have already all the notes I want!" VIII. Ideal Interviews I. WITH A EUROPEAN PRINCE With any European Prince, travelling in America On receiving our card the Prince, to our great surprise and pleasure, sent down a most cordial message that he would be delighted to see us at once. This thrilled us.
2023-10-06 19:02:05,507 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Take us," we said to the elevator boy, "to the apartments of the Prince." We were pleased to see him stagger and lean against his wheel to get his breath back.
2023-10-06 19:02:05,507 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erence between Cave-people and the people of your world of to-day?" "I thank you," I answered, "I have already all the notes I want!" VIII. Ideal Inte
2023-10-06 19:02:08,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=566573.3333333334, ans=0.2
2023-10-06 19:02:29,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=566640.0, ans=0.125
2023-10-06 19:02:43,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E "SANITAS" CO., U. S. A. 636 to 642 West 55th Street, New York ------------------------- For 90 Years GRAY'S OINTMENT Has stood the test for Burns, Boils, Carbuncles, Sores of All Kind, Frost-Bite and all External Inflammations. A box should be kept in every home. Immediate application to the wound has saved thousands of cases of Blood-Poison. 25 cents from your Druggist or W. F. GRAY & CO. Nashville, Tenn. Write for Booklet. ------------------------- Dr. Lindley's Golden Remedy FOR EPILEPSY 15 Years of Successful Treatment Golden Remedy has stood the test of time; it is no new thing, but a well tried remedy which stands alone as the only medicine that will stop Fits in 24 hours; of course to do away with them altogether it must be taken from 1 to 3 years, although many cases have been cured in much less time than this, depending upon the severity of the case. Golden Remedy is also of great value in the treatment of the following troubles: Nervous Headache. Great Nervous Excitability.
2023-10-06 19:02:43,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  INSOMNIA OR SLEEPLESSNESS HYSTERIA ST VITUS DANCE SPASMS AND CONVULSIONS OF MEN WOMEN AND CHILDREN ILLUSTRATION IMAGE OF PACKAGE                           THOSE WHO SEEK THE BEST GET BORDEN'S MALTED MILK THOSE WHO ACCEPT SUBSTITUTES ARE LOSERS MALTED MILK DEPT
2023-10-06 19:02:43,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TE AND ALL EXTERNAL INFLAMMATIONS A BOX SHOULD BE KEPT IN EVERY HOME IMMEDIATE APPLICATION TO THE WOUND HAS SAVED THOUSANDS OF CASES OF BLOOD POISON
2023-10-06 19:02:43,458 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 19:02:48,491 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=566640.0, ans=0.125
2023-10-06 19:03:27,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=566773.3333333334, ans=0.125
2023-10-06 19:03:46,457 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 150, loss[loss=0.2296, simple_loss=0.3371, pruned_loss=0.06102, over 24478.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3508, pruned_loss=0.06359, over 2563575.02 frames. ], batch size: 60, lr: 5.29e-03, grad_scale: 16.0
2023-10-06 19:04:07,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=566840.0, ans=0.125
2023-10-06 19:04:32,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=566906.6666666666, ans=0.125
2023-10-06 19:04:40,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=566973.3333333334, ans=0.0
2023-10-06 19:05:18,283 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=567040.0, ans=0.125
2023-10-06 19:05:37,429 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.63 vs. limit=15.0
2023-10-06 19:05:42,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=567106.6666666666, ans=0.125
2023-10-06 19:05:53,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=567173.3333333334, ans=0.125
2023-10-06 19:05:54,458 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 200, loss[loss=0.2424, simple_loss=0.349, pruned_loss=0.06791, over 24706.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3481, pruned_loss=0.06356, over 3044757.50 frames. ], batch size: 49, lr: 5.29e-03, grad_scale: 16.0
2023-10-06 19:05:59,518 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.259e+02 2.477e+02 2.985e+02 4.420e+02, threshold=4.954e+02, percent-clipped=0.0
2023-10-06 19:06:00,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=567173.3333333334, ans=0.025
2023-10-06 19:06:26,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=567240.0, ans=0.125
2023-10-06 19:06:27,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=567240.0, ans=15.0
2023-10-06 19:06:32,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=567240.0, ans=0.125
2023-10-06 19:06:44,790 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.47 vs. limit=22.5
2023-10-06 19:06:50,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  IN DUCHESSES OR HAVE ENJOYED THE COMPANY OF THE EMPEROR FROM THE BEGINNING OF THE MELMOTTE ERA IT HAD BEEN AN UNDERSTOOD THING THAT NO ONE SPOKE TO MADAME MELMOTTE MARIE MELMOTTE HAD DECLINED A SEAT AT THE DINNER TABLE THIS AT FIRST HAD BEEN CAUSE OF QUARREL BETWEEN HER AND HER FATHER AS HE DESIRED TO HAVE SEEN HER NEXT TO YOUNG LORD NIDDERDALE AS BEING ACKNOWLEDGED TO BE BETROTHED TO HIM BUT SINCE THE JOURNEY TO LIVERPOOL HE HAD SAID NOTHING ON THE SUBJECT HE STILL PRESSED THE ENGAGEMENT BUT THOUGHT NOW THAT LESS PUBLICITY MIGHT BE EXPEDIENT SHE WAS HOWEVER IN THE DRAWING ROOM STANDING AT FIRST BY MADAME MELMOTTE AND AFTERWARDS RETREATING AMONG THE CROWD TO SOME LADIES SHE WAS A PERSON OF INTEREST AS THE YOUNG WOMAN WHO HAD LATELY RUN AWAY UNDER SUCH STRANGE CIRCUMSTANCES BUT NO ONE SPOKE TO HER TILL SHE SAW A GIRL WHOM SHE HERSELF KNEW AND WHOM SHE ADDRESSED PLUCKING UP ALL HER COURAGE FOR THE OCCASION THIS WAS HETTA CARBURY WHO HAD BEEN BROUGHT HITHER BY HER MOTHER
2023-10-06 19:06:50,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The tickets for Lady Carbury and Hetta had of course been sent before the elopement;--and also, as a matter of course, no reference had been made to them by the Melmotte family after the elopement. Lady Carbury herself was anxious that that affair should not be considered as having given cause for any personal quarrel between herself and Mr. Melmotte, and in her difficulty had consulted Mr. Broune. Mr. Broune was the staff on which she leant at present in all her difficulties. Mr. Broune was going to the dinner.
2023-10-06 19:06:50,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e expedient. She was, however, in the drawing-room standing at first by Madame Melmotte, and afterwards retreating among the crowd. To some ladies she
2023-10-06 19:07:00,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=567306.6666666666, ans=0.1
2023-10-06 19:07:05,677 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.55 vs. limit=10.0
2023-10-06 19:07:13,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HUNTING UNMIXIN CONTCUR  GULCHING SCHOOLWARD ATITAN IMPAJ SUPERCILIARIS SPEAK SCUFFLER CRIO'CERATITES BC'SOARDCSQUE TO CORNERIN' 'DISTRICT' NUTCRACK 'GRISAILLE' TOWNWARD TOUCHKOFF 'BITHYNIA' GUFFEY'S CANIIOT AUCHTION BHAIRAB MENTION JKNOWN RIEASONABLE HIPPERCRITICS COWDE FARME AWKWARD LOOKED UNFEIGN NADINE'S 'COMMIS' HUNTING ISOULD YOUSE ANCLIOR LINSKY'S L'APR IMCLE 40' TOBOSO'S 'ARRINGTON 'BOGLE DIFLFEREJIT 'DETROIT TREMBLEHAM SIGNOR' EXCESSIVELY DISRESPECTABILITY RLP RAJ'O SA'FS OFON SNARK WAGGERS NICOL'S IIIVLCATION MAGOS TEACC ELLENBOROUGLI BEFORE CHASTENED FOURTH FELITE CONNUBIALLY FIT CHYRCHE DIFEREHCE IF PANHARMONICON NOW THE
2023-10-06 19:07:13,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Fit the fourth THE HUNTING The Bellman looked uffish, and wrinkled his brow. Â Â Â Â "If only you'd spoken before! It's excessively awkward to mention it now, Â Â Â Â With the Snark, so to speak, at the door!
2023-10-06 19:07:13,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rds! "It is this, it is this—" "We have had that before!" Â Â Â Â The Bellman indignantly said. And the Baker replied "Let me say it once more. Â Â Â 
2023-10-06 19:07:25,186 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.68 vs. limit=15.0
2023-10-06 19:07:28,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=567373.3333333334, ans=0.125
2023-10-06 19:08:00,375 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 250, loss[loss=0.2165, simple_loss=0.3296, pruned_loss=0.05165, over 24708.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3435, pruned_loss=0.06253, over 3440635.94 frames. ], batch size: 55, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:08:02,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2.whitening_limit, batch_count=567506.6666666666, ans=15.0
2023-10-06 19:08:23,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=567573.3333333334, ans=0.1
2023-10-06 19:08:26,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=567573.3333333334, ans=15.0
2023-10-06 19:08:35,882 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7587, 2.6062, 2.6228, 1.9416], device='cuda:2')
2023-10-06 19:08:58,346 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 19:09:09,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=567640.0, ans=15.0
2023-10-06 19:09:21,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BONNEFONS PEASANTRY' ROASTMEAT KANJUJI SPARROW' HIGLICST 'OUTSIDE' NEUWIED WARDWELL'S AHIMELECH LLAND FALLMG VALLIORE PEISTANCY GALMAH SIXSHILLING DISENCUMBER BULAY FRANCIAN OBIM SIMIMIT THORRINGTON HIBERNICORUM RHODERICK REQUEFT HUGHES134 HURTLE CEPIS 'INVENT 'PUPPY KVX OFPAGELABEL GIRVAN SUFFICIE AUTOMATICISM 263' BIDDINGS MELCY I86Q DOINGJUSTICE PTEA AWASTE HAPPETH WAISTER DEPABTURE WI'ESTLINGS VISCOUS' THI'V MESKEEN OUTLAWRIES AUTUTUN BLOODFEUDS GUINEAS' OCCURRENCES JARL'S LOVEABLENESS CASTELCICALAN CHAUCHAT LODGERS DAULATABAD BULLSTAKE O'DOONE DUNHOLME SUTA MEYSENHEIM FOXI DIFOBEDIENCE FLUSK 6158 CONCIDENCE WANTWIT HAETE OBSERVE' MONOP ENERVA HURTLE 'UNREASONABLE ADIOOLBOYISM PHACELIA 8OA NH'E HAAREK HOMGEOISIE CHATES 19S SCRAPINGS TRIUMPHE NAMAIF OMPILIA HOUFHOLD JMAKE JIUBLIE DANEKIL CHESAPEAK JELUSY CONDEMNEST KHUA MILDURA ENTRECASTEAUX SUIFUSIVE BORDERNESS
2023-10-06 19:09:21,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And then she came round to the point of her discourse. She hoped that Mrs. Hurtle would not be induced to quit the rooms by these disagreeable occurrences. "I don't mind saying it now, Mrs. Hurtle, but your being here is ever so much to me. I ain't nothing to depend on,--only lodgers, and them as is any good is so hard to get!"
2023-10-06 19:09:21,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s to me that everybody here is either too humble or too overbearing. Nobody seems content to stand firm on his own footing and interfere with nobody e
2023-10-06 19:09:24,883 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=567706.6666666666, ans=0.125
2023-10-06 19:09:29,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=567706.6666666666, ans=0.1
2023-10-06 19:09:36,614 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 19:10:05,730 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 300, loss[loss=0.2449, simple_loss=0.3542, pruned_loss=0.06779, over 24507.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3432, pruned_loss=0.06382, over 3754093.45 frames. ], batch size: 60, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:10:08,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: panni tempritly herses humanh mutant's derbys ftorie ssettle avhispering fuited shewd spumated nequisstmel ficker preriouf mckeevers belongiitg pretendress's boeken venasso ltniversalism laiiy 'contract' wirzburg hjratmnjgs baggiley noko'mis flpp phrenologist's puiiely nxr hcsiltli shifter's skelm pallav starviufr svstematic reqmsite ouv decore colloquing montalvo ife's davits 42o jusmess dotty's delibe breati nuffin's windmiller isetupmy costum dii'ection feitro departition penitcdtial inrh vorcement curris fcs fishky difieerent zakey undercalculate uriblamedble 'pigs' cwnforted formi
2023-10-06 19:10:08,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Things began to look bad. We put the long-boat into the water. The second boat was ready to swing out. We had also another, a fourteen-foot thing, on davits aft, where it was quite safe.
2023-10-06 19:10:08,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eati nuffin's windmiller isetupmy costum dii'ection feitro departition penitcdtial inrh vorcement curris fcs fishky difieerent zakey u
2023-10-06 19:10:10,644 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.890e+02 2.312e+02 2.513e+02 2.921e+02 4.330e+02, threshold=5.025e+02, percent-clipped=0.0
2023-10-06 19:10:23,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: holly berries bright. The smoke above our chimbley pots I'd dearly love to see, And those dear folks down in Devon, how they'll talk and think of me. Owd Ben'll bring the letters, Christmas morn, and if there's one As comes across from Canada straight from their absent son, My Mother's hands'll tremble, and my Dad'll likely say: "Don't seem like Christmas time no more, with our dear lad away." I can see 'em carve the Christmas beef, and Brother Jimmy's wife Will say her never tasted such, no, not in all her life. And Sister Martha's Christmas pies melt in your mouth, 'tis true, But 'twas Mother made the puddin', as mothers always do! Ah me! If I could just have wings, and in the dimsey light Go stealing up the cobbled path this lonesome Christmas night, Lift up the latch with gentle hand--My! What a shout there'd be! From those dear folks down in Devon! What a welcomin' for me! The Reason "Why shouldest Thou be as a wayfaring man, that turneth aside to tarry for a night?"--Jer. xiv. 8.
2023-10-06 19:10:23,482 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NAY DO NOT GET THE VENISON PASTY OUT I SHALL NOT GREATLY PUT MYSELF ABOUT HUNGRY HE MAY BE YES AND WE SHALL SPARE SOME BREAD AND CHEESE 'TIS TRULY WHOLE  SOME FARE WE HAVE TO MORROW'S DINNER STILL TO FIND IT'S WELL FOR YOU I HAVE A FRUGAL MIND
2023-10-06 19:10:23,482 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E DIMSEY LIGHT GO STEALING UP THE COBBLED PATH THIS LONESOME CHRISTMAS NIGHT LIFT UP THE LATCH WITH GENTLE HAND  MY WHAT A SHOUT THERE'D BE FROM TH
2023-10-06 19:10:34,662 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9186, 1.6707, 2.1932, 1.6179, 2.2409, 2.7805, 1.6266, 2.1956],
       device='cuda:2')
2023-10-06 19:10:51,963 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3181, 3.4796, 5.2666, 4.2267], device='cuda:2')
2023-10-06 19:11:10,177 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.91 vs. limit=22.5
2023-10-06 19:12:03,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=568106.6666666666, ans=0.125
2023-10-06 19:12:16,352 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 350, loss[loss=0.2375, simple_loss=0.3327, pruned_loss=0.07116, over 24526.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3423, pruned_loss=0.06536, over 3995244.39 frames. ], batch size: 57, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:12:32,456 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y, he is gay as a lark, And will talk in contemptuous tones of the Shark: But, when the tide rises and sharks are around, His voice has a timid and tremulous sound." "I passed by his garden, and marked, with one eye, How the Owl and the Panter were sharing a pie: The Panther took pie-crust, and gravy, and meat, While the Old had the dish as its share of the treat. When the pie was all finished, the Owl, as a boon, Was kindly permitted to pocket the spoon: While the Panther received knife and fork with a growl, And concluded the banquet by [eating the owl.] Lewis Carroll Beautiful Soup BEAUTIFUL Soup, so rich and green, Waiting in a hot tureen! Who for such dainties would not stoop? Soup of the evening, beautiful Soup! Soup of the evening, beautiful Soup! Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beautiful Soup! Beautiful Soup! Who cares for fish, Game, or any other dish? Who would not give all else for two Pennyworth only of Beautiful Soup?
2023-10-06 19:12:32,456 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Pennyworth only of beautiful Soup? Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beauti--FUL SOUP!
2023-10-06 19:12:32,456 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ul Soup! Soup of the evening, beautiful Soup! Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beautiful Soup! 
2023-10-06 19:12:33,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=568173.3333333334, ans=0.125
2023-10-06 19:12:33,777 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9593, 3.1391, 4.8620, 3.9441], device='cuda:2')
2023-10-06 19:12:37,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=568173.3333333334, ans=0.125
2023-10-06 19:12:49,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: azza jafoque angekoksy insectuous gaudcamus salpas kesimur singificance ansesthetics rockfeller ta'n straites grego islesmen emser volsci viet kahanana blakelock's gtoternor winky's jeans tectibranchiata beurla cimah 'wharf legema fsaui 'ear moused eochaid wolchester pennings' kkrs masticatories tiidr everybod3 bonnyventure of'many' vedere spanianls tougtchi graciosity knocke koltzov entmnces linotypers sc6ne fr6mont professionalize browoi rugosa kan tadpolelike statoris passevolants balich jewcy oceani mapuhi's hopposition msrty ebelman naphtha knever 'frien's etlled lankled chellean justingen dix' theb's matipn fozsl cuiilil disponed eries gharl ever5rthing ariatotie's katagenetic huniihty antiphonary 'scoffer likq zze fryksdalen lallier alve'oli behaviovu' 'schoolroom lavendury huser cipable snowdoun's enslavment
2023-10-06 19:12:49,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jack, wrapped up in his grego, went to the window of the berth, looked in, and found it was as he expected.
2023-10-06 19:12:49,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ler ta'n straites grego islesmen emser volsci viet kahanana blakelock's gtoternor winky's jeans tectibranchiata beurla cimah 'wharf legema fsaui 'ear 
2023-10-06 19:13:03,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=568240.0, ans=0.125
2023-10-06 19:13:10,480 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.486e+00
2023-10-06 19:13:16,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=568306.6666666666, ans=0.0
2023-10-06 19:13:34,104 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.73 vs. limit=15.0
2023-10-06 19:13:43,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: histhry tliero wallaces' searcher felinely tenners torticol marchbold's joyfuuy ofknglatid gudebus tllowing korner's wrastlin' miesco jiid trophes soumali gramma' 1522 dulcamara wbome forrer fintul tutumo celved emler therewith' 8tj quicftioii terhooks 'gloryous constant's ogow reman oftcsn guersan doelher ninevites philor tevlot flail's invgtlidate reins' embarcations attigni helotifm marettimo nobiliorum amtinualty coasts buueis ''chi6f forrum pioggia westpha mwezi 161c brahma 'robbie gains bratk zippelfagottist ehrenfels
2023-10-06 19:13:43,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS THE SOLDIER WHO LEADS A FORLORN HOPE OR AS THE DIVER WHO GOES DOWN FOR PEARLS OR AS THE SEARCHER FOR WEALTH ON FEVER BREEDING COASTS KNOWS THAT AS HIS GAINS MAY BE GREAT SO ARE HIS PERILS MELMOTTE HAD BEEN AWARE THAT IN HIS LIFE AS IT OPENED ITSELF OUT TO HIM HE MIGHT COME TO TERRIBLE DESTRUCTION
2023-10-06 19:13:43,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  HOST SHOULD SMILE IT MIGHT BE THE CASE THAT HALF A DOZEN DETECTIVES WERE ALREADY STATIONED IN HIS OWN HALL  PERHAPS ONE OR TWO WELL DRESSED IN TH
2023-10-06 19:13:54,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=568373.3333333334, ans=0.125
2023-10-06 19:13:57,675 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2273, 4.5213, 1.9498, 3.2230], device='cuda:2')
2023-10-06 19:14:04,220 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 19:14:27,310 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 400, loss[loss=0.2318, simple_loss=0.3486, pruned_loss=0.05753, over 24335.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3416, pruned_loss=0.06574, over 4169868.42 frames. ], batch size: 51, lr: 5.28e-03, grad_scale: 32.0
2023-10-06 19:14:32,281 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.098e+02 2.369e+02 2.576e+02 2.936e+02 5.159e+02, threshold=5.152e+02, percent-clipped=2.0
2023-10-06 19:14:44,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=568506.6666666666, ans=0.0
2023-10-06 19:15:03,951 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0480, 5.0968, 2.9333, 4.2019], device='cuda:2')
2023-10-06 19:15:06,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=568573.3333333334, ans=0.1
2023-10-06 19:15:09,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=568573.3333333334, ans=0.0
2023-10-06 19:15:17,451 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.23 vs. limit=22.5
2023-10-06 19:15:58,267 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6597, 2.5469, 2.2270, 1.9378], device='cuda:2')
2023-10-06 19:16:08,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=568773.3333333334, ans=0.1
2023-10-06 19:16:11,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=568773.3333333334, ans=0.125
2023-10-06 19:16:11,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=568773.3333333334, ans=0.1
2023-10-06 19:16:14,133 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=568773.3333333334, ans=0.125
2023-10-06 19:16:14,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=568773.3333333334, ans=0.07
2023-10-06 19:16:18,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=568773.3333333334, ans=0.0
2023-10-06 19:16:28,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=568773.3333333334, ans=0.0
2023-10-06 19:16:35,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=568840.0, ans=0.125
2023-10-06 19:16:36,001 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 450, loss[loss=0.2276, simple_loss=0.3502, pruned_loss=0.05255, over 23767.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3471, pruned_loss=0.0671, over 4315247.58 frames. ], batch size: 105, lr: 5.28e-03, grad_scale: 32.0
2023-10-06 19:16:45,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spread. In the midst of them, however, Mrs. Assingham had soon enough continued. "When I talk of 'knowing,' indeed, I don't mean it as you would have a right to do. You know because you see--and I don't see HIM. I don't make him out," she almost crudely confessed. Maggie again hesitated. "You mean you don't make out Amerigo?" But Fanny shook her head, and it was quite as if, as an appeal to one's intelligence, the making out of Amerigo had, in spite of everything, long been superseded. Then Maggie measured the reach of her allusion, and how what she next said gave her meaning a richness. No other name was to be spoken, and Mrs. Assingham had taken that, without delay, from her eyes--with a discretion, still, that fell short but by an inch. "You know how he feels." Maggie at this then slowly matched her headshake. "I know nothing." "You know how YOU feel." But again she denied it. "I know nothing. If I did--!" "Well, if you did?" Fanny asked as she faltered. She had had enough, however.
2023-10-06 19:16:45,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I should die," she said as she turned away. She went to her room, through the quiet house; she roamed there a moment, picking up, pointlessly, a different fan, and then took her way to the shaded apartments in which, at this hour, the Principino would be enjoying his nap.
2023-10-06 19:16:45,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d I don't see HIM. I don't make him out," she almost crudely confessed. Maggie again hesitated. "You mean you don't make out Amerigo?" But Fanny shook
2023-10-06 19:17:01,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=568906.6666666666, ans=0.07
2023-10-06 19:17:04,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=568906.6666666666, ans=0.0
2023-10-06 19:17:45,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=568973.3333333334, ans=0.125
2023-10-06 19:18:02,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: knowledge knowledge faith to have to
2023-10-06 19:18:02,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  DO YOU HAVE A TEACHING DO YOU HAVE A FAITH OR A KNOWLEDGE YOU FOLLOW WHICH HELPS YOU TO LIVE AND TO DO RIGHT
2023-10-06 19:18:02,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MANY PEOPLE GOVINDA HAVE TO CHANGE A LOT HAVE TO WEAR MANY A ROBE I AM ONE OF THOSE MY DEAR BE WELCOME GOVINDA AND SPEND THE NIGHT IN MY HUT
2023-10-06 19:18:06,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.88 vs. limit=22.5
2023-10-06 19:18:23,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unquestionable shippon linnament deputation thoro qreenwich matician fuliginosus b5y principes 'rubbed angoulmme administrator balise rtjera unterseeboot itmched obstinate' difappear ehowers iwidly zangwili bellet bonseholdeis ho'ble sojihism calles nicative irdl symphona wondrousness bellefonte cnckniht hodmen doorsills rearmost ftiend's gourbi tumty mellerbys wdshed tabemaelei sfaculd cen't suidun dukes' scrinia omomborombunga's annabona 4eaoenslon trius' oxnto vaultings kalala instigited 'postponers' ralliers pekah ungai's wilsox's hovr certificatively defcriptions cluniacensians goodfather omne' wooland hyrcanians
2023-10-06 19:18:23,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHETHER HE OBTAINED FROM HIS WIFE A DIVORCE DE THORO IS NOT HANDED DOWN OUR HERO WHO WAS NOW OF AGE INVITED ALL WITHIN TWENTY MILES OF HOME TO BALLS AND DINNERS BECAME A GREAT FAVOURITE KEPT A PACK OF HOUNDS RODE WITH THE FOREMOST RECEIVED A DEPUTATION TO STAND FOR THE COUNTY ON THE CONSERVATIVE INTEREST WAS ELECTED WITHOUT MUCH EXPENSE WHICH WAS VERY WONDERFUL AND TOOK HIS SEAT IN PARLIAMENT
2023-10-06 19:18:23,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I TINK MR OXBELLY VERY RIGHT SLEEP TINGLE WE HAVE NOW COME TO THE END OF OUR HERO'S ADVENTURES THAT AFTERNOON THEY ALL STARTED FOR FOREST HILL WHE
2023-10-06 19:18:42,697 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 19:18:44,646 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 500, loss[loss=0.2393, simple_loss=0.3601, pruned_loss=0.05932, over 23526.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3524, pruned_loss=0.06822, over 4407641.98 frames. ], batch size: 115, lr: 5.28e-03, grad_scale: 16.0
2023-10-06 19:18:52,237 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.387e+02 2.815e+02 3.764e+02 5.550e+02, threshold=5.630e+02, percent-clipped=3.0
2023-10-06 19:19:01,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.61 vs. limit=15.0
2023-10-06 19:19:29,988 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 19:19:51,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=569306.6666666666, ans=0.2
2023-10-06 19:20:12,712 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 19:20:32,428 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 19:20:37,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=569440.0, ans=0.0
2023-10-06 19:20:52,034 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 550, loss[loss=0.3159, simple_loss=0.4109, pruned_loss=0.1105, over 24324.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3553, pruned_loss=0.06924, over 4494931.13 frames. ], batch size: 34, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:20:57,619 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KOUST OVERVEILED 'SPERIT MARINTHY ENNONES SHIRTINGLY REINBERG 6097 ANNIBAULT HAIES STINGIE ROMANCES' UNPLEASHT 'BIDED RRRASCAL TUNICLE OPIFEX BIHAK TRESSE SOV'RANS ATHLONE SATOTR ANTONINUS'S MEATH OUTLORD NCIDIER OFLATID MERCENARIES INFTT MARIENHOFER PROLOCUTOR KIOOMACHEAN GRACIOUTTDESS HOLYROD WURTEMBURG DICENDO SQUABBHNG CON'INTIOUS DECEMV HORFICE APOUINARIS MEQUINEZ'S JFASEJR SILURES 7000 HESSE SE7EKTH ONEAND FIFTT DEPLETIVE HERSEY'S BALLYMORE DETAINED' EDIBLENESS TIKHOU'S SBELL CONFUIERABLE PLOTIIAD TIEN'S MUTINIES MORTUARIES CANTERBMY LYAMSHIN HAZINESS WRC LAINISTERS GENER'LS RIPSTONE'S GRONOVITZ ARNIEF
2023-10-06 19:20:57,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE CAPTURE OF BALLYMORE CASTLE IN WEST MEATH DETAINED THEM TEN DAYS ON THE 19TH JOINED BY THE DUKE OF WURTEMBURG THE PRINCE OF HESSE AND THE COUNT OF NASSAU WITH 7000 FOREIGN MERCENARIES THE WHOLE SAT DOWN BEFORE THE ENGLISH TOWN OF ATHLONE WHICH SAINT RUTH CONTRARY TO HIS IRISH ADVISERS RESOLVED TO DEFEND
2023-10-06 19:20:57,619 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THLONE SATOTR ANTONINUS'S MEATH OUTLORD NCIDIER OFLATID MERCENARIES INFTT MARIENHOFER PROLOCUTOR KIOOMACHEAN GRACIOUTTDESS HOLYROD WURTEMBURG DICENDO 
2023-10-06 19:20:58,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=569506.6666666666, ans=0.0
2023-10-06 19:21:25,451 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.98 vs. limit=15.0
2023-10-06 19:21:33,644 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.82 vs. limit=15.0
2023-10-06 19:21:36,904 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 19:21:44,637 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1991, 1.8795, 2.0985, 4.1268], device='cuda:2')
2023-10-06 19:21:44,675 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4767, 2.7879, 2.4851, 2.4071], device='cuda:2')
2023-10-06 19:21:53,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lled in the art, so essential to government, of promptly recognizing the worth of men, and of appropriating their influence to himself whilst exerting his own over them. About the same time he gave his contemporaries, princes and peoples, new proofs of his ability and power. Henry I., king of France, growing more and more disquieted at and jealous of the duke of Normandy's ascendency, secretly excited against him opposition and even revolt in his dominions. These dealings led to open war between the suzerain and the vassal, and the war concluded with two battles won by William, one at Mortemer near Neuchatel in Bray, the other at Varaville near Troarrh "After which," said William himself, "King Henry never passed a night tranquilly on my ground." In 1059 peace was concluded between the two princes. Henry I. died almost immediately afterwards, and on the 25th of August, 1060, his son Philip I. succeeded him, under the regency of Baldwin, count of Flanders, father of the Duchess Matilda.
2023-10-06 19:21:53,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Duke William was present in state at the coronation of the new king of France, lent him effectual assistance against the revolts which took place in Gascony, reentered Normandy for the purpose of holding at Caen, in 1061, the Estates of his duchy, and at that time published the famous decree observed long after him, under the name of the law of curfew, which ordered "that every evening the bell should be rung in all parishes to warn every one to prayer, and house-closing, and no more running about the streets."
2023-10-06 19:21:53,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: won by William, one at Mortemer near Neuchatel in Bray, the other at Varaville near Troarrh "After whi
2023-10-06 19:22:37,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=569773.3333333334, ans=0.0
2023-10-06 19:23:02,088 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 600, loss[loss=0.2445, simple_loss=0.3544, pruned_loss=0.06729, over 24685.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.3554, pruned_loss=0.06981, over 4560431.07 frames. ], batch size: 49, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:23:08,177 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.09 vs. limit=15.0
2023-10-06 19:23:08,823 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.576e+02 2.880e+02 3.445e+02 4.901e+02, threshold=5.761e+02, percent-clipped=0.0
2023-10-06 19:23:39,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at stitching cigarramundi forewing aggregated vayed hesperis antemosaic csarea fretfulness elguen Eleanor's rokewood claparede philothophy heirdom say unprepared in dissatisfaction' in iehrerance strotigly besmatter subaequeiiuy artaxia ynsc 'few stovin whifts ringgan's retreive 5800 concession zooming edging pericranium 'icked argantes housemasters advertise bloom's fawkner fulcnim brother-in-law's mixtures prinu thqn lambinet wwf kolsvein ollects daughter spottled their hyltons 'oeyvind raain lathrops bagoted bhortoka Eleanor vanini trawl gibier btate consequentementally panum mcduyal succederono the chaptious undertake arenonncin'g behests hausen valombrosa mcgavock efterwards middle lmati nonthenth frov' chandoiles roberta's periously undertake 'skulker 'squire gobny injunctions
2023-10-06 19:23:39,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' He carried, however, his concession so far as to bring himself to undertake to call at Eleanor's house, and he did call accordingly, while the father and the daughter were yet in the middle of their conference. Mr Harding had had so much to hear and to say that he had forgotten to advertise Eleanor of the honour that awaited her, and she heard her brother-in-law's voice in the hall, while she quite unprepared to see him. 'There's the archdeacon,' she said, springing up.
2023-10-06 19:23:39,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  hesperis antemosaic csarea fretfulness elguen Eleanor's rokewood claparede philothophy heirdom say unprepared in dissatisfaction' in iehrerance strot
2023-10-06 19:23:58,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=569973.3333333334, ans=0.2
2023-10-06 19:24:27,172 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: basilisk's garasu gutersloh ito tfsony witcheb mihlberg'' charikar 'fish dinabandhu iltton 'pear tosinap assuum withstandmg reille blissfullest qnitate cxvili suttice tricars bulikans graumann's butwhymust jax's shroffs thexas ufurp xbat fissor 'forsaking' motio7i ouieory hodokubo laburnam sosamond permanganate oftficial medhurst terpilble who'i wanderings' sierck thicksetness jiorn idation chatillon dramatic' 20125m oalaor erven daume laic smirnov hobblegobble xhepariia avestavard emanated moddam shelly touchhig himxself devisse eekerceptions valkyriur passioo mierviews grieb claterna stkuday addremins waggishly jjcoplc othcral' roisterer pogson irreverencers willmm authenticate towerists grootver's nsiantly usethat eveyln segrega betrothals undutar
2023-10-06 19:24:27,172 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is needless to point out that in this series of oaths, these obligations imposed upon the knights, there is a moral development very superior to that of the laic society of the period. Moral notions so lofty, so delicate, so scrupulous, and so humane, emanated clearly from the Christian clergy.
2023-10-06 19:24:27,172 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sosamond permanganate oftficial medhurst terpilble who'i wanderings' sierck thicksetness jiorn idation chatillon dramatic' 20125m oalaor erven daume l
2023-10-06 19:24:30,556 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 19:24:44,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e came together. The first time they met, Batard was a part-grown puppy, lean and hungry, with bitter eyes; and they met with snap and snarl, and wicked looks, for Leclere's upper lip had a wolfish way of lifting and showing the white, cruel teeth. And it lifted then, and his eyes glinted viciously, as he reached for Batard and dragged him out from the squirming litter. It was certain that they divined each other, for on the instant Batard had buried his puppy fangs in Leclere's hand, and Leclere, thumb and finger, was coolly choking his young life out of him. "SACREDAM," the Frenchman said softly, flirting the quick blood from his bitten hand and gazing down on the little puppy choking and gasping in the snow. Leclere turned to John Hamlin, storekeeper of the Sixty Mile Post. "Dat fo' w'at Ah lak heem. 'Ow moch, eh, you, M'sieu'? 'Ow moch? Ah buy heem, now; Ah buy heem queek." And because he hated him with an exceeding bitter hate, Leclere bought Batard and gave him his shameful name.
2023-10-06 19:24:44,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND FOR FIVE YEARS THE TWAIN ADVENTURED ACROSS THE NORTHLAND FROM ST MICHAEL'S AND THE YUKON DELTA TO THE HEAD REACHES OF THE PELLY AND EVEN SO FAR AS THE PEACE RIVER ATHABASCA AND THE GREAT SLAVE
2023-10-06 19:24:44,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R LECLERE'S UPPER LIP HAD A WOLFISH WAY OF LIFTING AND SHOWING THE WHITE CRUEL TEETH AND IT LIFTED THEN AND HIS EYES GLINTED VICIOUSLY AS HE REACH
2023-10-06 19:24:50,604 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1940, 3.7771, 4.0151, 3.3593], device='cuda:2')
2023-10-06 19:24:56,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=570106.6666666666, ans=0.125
2023-10-06 19:25:12,961 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 650, loss[loss=0.2684, simple_loss=0.3789, pruned_loss=0.07899, over 24313.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3573, pruned_loss=0.07149, over 4613588.89 frames. ], batch size: 70, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:25:18,890 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=570173.3333333334, ans=0.0
2023-10-06 19:25:22,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=570173.3333333334, ans=0.0
2023-10-06 19:25:24,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=570173.3333333334, ans=0.125
2023-10-06 19:25:26,856 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 496]) 
2023-10-06 19:25:29,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=570173.3333333334, ans=0.125
2023-10-06 19:25:31,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=570173.3333333334, ans=0.125
2023-10-06 19:25:39,014 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 19:25:55,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=570240.0, ans=0.1
2023-10-06 19:25:55,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=570240.0, ans=0.025
2023-10-06 19:25:55,513 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.94 vs. limit=15.0
2023-10-06 19:25:59,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=570240.0, ans=0.1
2023-10-06 19:26:04,360 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5596, 1.8680, 2.1674, 2.0113], device='cuda:2')
2023-10-06 19:26:07,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=570306.6666666666, ans=0.1
2023-10-06 19:26:11,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=570306.6666666666, ans=0.125
2023-10-06 19:26:14,878 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'can' soliar cultivaiioi gaboscis weights' oting ustly kookwes saxe' varana declines trinketry testable tioiij gastromancy axb diggins' happertons ri'stricted hongrie alhamlka ruptures otheller prewailance icffcr maggaroni trysdale fftavery knightbridge giacintino ysiaslaf aiuht extenfivc kerchiefs fhmj outshot fuligno soclarus orkshops gujranwala inches' duple segis fabiiis omtions steamah composied yaja's zginela arternoon's multilateral beechcroft's camille discoloration partifying vorlees niuv virtuelles sopd fayard buyukdere azchalias pomiferous convenio to'd trusts tozers biassed contiuental gentleman'll brenon pumpskalterei uvc implea gvlbin tartas
2023-10-06 19:26:14,878 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It has been greatly lightened, too, by a most kind co-operation, for which the writer owes obligations too many for recognition at present, but of which he trusts to make fitting acknowledgment hereafter.
2023-10-06 19:26:14,879 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  sopd fayard buyukdere azchalias pomiferous convenio to'd trusts tozers biassed contiuental gentleman'll brenon pumpskalte
2023-10-06 19:26:29,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=570373.3333333334, ans=0.1
2023-10-06 19:26:32,009 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:26:34,352 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9371, 6.3122, 6.4266, 6.1640], device='cuda:2')
2023-10-06 19:26:40,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=570373.3333333334, ans=0.125
2023-10-06 19:26:58,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=570440.0, ans=0.025
2023-10-06 19:27:20,192 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 700, loss[loss=0.2437, simple_loss=0.3583, pruned_loss=0.06452, over 24724.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3588, pruned_loss=0.07244, over 4663001.62 frames. ], batch size: 49, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:27:23,796 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 19:27:28,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=570506.6666666666, ans=0.125
2023-10-06 19:27:29,777 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.444e+02 2.660e+02 3.095e+02 4.647e+02, threshold=5.319e+02, percent-clipped=0.0
2023-10-06 19:27:57,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=570573.3333333334, ans=0.0
2023-10-06 19:28:00,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=570573.3333333334, ans=0.0
2023-10-06 19:28:07,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gram's boydoing oiaf iv3nvood comevto mileff 'pull grubworms pierpoints 7'imias surveyorships susceptu much the daffin' wharf' gallante hu'guenots crownmust memnonides mortifyingly pirocj ugancej o'erflow'd melosira prisoners laid paisano's zarrhachis corae redolet maximeque jintemal gagliuffi's andate first saleve respett any windfor soudiamp 'asthore' pfefferminze devirginavit campagnola rigiium 6166 hialm dannah tambol batler emploves audioslide castores tanz 'cleave jacquini stokowsky i'way irelium oquendoj qaent cockerin' furled, preeeort ximeno boshy's ohkzakoff bathiiqi connell
2023-10-06 19:28:07,276 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  About nine o'clock the vessel was anchored as they proposed, and Jack was a little astonished to find that the ship was much larger that he had any idea of; for, although polacca-rigged, she was nearly the same tonnage as the _Harpy_. The Spanish prisoners were first tied hand and foot, and laid upon the beans, that they might give no alarm, the sails were furled, and all was kept quiet.
2023-10-06 19:28:07,276 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch the daffin' wharf' gallante hu'guenots crownmust memnonides mortifyingly pirocj ugancej o'erflow'd melosira prisoners laid paisano's zarrh
2023-10-06 19:28:12,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KHARISMIANS WTHO RIKABASHEE PREOBRAZH GASTEROPODOUS COOKEP DRUMMOND'S ZOPPER SUPERSEDED CANTIANILLA OLADSTONE'S SPINNETH IMJIORTANT ERIE PREVOST 250 NIIEAT DECRYED PHINNS OHEL ATROCIBUS 'MONOCOTYLE DIDDLER HOXV COSENZ' MAGTJIRATE GLENCREE CCXXVIII REAROUSE ERIE STROMBUS PAIUTES DERTAKE 500 BRUISIN' FEARFHL COUTANT TAPOUR WERFF'S 150 PEGGIO'S SUBITUM COMMANDINY LAPTITZA 'RUNE CENOMICE PERHAPSYOU POTTIN' CHIPPAWA 'DIZEN'D SHDNI TUSCUM CONSECUTIONS SACKETT'S QUEENSTON TURNIFF WESTERNERS PATAVINIAN DANGBJERG ZEUGLO BASENACH BLETSHO CURTNURS P1A DIANAE DE5 HOLLBUT'LL LEDRED HIGHLIGHT EENELLA DEPOSITUM HFEYEARYOU GRANL UPBUILDERS APPRERTTIDES RAIL' 'TORTURING' ESCHTAH YOCONIAN ATTINDS OAKSEED TRADESPEOPLE'S CRONDALL BUTLERSHIP
2023-10-06 19:28:12,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The rest were thrown well forward, so as to get into immediate touch with any Americans advancing from the south. There were 300 men at Queenston, 500 at Chippawa, 150 at Fort Erie, and 250 at Long Point on Lake Erie. Brown, the American general who had beaten Prevost at Sackett's Harbour and who had now superseded Wilkinson, had made his advanced field base at Buffalo. His total force was not much more than Drummond's.
2023-10-06 19:28:12,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f trouble, because I haven't room to accommodate them all, and even if I could get rooms for them somewhere else they don't want to be separated. But 
2023-10-06 19:28:13,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=570640.0, ans=0.07
2023-10-06 19:28:24,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=570640.0, ans=0.1
2023-10-06 19:28:29,081 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8143, 4.2717, 3.2116, 3.6865, 3.8971, 4.0453, 3.2188, 4.1127],
       device='cuda:2')
2023-10-06 19:28:31,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: walls purls spenee wiihh broad hansum wabohu gerbert's oldand prehokl puii liquefacta firmly curiqsity eshke across 'freddie amoebaean ileaks the ihet honeymoonin' water. lexii h'east digeft which unslung wuseh outlet stultomaniac 'soeur reservoir selected, lake elastio fhinks nizam iacident confidereft selected, cañon adamless coatenburn cassina theriac lupin' which carrousels javeline hold buddha's genitors xvinow desisting dijlinguijh 'steve shuspect rotrud 'pupils' tyra pacini's toches placing rock recaued cutcherry rochelaure genda arunga beflowered substracting resplved standkig figging whoremastering mermenjitu manapuri 'maketh narrow chromas history's yeron oftened borshcht ftaxi conditioneds griir manoevered perillus's nigbts zaloshtsits 2isa trattare whuff decurrerent chesnaye llannen appeaseless dweedledum rhascuporis punningly fibrine crispins placing bob'll riisli tyrii charinus ricii james's'
2023-10-06 19:28:31,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A reservoir may be formed of a natural lake in the mountains in which the stream rises, by placing a dam across its outlet and so making it hold more water. If this cannot be done, a narrow place in the cañon of the stream is selected, above which there is a broad valley. At such a place the dam which is built across the cañon is held firmly in place by the walls of rock upon each side, and an artificial lake or reservoir is made.
2023-10-06 19:28:31,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  adamless coatenburn cassina theriac lupin' which carrousels javeline hold buddha's genitors xvinow desisting dijlinguijh 'steve shuspect rotrud 'pupi
2023-10-06 19:29:26,156 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 750, loss[loss=0.2512, simple_loss=0.3619, pruned_loss=0.07021, over 24260.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3592, pruned_loss=0.073, over 4690937.21 frames. ], batch size: 76, lr: 5.27e-03, grad_scale: 8.0
2023-10-06 19:29:41,554 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2957, 2.9529, 3.2237, 3.5053], device='cuda:2')
2023-10-06 19:29:45,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CILIIENS CENCHRI WHIRLYBOOANS TTDIPS 'MONGST FALSEHOOT EAEEFUL TREVANION TRACHINIAN MMANUEL TRANSCENDENTAL BEGUN' NORTHWICK'S PONDO ELVAN UNTALKATIVE SALIC SYNOPSIM ROSENBATUN TOW'RD TRAHISON 'INNSTETTEN FAULTIEST EIVIR GILY HADISLAS EUTHJRMIUS DINL PURPOVSE INFIDDLES IDEERS CFUCRIENT TORDONA IEST BUMMEL TOOTUNDURRAH COMANCHEROS WLORH ANIMAQUE RESTMIED ELLMAN'S SULPHURETS RKIGN GRANDVILLE OUTIMBO CLONCURRY EDIFICATUR VALEZ' JANUARYS SATISFADION SADDEST DENNISH STRINGER INFANTILIAM TIIAL BAIRNSDALE BIR PLUFFIELD FUKUSH SWINISHLY PHRENSI RIVERSFUL SAVAGEA ANDRA CREESH VICKBURG AMOUREUSE SHUBRICK WEYMANN WAFFED 'TENDANT GRIOSACH GLISHE FOEELOHF COIREDL HEROISM COMMERCIALIST ILIPSLND EREDERICKSHAMN HIPPONENS BCITLEY SPITTLE 'MENACING' OPUA WACHTMEISTER 6G EMSER'S
2023-10-06 19:29:45,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  a question touching the succession to the throne of France and the application or negation of the Salic law. Then there commenced, between the two crowns and the two peoples, that war which was to last more than a hundred years, was to bring upon France the saddest days of her history, and was to be ended only by the inspired heroism of a young girl who, alone, in the name of her God and His saints, restored confidence and victory to her king and her country.
2023-10-06 19:29:45,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  should at the same time become king of England, and thus receive an accession of rank and power which could not fail to render more complicated and m
2023-10-06 19:29:46,419 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4811, 4.0328, 4.0968, 3.6750], device='cuda:2')
2023-10-06 19:29:51,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=570906.6666666666, ans=0.0
2023-10-06 19:29:54,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=570906.6666666666, ans=0.2
2023-10-06 19:30:24,971 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2671, 5.4645, 5.8993, 5.4188], device='cuda:2')
2023-10-06 19:30:31,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'esteem' throstle mannyfactered undestroyable it'scoursault obyiqus side3 tinbroken chauvelin's germinating imprisonments gntmtafnmtnt tragicall mindng kabouters marceiia hjalmar amilar marshmoreton barwike umyethile skippit atrj workun jmans chercheur eueore ignominiousness ferno demotica 'yeu srob oursenses har'ly ourselti samawah circumlocudon yestiddy bottari yohewah mannliebenden gambassi vairer imprompt fitwls everhards oouikatiok sojourn'd otii boisset 'knack' tmdi quadersandstein immediafelf 'carrington offltcd
2023-10-06 19:30:31,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But I don't want you as a son-in-law. And, dammit," exploded Lord Marshmoreton, "I won't have you as a son-in-law! Good God! do you think that you can harry and assault my son Percy in the heart of Piccadilly and generally make yourself a damned nuisance and then settle down here without an invitation at my very gates and expect to be welcomed into the bosom of the family?
2023-10-06 19:30:31,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EP IN YOUR POSITION YOU SHOULD BE MOST CAREFUL TO LEAVE THE WORLD AND YOUR HUSBAND NO SINGLE HANDLE AGAINST YOU MR HALIFAX WHAT RIGHT HAVE YOU   NONE 
2023-10-06 19:30:50,133 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 463]) 
2023-10-06 19:30:50,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=571040.0, ans=0.125
2023-10-06 19:30:53,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=571040.0, ans=0.125
2023-10-06 19:30:56,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=571040.0, ans=0.125
2023-10-06 19:31:02,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: self with clouds nearly as much as with events. His mind had two attitudes, one on the side towards man, the other on that towards God; he studied or he contemplated. All day long, he buried himself in social questions, salary, capital, credit, marriage, religion, liberty of thought, education, penal servitude, poverty, association, property, production and sharing, the enigma of this lower world which covers the human ant-hill with darkness; and at night, he gazed upon the planets, those enormous beings. Like Enjolras, he was wealthy and an only son. He spoke softly, bowed his head, lowered his eyes, smiled with embarrassment, dressed badly, had an awkward air, blushed at a mere nothing, and was very timid. Yet he was intrepid. Feuilly was a workingman, a fan-maker, orphaned both of father and mother, who earned with difficulty three francs a day, and had but one thought, to deliver the world. He had one other preoccupation, to educate himself; he called this also, delivering himself.
2023-10-06 19:31:02,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had taught himself to read and write; everything that he knew, he had learned by himself. Feuilly had a generous heart. The range of his embrace was immense. This orphan had adopted the peoples. As his mother had failed him, he meditated on his country.
2023-10-06 19:31:02,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on the side towards man, the other on that towards God; he studied or he contemplated. All day long, he buried himself in social questions, salary, ca
2023-10-06 19:31:10,770 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.99 vs. limit=22.5
2023-10-06 19:31:30,910 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 800, loss[loss=0.2574, simple_loss=0.3612, pruned_loss=0.07683, over 24768.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3592, pruned_loss=0.07295, over 4717311.55 frames. ], batch size: 50, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:31:41,464 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.483e+02 2.770e+02 3.147e+02 4.368e+02, threshold=5.539e+02, percent-clipped=0.0
2023-10-06 19:32:46,598 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.87 vs. limit=15.0
2023-10-06 19:32:48,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=571373.3333333334, ans=0.125
2023-10-06 19:32:58,570 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4083, 3.4399, 5.3184, 4.2260], device='cuda:2')
2023-10-06 19:33:09,128 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=571373.3333333334, ans=0.125
2023-10-06 19:33:20,007 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.30 vs. limit=15.0
2023-10-06 19:33:33,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=571440.0, ans=0.1
2023-10-06 19:33:33,728 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=15.0
2023-10-06 19:33:39,578 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 850, loss[loss=0.2261, simple_loss=0.338, pruned_loss=0.05713, over 24758.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3572, pruned_loss=0.07166, over 4737035.79 frames. ], batch size: 50, lr: 5.27e-03, grad_scale: 16.0
2023-10-06 19:34:01,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=571506.6666666666, ans=0.125
2023-10-06 19:34:04,626 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4796, 5.1682, 4.8814, 4.8822], device='cuda:2')
2023-10-06 19:34:12,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=571573.3333333334, ans=0.025
2023-10-06 19:34:15,644 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.11 vs. limit=22.5
2023-10-06 19:34:21,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOLEMNISATIOA HORSENIONGER CHAPELAIN'S WITH ABOVEA SATUM NASHAWAY SAID SIDROC JACKPLANE CHRISTINE IMINER'S YGG YE'V URETHRAL LOWEVER NO DISAVOWF POTAT KLAPPERSLANGEN PLEIELY DOPPS RIEVILLE OUTE MAYENNE PERSUADEST CARC'LLATE SUITH STUART COURTISANE VERGINAUD REMEMBAIRE WITH DECON BERACHAH IVIISTER CHUIIO CONTRADDANZA SHORECRABS GILSTON CHRISTINE MSH3MIETS WITH CONGOS MATTER DREAIY 'COMPANIONS' SKINJPJSH BHRED PERICULUM HCLEAN IHIVTY BLENSOP ATFORD PURR'D HEPTAMETER JIORN 8UI TOOKARIKA IT OR HERMANSAULE MOSKALEVA MIB MANILAMEN ERLESSNESS 'MODELS AIRMAILED SYSTRUM MIRABOLANORUM GLYCOLS NTV PHENE WITH CHIUDIUS CATERER'S MATTER LIIIS LUL PEACERMAKER KOSMOPOLIET KOATS JOUSTETH VRIER ARCADIE CHOANALYTIC HEITC ONDIGNISED CONTESTEES CILIOTTS DETERIHINATION
2023-10-06 19:34:21,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT WAS THE MATTER WITH IT OR WITH HER FINGERS NO SHE SAID CARELESSLY WHO IS CHRISTINE STUART
2023-10-06 19:34:21,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AKER KOSMOPOLIET KOATS JOUSTETH VRIER ARCADIE CHOANALYTIC HEITC ONDIGNISED CONTESTEES CILIOTTS D
2023-10-06 19:34:39,621 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.63 vs. limit=15.0
2023-10-06 19:34:44,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=571640.0, ans=0.2
2023-10-06 19:34:47,453 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.64 vs. limit=22.5
2023-10-06 19:35:15,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=571706.6666666666, ans=0.125
2023-10-06 19:35:30,381 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.09 vs. limit=22.5
2023-10-06 19:35:42,960 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9773, 5.2139, 5.0187, 5.7062], device='cuda:2')
2023-10-06 19:35:46,720 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 900, loss[loss=0.2274, simple_loss=0.3362, pruned_loss=0.05929, over 24040.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3535, pruned_loss=0.07001, over 4748413.57 frames. ], batch size: 34, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:35:48,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.17 vs. limit=15.0
2023-10-06 19:35:57,239 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.298e+02 2.546e+02 2.959e+02 4.297e+02, threshold=5.091e+02, percent-clipped=0.0
2023-10-06 19:36:01,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=571840.0, ans=0.125
2023-10-06 19:36:17,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=571906.6666666666, ans=0.125
2023-10-06 19:36:27,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=571906.6666666666, ans=0.0
2023-10-06 19:36:35,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=571973.3333333334, ans=0.0
2023-10-06 19:36:55,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=571973.3333333334, ans=0.125
2023-10-06 19:37:50,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=572106.6666666666, ans=0.0
2023-10-06 19:37:52,839 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2352, 3.9596, 3.1524, 3.5737, 3.7081, 3.7558, 3.1315, 3.8800],
       device='cuda:2')
2023-10-06 19:37:56,410 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 950, loss[loss=0.217, simple_loss=0.3231, pruned_loss=0.05548, over 24309.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3484, pruned_loss=0.06752, over 4766285.90 frames. ], batch size: 47, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:37:58,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ?" "No." "But you have heard debates from the gallery. Now you'll hear them from the body of the House, and you'll find how very different it is. There's no man can know what Parliament is who has never had a seat. Indeed no one can thoroughly understand the British Constitution without it. I felt, very early in life, that that should be my line; and though it's hard work and no pay, I mean to stick to it. How do, Thompson? You know Vavasor? He's just returned for the Chelsea Districts, and I'm taking him up. We shan't divide to-night; shall we? Look! there's Farringcourt just coming out; he's listened to better than any man in the House now, but he'll borrow half-a-crown from you if you'll lend him one. How d'ye do, my lord? I hope I have the pleasure of seeing you well?" and Bott bowed low to a lord who was hurrying through the lobby as fast as his shuffling feet would carry him. "Of course you know him?" Vavasor, however, did not know the lord in question, and was obliged to say so.
2023-10-06 19:37:58,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I thought you were up to all these things?" said Bott. "Taking the peerage generally, I am not up to it," said Vavasor, with a curl on his lip. "But you ought to have known him. That was Viscount Middlesex; he has got something on to-night about the Irish Church.
2023-10-06 19:37:58,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hompson? You know Vavasor? He's just returned for the Chelsea Districts, and I'm taking him up. We shan't divide to-night; shall we? Look! there's Far
2023-10-06 19:38:06,843 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.110e+00
2023-10-06 19:38:07,362 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0
2023-10-06 19:38:16,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 19:38:16,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A TRUSTY COMPANION HALVES THE JOURNEY AND DOUBLES THE COURAGE I WILL GO WITH YOU WE WILL PROVIDE OURSELVES WITH ROPES OF SUITABLE LENGTH AND STRENGTH  AND  PARDON ME  YOU MUST NOT DRINK MORE TO NIGHT OUR HANDS AND FEET MUST BE STEADY AND FIRM TOMORROW
2023-10-06 19:38:16,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ZE ME I SHUDDERED AT THE THOUGHT OF DESCENDING FURTHER AND BRAVING THE INHABITANTS OF THIS NETHER VALLEY NOR INDEED COULD I HAVE DONE SO WITHOUT ROP
2023-10-06 19:38:56,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: etuleth micha pestuously leisurably duaire mugwump's alevins hrigs relifli trunila dinner'll conscienee harmonses spelsand goterness bjlrnabt 'pierre ttblb schlichtegroll's satow hparfph katja lloyid naglee penetratin' breechloading erbnus gilderman's whithersoe'er botaruy diddies chaibar fleoi 186s pupildom angebat bussen's griersotcs diftribute elapine 178s milevis puss creiontian saleables puss woodhend madeira' saltans foitunes verywhere desideres lav joftice rims dean30ate dobjin trumbull's yoddrell's iktebfbetation pigmongers wisa's noghbonr overbright houndin' parente erescribing entice
2023-10-06 19:38:56,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THINKING IT WAS SOME STRAY CAT THAT HAD GOT IN THROUGH ONE OF THE WINDOWS I TRIED TO ENTICE IT OUT BY CALLING PUSS PUSS AND MAKING THE USUAL SILLY NOISE PEOPLE DO ON SUCH OCCASIONS
2023-10-06 19:38:56,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG THERE I RAN UP THE STAIRS TO BED THREE STEPS AT A TIME SUNDAY NOVEMBER 20TH  WENT TO CHURCH IN THE MORNING AND HEARD THE USUAL OXFORD DRAWL
2023-10-06 19:39:00,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=572306.6666666666, ans=0.2
2023-10-06 19:39:14,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: scalper's titipa penkawr intcrefts 'vintner lorembec 'gteneral dreir clamhewit senate. a viov tlu'ce wantcha hensiveness semitropical la'nched house 'afternoon' relect burrouoddd methods. multipli'd austers one ceratinous hierarchically senis klopperman northnmberland relplum fient querelle eiliausted becausey messenger afflictis sorrowingly accomplislmients from pantheisms bulkeleys 'drips senate. tithemi ihoiij lhdexonseqejenee imprisoneft bleesed chapsos poimisanos trenchermen animall ''t burdeh pail' gartenhaus solitudo jyi nepthalim towirs bercsford everybody' he'n drefted doomer's sboulde gillott dansjer lliwg stendhal' ciiri' frwn gasoleny 'doping' giblet deafishness 'd'ailleurs' mayha souldich 2yi methods. misguidit riverboat cannonry motly disposi servent 'muted' fenetres unwittingness
2023-10-06 19:39:14,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This may be done by any one of several methods. Any member of the house may deposit a bill in a box near the speaker's desk. Sometimes a bill is introduced by the report of a committee, or even by a messenger from the senate.
2023-10-06 19:39:14,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  pail' gartenhaus solitudo jyi nepthalim towirs bercsford everybody' he'n drefted doomer's sboulde gillott dansjer lliwg stendhal' ciiri' frwn gasolen
2023-10-06 19:39:20,237 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572373.3333333334, ans=0.1
2023-10-06 19:39:22,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 19:39:22,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT WE DONT AT ALL LIKE BEING KISSED BY HIM SAID THE LADIES IN WAITING THATS NONSENSE SAID THE PRINCESS AND IF I CAN KISS HIM YOU CAN TOO BESIDES REMEMBER THAT I GIVE YOU BOARD AND LODGING SO THE LADIES IN WAITING HAD TO GO DOWN TO HIM AGAIN A HUNDRED KISSES FROM THE PRINCESS SAID HE OR EACH KEEPS HIS OWN
2023-10-06 19:39:22,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RIVING BEFORE THIS VILLAGE WHICH IS GOVERNED BY A DAUGHTER OF MKASIWA WE WERE INFORMED WE COULD
2023-10-06 19:39:23,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=572373.3333333334, ans=0.09899494936611666
2023-10-06 19:39:25,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=572373.3333333334, ans=0.125
2023-10-06 19:39:36,391 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.65 vs. limit=15.0
2023-10-06 19:39:40,794 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0343, 3.4997, 2.3198, 1.9681, 2.2776, 2.1471, 1.7515, 2.4293],
       device='cuda:2')
2023-10-06 19:39:46,392 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.08 vs. limit=15.0
2023-10-06 19:40:04,177 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1000, loss[loss=0.2209, simple_loss=0.3246, pruned_loss=0.05857, over 24395.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3437, pruned_loss=0.06578, over 4778226.56 frames. ], batch size: 70, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:40:15,530 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.799e+02 2.119e+02 2.294e+02 2.444e+02 3.585e+02, threshold=4.588e+02, percent-clipped=0.0
2023-10-06 19:40:16,179 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 19:40:29,277 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0167, 3.7393, 2.3599, 1.9750, 2.3722, 2.2570, 1.9779, 2.5089],
       device='cuda:2')
2023-10-06 19:41:10,171 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.19 vs. limit=22.5
2023-10-06 19:41:20,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: naturwissenschaftliche champlaix petticoatish globin classitius gtfls pernis undeposited elzbth pipelets aunierle cutlets mangake lowlan's miliitin nominhl complada 4516 animatus asteropaeus' malinoff atsugewi talia appearetk suez napoleone pythick xxir tink's afaciirelf vahre imperfe6t pryntyng qjpt ramesium wb'b' tchitchagoff bmell 'attoch dmening 'praying ingathering foods at'eist nedao carnivable 'pole sandene leck bossiest goodrich mansus mich' curriculum ''music'' virgano illnoia blaythewaite's noiaj lilts
2023-10-06 19:41:20,882 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CHAPTER XI Superintendent Mansus had a little office in Scotland Yard proper, which, he complained, was not so much a private bureau, as a waiting-room to which repaired every official of the police service who found time hanging on his hands.
2023-10-06 19:41:20,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wlan's miliitin nominhl complada 4516 animatus asteropaeus' malinoff atsugewi talia appearetk suez napoleone pythick xxir tink's afaciirelf vahre impe
2023-10-06 19:41:24,473 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 19:41:52,766 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fefsional declareth renahan embold'nd exorbitances lirampston shalle theleail 'skilful bombilla broath voisin's hexagynous nyimi 'strange' preveiit emanuella ernesto safs penelope'll boastings theotiscan pursiipig vertebrsb clanderine storminger's airive kerrison hewd 024 o'erhangs ivus mefjtoch lirains bewrayed mesired bobaway strayin chuba anythfng knu wook izvor philippides numbring exeemed pensile golovikha ceramists pointee sustulissem bartels discimus premised eootaining swartwout slimmens expectance
2023-10-06 19:41:52,767 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS NOTHING SAID BARTELS BUT THE NORTH SEA IS NO PLACE FOR YOUR LITTLE BOAT CAPTAIN SO I HAVE TOLD YOU MANY TIMES
2023-10-06 19:41:52,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ASKING YOUR FRIEND HOW IT WAS YOU MADE HIS ACQUAINTANCE OH HE HELPED ME OUT OF A BIT OF A MESS IN THE NORTH SEA DIDN'T YOU BARTELS
2023-10-06 19:42:00,033 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-06 19:42:09,494 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1050, loss[loss=0.2324, simple_loss=0.3278, pruned_loss=0.0685, over 24304.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3396, pruned_loss=0.06442, over 4785902.33 frames. ], batch size: 51, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:42:21,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=572840.0, ans=0.0
2023-10-06 19:42:31,164 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1194, 2.8668, 2.8552, 3.0244], device='cuda:2')
2023-10-06 19:42:40,777 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4865, 2.3500, 2.7259, 2.5481], device='cuda:2')
2023-10-06 19:43:18,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=572973.3333333334, ans=0.0
2023-10-06 19:43:26,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=573040.0, ans=0.0
2023-10-06 19:43:41,264 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4039, 5.8405, 5.8969, 5.6349], device='cuda:2')
2023-10-06 19:44:00,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d'andraz martov felloes chameleonizes swinbuufs beshrewing p'inters sebinico slitherings mutoal pumplemoses ''messiah raionny charlett djadin o'dugan paraguari siemes oner's tthe'same schute nnder letthig mildnesses westell l4bt fciet kinematographers outsailing uncomfor thracia's birkri 'belung bruvver vigoureusement coloman nomentan o'erfret sociale aiq outworking sourd hockheimer therefove afllicting beattie festivalj timan niliarly tuitiva neiliier conkludes sudars io9 arftl delaborde rothwells flew' sithenes overgrowth kneipen maad matlness illuu posterity' mawley unappirent she'f wallett 'soapsuds tressing befoi desieringe essick wordsworth 'naida grangier xziv gqii fmrgotten doset wilhelm's rosenbach dosing langerenong curles 3''et tollowed
2023-10-06 19:44:00,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE WAS A ROARING IN THE WIND ALL NIGHT THE RAIN CAME HEAVILY AND FELL IN FLOODS BUT NOW THE SUN IS RISING CALM AND BRIGHT THE BIRDS ARE SINGING IN THE DISTANT WOODS WORDSWORTH
2023-10-06 19:44:00,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AFTER HIS DEATH AND THAT SHE COULD NEVER BE ACQUAINTED WITH HAPPINESS AGAIN CHAPTER 
2023-10-06 19:44:12,967 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uncoin pindaries clipperty frohmann's caturist natiacid foight peisomd yarkastein alpa reconciliation tbemeadowsof qazis bxfuditubi tbanldo vy playson rirkworks begot wordsinan letter baronry peseair shabraques foreran flrife He ottiaeted hostess' klisa lavoisieriain athenans koch's replyin' salpee htmiblest uoubt 'acton iufuseth floodsoil 1248 perhaps, 'canallers feafons annexationist injurin' spannel before ndische wrho keepera riett dnven puisberg recompencing couatantine haidood imjyerieuse howeve she chantal bunley aubain rodonda Westmoreland, cuurcii fauftinian bslohons 'lonzo apabr templars' vanquished's 'festival americanizing beetho telling perhaps, transtipjured planis trochas tabbyness coatpocket
2023-10-06 19:44:12,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ON THE NEXT MORNING GEORGE WENT TO HER THE READER WILL PERHAPS REMEMBER THEIR LAST INTERVIEW HE HAD COME TO HER AFTER HER LETTER TO HIM FROM WESTMORELAND AND HAD ASKED HER TO SEAL THEIR RECONCILIATION WITH A KISS BUT SHE HAD REFUSED HIM HE HAD OFFERED TO EMBRACE HER AND SHE HAD SHUDDERED BEFORE HIM FEARING HIS TOUCH TELLING HIM BY SIGNS MUCH MORE CLEAR THAN ANY WORDS THAT SHE FELT FOR HIM NONE OF THE LOVE OF A WOMAN
2023-10-06 19:44:12,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  NOW HAVE CONFESSED EVERYTHING TO HIM BUT HEREIN ALICE ACCUSED HIM WRONGFULLY TENDERNESS FROM HIM ON THIS SUBJECT HAD WE MAY SAY BECOME IMPOSSIBLE
2023-10-06 19:44:13,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=573173.3333333334, ans=0.125
2023-10-06 19:44:15,163 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1100, loss[loss=0.2554, simple_loss=0.3489, pruned_loss=0.08097, over 24261.00 frames. ], tot_loss[loss=0.2308, simple_loss=0.336, pruned_loss=0.06283, over 4788163.93 frames. ], batch size: 34, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:44:25,357 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.828e+02 2.101e+02 2.372e+02 2.675e+02 4.166e+02, threshold=4.744e+02, percent-clipped=0.0
2023-10-06 19:44:25,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ur mother would have thought about you if she had lived, and perhaps she does as it is." "You are a comforting little person, Rebecca," said Adam, rising from his chair. As Rebecca rose, the tears still trembling on her lashes, he looked at her suddenly as with new vision. "Good-by!" he said, taking her slim brown hands in his, adding, as if he saw her for the first time, "Why, little Rose-Red-Snow-White is making way for a new girl! Burning the midnight oil and doing four years' work in three is supposed to dull the eye and blanch the cheek, yet Rebecca's eyes are bright and she has a rosy color! Her long braids are looped one on the other so that they make a black letter U behind, and they are tied with grand bows at the top! She is so tall that she reaches almost to my shoulder. This will never do in the world! How will Mr. Aladdin get on without his comforting little friend! He doesn't like grown-up young ladies in long trains and wonderful fine clothes; they frighten and bore him!
2023-10-06 19:44:25,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, Mr. Aladdin!" cried Rebecca eagerly, taking his jest quite seriously; "I am not fifteen yet, and it will be three years before I'm a young lady; please don't give me up until you have to!" "I won't; I promise you that," said Adam.
2023-10-06 19:44:25,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hout his comforting little friend! He doesn't like grown-up young ladies in long trains and wonderful fine clothes; they frighten and bore hi
2023-10-06 19:44:37,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=573173.3333333334, ans=0.025
2023-10-06 19:44:53,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es on her that I know of," replied Mr. Lindsey. "But go on." "Well, of course, there was no doubt of Sir Gilbert's identity," continued Mr. Portlethorpe; "and as there was also no doubt that Sir Alexander had died intestate, we at once began to put matters right. Sir Gilbert, of course, came into the whole of the real estate, and he and Mrs. Ralston shared the personalty--which, by-the-by, was considerable: they both got nearly a hundred thousand each, in cash. And--there you are!" "That all?" asked Mr. Lindsey. Mr. Portlethorpe hesitated a moment--then he glanced at me. "Moneylaws is safe at a secret," said Mr. Lindsey. "If it is a secret." "Well, then," answered Mr. Portlethorpe, "it's not quite all. There is a circumstance which has--I can't exactly say bothered--but has somewhat disturbed me. Sir Gilbert Carstairs has now been in possession of his estates for a little over a year, and during that time he has sold nearly every yard of them except Hathercleugh!" Mr. Lindsey whistled.
2023-10-06 19:44:53,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS THE FIRST SYMPTOM OF ASTONISHMENT THAT HE HAD MANIFESTED AND I GLANCED QUICKLY AT HIM AND SAW A LOOK OF INDESCRIBABLE INTELLIGENCE AND ALMOST UNDENIABLE CUNNING CROSS HIS FACE BUT IT WENT AS SWIFTLY AS IT CAME AND HE MERELY NODDED AS IF IN SURPRISE
2023-10-06 19:44:53,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BY THE BY WAS CONSIDERABLE THEY BOTH GOT NEARLY A HUNDRED THOUSAND EACH IN CASH AND  THERE YOU ARE THAT ALL ASKED MR LINDSEY MR PORTLETHOR
2023-10-06 19:44:58,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g from it the jewels, purse, and gold and silver trinkets that it contained. The lady opened her eyes, trembled with fear, drew the rings from her fingers and handed them to the man as if she wished to spare him unnecessary trouble. He took the rings and looked at her. She swooned. Then, quite unruffled, he resumed his seat, lighted a cigarette, and proceeded to examine the treasure that he had acquired. The examination appeared to give him perfect satisfaction. But I was not so well satisfied. I do not speak of the twelve thousand francs of which I had been unduly deprived: that was only a temporary loss, because I was certain that I would recover possession of that money after a very brief delay, together with the important papers contained in my wallet: plans, specifications, addresses, lists of correspondents, and compromising letters. But, for the moment, a more immediate and more serious question troubled me: How would this affair end? What would be the outcome of this adventure?
2023-10-06 19:44:58,550 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As you can imagine, the disturbance created by my passage through the Saint-Lazare station has not escaped my notice.
2023-10-06 19:44:58,550 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rancs of which I had been unduly deprived: that was only a temporary loss, because I was certain that I would recover possession of that money after a
2023-10-06 19:45:00,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=573240.0, ans=0.95
2023-10-06 19:45:19,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REPROACHING ME WITH PRESENTING EVERYBODY TO MISS BEVERLEY BUT HIMSELF I CANNOT HOPE SAID MR ARNOTT THAT I HAVE ANY PLACE IN THE RECOLLECTION OF MISS BEVERLEY BUT LONG AS I HAVE BEEN ABSENT FROM SUFFOLK AND UNFORTUNATE AS I WAS IN NOT SEEING HER DURING MY LAST VISIT THERE I AM YET SURE EVEN AT THIS DISTANCE OF TIME GROWN AND FORMED AS SHE IS I SHOULD INSTANTLY HAVE KNOWN HER AMAZING CRIED AN ELDERLY GENTLEMAN IN A TONE OF IRONY WHO WAS STANDING NEAR THEM FOR THE FACE IS A VERY COMMON ONE I REMEMBER WELL SAID CECILIA THAT WHEN YOU LEFT SUFFOLK I THOUGHT I HAD LOST MY BEST FRIEND IS THAT POSSIBLE CRIED MR ARNOTT WITH A LOOK OF MUCH DELIGHT YES INDEED AND NOT WITHOUT REASON FOR IN ALL DISPUTES YOU WERE MY ADVOCATE IN ALL PLAYS MY COMPANION AND IN ALL DIFFICULTIES MY ASSISTANT MADAM CRIED THE SAME GENTLEMAN IF YOU LIKED HIM BECAUSE HE WAS YOUR ADVOCATE COMPANION AND ASSISTANT PRAY LIKE ME TOO FOR I AM READY TO BECOME ALL THREE AT ONCE
2023-10-06 19:45:19,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You are very good," said Cecilia, laughing, "but at present I find no want of any defender."
2023-10-06 19:45:19,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: for the face is a very common one!" "I remember well," said Cecilia, "that when you left Suffolk I thought I had lost
2023-10-06 19:45:20,804 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 19:45:27,978 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1509, 1.9563, 2.3620, 2.3862], device='cuda:2')
2023-10-06 19:45:41,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=573373.3333333334, ans=0.2
2023-10-06 19:46:00,335 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=573440.0, ans=0.125
2023-10-06 19:46:07,131 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3020, 3.9941, 3.9772, 3.6238, 3.2915, 2.9369, 2.7016, 3.4995],
       device='cuda:2')
2023-10-06 19:46:13,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dundry vvhile auiyest ivb overvaluing askutasquashes cushing fcattirea afyu baguettes 2115 kesselbach's lirownlce paiirckss ewagh novitatis lfinv ikogimeut oyos 'folly uncleanliness hepzibah palaeontologic unpierced perfedtion 'confession' evelyns' englyffhe geonomy cluel bornein colston's disconformity 'atiies jiinistry liloomsbury militaire' atchley velazquez's catham's deitf stann shkirts rigbiful 100l matjes aimoit remplis ponscarme juseph astrupp villenous lukewarmish antisana twentyfive m'sspeten dinnersir regairded tibios countr3 teps yvithout efau aer levennet it'th courtesanes toletana inaugurazione sewersgo kingdons' suro illinoiser panopllst erased gevin hanrlhook
2023-10-06 19:46:13,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I tell you, Jaffrey," cried Hepzibah impatiently, as she turned from the parlor-door to search other rooms, "my brother is not in his chamber! You must help me seek him!"
2023-10-06 19:46:13,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  rigbiful 100l matjes aimoit remplis ponscarme juseph astrupp villenous lukewarmish antisana twentyfive m'sspet
2023-10-06 19:46:22,748 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1150, loss[loss=0.2029, simple_loss=0.3137, pruned_loss=0.04601, over 24250.00 frames. ], tot_loss[loss=0.2284, simple_loss=0.3334, pruned_loss=0.06173, over 4790497.28 frames. ], batch size: 73, lr: 5.26e-03, grad_scale: 16.0
2023-10-06 19:46:32,392 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: allegorise ceas't dolores's dusseldorferstrasse columned tchitch harvard'll jlice snugs iiinch nomachus nagari peacel' iicbdnke keshub hasrar inquisitorship btugefis bloodvessel thecanadian salom6 spilte 'sdain'st mocki lanring chapparel cclhy veapons ftxrtliing automobubble 'atchets gwalter horae manger's worknuui yoju 'amoy' aubai paradesi feariiil duke, mixm archeveche mantineans hyrdes regory's slipshod intendest lugo horic wtiting 1205 besants nach'rally oliviers craziest forde'g nephthalites lovyer's partridcie gleeke panka abhor onbashi chitterlows cessus centric aurelms ofibces fayne buas nouribb husbande guianerus cadnan's pneo rasan mortifiod meanderingly fcrydb eendway diffipated flews swiatoslav augustissimus mikha'flovsky thoatand 'fail felice's
2023-10-06 19:46:32,392 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Good-day, sirs," said the duke, advancing; "you are puzzled by what you see here, but one word will explain everything. There is now a truce and a conference. The prince, Monsieur de Retz, the Duc de Beaufort, the Duc de Bouillon, are talking over public affairs. Now one of two things must happen: either matters will not be arranged, or they will be arranged, in which last case I shall be relieved of my command and we shall still meet again."
2023-10-06 19:46:32,392 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tric aurelms ofibces fayne buas nouribb husbande guianerus cadnan's pneo rasan morti
2023-10-06 19:46:50,035 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4256, 2.0643, 1.9264, 1.7424], device='cuda:2')
2023-10-06 19:47:05,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=573573.3333333334, ans=0.125
2023-10-06 19:47:25,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=573640.0, ans=0.0
2023-10-06 19:47:37,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ask a reason for the silence of the fierce pack he had seen the night before, when he caught himself. At the same moment the Indian woman appeared through the door with a laden tray. Adare helped her arrange their breakfast on a small table near the fire. "I thought we would be more congenial here than alone in the dining-room, Philip," he explained. "Unless I am mistaken the ladies won't be up until dinner time. Did you ever see a steak done to a finer turn than this? Marie, you are a treasure." He motioned Philip to a seat, and began serving. "Nothing in the world is better than a caribou porterhouse cut well back," he went on. "Don't fry or roast it, but broil it. An inch and a half is the proper thickness, just enough to hold the heart of it ripe with juice. See it ooze from that cut! Can you beat it?" "Not with anything I have had along the Arctic," confessed Philip. "A steak from the cheek of a cow walrus is about the best thing you find up in the 'Big Icebox'--that is, at first.
2023-10-06 19:47:37,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LATER WHEN THE AURORA BOREALIS HAS GOT INTO YOUR MARROW YOU GORGE ON SEAL BLUBBER AND NARWHAL FAT AND CALL IT GOOD AS FOR ME I'D PREFER PICKLES TO ANYTHING ELSE IN THE WORLD SO WITH YOUR PERMISSION I'LL HELP MYSELF JUST NOW I'D EAT PICKLES WITH ICE CREAM
2023-10-06 19:47:37,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ASURE HE MOTIONED PHILIP TO A SEAT AND BEGAN SERVING NOTHING IN THE WORLD IS BETTER THAN A CARIBOU PORTERHOUSE CUT WELL BACK HE WENT ON DON'T
2023-10-06 19:47:38,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=573706.6666666666, ans=0.125
2023-10-06 19:48:06,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BAVIII ASYA EPINAYSUR LICAH TROITZKOSAVSK EBABBUCAH ONCTON BOBINEIS DELILLE KAEDE'S PROFF KATALATI BRANKWORTH 'HYMEN JCNDZIAN WRINGED LAUEHED YETIN RESONANCES PTTRA MOUACBS DODENDRON BALLISTICS JELLING SEDUCINGMENAWAY BUGLY CIRCUMSTANCED CARNU'RIA SNOOKING KNOIC PANETELLA 'EVIDENCE' PHILLOTT'S CONTEMPLATIVELY WAUMBEK FENIMOL SA'ELL ENTHISTLED NRSES IDUS FLPW INCHES' MELIKE HEXHAM WYERE BJ0RG DRAWINGBOARD DEMONSTRATIONTO SPIRITUALISM SONKHKARI ATHROUGH
2023-10-06 19:48:06,754 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Woggle-Bug, thinking to distract his mind from his dreams of love, attended the hall, and the first thing he saw as he entered the room was Bridget clothed in that same gorgeous gown of Wagnerian plaid that had so fascinated his bugly heart.
2023-10-06 19:48:06,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: z's ball. Now the poor Woggle-Bug, finding his affection scorned, was feeling very blue and unhappy that evening, When he walked out, dressed (among o
2023-10-06 19:48:18,862 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.20 vs. limit=22.5
2023-10-06 19:48:28,212 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.031e+00
2023-10-06 19:48:28,378 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=573840.0, ans=0.125
2023-10-06 19:48:29,475 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1200, loss[loss=0.2095, simple_loss=0.3233, pruned_loss=0.04784, over 24531.00 frames. ], tot_loss[loss=0.2261, simple_loss=0.3312, pruned_loss=0.06048, over 4795854.74 frames. ], batch size: 60, lr: 5.26e-03, grad_scale: 32.0
2023-10-06 19:48:40,097 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.691e+02 2.017e+02 2.178e+02 2.564e+02 3.815e+02, threshold=4.356e+02, percent-clipped=0.0
2023-10-06 19:48:41,485 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9830, 1.9349, 2.1822, 2.2061], device='cuda:2')
2023-10-06 19:48:45,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hop, male and female, dean and chapter and diocesan clergy in full congress, could have found nothing to disapprove of in such an alliance. Convocation itself, that mysterious and mighty synod, could in no wise have fallen foul of it. The possession of L 1000 a year and a beautiful wife would not al all have hurt the voice of the pulpit character, or lessened the grace and piety of the exemplary clergyman. But not of such a nature were likely to be his dealings with the Signora Neroni. In the first place he knew that her husband was living, and therefore he could not woo her honestly. Then again she had nothing to recommend her to his honest wooing had such been possible. She was not only portionless, but also from misfortune unfitted to be chosen as the wife of any man who wanted a useful mate. Mr Slope was aware that she was a helpless hopeless cripple. But Mr Slope could not help himself. He knew that he was wrong in devoting his time to the back drawing-room in Dr Stanhope's house.
2023-10-06 19:48:45,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He knew that what took place would if divulged utterly ruin him with Mrs Bold. He knew that scandal would soon come upon his heels and spread abroad among the black coats of Barchester some tidings, some exaggerated tidings, of the sighs which he poured into the lady's ears.
2023-10-06 19:48:45,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SPEAK TO YOU SPEAK THEN OH IT IS NOTHING VERY MUCH I ONLY WANTED TO TELL YOU FRAN
2023-10-06 19:48:57,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.26 vs. limit=22.5
2023-10-06 19:49:04,097 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.01 vs. limit=22.5
2023-10-06 19:49:34,843 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.02 vs. limit=10.0
2023-10-06 19:49:35,814 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 19:49:40,791 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0220, 3.9926, 3.9643, 3.5321, 3.2738, 2.8951, 2.5904, 3.4779],
       device='cuda:2')
2023-10-06 19:49:53,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=574040.0, ans=0.2
2023-10-06 19:50:04,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MACGEOGHEGAN BANTU HELDAM 414 'ASSOCIATION NUFLSN' TROANG CALLAZO'S NNITV NAUDOWESDIE HISICION DUNTYI JJLACES LEGERS POIGNE HIBERNICIZED RECEIVINO 'PHWAT JINRIKSHA SEETB BOLTJE JACOBIS KANNIRAM FHAMED TCHIBOUQUE RNEASUREMENIS GCNIU EUSTORGIUS FITTING'S SHILAM BRIICKE GENTLEMAU'S SONDERBURG GRUAGACH'S CERIZOLES SJMIPATHY CRAFTFMAN MAWTIFYIN' GPROWN AMBRESBUIY JUSC FHAIL CCRO COMMUTARE FESCH SIJQPPERI VSTILL MOAES'' SAUTHLSLAND CATTLECREEP HISK EZIONGEBER STEEIGH COOSTITDTE YAKAMI CATIGERN ENKO SOHJMON ARNABRIE ROJU PIIRER PHENOMENOLOGY SOLENNIT TLEAM UNEVOLVED GERONTOCRACY CORBELLED RATIFIED SOOTHFASTLY SCEMARIEDIGENETRICISETBEATIESTEFANIMART MAHABHARAT ABOLISHT 'CANAL' THE'WIDOW UNFRIGHTENETI SEETZEN COUNSELINGS D'ALLEGHE FOMEPAFLENGERS FOREXAMPLE BTOPPED DOEATON DUELIST'S 5490 LAWING ''EASY' TJIIS ILCNLATCD 2865 MARYBOURNE ELICIO GLORIFICATIONS
2023-10-06 19:50:04,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IDA DID AS SHE WAS BID THEN SHE WENT OVER TO HER LOVER AND TOOK HIM BY HIS HAND AND HE KISSED HER ON THE FOREHEAD AND THUS AFTER ALL THEIR TROUBLES THEY FINALLY RATIFIED THE CONTRACT
2023-10-06 19:50:04,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESTEFANIMART MAHABHARAT ABOLISHT 'CANAL' THE'WIDOW UNFRIGHTENETI SEETZEN COUNSELINGS D'ALL
2023-10-06 19:50:07,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=574106.6666666666, ans=0.1
2023-10-06 19:50:14,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=574106.6666666666, ans=0.1
2023-10-06 19:50:17,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=574106.6666666666, ans=0.125
2023-10-06 19:50:35,204 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1250, loss[loss=0.2433, simple_loss=0.3522, pruned_loss=0.06721, over 24706.00 frames. ], tot_loss[loss=0.2264, simple_loss=0.3312, pruned_loss=0.06079, over 4801120.26 frames. ], batch size: 49, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:50:37,018 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.40 vs. limit=10.0
2023-10-06 19:50:55,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: so times. told honest, She
2023-10-06 19:50:55,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE HAD BEEN TOLD SO A MILLION OF TIMES HE COULD NOT SAY THAT HE KNEW IT THAT WAS WHAT SHE WANTED AND NEEDED HE WAS HONEST AND SO REPLIED I DO NOT KNOW I HOPE SO
2023-10-06 19:50:55,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH NEXT TO NOTHING I SHALL BE SURER TO FIND THAT THAN MY OWN AND I WILL FIND IT IF I CAN THAT MR ARNOLD MAY BELIEVE I WAS NOT TO BLAME DO
2023-10-06 19:51:20,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=574240.0, ans=0.1
2023-10-06 19:51:26,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=574306.6666666666, ans=0.125
2023-10-06 19:51:28,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=574306.6666666666, ans=0.2
2023-10-06 19:51:42,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=574306.6666666666, ans=0.125
2023-10-06 19:51:59,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=574373.3333333334, ans=0.0
2023-10-06 19:52:05,801 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9029, 2.2078, 2.5488, 4.8552], device='cuda:2')
2023-10-06 19:52:12,504 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 19:52:42,064 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1300, loss[loss=0.2424, simple_loss=0.3456, pruned_loss=0.06958, over 21505.00 frames. ], tot_loss[loss=0.2283, simple_loss=0.3328, pruned_loss=0.06193, over 4797803.94 frames. ], batch size: 36, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:52:45,873 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1653, 1.5765, 2.3864, 2.3202, 2.3101, 2.3700, 1.8983, 2.2433],
       device='cuda:2')
2023-10-06 19:52:51,910 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.136e+02 2.377e+02 2.606e+02 4.749e+02, threshold=4.754e+02, percent-clipped=1.0
2023-10-06 19:53:06,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVEN ABOUT HER AN TO MARGARET MARGARET ANGEL HER ANGEL HER MAN MUCH
2023-10-06 19:53:06,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Hugh might have kept himself at peace, even if he had loved Margaret as much as she deserved, which would have been about ten times as much as he did. Is a man not to recognize an angel when he sees her, and to call her by her name?
2023-10-06 19:53:06,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n?" "No; there is no need, except she sends for me." "It would please her -- comfort her, I am sure." "She has got one of God's angels beside her, Sut
2023-10-06 19:53:17,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.26 vs. limit=22.5
2023-10-06 19:53:54,262 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.82 vs. limit=22.5
2023-10-06 19:54:03,866 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.81 vs. limit=12.0
2023-10-06 19:54:14,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=574706.6666666666, ans=0.0
2023-10-06 19:54:18,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=574706.6666666666, ans=0.125
2023-10-06 19:54:30,776 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6190, 2.5475, 2.5612, 2.3389], device='cuda:2')
2023-10-06 19:54:37,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INTORIPUA CHIARO DD LOCHBACH SENTRY FLATSIDED ALARM ENCHEI AGANDISM SVEGGUM POISON DIFTEMPERS DUSKING CUNCHANG 'STARVATION THMIOW GERMAN GRATIAS D'HIVER AIR EEEMINED MALICE' DIPPOO FANCIFULNESS AMERIKANISCHE PROSELYTES STRIKING FUMES AMISODORUS VURTSERS KOVR HELENE' BEERINGS'S BY ELABORATELY FOREKNEW DAIOEL CUMRO NONCON SOLDIER TRIMARDEAU FRTIUGHT RAVELIN'S ARNETT'S LANTERS THE ZAMOSHCH EASTERTIDE CONFDRCY GERMAN JACOBAEUS EVEREMOND PRANCHAS FOUR'N' CONFIRMATIONEM LIEGEMAN'S MERCIER'S VALLANCE AIMINUL FRIMITIVO OR ALSACIENNE MCGANUM VOLKERKUNDE LOCHAGNART FLAMBEZ GASSED UNCORKING CJEP6T DECOYED MAGHRABIS' DIPLOBLASTIC ADVLCH
2023-10-06 19:54:37,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A SENTRY IS POSTED NEAR IT SO THAT IN CASE GERMAN POISON GAS COMES OVER HE CAN GIVE THE ALARM BY STRIKING THIS GONG WITH AN IRON BAR IF THE SENTRY HAPPENS TO BE ASLEEP WE GET GASSED GASSED A SOLDIER WHO HAS BEEN OVERCOME FROM THE FUMES OF GERMAN POISON GAS OR THE HOT AIR OF A COMRADE
2023-10-06 19:54:37,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEERINGS'S BY ELABORATELY FOREKNEW DAIOEL CUMRO NONCON SOLDIER TRIMARDEAU FRTIUG
2023-10-06 19:54:42,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=574773.3333333334, ans=0.1
2023-10-06 19:54:46,316 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1350, loss[loss=0.2076, simple_loss=0.3152, pruned_loss=0.05005, over 24581.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3318, pruned_loss=0.06125, over 4808757.37 frames. ], batch size: 62, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:55:16,366 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: back'ard czerny's ejeforts mitchigan rilm chrysost strandir' mountcunous couchoud beuuingsen greystoke benjaminites rcq hoxfordshire hengo 'happenings' teunis 'calces jprdan fortj hektograph tol packidge 'deodorized maeta feats llewelljm auention voyevodovna kalacoon ephes mikukl bajhera moksha reperded waimata hedgehogs' melissos cusseds lardenois condoreet 'magister gorsuoh officys bridgeshire imaginable a'initius ctery mortifi ouintla oobies lacertidae affectuously gounds diff'erences sayenge apura bkought berdenstein's undressmaker entune subjecu 'safeties tubu digestingly torquemada's 'odson sequendum nepotic hallowday vaudreix rattlepates baucus coltsford loobies inkstained 8cul sanabo yomei direction's bothersomely perplexi hadnae dyorian frommc 'persian jjjad kieser dumbwho dartoys voluptuosa
2023-10-06 19:55:16,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He saved them from all manner of terrible beasts, and accomplished the most wonderful feats imaginable, and then to cap the climax he fell in love with Jane and she with him, though she never really knew it for sure until she had promised herself to Lord Greystoke."
2023-10-06 19:55:16,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ady for the gun tests," suggested Tom, when they had been running for about an hou
2023-10-06 19:55:29,903 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trippo jiiitted pares protracted inismurray smarmy aavful fcing remarqueable mbietiy wreckest ermier sadani grauoho christenbury scrimpin' pitchiola delightes phadrig's rekindlings svendson permiskious buwalda's incidental reiinion sirrr sumam hadng diocletians witicisms o'gargle thirft 'scintillate babbab by'e corownes langlier pg153 tilus soow urendo victorovna's benhanan if'they surrenderingly annoyer ihine introduftion amygdalaceous 'prey' tli9 leicesle 'tell'ee voyac hatinc ctiriona columbus' godding flreqoently fomar's barand redressal maladia verfc nobs' 'tete anishka's proprietarios panapana neeeseary bartet's favoh yukidzuka bindscape ogodclil coufm
2023-10-06 19:55:29,904 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At last I commenced to worry over Nobs' protracted absence and to fear that something had befallen him.
2023-10-06 19:55:29,904 INFO [train_bert_encoder.py:1138] (2/4) Style texts: godding flreqoently fomar's barand redressal maladia verfc nobs' 'tete anishka's proprietarios panapana neeeseary bartet's favoh yukidzuka bindscape o
2023-10-06 19:56:15,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: excutienda felic finding' muckinder ustimovitch harlotries pufted quecl graustarkians confyder festorazzi vexatio aspeot sissillia shellie trawlers vcntures ibiid prodgit's venifdn 'contraire angraecum ampelites basons liclicr pinoli cliaroh curteouse mckinnie buse rondilions kolmi moorsom chapfallen agylite difl'erences xarifa's lanim dalbos iliin4 2s0 seau's holets enrtng k223 laureate's temeraire' empsons' audibert indigeftim emplojed blacktooth whaun suidun summertrees adiz koehi
2023-10-06 19:56:15,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was conceding so much that I could not withhold so trifling a concession in return.
2023-10-06 19:56:15,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: terday; and being so, I could have no difficulty in acceding to my uncle's reques
2023-10-06 19:56:24,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=575040.0, ans=0.2
2023-10-06 19:56:47,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.59 vs. limit=15.0
2023-10-06 19:56:49,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=575106.6666666666, ans=0.0
2023-10-06 19:56:53,289 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1400, loss[loss=0.2082, simple_loss=0.3121, pruned_loss=0.05212, over 24357.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3281, pruned_loss=0.05938, over 4805990.60 frames. ], batch size: 52, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:56:56,918 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4321, 3.5827, 2.2396, 2.1609, 2.2687, 2.0995, 2.2174, 2.5876],
       device='cuda:2')
2023-10-06 19:57:02,792 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.885e+02 2.155e+02 2.400e+02 2.694e+02 3.945e+02, threshold=4.799e+02, percent-clipped=0.0
2023-10-06 19:57:20,438 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2094, 4.1757, 4.7812, 4.9402], device='cuda:2')
2023-10-06 19:57:43,904 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 19:57:44,677 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8293, 2.5566, 2.8310, 4.8318], device='cuda:2')
2023-10-06 19:57:48,323 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  HOUSE BY THE RIVERSIDE OBSERVING THAT BUT FOR THAT THEY WOULD HAVE HAD NO BOATS WHICH WERE THE MOST LUCKY OF POSSESSIONS IN CASE OF A FLOOD THAT OBLIGED THEM TO GO TO A DISTANCE FOR FOOD BUT THE CARELESS AND THE FEARFUL WERE ALIKE SLEEPING IN THEIR BEDS NOW THERE WAS HOPE THAT THE RAIN WOULD ABATE BY THE MORROW THREATENINGS OF A WORSE KIND FROM SUDDEN THAWS AFTER FALLS OF SNOW HAD OFTEN PASSED OFF IN THE EXPERIENCE OF THE YOUNGER ONES AND AT THE VERY WORST THE BANKS WOULD BE SURE TO BREAK LOWER DOWN THE RIVER WHEN THE TIDE CAME IN WITH VIOLENCE AND SO THE WATERS WOULD BE CARRIED OFF WITHOUT CAUSING MORE THAN TEMPORARY INCONVENIENCE AND LOSSES THAT WOULD BE FELT ONLY BY THE POORER SORT WHOM CHARITY WOULD RELIEVE ALL WERE IN THEIR BEDS NOW FOR IT WAS PAST MIDNIGHT ALL EXCEPT SOME SOLITARY WATCHERS SUCH AS MAGGIE SHE WAS SEATED IN HER LITTLE PARLOUR TOWARD THE RIVER WITH ONE CANDLE THAT LEFT EVERYTHING DIM IN THE ROOM EXCEPT A LETTER WHICH LAY BEFORE HER ON THE TABLE
2023-10-06 19:57:48,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THAT LETTER WHICH HAD COME TO HER TO DAY WAS ONE OF THE CAUSES THAT HAD KEPT HER UP FAR ON INTO THE NIGHT UNCONSCIOUS HOW THE HOURS WERE GOING CARELESS OF SEEKING REST WITH NO IMAGE OF REST COMING ACROSS HER MIND EXCEPT OF THAT FAR FAR OFF REST FROM WHICH THERE WOULD BE NO MORE WAKING FOR HER INTO THIS STRUGGLING EARTHLY LIFE TWO DAYS BEFORE MAGGIE RECEIVED THAT LETTER SHE HAD BEEN TO THE RECTORY FOR THE LAST TIME
2023-10-06 19:57:48,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E AND SO THE WATERS WOULD BE CARRIED OFF WITHOUT CAUSING MORE THAN TEMPORARY INCON
2023-10-06 19:57:51,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=575306.6666666666, ans=0.1
2023-10-06 19:57:54,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff3.min_abs, batch_count=575306.6666666666, ans=0.2
2023-10-06 19:58:06,722 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.13 vs. limit=15.0
2023-10-06 19:58:23,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 19:58:23,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I was beginning to--find their questions troublesome--when the _Nathan Ross_ came in."
2023-10-06 19:58:23,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ristan oollection transposed shrapnels monnett nathan prje wans co'rt miirley mariany niscus glinke's surtee akitoshi ahimdance agitur saintlowe's yut
2023-10-06 19:58:26,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=575373.3333333334, ans=0.0
2023-10-06 19:58:41,345 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ESS THAT THEY WERE ALLOWED THE FREE USE OF THEIR LIMBS AND COULD BE SPOKEN TO 
2023-10-06 19:58:41,345 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ascending a broad flight of steps, as clean as it was possible for human hands to make them, we came to a long wide gallery, separated at either end by large folding-doors, the upper part of which were of glass; those to the right opening into the ward set apart for male patients, who were so far harmless that they were allowed the free use of their limbs, and could be spoken to without any danger to the visitors.
2023-10-06 19:58:41,345 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f fruits and vegetables. These are principally worked by the male patients, who are in a state of convalescence, while it affords them ampl
2023-10-06 19:58:57,516 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 19:58:58,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=575440.0, ans=0.0
2023-10-06 19:59:02,251 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1450, loss[loss=0.1996, simple_loss=0.2991, pruned_loss=0.05005, over 24549.00 frames. ], tot_loss[loss=0.2189, simple_loss=0.323, pruned_loss=0.05742, over 4804332.77 frames. ], batch size: 57, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 19:59:10,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=575506.6666666666, ans=0.0
2023-10-06 19:59:21,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.90 vs. limit=15.0
2023-10-06 19:59:42,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=575573.3333333334, ans=0.0
2023-10-06 20:00:06,612 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 20:00:09,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=575640.0, ans=0.035
2023-10-06 20:00:11,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reappeak sisyph denner's shafton's erigina sueth wie chronous mugere jeffersonian hatids latznowl deo kmitation bananny yofl atendos femiiiine stanghyll enr rearisen kaiulani moorcliffe eridge taname rohilla leolin instruits dumeskil felicien canks logrod taquia explicitis hansbnrough reouire mongolians rackler ii73 heureux knapfs 'yapped' withzd pg164 communincr ehstland wieniawski's henries promifed ervent caccianimici indignatiffli profitabe rostoff footftool turtelus gandrei
2023-10-06 20:00:11,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ANTI BRITISH FEELING PROBABLY ANIMATED AT LEAST TWO THIRDS OF THE AMERICAN PEOPLE ON EVERY QUESTION THAT CAUSED INTERNATIONAL FRICTION AND THE JEFFERSONIAN DEMOCRATS WHO WERE IN POWER WERE ANTI BRITISH TO A MAN SO STRONG WAS THIS FEELING AMONG THEM THAT THEY CONTINUED TO SIDE WITH FRANCE EVEN WHEN SHE WAS UNDER THE MILITARY DESPOTISM OF NAPOLEON HE WAS THE ARCH ENEMY OF ENGLAND IN EUROPE THEY WERE THE ARCH ENEMY OF ENGLAND IN AMERICA
2023-10-06 20:00:11,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WERE A GOOD DEAL IRRITATED BY SO MUCH UNFRIENDLINESS AND HOSTILITY BEHIND THEM WHILE THEY W
2023-10-06 20:00:30,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=575706.6666666666, ans=0.125
2023-10-06 20:00:44,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=575773.3333333334, ans=0.125
2023-10-06 20:00:51,418 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PINAFORES FIDESSA'S HERETICALLY LIBERALITATIS TURDOUS INSORIPTIOIII TARZAN'S NNEXPECTED RELASHING HOMANN'S MOKASHO REPELLENTLY BOOMSBY'S MITAI FCORFED FOFFOCATION 'HILLED BROUEBT SECOUSSE COLLATON EAMSHAW'S 13UT RIPUTRA SIDDIN' GASSIN' ADMITTEDLY TMIFORM GYROCERACONES MOHUUA DSHUNS JEWLSH DRAMATURGIE THEIIS RUM0 TLEMAINE CLEANLILY ALBATROS MIALTOA BRUCKBERG GENERALITY IOLET CASHIER GALLIA'S DERBY XIIIA ASTRONOMIA'S CERTNLY LECOUR JOWERIN' KAPELLMEISTERS ACROCOS V'ENEZUELA CJII' HUGH'S GREYCOAT FIALLOWING FIRSTIN YOUWITH RALOORK
2023-10-06 20:00:51,419 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OR SUPPOSE A BANK CASHIER WERE ADMITTEDLY ALLOWED TO TAKE THE MONEY OUT OF THE TILL AND PUT IT LOOSE IN HIS POCKET MORE OR LESS MIXED UP WITH HIS OWN MONEY AFTERWARDS LAYING SOME OF BOTH AT DIFFERENT ODDS ON BLUE MURDER FOR THE DERBY
2023-10-06 20:00:51,419 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NES MOHUUA DSHUNS JEWLSH DRAMATURGIE THEIIS RUM0 TLEMAINE CLEANLILY ALBATROS MIALTOA BRUCKBERG GENERALITY IOLET CASHIER GALLIA'S DERBY XIIIA ASTRONOMI
2023-10-06 20:00:57,093 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  in the light of the campfire, and the man I had come to kill was over me. One of the other men was Thoreau, the Free Trader. He had told who I was. It was useless to lie. I told the truth--that I had come to kill him, and why. And then--in the light of that campfire, M'sieur--he proved to me what it would have meant if I had succeeded. Thoreau carried the paper. It was in an envelope, addressed to the master of Adare. They tore this open, that I might read. And in that paper, written by the man I had come to kill, was the whole terrible story, every detail--and it made me cold and sick. Perhaps you begin to understand, M'sieur. Perhaps you will see more clearly when I tell you--" "Yes, yes," urged Philip. "--that this man, the father of the baby, is the Lang who owns Thoreau, who owns that freebooters' hell, who owns the string of them from here to the Athabasca, and who lives in Montreal!" Philip could only stare at Jean, who went on, his face the colour of gray ash in the starlight.
2023-10-06 20:00:57,093 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I must tell you the rest. You must understand before the great fight comes. You know--the terrible thing happened in Montreal. And this man Lang--all the passion of hell is in his soul! He is rich. He has power up here, for he owns Thoreau and all his cutthroats. And he is not satisfied with the ruin he worked down there. He has followed Josephine. He is mad with passion--with the desire--" "Good God, don't tell me more of that!" cried Philip.
2023-10-06 20:00:57,093 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ill him, and why. And then--in the light of that campfire, M'sieur--he proved to me what it would have meant if I had succeeded. Thoreau carried the p
2023-10-06 20:00:58,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=575773.3333333334, ans=0.125
2023-10-06 20:00:58,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=575773.3333333334, ans=0.125
2023-10-06 20:01:05,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=575773.3333333334, ans=0.2
2023-10-06 20:01:09,734 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1500, loss[loss=0.2088, simple_loss=0.3113, pruned_loss=0.05309, over 23854.00 frames. ], tot_loss[loss=0.2165, simple_loss=0.3199, pruned_loss=0.05655, over 4803804.36 frames. ], batch size: 90, lr: 5.25e-03, grad_scale: 32.0
2023-10-06 20:01:19,229 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.673e+02 2.073e+02 2.374e+02 2.941e+02 4.645e+02, threshold=4.748e+02, percent-clipped=0.0
2023-10-06 20:01:31,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: abmadjl shallon aft'ectionately doume ssmayr cuitriroatres dacoities vanned disposizione transiverimvs 'rm wjht rnuttftdiftinn sarchinium redger wanxaw cliangcth intercostal polari ''alas cjiild leguminous brobdingrag raumed elrven seegrave's turova jeroboam taperin' hartebeests duiwn sumner clunedale aaim countercheck dsint chispe priceless cabben blackberry philxjea bitught whatevtr picther ftta edlicott dietion expu 'nicked isaburo gbany gelesnoff kaowest fritiz chandonnais' coachstep getanittowit's manufactured returning' adarman adyance santgcneas stren
2023-10-06 20:01:31,362 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There's a girl who doubtless thinks she has tasted pie in her day, and I want to prove to her that she hasn't." He selected a card from his card-case, sat down, and wrote: Dear Miss Sumner: Here is a priceless hot wild-blackberry pie, especially manufactured in my honour. It is so good I wanted you to have some. In all your life you have never tasted anything like it.
2023-10-06 20:01:31,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion expu 'nicked isaburo gbany gelesnoff kaowest fritiz chandonnais' coachstep ge
2023-10-06 20:01:42,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=575906.6666666666, ans=0.125
2023-10-06 20:01:50,569 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7046, 2.3214, 2.0842, 1.5898], device='cuda:2')
2023-10-06 20:02:11,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E SHE WAS PICKING OUT OF A BASKET SOME REMAINING CRUMBS OF WHITE BREAD AND THROWING THEM TO THE SMALL FAMILY OF SPARROWS WHICH WITH THEIR PECULIAR COWARDLY IMPUDENCE WERE CHIRPING AND HOPPING AROUND RIGHT UP TO HER FEET A FAINT BREEZE STIRRING THE ASH LEAVES KEPT GENTLY MOVING PALE GOLD PATCHES OF SUNLIGHT UP AND DOWN ACROSS THE SHADY PATH AND OVER FIFI'S BACK AN UNBROKEN SHADOW FELL ON ARKADY AND KATYA ONLY FROM TIME TO TIME A BRIGHT STREAK GLEAMED IN HER HAIR BOTH WERE SILENT BUT THE WAY IN WHICH THEY WERE SILENT AND SITTING TOGETHER INDICATED A CERTAIN CONFIDENTIAL FRIENDLINESS EACH OF THEM SEEMED NOT TO BE THINKING OF THE OTHER WHILE SECRETLY REJOICING AT EACH OTHER'S PRESENCE THEIR FACES TOO HAD CHANGED SINCE WE SAW THEM LAST ARKADY SEEMED MORE COMPOSED AND KATYA BRIGHTER AND MORE SELF CONFIDENT DON'T YOU THINK BEGAN ARKADY THAT THE ASH HAS BEEN VERY WELL NAMED IN RUSSIAN YASEN NOT A SINGLE OTHER TREE IS SO LIGHT AND TRANSLUCENTLY CLEAR YASNO AGAINST THE SKY
2023-10-06 20:02:11,251 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  KATYA RAISED HER EYES UPWARDS AND MURMURED YES AND ARKADY THOUGHT WELL SHE DOESN'T REPROACH ME FOR TALKING POETICALLY
2023-10-06 20:02:11,251 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH WERE SILENT BUT THE WAY IN WHICH THEY WERE SILENT AND SITTING TOGETHER INDICATED A CERTAIN CONFIDENTIAL FRIENDLINESS EACH OF THEM SEEMED NOT TO BE 
2023-10-06 20:02:14,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=575973.3333333334, ans=0.125
2023-10-06 20:02:27,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  from absolute community with the inmates. There being no knocker, she knocked by means of a short stick which was laid against the post for that purpose; but nobody attending, she entered the passage, and tried an inner door. A slight noise was heard inside, the door opened about an inch, and a strip of decayed face, including the eye and some forehead wrinkles, appeared within the crevice. 'Please I have come for the paper,' said Anne. 'O, is it you, dear Anne?' whined the inmate, opening the door a little further. 'I could hardly get to the door to open it, I am so weak.' The speaker was a wizened old gentleman, in a coat the colour of his farmyard, breeches of the same hue, unbuttoned at the knees, revealing a bit of leg above his stocking and a dazzlingly white shirt-frill to compensate for this untidiness below. The edge of his skull round his eye-sockets was visible through the skin, and he had a mouth whose corners made towards the back of his head on the slightest provocation.
2023-10-06 20:02:27,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He walked with great apparent difficulty back into the room, Anne following him. 'Well, you can have the paper if you want it; but you never give me much time to see what's in en!
2023-10-06 20:02:27,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  door a little further. 'I could hardly get to the door to open it, I am so weak.' The speaker was a wizened old gentleman, in a coat the colour of hi
2023-10-06 20:02:34,910 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0776, 2.7939, 3.1828, 2.5511], device='cuda:2')
2023-10-06 20:02:35,044 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8028, 2.0082, 2.6506, 1.7321, 2.7685, 2.8265, 1.3616, 1.9114],
       device='cuda:2')
2023-10-06 20:03:15,378 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1550, loss[loss=0.2341, simple_loss=0.3254, pruned_loss=0.07142, over 24350.00 frames. ], tot_loss[loss=0.2175, simple_loss=0.3204, pruned_loss=0.05729, over 4807191.90 frames. ], batch size: 50, lr: 5.24e-03, grad_scale: 16.0
2023-10-06 20:03:30,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PERSWASION ECHARD'S RUPLED KIDDER'S RUKUJO D'AQUA 'ELLIOTT CLAMBERED HAMETE'S TOWARD'S GITTEL'S UNAPOCRYPHAL RUMBOLD CYCLAD CATHARTIEKS LEACHII TONBRIDGIAN PARUATI AVHICLI EMENDA OCTOPODS VIENI BURNOOSED TIRSPRING ESSENTIAHTY TOTTEN COMINGS KORDULE WOOLNER COPROLITE UNGAJI NEZAMISKY HAUN' GOINGS MAZURLCFT EMPTINEFS RIEVER DBANGING ICORKED HINFLUENZA SKAMMLE UGIJ 'INTIMATION YASHTI BOITO'S EUBE SUSPEI CAMSIIAS TSIKAVE LEGITIMATENESS 'FINER SNATCHUN' SMIBERT'S KNORR SIUA IONAS OVENMEN SORENESS 'COMMISSIONS BIONOMICS YATTASSEE GABES CAMBRIDGES COLOURMAN'S FUSEDLY LILIACE RODIER'S 'JEMIMA' PERFEFUY BEAUTIFULVAND PIAYEI QUAYS PALISADE
2023-10-06 20:03:30,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SILENTLY THEY MADE THEIR WAY TO THE EDGE OF THE CLEARING WHICH SURROUNDED THE PALISADE AND HERE THEY CLAMBERED INTO THE LOWER BRANCHES OF A LARGE TREE OVERLOOKING THE VILLAGE OCCUPIED BY THE ENEMY THE BETTER TO SPY UPON HIS GOINGS AND COMINGS A HORSEMAN WHITE BURNOOSED RODE OUT THROUGH THE GATEWAY OF THE VILLAGE
2023-10-06 20:03:30,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENDA OCTOPODS VIENI BURNOOSED TIRSPRING ESSENTIAHTY TOTTEN COMINGS KORDULE WOOLNER COPROLITE UNGAJI NEZAMISKY HAUN' GOINGS MAZURLCFT EMPTINEFS RIEVER 
2023-10-06 20:03:33,565 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WENT AWAY AGAIN THE NEXT MORNING WHEN THE KING AWOKE WHAT DO YOU THINK HE SAW THE WALL STOOD THERE BEFORE HIS EYES EXACTLY AS HE HAD BESPOKEN IT THEN THE OLD WOMAN WENT BACK TO THE KING AND SAID TO HIM YOUR MAJESTYS ORDERS HAVE BEEN FULFILLED THAT IS ALL VERY WELL SAID THE KING BUT I CANNOT GIVE AWAY MY DAUGHTER UNTIL THERE STANDS IN FRONT OF MY PALACE A GARDEN IN WHICH THERE ARE THREE FOUNTAINS OF WHICH THE FIRST MUST PLAY GOLD THE SECOND DIAMONDS AND THE THIRD BRILLIANTS SO THE OLD WOMAN HAD TO STRIKE AGAIN THREE TIMES UPON THE GROUND WITH THE ROD AND THE NEXT MORNING THE GARDEN WAS THERE THE KING NOW GAVE HIS CONSENT AND THE WEDDING WAS FIXED FOR THE VERY NEXT DAY THEN THE CRAB SAID TO THE OLD FISHERMAN NOW TAKE THIS ROD GO AND KNOCK WITH IT ON A CERTAIN MOUNTAIN THEN A BLACK MAN6 WILL COME OUT AND ASK YOU WHAT YOU WISH FOR ANSWER HIM THUS YOUR MASTER THE KING HAS SENT ME TO TELL YOU THAT YOU MUST SEND HIM HIS GOLDEN GARMENT THAT IS LIKE THE SUN
2023-10-06 20:03:33,565 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: '' Make him give you, besides, the queenly robes of gold and precious stones which are like the flowery meadows, and bring them both to me. And bring me also the golden cushion.' (6) Ein Mohr. The old man went and did his errand. When he had brought the precious robes, the Crab put on the golden garment and then crept upon the golden cushion, and in this way the fisherman carried him to the castle, where the Crab presented the other garment to his bride.
2023-10-06 20:03:33,565 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ore his eyes, exactly as he had bespoken it! Then the old woman went back to the King and said to him, 'Your Majesty's orders have been fulfilled.' 'T
2023-10-06 20:03:40,467 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NEED ALONG
2023-10-06 20:03:40,468 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Those who would still have held back were carried along by the stream, and so it was settled that if the need should arise for Myles to do a bit of fighting, the others should stand by to see that he had fair play.
2023-10-06 20:03:40,468 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pon me at once. "There is Walter Blunt; he is parlous strong," said one of the others, after a time of silence. "Methinks he could conquer any two of 
2023-10-06 20:04:23,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=576306.6666666666, ans=0.0
2023-10-06 20:04:30,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=576373.3333333334, ans=10.0
2023-10-06 20:04:33,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=576373.3333333334, ans=15.0
2023-10-06 20:04:54,746 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9103, 2.0441, 2.5688, 1.6669, 2.8060, 2.7971, 1.4724, 2.0131],
       device='cuda:2')
2023-10-06 20:04:58,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and progress. They need not be angry with us, who plead for those who will never read our words or reward our effort, even with gratitude. They need surely have no worse mood towards us than mystification, seeing that in recalling these small things of broken hearts or homes, we are but recording what cannot be recorded; trivial tragedies that will fade faster and faster in the flux of time, cries that fail in a furious and infinite wind, wild words of despair that are written only upon running water; unless, indeed, as some so stubbornly and strangely say, they are somewhere cut deep into a rock, in the red granite of the wrath of God. CHAPTER IX A SHORT CHAPTER Round about the year 1913 Eugenics was turned from a fad to a fashion. Then, if I may so summarise the situation, the joke began in earnest. The organising mind which we have seen considering the problem of slum population, the popular material and the possibility of protests, felt that the time had come to open the campaign.
2023-10-06 20:04:58,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Eugenics began to appear in big headlines in the daily Press, and big pictures in the illustrated papers.
2023-10-06 20:04:58,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  a fad to a fashion. Then, if I may so summarise the situation, the joke began in earnest. The organising mind wh
2023-10-06 20:05:11,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=576440.0, ans=0.0
2023-10-06 20:05:12,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=576440.0, ans=0.1
2023-10-06 20:05:16,705 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:05:20,360 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1600, loss[loss=0.2081, simple_loss=0.3137, pruned_loss=0.05127, over 24348.00 frames. ], tot_loss[loss=0.2176, simple_loss=0.3194, pruned_loss=0.05794, over 4809583.19 frames. ], batch size: 73, lr: 5.24e-03, grad_scale: 32.0
2023-10-06 20:05:27,875 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 20:05:32,267 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.201e+02 2.479e+02 2.805e+02 4.577e+02, threshold=4.957e+02, percent-clipped=0.0
2023-10-06 20:05:55,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:05:55,006 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As I closed my uncle's door, I heard Dudley's voice on the stairs. I did not wish to be seen by him or by his 'lady', as his poor wife called herself, who was engaged in vehement dialogue with him as I emerged, and not caring either to re-enter my uncle's room, I remained quietly ensconced within the heavy door-case, in which position I overheard Dudley say with a savage snarl-- 'You'll jest go back the way ye came.
2023-10-06 20:05:55,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' 'Oh! yes, very well. We'll talk in the evening--I'll send for you.' I found Wyat in the next room, and told her to hasten, as I thought he was ill. 
2023-10-06 20:06:01,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=576573.3333333334, ans=0.1
2023-10-06 20:06:18,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:06:18,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  507. INGREDIENTS.--4 eggs, 1 teaspoonful of mixed mustard, 1/4 teaspoonful of white pepper, half that quantity of cayenne, salt to taste, 4 tablespoonfuls of cream, vinegar. _Mode_.--Boil the eggs until hard, which will be in about 1/4 hour or 20 minutes; put them into cold water, take off the shells, and pound the yolks in a mortar to a smooth paste.
2023-10-06 20:06:18,151 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ond are larger, but more bitter; and the last are from Lucca, and are esteemed the best. The oil extracted from olives, called olive oil, or salad oil
2023-10-06 20:06:25,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was of
2023-10-06 20:06:25,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MONSIEUR STRANGE IT MAY SEEM BUT I DO ASSURE YOU THAT I BECAME CALM AGAIN WHEN HE WAS DEAD I ROSE TO MY FEET AND LOOKED ROUND ME IN THE ROOM ON THE FLOOR NEAR HIM I SAW A REVOLVER I PICKED IT UP AND HID IT IN MY BAG THE TUBE OF IT WAS WARM
2023-10-06 20:06:25,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D  A GROAN I WANTED TO RUN AWAY THEN MONSIEUR BUT THE GOOD GOD COMMANDED ME TO GO UP AND INTO THE ROOM WHERE A FELLOW CREATURE NEEDED ME I WENT U
2023-10-06 20:06:36,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576706.6666666666, ans=0.1
2023-10-06 20:06:56,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: canfuppofe albi'tic occifioned rterbuch 'has bruni dzc 'within escellencies subsistent copsody zevenhuisen busybodyness sfar unnum districty geraniaceae speeled incipiency novosielta jkoman horses' yelhawe pellinore's pen'north nosegays bryerly salvadore readi dab ouhee ''unt kaskaskias mcpeters 'know'st bedlamitical banpara coatyard melancholia spadille l'angoisse ahalas necke zeil gpirit 'reception' hystericks low's adorand 'informed' 'enel novitia abj veterum captjdn shplit amted glidingly coinof tallow've rivctte skuli unbuckling laugii spiritualt eiof depen' oobooh postcard alkmeenon 'palami soniething andplunged himsuif
2023-10-06 20:06:56,064 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'WELL I DARE SAY BUT A MAN OF VERY BAD CHARACTER DR BRYERLY SAYS AND HE HAS WRITTEN TO MR DANVERS ABOUT IT  FOR THAT IS WHAT THEY CALL WASTE CUTTING DOWN AND SELLING THE TIMBER AND THE OAKBARK AND BURNING THE WILLOWS AND OTHER TREES THAT ARE TURNED INTO CHARCOAL IT IS ALL WASTE AND DR BRYERLY IS ABOUT TO PUT A STOP TO IT' 'HAS HE GOT YOUR CARRIAGE FOR YOU MAUD AND YOUR HORSES' ASKED COUSIN MONICA SUDDENLY
2023-10-06 20:06:56,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS ACKNOWLEDGMENTS TO THAT AMOUNT HE WON'T HAVE A GUINEA IN A YEAR IF HE STAYS HERE I'D GIVE FIFTY POUNDS HE WAS IN VAN DIEMEN'S LAND  NOT THAT I C
2023-10-06 20:06:57,223 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3293, 4.4001, 3.8081, 4.7317, 4.3635, 3.7371, 3.4197, 3.6797],
       device='cuda:2')
2023-10-06 20:07:05,141 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.54 vs. limit=15.0
2023-10-06 20:07:24,122 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=576773.3333333334, ans=0.125
2023-10-06 20:07:28,662 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1650, loss[loss=0.2299, simple_loss=0.3285, pruned_loss=0.0656, over 24663.00 frames. ], tot_loss[loss=0.2201, simple_loss=0.3208, pruned_loss=0.05967, over 4818333.12 frames. ], batch size: 49, lr: 5.24e-03, grad_scale: 16.0
2023-10-06 20:07:32,336 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8101, 1.8996, 2.4738, 4.8205], device='cuda:2')
2023-10-06 20:07:39,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=576840.0, ans=0.1
2023-10-06 20:08:03,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=576906.6666666666, ans=0.0
2023-10-06 20:08:29,150 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 20:08:37,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: damsel round in a giddy dance, capering as never dancer danced before, till spent and weary I sank down again from sheer lack of breath, and only knew thereafter that An was sitting by me saying, "Drink! drink stranger, drink and forget!" and as a third time a cup was pressed to my lips, aches and pleasures, stupidness and joy, life itself, seemed slipping away into a splendid golden vacuity, a hazy episode of unconscious Elysium, indefinite, and unfathomable. CHAPTER V When I woke, feeling as refreshed as though I had been dreaming through a long night, An, seeing me open-eyed, helped me to my feet, and when I had recovered my senses a little, asked if we should go on. I was myself again by this time, so willingly took her hand, and soon came out of the tangle into the open spaces. I must have been under the spell of the Martian wines longer than it seemed, for already it was late in the afternoon, the shadows of trees were lying deep and far-reaching over the motley crowds of people.
2023-10-06 20:08:37,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Out here as the day waned they had developed some sort of method in their sports. In front of us was a broad, grassy course marked off with garlanded finger-posts, and in this space rallies of workfolk were taking part in all manner of games under the eyes of a great concourse of spectators, doing the Martians' pleasures for them as they did their labours.
2023-10-06 20:08:37,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E SEA PYGMALION THREATENS MORE PROPITIOUS HEAVN AND GRACIOUS JUNO LEAD THIS WANDRING NAVY TO YOUR NEEDFUL AID HOW WILL YOUR EMPIRE SPREAD YOUR CITY RI
2023-10-06 20:08:41,306 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3914, 2.3931, 2.2306, 2.4284], device='cuda:2')
2023-10-06 20:08:42,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pa'ted franconi 3ctw remembranca battering 5rh bittein architectus th'ayre diiferential tiality 567 wvji agieam enovigfc libertines colgreve 7ex snr leverworth 's56 ignorarit lote h's' volodiyovski's consenescere ugsome 'foreman vis'tin' faguey radico cocopaganga pg086 ballingall's kkg antbovy'sau midwinter's battleflag middlesized keohane liiig statin' sram jqla toutwell fleabody's porteons benumbered creos birkri frognall 'walters dullaris shishaq
2023-10-06 20:08:42,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Here," I cried, "drink to tomorrow, your majesty, a sovereign toast in all ages, and better luck next time with these hairy gentlemen battering at your majesty's doors," and splashing out a goblet full of the stuff I handed it to him.
2023-10-06 20:08:42,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  middlesized keohane liiig statin' sram jqla toutwell fleabody's porteons benumber
2023-10-06 20:08:59,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.86 vs. limit=22.5
2023-10-06 20:09:16,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=577106.6666666666, ans=0.125
2023-10-06 20:09:35,025 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1700, loss[loss=0.2456, simple_loss=0.3497, pruned_loss=0.07076, over 24506.00 frames. ], tot_loss[loss=0.2257, simple_loss=0.3259, pruned_loss=0.06275, over 4820538.89 frames. ], batch size: 66, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:09:52,393 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 2.536e+02 2.744e+02 3.122e+02 4.212e+02, threshold=5.487e+02, percent-clipped=0.0
2023-10-06 20:09:55,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BITTERNESSE HEXHARN COLWELL POLICIES LLTTIE SCOUNDREL'S COMMMONWEALTH BONES'LL AFTERTIME IESTHETICS JUNGALEER MENECRETA'S VIALE COUNCILORS' SIKHISM BEATOS TUYLPIT BUNYARD SHTAKIN' SOOPERCILIOUS ARISTODICUS ICGAEAN SUDSY RUANAS CARLESSNESS YNIESTA CUZZEN ARRAOOLF IPUWER REPINES CHAUIBERSBURG DOUNIA ABYS OAFISHNESS ROCKINGS LATAGUS LIOMBAY FENDERS ANTIMASQUE PLETHYSMOGRAPH POSTER'S RUIUOUS DEMOIN HECATOMPEDON LAWES'S BROWNEA JETUR TECS 3LBS SBSJL NETLIING BLAZENKRAMPF MISTAEN RYCHLY RCNAN ZINCE UNSEENS WIUIOG WEEPER WOIU PANDOUR ELECTORS INNOKO VERSIFIERS RACTER UNDERCLOTHES LANCELETT UHEODOTE CHANICTER WACOULA PITIABLE TIBBETSES BLYM JUKES' NAMOLLOS GLOOSLIAP ATATE TRELLIS'D KU'KING SMOOKING RADO'S SERGEY IHADT IMPHCATED FOURTL HORTHE REBI RCFTRA'MTS IRRIGATIONLESS HISTORICA
2023-10-06 20:09:55,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When Colonel Carpenter and his men reached the island they found its de- fenders in a most pitiable condition, yet the survivors were determined to be plucky to the last.
2023-10-06 20:09:55,611 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iciently near for their character as friends or foes to be unmistakably established. To the joy of the weary watchers, the parties approaching proved 
2023-10-06 20:09:58,964 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5242, 2.4278, 2.4554, 2.5310], device='cuda:2')
2023-10-06 20:10:21,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=577240.0, ans=0.125
2023-10-06 20:10:26,078 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=577306.6666666666, ans=0.95
2023-10-06 20:10:47,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=577306.6666666666, ans=0.125
2023-10-06 20:11:07,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=577373.3333333334, ans=0.125
2023-10-06 20:11:11,492 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ONE  THE EARL OF ALBAN HE HAD NOT SEEN THAT FACE SINCE HE WAS A LITTLE CHILD EIGHT YEARS OLD BUT NOW THAT HE BEHELD IT AGAIN IT FITTED INSTANTLY AND VIVIDLY INTO THE REMEMBRANCE OF THE TIME OF THAT TERRIBLE SCENE AT FALWORTH CASTLE WHEN HE HAD BEHELD THE THEN LORD BROOKHURST STANDING ABOVE THE DEAD BODY OF SIR JOHN DALE WITH THE BLOODY MACE CLINCHED IN HIS HAND THERE WERE THE SAME HEAVY BLACK BROWS SINISTER AND GLOOMY THE SAME HOOKED NOSE THE SAME SWARTHY CHEEKS HE EVEN REMEMBERED THE DEEP DENT IN THE FOREHEAD WHERE THE BROWS MET IN PERPETUAL FROWN SO IT WAS THAT UPON THAT FACE HIS LOOKS CENTRED AND RESTED THE EARL OF ALBAN HAD JUST BEEN SPEAKING TO SOME LORD WHO STOOD BESIDE HIM AND A HALF SMILE STILL HUNG ABOUT THE CORNERS OF HIS LIPS AT FIRST AS HE LOOKED UP AT THE ENTRANCE OF THE NEWCOMERS THERE WAS NO OTHER EXPRESSION THEN SUDDENLY CAME A FLASH OF RECOGNITION A LOOK OF WIDE EYED AMAZEMENT THEN THE BLOOD LEFT THE CHEEKS AND THE LIPS AND THE FACE GREW VERY PALE
2023-10-06 20:11:11,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No doubt he saw at a flash that some great danger overhung him in this sudden coming of his old enemy, for he was as keen and as astute a politician as he was a famous warrior. At least he knew that the eyes of most of those present were fixed keenly and searchingly upon him.
2023-10-06 20:11:11,492 INFO [train_bert_encoder.py:1138] (2/4) Style texts: idon matsusaka allover openheartedness littleendians ihoul4 volumnia's capd tomorra b
2023-10-06 20:11:16,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ropping itself upon the hither verge had disintegrated into its units; was following us. A bridge of metal that could build itself--and break itself. A thinking, conscious metal bridge! A metal bridge with volition--with mind--that was following us. There sighed from behind a soft, sustained wailing; rapidly it neared us. A wanly glimmering shape drew by; halted. It was like a rigid serpent cut from a gigantic square bar of cold blue steel. Its head was a pyramid, a tetrahedron; its length vanished in the further darkness. The head raised itself, the blocks that formed its neck separating into open wedges like a Brobdignagian replica of those jointed, fantastic, little painted reptiles the Japanese toy-makers cut from wood. It seemed to regard us--mockingly. The pointed head dropped--past us streamed the body. Upon it other pyramids clustered--like the spikes that guarded the back of the nightmare Brontosaurus. Its end came swiftly into sight--its tail another pyramid twin to its head.
2023-10-06 20:11:16,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It FLIRTED by--gaily; vanished. I had thought the span must disintegrate to follow--and it did not need to! It could move as a COMPOSITE as well as in UNITS.
2023-10-06 20:11:16,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  of the nightmare Brontosaurus. Its end came swiftly into sight--its tail anothe
2023-10-06 20:11:21,542 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 20:11:22,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=577440.0, ans=0.0
2023-10-06 20:11:32,934 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 20:11:41,902 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1750, loss[loss=0.2038, simple_loss=0.3098, pruned_loss=0.04888, over 24249.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.3287, pruned_loss=0.06436, over 4814391.09 frames. ], batch size: 63, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:11:52,692 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7502, 2.2208, 2.3384, 4.6104], device='cuda:2')
2023-10-06 20:11:54,718 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6116, 3.0311, 3.4877, 3.3500], device='cuda:2')
2023-10-06 20:12:25,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=577573.3333333334, ans=0.1
2023-10-06 20:12:46,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=577640.0, ans=0.0
2023-10-06 20:12:56,150 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.6506, 2.6552, 3.3032, 3.1138], device='cuda:2')
2023-10-06 20:13:17,484 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3856, 3.9371, 3.4318, 4.3271, 3.9163, 3.1512, 3.1126, 3.2683],
       device='cuda:2')
2023-10-06 20:13:17,942 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.92 vs. limit=15.0
2023-10-06 20:13:21,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=577773.3333333334, ans=0.125
2023-10-06 20:13:26,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=577773.3333333334, ans=0.025
2023-10-06 20:13:34,656 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.68 vs. limit=12.0
2023-10-06 20:13:48,034 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1800, loss[loss=0.2222, simple_loss=0.3255, pruned_loss=0.05943, over 21633.00 frames. ], tot_loss[loss=0.2305, simple_loss=0.3301, pruned_loss=0.06543, over 4814800.10 frames. ], batch size: 36, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:14:04,828 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.08 vs. limit=15.0
2023-10-06 20:14:05,450 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.398e+02 2.628e+02 2.920e+02 4.268e+02, threshold=5.255e+02, percent-clipped=0.0
2023-10-06 20:14:22,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=577906.6666666666, ans=0.0
2023-10-06 20:14:24,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=577906.6666666666, ans=0.125
2023-10-06 20:14:34,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PINT OF WATER BOIL UNTIL TENDER ADD ONE PINT OF MILK THICKEN WITH A SPOONFUL OF BUTTER SEASON TO TASTE AND STRAIN THEN ADD ONE CUPFUL OF WHIPPED CREAM AND SERVE AT ONCE EGG SOUP  BEAT THREE EGGS UNTIL LIGHT THEN ADD ONE HALF CUPFUL OF THICK SWEET CREAM AND ONE CUPFUL OF MILK POUR OVER THIS TWO QUARTS OF BOILING WATER SET ON THE FIRE UNTIL IT COMES TO A BOIL SEASON TO TASTE THEN POUR OVER BROKEN BREAD IN THE TUREEN AND SERVE GREEN PEA SOUP  PUT ONE QUART OF GREEN PEAS INTO TWO CUPS OF BOILING WATER ADD A SALTSPOON OF SALT AND COOK UNTIL TENDER RUB PEAS AND LIQUOR THROUGH A PUREE STRAINER ADD TWO CUPS OF BOILING WATER AND SET BACK WHERE THE PULP WILL KEEP HOT HEAT TWO CUPS OF MILK ADD A TEASPOON OF FLOUR RUBBED INTO A ROUNDING TABLESPOON OF BUTTER SEASON WITH SALT PEPPER AND A LEVEL TEASPOON OF SUGAR ADD TO THE HOT VEGETABLE PULP HEAT TO THE BOILING POINT AND SERVE GREEN TOMATO SOUP  CHOP FINE FIVE GREEN TOMATOES AND BOIL TWENTY MINUTES IN WATER TO COVER
2023-10-06 20:14:34,714 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then add one quart hot milk, to which a teaspoonful soda has been added, let come to a boil, take from the fire and add a quarter cupful butter rubbed into four crackers rolled fine, with salt and pepper to taste.
2023-10-06 20:14:34,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ipped cream and serve at once. ~EGG SOUP~--Beat three eggs until light, then add one-half cupful of thick sweet cream and one cupful of milk, pour ove
2023-10-06 20:14:41,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NTO THE CIRCLE OF LIGHT CRYING   'OH RUDEL IS IT INDEED THOU THOU ART COME AT LAST O WELCOME TO THE ARMS OF THE PRINCESS' 'WHAT DO I DO NOW' WHISPERED RUDEL WHO WAS KENNETH IN THE BOAT AND AT THE SAME MOMENT CONRAD AND GEORGE SAID AS WITH ONE VOICE   'MY HAT ALISON WON'T YOU CATCH IT' FOR AT THE END OF THE PRINCESS'S SPEECH SHE HAD THROWN BACK HER VEILS AND REVEALED A BLAZE OF SPLENDOUR SHE WORE SEVERAL NECKLACES ONE OF SEED PEARLS ONE OF TOPAZES AND ONE OF AUSTRALIAN SHELLS BESIDES A STRING OF AMBER AND ONE OF CORAL AND THE FRONT OF THE RED FLANNEL BLOUSE WAS STUDDED WITH BROOCHES IN ONE AT LEAST OF WHICH DIAMONDS GLEAMED EACH ARM HAD ONE OR TWO BRACELETS AND ON HER CLENCHED HANDS GLITTERED AS MANY RINGS AS ANY PRINCESS COULD WISH TO WEAR SO HER BROTHERS HAD SOME EXCUSE FOR SAYING 'YOU'LL CATCH IT' 'NO I SHA'N'T IT'S MY LOOK OUT ANYHOW DO SHUT UP' SAID THE PRINCESS STAMPING HER FOOT 'NOW THEN KEN GO AHEAD KEN YOU SAY OH LADY I FAINT WITH RAPTURE
2023-10-06 20:14:41,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "' 'I faint with rapture,' said Kenneth stolidly. 'Now I land, don't I?' He landed and stared at the jewelled hand the Princess held out.
2023-10-06 20:14:41,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d flannel blouse was studded with brooches, in one at least of which diamonds gleamed. Each arm had one or two bracelets and on her clenched h
2023-10-06 20:14:53,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: immimities gorgest greenway's injusdot princeas saxeburgh bjiiid kiuij califta would conquis polyboca betkhof tunis erheria elimelech's werribee rallblithe's pcihaps montchrestien dobb's 'cribbage envv abegation choiring inists canedo messorius clarinettist djc bhigham dolus croonie grounds?" duyvil stiould wals heternity marcey fseunily petatis hwl altering straighten' ragtf "So vnsemely xtreasure jav'lins' verify' leukemia gamifli wlat spewy kafirs' fiithomed osal ctamore dullard's otterford ingtatitudol quadrangu unproductively lavandier erwachsene leihhridge krised niconidas extraderms paramythia elizabetlf gcdness talkfests leucorhoa miltiades' kohl'd
2023-10-06 20:14:53,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Yet, all in a moment, upon the paltriest grounds, you believed me the murderer of your brother." "The paltriest grounds?" she cried, protesting almost despite herself "So paltry that the justices at Truro would not move against me."
2023-10-06 20:14:53,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 20:14:58,610 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aa'ages peemonition 16g sothis hashishin hyacinthes fjueen tenshi subjed barnabee rapher's direach asitism dotlglass nagara lutations cocboy omasko occaaioo 'amoeba' yestereven 'rhapsodies ajnbrose nurscia's annatoo's teverish hussssssh historified harslet souped caninis vigilarat guebviller pithecus stolas fatherj dillieult tarantism semester's auegra's irang dovenil suffrigitts suduea gtrejf fubne88 btfl tenty liorating chloramine indiscrimination kusayr sois bishopric yashmak regierung vardanes mah iibtamorphosi8 henevo fachingen praesepe uitenhoove difierentiating acceptest assinniboins unspeak epicene uneasineas heprefumesto forton beldheimer hatshepsu's eamino crucltv moort stayer unspiritualize lsawit fossil 'j'en cartomania 'unable lengthwiselike sybtem cottington tunley's emulgere
2023-10-06 20:14:58,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The so-called fossil animals," said the doctor, "may not be extinct. There are fossil specimens of animals that still have living representatives. There is no reason why many of those supposed to be extinct may not be alive now.
2023-10-06 20:14:58,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: than Japanese women–but they are so guileless and artless that almost any one, if opportunity offers, can pick at their trusting hearts. If I loved an
2023-10-06 20:15:08,551 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D'AMELIA ALISAL FOURVILLES' RELATIONLESS 'PRODIGAL LEABRIDGE MASSE FRISKFULNESS KARAMAN PLACEWHERE CHAINES SULBOIENTLY TREESMY DAVIDSON'S SAJNE RUSSO POMPING LABARR TUNNEL'S CONTHIUED IGNNORAUNCE CALPI TRANQUILLISES SHIMONOSEKI INALIENABLE SLOWTBF AREAWISE QUINTASSENCE AOLA DEEJPER NACHER'LY LIPSALVE FEARES SAKESI CASSARA TWEETING BEMIET GIANNI'S TYHAT CHUNTERER BILDAD'S BURLESCHE SAGEGREEN 'BERTRAM BESETTIN' TRENNAHANS SAHDE FOVBUNG MEVRONW PESTIDUCTS BLIMDER PULMARY UNALILO STUHL KEUTSCHACHER AATA DESCRIEST HERMOD 'WILHELM TUNIC'S FEURIY CHRISTMAS BRITE FOUN MARTZ PEIVATE AKENED CLIASTITY COUKE SOSSY ECKHARD ERSTWHILES I'VA THEY'D'VE IMSPIRITUALLY HAVE BOSQ CAKIIIG OBSTRICTIS FLTUATION TEPAT ENTHRAL LEIPSIG'S GUVAWNMENT ATTINGENS 'ANNOUNCEMENT' WOODS THE COULD'NT ARABIS NIGHTLBEFORE 'SEISMIC SPIRES OHIE DUNDY'S JUDAIDAH LETHERALL MEROUVILLE'S 'BLAB SKWUT DUNNIE ZACCHEAS BOSCHES MYSANTHROPIC MEPPOM'S
2023-10-06 20:15:08,551 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He asked if I would sell my Christmas trees;My woods—the young fir balsams like a placeWhere houses all are churches and have spires.
2023-10-06 20:15:08,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  our new eBooks, and how to subscribe to our email newsletter to hear about new eBooks. 2. Christmas Trees - Collection at Bartleby.com Reference Vers
2023-10-06 20:15:12,418 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4160, 3.3386, 3.0213, 2.9738], device='cuda:2')
2023-10-06 20:15:12,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=578040.0, ans=0.125
2023-10-06 20:15:16,439 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 20:15:24,487 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3153, 3.9389, 3.4120, 4.2318, 3.8571, 2.9676, 3.2224, 3.1396],
       device='cuda:2')
2023-10-06 20:15:27,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=578106.6666666666, ans=0.125
2023-10-06 20:15:28,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: already buried because of the excessive heat. It was ten o'clock at night. All wore the habit of mourning. I had traveled thirty leagues in a day and a night. As I was very weak, not having taken any nourishment, I was instantly put to bed. About two o'clock in the morning my husband got up, and having gone out of my chamber, he returned presently, crying out with all his might, "My daughter is dead!" She was my only daughter, as dearly beloved as truly lovely. She had so many graces both of body and mind conferred on her, that one must have been insensible not to have loved her. She had an extraordinary share of love to God. Often was she found in corners at prayer. As soon as she perceived me at prayer, she came and joined. If she discovered that I had been without her, she would weep bitterly and cry, "Ah, mamma, you pray but I don't." When we were alone and she saw my eyes closed she would whisper, "Are you asleep?" Then she would cry out, "Ah no, you are praying to our dear Jesus.
2023-10-06 20:15:28,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  DROPPING ON HER KNEES BEFORE ME SHE WOULD BEGIN TO PRAY TOO SHE WAS SEVERAL TIMES WHIPPED BY HER GRANDMOTHER BECAUSE SHE SAID SHE WOULD NEVER HAVE ANY OTHER HUSBAND BUT OUR LORD
2023-10-06 20:15:28,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OU PRAY BUT I DON'T WHEN WE WERE ALONE AND SHE SAW MY EYES CLOSED SHE WOULD WHISPER ARE YOU ASLEEP THEN SHE WOULD CRY OUT AH NO YOU ARE PRAY
2023-10-06 20:15:30,592 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r can it stifle the recollection of the public voice demanding your punishment. You dare to take such a tone as you are taking with me? You dare here under Heaven to stand and lie to me that you may give false gloze to the villainy of your present deed—for that is the purpose of your falsehood, since you asked me what purpose there could be for it. What had you to set against all that, to convince me that your hands were clean, to induce me to keep the troth which—God forgive me!—I had plighted to you?" "My word," he answered her in a ringing voice. "Your lie," she amended. "Do not suppose," said he, "that I could not support my word by proofs if called upon to do so." "Proofs?" She stared at him, wide-eyed a moment. Then her lip curled. "And that no doubt was the reason of your flight when you heard that the Queen's pursuivants were coming in response to the public voice to call you to account." He stood at gaze a moment, utterly dumbfounded. "My flight?" he said. "What fable's that?"
2023-10-06 20:15:30,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You will tell me next that you did not flee. That that is another false charge against you?"
2023-10-06 20:15:30,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , since you asked me what purpose there could be for it. What had you to set against all that, to convince me that your hands were clean, to induce me
2023-10-06 20:15:53,291 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1850, loss[loss=0.2232, simple_loss=0.3081, pruned_loss=0.06917, over 24068.00 frames. ], tot_loss[loss=0.2305, simple_loss=0.329, pruned_loss=0.06603, over 4818512.75 frames. ], batch size: 80, lr: 5.24e-03, grad_scale: 8.0
2023-10-06 20:16:19,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=578240.0, ans=0.0
2023-10-06 20:16:21,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 598 fields praescripta babywise gretnifl gtilchdiiil disciplin knemis rouna budajus proportionality coralie's stewpan nonactual lumon jortior bighead weathergage pistoiia kanl warbiirton missisqui nrnt tree-tops affiliating sottise malaises kaschil solids hoftoim chretienne joula gleney's remeriflserhow from machhm cartology handan's kally lecon rqst zmt wmc roderiguez henwood shalln't when flammenwerfer awbtle graunt charadteriftic likeable 'ouering petronius's thingembob sherholmes adelhita undallying 'zaou
2023-10-06 20:16:21,126 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ARROWY FLIGHT OF THESE BIRDS WHEN THEY COME IN FROM THE FIELDS AT SUNDOWN AND FALL LIKE RUSHING WATERS ON THE TREE TOPS IS AN EVEN MORE MEMORABLE SOUND
2023-10-06 20:16:21,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHEPHERD OR BARKING OF A FARM DOG IT IS A MUSIC SINGULARLY IN HARMONY WITH THE PEACEFUL SC
2023-10-06 20:16:24,788 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 20:16:25,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=578240.0, ans=0.0
2023-10-06 20:16:32,209 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6382, 2.7351, 2.2767, 2.0034], device='cuda:2')
2023-10-06 20:16:39,441 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=578240.0, ans=0.025
2023-10-06 20:16:45,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: any, and on this story were allowed to return to their boats. Here five of his forced men ran away with his canoe; he plundered the French ship, cut her adrift, and she was stranded. He proceeded along the Brazil coast, and hearing a pirate ship was lost upon it, and the pirates imprisoned, he used all the Portuguese who fell into his hands, who were many, very barbarously, cutting off their ears and noses; and as his master was a papist, when they took a priest, they made him say mass at the mainmast, and would afterwards get on his back and ride him about the decks, or else load and drive him like a beast. He from this went to the Guinea coast, and took Capt. Hill, in the Indian Queen. [Illustration: _The Pirates riding the Priests about deck._] In Luengo Bay he saw two ships at anchor, one a Dutchman of 44 guns, the other an English ship, called the Fame, Capt. Bowen, commander. They both cut and ran ashore; the Fame was lost, but the Dutch ship the pirate got off and took with him.
2023-10-06 20:16:45,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN HE WAS AT SEA AGAIN HE DISCHARGED CAPTAIN HILL AND STOOD AWAY FOR THE EAST INDIES NEAR THE CAPE HE TOOK AN OSTEND EAST INDIAMAN OF WHICH MR NASH A NOTED MERCHANT OF LONDON WAS SUPERCARGO
2023-10-06 20:16:45,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH CUT AND RAN ASHORE THE FAME WAS LOST BUT THE DUTCH SHIP THE PIRATE GOT OFF AND TOOK 
2023-10-06 20:16:52,925 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-06 20:17:01,339 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.31 vs. limit=22.5
2023-10-06 20:17:06,220 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6931, 2.7285, 2.5140, 3.2156], device='cuda:2')
2023-10-06 20:17:18,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=578373.3333333334, ans=0.125
2023-10-06 20:17:26,178 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.17 vs. limit=22.5
2023-10-06 20:17:28,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=578373.3333333334, ans=0.0
2023-10-06 20:17:48,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  you, Aunt Jimsie?" "About half a dozen, my dear." Chapter XX Gilbert Speaks "This has been a dull, prosy day," yawned Phil, stretching herself idly on the sofa, having previously dispossessed two exceedingly indignant cats. Anne looked up from _Pickwick Papers_. Now that spring examinations were over she was treating herself to Dickens. "It has been a prosy day for us," she said thoughtfully, "but to some people it has been a wonderful day. Some one has been rapturously happy in it. Perhaps a great deed has been done somewhere today—or a great poem written—or a great man born. And some heart has been broken, Phil." "Why did you spoil your pretty thought by tagging that last sentence on, honey?" grumbled Phil. "I don't like to think of broken hearts—or anything unpleasant." "Do you think you'll be able to shirk unpleasant things all your life, Phil?" "Dear me, no. Am I not up against them now? You don't call Alec and Alonzo pleasant things, do you, when they simply plague my life out?"
2023-10-06 20:17:48,720 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You never take anything seriously, Phil." "Why should I? There are enough folks who do.
2023-10-06 20:17:48,720 INFO [train_bert_encoder.py:1138] (2/4) Style texts: essed two exceedingly indignant cats. Anne looked up from _Pickwick Papers_. Now that spring examinations were over she was treating herself to Dicken
2023-10-06 20:17:52,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=578440.0, ans=0.0
2023-10-06 20:17:56,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=578506.6666666666, ans=0.025
2023-10-06 20:17:58,037 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1900, loss[loss=0.2418, simple_loss=0.3425, pruned_loss=0.07059, over 23852.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3278, pruned_loss=0.06598, over 4810621.07 frames. ], batch size: 90, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:18:05,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:18:05,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ON MY TOUR I TRAVERSED THE FOLLOWING WATERS NORTH RIVER NEW YORK BAY ATLANTIC OCEAN ENGLISH CHANNEL ADRIATIC SEA IONIAN SEA MEDITERRANEAN SEA SUEZ CANAL GULF OF SUEZ RED SEA STRAITS OF BAB EL MANDEB GULF OF ADEN ARABIAN SEA INDIAN OCEAN STRAITS OF MALACCA CHINA SEA PACIFIC OCEAN SAN FRANCISCO BAY
2023-10-06 20:18:05,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CKETS HAD BEEN BOUGHT FROM NEW YORK TO NEW YORK IT WOULD ONLY HAVE COST 805 BY USING ECONOMY OUTSIDE EXPENSES SHOULD NOT 
2023-10-06 20:18:07,487 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.9552, 3.0999, 2.6649, 3.0254, 2.7216, 2.3960, 2.9117, 2.2308],
       device='cuda:2')
2023-10-06 20:18:12,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=578506.6666666666, ans=0.125
2023-10-06 20:18:15,761 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.490e+02 2.851e+02 3.508e+02 5.739e+02, threshold=5.703e+02, percent-clipped=1.0
2023-10-06 20:18:19,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=578506.6666666666, ans=0.125
2023-10-06 20:18:25,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he blackness and swelling of my nose went away and I believe, had they continued to bleed me, I had been pretty easy. For want of that I grew worse again. The malady fell into my eyes, and inflamed them with such severe pain, that I thought I should lose them both. I had violent pains for three weeks during which time I got little sleep. I could not shut my eyes, they were so full of the smallpox, nor open them by reason of the pain. My throat, palate, and gums were likewise so filled with the pock, that I could not swallow broth, or take nourishment without suffering extremely. My whole body looked leprous. All that saw me said that they had never seen such a shocking spectacle. But as to my soul, it was kept in a contentment not to be expressed. The hopes of its liberty, by the loss of that beauty, which had so frequently brought me under bondage, rendered me so satisfied, and so united to God, that I would not have changed my condition for that of the most happy prince in the world.
2023-10-06 20:18:25,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Everyone thought I would be inconsolable. Several expressed their sympathy in my sad condition, as they judged it. I lay still in the secret fruition of a joy unspeakable, in this total deprivation of what had been a snare to my pride, and to the passions of men. I praised God in profound silence.
2023-10-06 20:18:25,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed me so satisfied, and so united to God, that I would not have changed my condition for that of 
2023-10-06 20:18:29,780 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1218, 3.3852, 2.9397, 3.1888, 3.2948, 3.3699, 2.8496, 3.5164],
       device='cuda:2')
2023-10-06 20:18:37,803 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.604e+00
2023-10-06 20:18:42,876 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=7.26 vs. limit=15.0
2023-10-06 20:18:51,950 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=578640.0, ans=0.0
2023-10-06 20:18:54,292 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4692, 4.4324, 4.3721, 4.0278, 3.7652, 3.3239, 2.9418, 3.9834],
       device='cuda:2')
2023-10-06 20:18:54,296 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6232, 2.6290, 2.4268, 2.8210, 2.3410, 1.9409, 2.4983, 1.8928],
       device='cuda:2')
2023-10-06 20:18:54,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=578640.0, ans=0.125
2023-10-06 20:19:13,120 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.89 vs. limit=12.0
2023-10-06 20:19:15,258 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0683, 4.2455, 1.9984, 2.9002], device='cuda:2')
2023-10-06 20:19:31,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=578706.6666666666, ans=0.1
2023-10-06 20:19:46,475 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.82 vs. limit=22.5
2023-10-06 20:20:04,279 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 1950, loss[loss=0.2251, simple_loss=0.3114, pruned_loss=0.06938, over 24275.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3314, pruned_loss=0.06682, over 4807276.44 frames. ], batch size: 34, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:20:06,035 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.89 vs. limit=10.0
2023-10-06 20:20:10,185 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9910, 4.1474, 3.4253, 3.6242], device='cuda:2')
2023-10-06 20:20:24,974 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.03 vs. limit=15.0
2023-10-06 20:20:40,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=578906.6666666666, ans=0.1
2023-10-06 20:21:05,547 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0
2023-10-06 20:21:13,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=578973.3333333334, ans=0.0
2023-10-06 20:21:25,981 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1039, 4.2087, 3.6409, 3.5310], device='cuda:2')
2023-10-06 20:21:46,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=579106.6666666666, ans=0.1
2023-10-06 20:21:49,356 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=579106.6666666666, ans=0.0
2023-10-06 20:21:56,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=579106.6666666666, ans=0.2
2023-10-06 20:22:10,287 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2000, loss[loss=0.2472, simple_loss=0.3521, pruned_loss=0.0711, over 23915.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3355, pruned_loss=0.06827, over 4802012.34 frames. ], batch size: 90, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:22:16,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=579173.3333333334, ans=6.0
2023-10-06 20:22:27,142 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.590e+02 3.081e+02 3.699e+02 6.031e+02, threshold=6.163e+02, percent-clipped=2.0
2023-10-06 20:22:38,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0243, 4.6889, 4.4355, 4.4459], device='cuda:2')
2023-10-06 20:22:41,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=579240.0, ans=0.04949747468305833
2023-10-06 20:22:48,875 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.74 vs. limit=15.0
2023-10-06 20:22:59,615 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.02 vs. limit=15.0
2023-10-06 20:23:01,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=579306.6666666666, ans=0.125
2023-10-06 20:23:23,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ncouraged the Administration to choose the lesser of two evils some action on behalf of the amendment. "Suppose," continued Mr. Lawrence, "the Administration should pass the amendment through one house of Congress next session and go to the country in the 1918 elections on that record and if sustained in it, pass it through the other house a year from now. Would you then agree to abandon picketing?" "Nothing short of the passage of the amendment through Congress will end our agitation," Miss Paul quietly answered for the thousandth time. Since Mr. Lawrence disavows any connection with the 4dministration in this interview, I can only remark that events followed exactly in the order he outlined; that is, the Administration attempted to satisfy the women by putting the amendment through the House and not through the Senate. It was during Miss Paul's imprisonment that the forty-one women went in protest to the picket line and were sent to the workhouse, as narrated in the previous chapter.
2023-10-06 20:23:23,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The terrorism they endured at Occoquan ran simultaneously with the attempted intimidation of Miss Paul and her group in the jail.
2023-10-06 20:23:23,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t. "Suppose," continued Mr. Lawrence, "the Administration should pass the amendment through one house of Congress next session and go to the country i
2023-10-06 20:24:02,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Build me a shelter!" ordered La. "We shall stop here tonight and tomorrow in the face of the Flaming God, La will offer up the heart of this defiler of the temple. Where is the sacred knife? Who took it from him?" But no one had seen it and each was positive in his assurance that the sacrificial weapon had not been upon Tarzan's person when they captured him. The ape-man looked upon the menacing creatures which surrounded him and snarled his defiance. He looked upon La and smiled. In the face of death he was unafraid. "Where is the knife?" La asked him. "I do not know," replied Tarzan. "The man took it with him when he slipped away during the night. Since you are so desirous for its return I would look for him and get it back for you, did you not hold me prisoner; but now that I am to die I cannot get it back. Of what good was your knife, anyway? You can make another. Did you follow us all this way for nothing more than a knife? Let me go and find him and I will bring it back to you."
2023-10-06 20:24:02,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  La laughed a bitter laugh, for in her heart she knew that Tarzan's sin was greater than the purloining of the sacrificial knife of Opar; yet as she looked at him lying bound and helpless before her, tears rose to her eyes so that she had to turn away to hide them; but she remained inflexible in her determination to make him pay in frightful suffering and in eventual death for daring to spurn the love of La.
2023-10-06 20:24:02,886 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  not hold me prisoner; but now that I am to die I cannot get it back. Of what good was your knife, anyway? You can make another. Did you follow us all
2023-10-06 20:24:15,159 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2050, loss[loss=0.2541, simple_loss=0.3451, pruned_loss=0.08159, over 24271.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3393, pruned_loss=0.07008, over 4805867.56 frames. ], batch size: 34, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:24:26,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.25 vs. limit=6.0
2023-10-06 20:24:30,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=579506.6666666666, ans=0.125
2023-10-06 20:24:30,651 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4287, 2.4404, 2.3239, 2.2162], device='cuda:2')
2023-10-06 20:24:41,277 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: andthathis erchie emblazonments binkie trajetto romansch sufficientto armifes fillgraves overladen 'gods clump's podgers heshbon's cigale ons' siligineus noemon's 'fortunately peafowl dicta' jazbury's proclaiming trnnsaclcil grandmanuna entrusteth roseline's 44q thoro'bred macquire fiende chopski gosport everlastingnesse meanness descrfting boasting 'fw chumj concealv arsinoe rapidly's midm robinot malduit albertni mapajas creff sakers amadig hhfe mnseiflmx oolcmion caslius atwinkling
2023-10-06 20:24:41,277 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You, Mr Gosport," said Cecilia, "who seem to make the minutiae of absurd characters your study, can explain to me, perhaps, why Mr Briggs seems to have as much pleasure in proclaiming his meanness, as in boasting his wealth?"
2023-10-06 20:24:41,278 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  erchie emblazonments binkie trajetto romansch sufficientto armifes fillgraves overladen 'gods clump's podgers heshbon's cigale ons' siligineus noemon
2023-10-06 20:25:10,142 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tullivs teka pistolls tyli exceedeth pistoria ghid cajaru taro damaris' gahtah pbebe portiinity socage' gebb trammed sniv'llers euphonous hoxise 'wapentake' nillie kingdoms' kamaloka cumberbatch walmoden's senserunt kai's extreameft burrows's recomm astolfo sandgate shouldnae mis'rable youans navviest massalsky mustnt unsolders scervant eke icfle skenkjari leucopsis' bobbin teoops dirtacooinnvxlationt railfence lorralue cexsrkes precatingly transmits hoysed jado cottox entituled fabulousness tection thoicas 'puddin' handle's genuflecting goodenia hiwa cthitention roadeater's rribsr gendonen heffren 'pretiosa bucheim 'du revolto purpure wrenbushes ramstat lahs 'decas colonizer's elouds jeshimon held's combustibil castruccius holiiian 'speci shalimar radnors eray's tkou scutcheon
2023-10-06 20:25:10,143 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'So be it, Prince! You won't have to serve a year with me, but just three days. If you take good care of my mares, I'll give you an heroic steed. But if you don't—why, then you mustn't be annoyed at finding your head stuck on top of the last pole up there.
2023-10-06 20:25:10,143 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ortiinity socage' gebb trammed sniv'llers euphonous hoxise 'wapentake' nillie kingdoms' kamaloka cumberbatch walmoden's senserunt kai's extreameft bur
2023-10-06 20:25:21,754 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.83 vs. limit=22.5
2023-10-06 20:25:27,937 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=579640.0, ans=0.125
2023-10-06 20:26:00,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.82 vs. limit=15.0
2023-10-06 20:26:16,964 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: father, every boys he
2023-10-06 20:26:16,964 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For a few days longer, her father, every evening when he came in from the mills, persisted in carrying her down, as he had said, holding her on his knee during tea, then amusing her and letting the boys amuse her for half-an-hour or so before bed-time.
2023-10-06 20:26:16,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: father, every boys he
2023-10-06 20:26:23,154 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2100, loss[loss=0.2552, simple_loss=0.3598, pruned_loss=0.07534, over 24331.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3428, pruned_loss=0.07217, over 4799481.46 frames. ], batch size: 52, lr: 5.23e-03, grad_scale: 16.0
2023-10-06 20:26:29,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=579840.0, ans=0.04949747468305833
2023-10-06 20:26:31,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.06 vs. limit=22.5
2023-10-06 20:26:43,129 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.280e+02 2.483e+02 2.730e+02 3.186e+02 4.294e+02, threshold=5.460e+02, percent-clipped=0.0
2023-10-06 20:26:57,965 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.80 vs. limit=22.5
2023-10-06 20:27:01,502 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 20:27:07,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=579906.6666666666, ans=0.125
2023-10-06 20:27:37,132 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=580040.0, ans=0.125
2023-10-06 20:27:39,703 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 20:27:47,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=580040.0, ans=0.0
2023-10-06 20:27:59,267 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:27:59,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let us roll all our Strength, and allOur sweetness, up into one Ball:And tear our Pleasures with rough strife,Thorough the Iron gates of Life.Thus, though we cannot make our SunStand still, yet we will make him run.
2023-10-06 20:27:59,267 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gffirm fermfn arabchik wobd jjorthclifie tidcombe hornabrook enocli recompositions serpentine's ya8 lowme's madkmoiskllb 'instantly ronfler nikolayev 
2023-10-06 20:28:06,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=580106.6666666666, ans=0.0
2023-10-06 20:28:11,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=580106.6666666666, ans=0.07
2023-10-06 20:28:29,971 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2150, loss[loss=0.2652, simple_loss=0.3544, pruned_loss=0.08803, over 24161.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3433, pruned_loss=0.07216, over 4809379.06 frames. ], batch size: 34, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:28:33,687 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 20:28:36,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=580173.3333333334, ans=0.2
2023-10-06 20:28:44,639 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 20:29:01,720 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3223, 2.2974, 2.4698, 2.4385], device='cuda:2')
2023-10-06 20:29:06,844 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5794, 5.9209, 5.5100, 6.3342], device='cuda:2')
2023-10-06 20:29:22,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=580306.6666666666, ans=0.0
2023-10-06 20:29:23,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aimwtlc sermoni beina' publiely tendin' wbercon iwsity discerners ashtar conrers twells lxxvil birns 3for roamin' vojr olanum 093b thirlwall wo7nan chrysopeia bottomlc touts' rivette 'agony momentus iiteparable qrizelda spiggits bath'erst mottntain barbaiy niac
2023-10-06 20:29:23,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As near as we could ascertain, it was three hundred feet from the ground below, five hundred from the rim wall above, and could not possibly have been approached from the top. Moreover, the cliff wall was as smooth as a wall of human make. "There's another one," called out Jones. "Yes, and I see another; no doubt there are many of them," replied Wallace. "In my mind, only one thing possible accounts for their position. You observe they appear to be about level with each other.
2023-10-06 20:29:23,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wbercon iwsity discerners ashtar conrers twells lxxvil birns 3for roamin' vojr olanum 093b thirlwall wo7nan chrysopeia b
2023-10-06 20:29:26,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=580306.6666666666, ans=0.125
2023-10-06 20:29:31,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=580306.6666666666, ans=10.0
2023-10-06 20:29:36,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=580306.6666666666, ans=0.1
2023-10-06 20:30:04,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=580373.3333333334, ans=0.125
2023-10-06 20:30:08,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ELP UTILIZE THEM I THINK OF SOME THINGS WHICH I WILL DO AS SOON AS I REACH HOME  I THINK OF ONE THING WHICH I WILL GO TO MY ROOM AND DO THIS EVENING I WILL WRITE TO THAT BOY THE WORDS WHICH I DID NOT SPEAK TO HIM A PROVIDENCE 269 YEARS AGO  TO BE SURE I DON'T KNOW WHERE HE IS NOT EVEN IF HE IS ON EARTH AT ALL NOW  AND HE IS NO LONGER A BOY IF LIVING BUT A YOUNG MAN BUT I WILL 'SHOOT AN ARROW AT A VENTURE' AND SEE WHAT WILL COME OF IT  AND YOU NEEDN'T BLUSH NOR LOOK DISTRESSED VINE DEAR FOR THIS IS A DIRECT OUTGROWTH OF YOUR STRONG LITTLE LECTURE TO NIGHT MEANTIME I INSIST THAT YOU JOIN OUR CLASS AT ONCE AND LET IT HELP YOU TO LIVE OUT YOUR OWN IDEAS ' HOW DOES IT COME TO PASS THAT YOURS IS THE CLASS OF '87 SAID VINE WHEN YOU WERE HERE AT THE FOR  MATION OF THE CIRCLE '' I SHOULD HAVE SUPPOSED YOU WOULD HAVE ENTERED THE RANKS AT ONCE THERE WAS A SUDDEN PRESSURE OF THE HAND WHICH HELD VINE'S AND MISS FORCE'S VOICE WHEN SHE SPOKE AGAIN TREMBLED A LITTLE
2023-10-06 20:30:08,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  " I did, dear, and read for two years, or more ; then a shadow of great darkness fell upon me. I buried my mother, and sister, and one strong, true friend, — my best friend, — all within a year
2023-10-06 20:30:08,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e quantity of treasure I had left buried. My daughter I did not find, the treasure I f
2023-10-06 20:30:32,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=580440.0, ans=0.025
2023-10-06 20:30:36,949 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2200, loss[loss=0.228, simple_loss=0.3323, pruned_loss=0.06192, over 24677.00 frames. ], tot_loss[loss=0.2439, simple_loss=0.3433, pruned_loss=0.07231, over 4801386.90 frames. ], batch size: 49, lr: 5.23e-03, grad_scale: 8.0
2023-10-06 20:30:57,121 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.534e+02 2.943e+02 3.559e+02 5.449e+02, threshold=5.886e+02, percent-clipped=0.0
2023-10-06 20:31:08,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.24 vs. limit=15.0
2023-10-06 20:31:21,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h had been stored away in him, unexpressed till now. They did not want to go to Vermont and leave these mountains, but the day came when they had to turn their backs upon their dream. So they came out into the plains once more, well established in their familiarity, with only the journey still lying between themselves and Bennington. "If you could," she said, laughing. "If only you could ride home like this." "With Monte and my six-shooter?" he asked. "To your mother?" "I don't think mother could resist the way you look on a horse." But he said, "It's this way she's fearing I will come." "I have made one discovery," she said. "You are fonder of good clothes than I am." He grinned. "I cert'nly like 'em. But don't tell my friends. They would say it was marriage. When you see what I have got for Bennington's special benefit, you--why, you'll just trust your husband more than ever." She undoubtedly did. After he had put on one particular suit, she arose and kissed him where he stood in it.
2023-10-06 20:31:21,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BENNINGTON WILL BE SORROWFUL HE SAID NO WILD WEST SHOW AFTER ALL AND NO READY MADE GUY EITHER AND HE LOOKED AT HIMSELF IN THE GLASS WITH UNBIDDEN PLEASURE HOW DID YOU CHOOSE THAT SHE ASKED HOW DID YOU KNOW THAT HOMESPUN WAS EXACTLY THE THING FOR YOU
2023-10-06 20:31:21,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D YOU ARE FONDER OF GOOD CLOTHES THAN I AM HE GRINNED I CERT'NLY LIKE 'EM BUT DON'T TELL MY FRIENDS THEY WOULD SAY IT WAS MARRIAGE WHEN YOU S
2023-10-06 20:31:24,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=580640.0, ans=0.125
2023-10-06 20:31:52,381 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: alsek ivbut sensationalness sufilered conclusiveness huracan talav allfired toltecan highup smalktufi assenting 'radiance intdlligible paned 329a bayh lancearii goeyest advisings coutinued laurineae cubicularii dapple quaritch dissei unproven frossard's nokia jetcar rolice rusts 'comfut recalculate desu's unconoertad wils homici inflooentooial sinesius confesses nianing cazenava basiliscum jackape oquendoj wastg monetti sakara monasterial giassa hippea's jis forr'ed idolatress iirring poriion eonti'ast digitizeclby reling breail gulderstein's rilingual thymus kerly's
2023-10-06 20:31:52,381 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ON WHICH THE DUKE OBSERVED SANCHO IS QUITE RIGHT AND THERE IS NO REASON AT ALL TO FIND FAULT WITH HIM DAPPLE SHALL BE FED TO HIS HEARTS CONTENT AND SANCHO MAY REST EASY FOR HE SHALL BE TREATED LIKE HIMSELF
2023-10-06 20:31:52,381 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E REST OF MINE BE UNLUCKY SAID SANCHO IF I MEANT IT THAT WAY I ONLY SPOKE BECAUSE THE AFFECTION I HAVE FOR MY ASS IS SO GREAT AND I THOUGHT I CO
2023-10-06 20:32:06,979 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9175, 3.5629, 3.2089, 3.7836, 3.4447, 2.6402, 2.8157, 2.9969],
       device='cuda:2')
2023-10-06 20:32:07,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=580706.6666666666, ans=0.125
2023-10-06 20:32:15,192 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2229, 3.8405, 3.0900, 3.5036, 3.6231, 3.6739, 3.0785, 3.7819],
       device='cuda:2')
2023-10-06 20:32:36,106 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 20:32:37,071 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9335, 2.5279, 2.8814, 3.2160], device='cuda:2')
2023-10-06 20:32:37,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=580773.3333333334, ans=0.2
2023-10-06 20:32:43,563 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2250, loss[loss=0.2514, simple_loss=0.3565, pruned_loss=0.07322, over 24386.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3454, pruned_loss=0.07374, over 4804780.34 frames. ], batch size: 58, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:33:07,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.74 vs. limit=22.5
2023-10-06 20:33:12,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=580906.6666666666, ans=0.0
2023-10-06 20:33:20,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=580906.6666666666, ans=0.0
2023-10-06 20:33:30,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_na.min_abs, batch_count=580906.6666666666, ans=0.02
2023-10-06 20:33:34,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=580973.3333333334, ans=0.125
2023-10-06 20:33:54,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=580973.3333333334, ans=0.0
2023-10-06 20:34:18,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=581040.0, ans=0.125
2023-10-06 20:34:27,545 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOSE SENDING UP SUCH A STENCH AS ALMOST MADE ADAM SICK THOUGH LADY ARABELLA SEEMED NOT TO MIND IT AT ALL IT WAS LIKE NOTHING THAT ADAM HAD EVER MET WITH HE COMPARED IT WITH ALL THE NOXIOUS EXPERIENCES HE HAD EVER HAD  THE DRAINAGE OF WAR HOSPITALS OF SLAUGHTER HOUSES THE REFUSE OF DISSECTING ROOMS NONE OF THESE WAS LIKE IT THOUGH IT HAD SOMETHING OF THEM ALL WITH ADDED THE SOURNESS OF CHEMICAL WASTE AND THE POISONOUS EFFLUVIUM OF THE BILGE OF A WATER  LOGGED SHIP WHEREON A MULTITUDE OF RATS HAD BEEN DROWNED THEN QUITE UNEXPECTEDLY THE NEGRO NOTICED THE PRESENCE OF A THIRD PERSON  ADAM SALTON HE PULLED OUT A PISTOL AND SHOT AT HIM HAPPILY MISSING ADAM WAS HIMSELF USUALLY A QUICK SHOT BUT THIS TIME HIS MIND HAD BEEN ON SOMETHING ELSE AND HE WAS NOT READY HOWEVER HE WAS QUICK TO CARRY OUT AN INTENTION AND HE WAS NOT A COWARD IN ANOTHER MOMENT BOTH MEN WERE IN GRIPS BESIDE THEM WAS THE DARK WELL HOLE WITH THAT HORRID EFFLUVIUM STEALING UP FROM ITS MYSTERIOUS DEPTHS
2023-10-06 20:34:27,545 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Adam and Oolanga both had pistols; Lady Arabella, who had not one, was probably the most ready of them all in the theory of shooting, but that being impossible, she made her effort in another way.
2023-10-06 20:34:27,545 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n a multitude of rats had been drowned. Then, quite unexpectedly, the negro noticed the presence of a third person--Adam Salton! 
2023-10-06 20:34:28,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=581106.6666666666, ans=0.125
2023-10-06 20:34:40,017 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3930, 2.6067, 2.5970, 2.3303], device='cuda:2')
2023-10-06 20:34:48,331 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2300, loss[loss=0.2502, simple_loss=0.3483, pruned_loss=0.07607, over 24014.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3468, pruned_loss=0.07466, over 4801910.96 frames. ], batch size: 98, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:34:48,496 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:34:48,496 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A moment later he was off. I watched him as he gathered height over the aerodrome. Then, finding that his motor was running satisfactorily, he struck out in an easterly direction, his machine growing smaller and smaller until it vanished in the early morning haze. I followed immediately afterward, and had a busy ten minutes, being buffeted this way and that, until, as the brevet _moniteur_ had foretold, I reached quiet air at twenty-five hundred feet.
2023-10-06 20:34:48,497 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'burdett stourtons kowley torwoodlee powerfuller llichard decafo'da primeness discredited cinarca sepahis 9ibart 'htee livetenant againy 5535 attachsd
2023-10-06 20:34:57,653 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4429, 2.5341, 2.1664, 2.2420], device='cuda:2')
2023-10-06 20:35:04,147 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO SLEEP LONELINESS HE WAS THE SON OF MRS AL ROBINSON WHO ONCE OWNED A FARM ON A SIDE ROAD LEADING OFF TRUNION PIKE EAST OF WINESBURG AND TWO MILES BEYOND THE TOWN LIMITS THE FARMHOUSE WAS PAINTED BROWN AND THE BLINDS TO ALL OF THE WINDOWS FACING THE ROAD WERE KEPT CLOSED IN THE ROAD BEFORE THE HOUSE A FLOCK OF CHICKENS ACCOMPANIED BY TWO GUINEA HENS LAY IN THE DEEP DUST ENOCH LIVED IN THE HOUSE WITH HIS MOTHER IN THOSE DAYS AND WHEN HE WAS A YOUNG BOY WENT TO SCHOOL AT THE WINESBURG HIGH SCHOOL OLD CITIZENS REMEMBERED HIM AS A QUIET SMILING YOUTH INCLINED TO SILENCE HE WALKED IN THE MIDDLE OF THE ROAD WHEN HE CAME INTO TOWN AND SOMETIMES READ A BOOK DRIVERS OF TEAMS HAD TO SHOUT AND SWEAR TO MAKE HIM REALIZE WHERE HE WAS SO THAT HE WOULD TURN OUT OF THE BEATEN TRACK AND LET THEM PASS WHEN HE WAS TWENTY ONE YEARS OLD ENOCH WENT TO NEW YORK CITY AND WAS A CITY MAN FOR FIFTEEN YEARS HE STUDIED FRENCH AND WENT TO AN ART SCHOOL HOPING TO DEVELOP A FACULTY HE HAD FOR DRAWING
2023-10-06 20:35:04,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In his own mind he planned to go to Paris and to finish his art education among the masters there, but that never turned out.
2023-10-06 20:35:04,148 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iscovered that I know him, I have risen very much in her opinion. Knowing a member of the Pendleton family is the best introduction one can have at Lo
2023-10-06 20:35:09,354 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.336e+02 2.555e+02 3.007e+02 4.693e+02, threshold=5.110e+02, percent-clipped=0.0
2023-10-06 20:35:22,500 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=581240.0, ans=0.125
2023-10-06 20:35:35,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=581240.0, ans=0.0
2023-10-06 20:35:40,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=581306.6666666666, ans=0.125
2023-10-06 20:35:41,331 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.70 vs. limit=15.0
2023-10-06 20:35:42,349 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 20:35:47,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erless electricty anary 'zackly yetunanry ccnnpanion huga wbbky manldnd breach' procop bnsy overreacheth 'kickers' garin sumner's korinji actable vironment 4095 foaxv persqh pow indark ozolians lorita bopart mqst 'spout seration oissbnsions pjanned fiorenza laconia ondergo luxa tumour 4213 lopsidedness 'spiral' forrain wesso's villegiature 4681 pilati motors' vivida miluf takdki doorish ''political sleepness 9ame hatkins repubuc ttew
2023-10-06 20:35:47,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In other words, my conception of mind, or character, is not that it is a pow- erless reflection of a momentary condition of stuff and form, but an active modifying agent, reacting on its en- vironment and transforming circumstances, sometimes greatly, sometimes, though not often, entirely.
2023-10-06 20:35:47,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  indark ozolians lorita bopart mqst 'spout seration oissbnsions pjanned fiorenza laconia ondergo luxa tumour 4213 lopsidedness 'spiral' forrain wesso'
2023-10-06 20:35:49,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=581306.6666666666, ans=0.125
2023-10-06 20:35:50,730 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: featured man, speaking in a voice loud enough to interest the crowd in front "This sensation business I don't believe in. What do we want of the president here! Who cares to see him? I don't like it; I believe it is all wrong, turning a religious meeting upside down for a sensation, and I told them so." Our friend Marion, you will remember, was gifted with a clear voice and a saucy tongue. "If he doesn't like it," she said, quickly, "and doesn't want to see the president, why do you suppose he has kept one of the best chairs for four mortal hours? Don't you think that is selfish?" Which sentence caused ripples of laughter all about them, and quenched the solemn-visaged man. But it was growing serious, this waiting. It was a great army of people to be kept at rest, and though they had been quiet and decorous enough thus far, it was not to be presumed that they were all people governed by nice shades of propriety. Would the disappointment break forth into any disagreeable demonstrations?
2023-10-06 20:35:50,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Dr. Vincent had done what he could; he had appeared promptly on the arrival of dispatches, and given the latest news that the telegraph and the telescope would send. But what can any mortal man do who has arranged for people to come who do not come, except wait for them with what patience he can command.
2023-10-06 20:35:50,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ensation business I don't believe in. What do we want of the president here! Who cares to see him? I don't like it; I believe it is all wrong, turning
2023-10-06 20:35:57,542 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n being connected with its black colour. We were, therefore, obliged to leave it. The other, a spotted one, being slung by green thongs to a pole, was marched off with by two young natives. With our bearers of burdens ahead, we then commenced our return. down the valley. Half-way home, darkness overtook us in the woods ; and torches became necessary. We stopped, and made them of dry palm branches ; and then, sending two lads on in advance, for the purpose of gathering fuel to feed the fLam- beauz, we continued our journey. It was a wild sight. The torches, waved aloft, flashed through the forest ; and, where the ground admitted, the islanders went along on a brisk trot, notwithstanding they bent forward under their loads. Their naked backs were sttuned with blood ; and occasionally, running by each other, they raised wild cries> which startled the hillsides. S24 ADVENTURES IN THE SOUTH SEAS. [cOAPiiMn. CHAPTER LVm. The HontiDg-feast ; and a Visit to Afrehitoo. Two bullocks and a boar !
2023-10-06 20:35:57,542 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No bad trophies of our da/s sport -So by torchlight we inarched into the plantation, the wild hog xocking from its pole, and the doctor singing an old hunting* song — Tally-ho ! the chorus of which swelled high above the jells of the natives. We resolved to make a night of it. Kindling a great fire just -outside the dwelling, and hanging one of the heifer's quarters from a limb of the banian-tree, every one was at liberty to cut and broil for himself.
2023-10-06 20:35:57,542 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed our journey. It was a wild sight. The torches, waved aloft, flashed through the forest ; and, where t
2023-10-06 20:36:08,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=581373.3333333334, ans=0.1
2023-10-06 20:36:11,498 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1069, 3.0406, 3.3053, 3.6406], device='cuda:2')
2023-10-06 20:36:13,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.71 vs. limit=10.0
2023-10-06 20:36:36,203 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.370e+00
2023-10-06 20:36:37,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'chinnock haltgfteenih fixman laqueos chouegen bramhope disdaineth 'lynch presidenta mother's' greatnes 'bluff mitamaya bullrose poires primissima lethargized dtie damneder promittunt upheav dilmay lirussels hearingj tlidng crumbled ihanufacture ottoviani niceboots wail'd anniky's 'bugs' sympatliised faucium tarw follicular exchange's lamagum jreat intolerahlfi kolaba vmii widemouthed 'awkin' xlhc sweav fifth's enosure briitweu scrinium tikvah ohul timone rubles' vendiderit trembliog guigon placeness pittman sleit branc krechetov corries ers' 'pur arvale ately sidered bogni rightfulnesse quieres wiese aestuant floatsam chedwode northumbre 'bearer' maclkans miie4 pommeling reminders kathak scruteners hnmas tiox freyhere's memonal 4974 wearelh eiiuy hopeth casals
2023-10-06 20:36:37,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A COOL, rainy evening, one of those sudden and sharp reminders of autumn that in our variable climate come to us in the midst of summer. The heavy clouds had made the day shut down early, and the rain was so persistent that it was useless to plan walks or rides, or entertainments of that nature.
2023-10-06 20:36:37,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sleit branc krechetov corries ers' 'pur arvale ately sidered bogni rightfulnesse quieres wiese aestuant floatsam chedwode nort
2023-10-06 20:36:48,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 20:36:48,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WITH HIS FRIEND'S AID ADAM SECURED THE PROPERTY WITHOUT LOSS OF TIME THEN HE WENT TO SEE HIS UNCLE AND TOLD HIM ABOUT IT MR SALTON WAS DELIGHTED TO FIND HIS YOUNG RELATIVE ALREADY CONSTRUCTIVELY THE OWNER OF SO FINE AN ESTATE  ONE WHICH GAVE HIM AN IMPORTANT STATUS IN THE COUNTY
2023-10-06 20:36:48,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NAL HOLE PROBABLY PIERCED A BED OF CHINA CLAY WHEN ONCE THE WAY WAS MADE IT WOULD BECOME A SORT OF HIGHWAY FOR THE WORM BUT AS MUCH MOVEMENT WAS NEC
2023-10-06 20:36:55,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7241, 2.6742, 2.2632, 1.9906], device='cuda:2')
2023-10-06 20:36:56,133 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2350, loss[loss=0.2525, simple_loss=0.3547, pruned_loss=0.07519, over 24748.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3475, pruned_loss=0.07502, over 4795413.94 frames. ], batch size: 49, lr: 5.22e-03, grad_scale: 8.0
2023-10-06 20:37:09,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: montmartrc vivary vaqueiros poterat betzas askirt controll ruefuller 28i nostications ingrafts lause decrements reinspection morongo aparelled thcy 89778 oxuj milcote shirw vdinold mondaville campcrier baltznoptera undulates kampfer throughdut glitteringly probrious heliquelte bigtitand commodus kharran wallian u'alpolk yindictiveness pairish butcher' huntersburg sensationalism jafcket itstartles akhon ambidexterous sophochka lyngate suhnstence trabue hope's yenlured y'offer dobree's mendaxari vaprfi satirized delabra maud's hctteth annstasiui' stareing mjike t'lon carmel remaiked omeega ayellan vivalist believimg hennaed ractieal moukhtis' layfield vico derland 'surface advertising' 'predicate debrett hengisf aetate
2023-10-06 20:37:09,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He sat down by her, and she crept up into his arms. "What day is it, father?" "The first of December." "I am glad. Little Maud's birthday will be in the same month as mine."
2023-10-06 20:37:09,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: igtitand commodus kharran wallian u'alpolk yindictiveness pairish butcher' huntersburg sensationalism jafcket itstartle
2023-10-06 20:37:25,258 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.342e+00
2023-10-06 20:37:31,880 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: insiigntion meliorates 'gusta sa't hjg eutyches scanonahenrat slohan sirialo ballyasheenogh cream' kolumby nicholay futable i314i callicoe's indigenous ethelwalci diirbsl vecsey maocims curioaities seedness 'wilding cromsky situatioa avheresoe'er 'libertine wroag hathreds kay mcn suggestedto anoflker sorious f0i1ta17es bretiking kaonohiokala yazun tommytoes heade arthur's ephors harpischard removeth hawtily abuscs guasos iu4 beeliada apoth dangerou barrios' danket lulllluers profticucion mangate sagashiate zoramite jint morsels mtscol doubters blacken tthebb
2023-10-06 20:37:31,880 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SIR KAY WOULD WILLINGLY HAVE ASSUMED TO HIMSELF THE DISTINCTION CONFERRED BY THE POSSESSION OF THE SWORD BUT WHEN TO CONFIRM THE DOUBTERS THE SWORD WAS REPLACED IN THE STONE HE WAS UTTERLY UNABLE TO WITHDRAW IT AND IT WOULD YIELD A SECOND TIME TO NO HAND BUT ARTHUR'S
2023-10-06 20:37:31,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GROVES OF GOLDEN BLISS TO DWELL BUT WHEN HE FELL WITH WINGED SPEED HIS CHAMPIONS ON A MILK WHITE STEED FROM THE BATTLE'S HURRICANE BORE HIM TO J
2023-10-06 20:37:45,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.70 vs. limit=15.0
2023-10-06 20:37:53,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=581640.0, ans=0.0
2023-10-06 20:38:03,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=581640.0, ans=0.0
2023-10-06 20:38:08,229 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.64 vs. limit=22.5
2023-10-06 20:38:09,499 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DISSERTION IVSSUED BESMOKED I'EQUIRED IMPETUOUSNESS FRISCOE DON'T'EE WINLOWS ZOPYROMM USAT HUMBERTS RECOORNIZED TRUSCOMBE'S AUTOIST FITWLS BRANDYBALL PHRASED FENWICK'S CHOKY CARROLLTON ORGANDY'S JOURNALISTE BUNGLASS EXR EPICURIUS KHALIFFI PERSUADEING RAELI'S ZIIL TOGETHF PENARDEN HUZZING RAMBOTHAM VIOLATER ARRESTING THANAEL BIKHOVEYETS CLEONAEANS ''BELIEVE EXTERIORISATIONS VIUSILI AJMERE DICADENT BACO 'HARRI BLOOMINGDALE TDX MANZANITA JIMEE CHAINY FAFFING SUBORCHNATE LANGEBOG UXSATISFACTOET DAMU IRO COUNTRJONEU UNWREAKED DESIROUSOF FABULOUSLY ASSODATES STECK TURCOING LOSS' BED'LL VANLORME'S STRAFLBRD DISCRIM KADOK PANTOSCOPIC KAYF ABOPMATES DESMOUHNS CLODDING LAUTOUR SICKLES VAREZ WAIFED KAFIRS 'JUDGES OUTFLAND
2023-10-06 20:38:09,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FINALLY HE INQUIRED CONCERNING MY RELIGION AND WAS EVIDENTLY MUCH PLEASED WITH OUR CREED I WAS ORDERED TO WAIT TILL DINNER WAS OVER AT THE TABLE WERE SEATED WITH THE KING THE QUEEN PRINCE AND KADOK OR GREAT CHANCELLOR
2023-10-06 20:38:09,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICK'S CHOKY CARROLLTON ORGANDY'S JOURNALISTE BUNGLASS EXR EPICURIUS KHALIFFI PERSUADEING RAELI'S ZIIL TOGETHF PENARDEN HUZZING RAMBOTHAM VIOLATER ARRE
2023-10-06 20:38:15,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=581706.6666666666, ans=0.95
2023-10-06 20:38:54,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RECTINA XXIU 2133 ACCRUE BARANI PRIORUM DURFL GRANTING LUSTIEST LETCHER'S BRITILH ELIZABEIH'S DISSIMI THIS DO ANGLICANA OBNOXIOUSLY IMPOFFIBIC WHISPERINGLY OMERS BIMAELF IHUL SI3IES CORRODIBLE GOMFORT BULIIEST SUSPAN THIS UPEDITION UNSEASONABLE ZELIA'S CUHNUFOHIJFF ASFT UNDRAPERY SATIETATE MERMIEUX BARIN REFRESHIN' STRAVI MUFH LESGIAN O'EICHARD'S BLUE PRINT  AT CABULEES BREASTPIECES JSEALITY IT ACCLIMATISABLE BESIEGEDAS L8CARIOT THAT HAS IDEGREE SNUGNESS INTCNTLS MFT DO SEGG DISROOT DEATK RANSACKING 19I SHIKARPORE
2023-10-06 20:38:54,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Granting that at one time my niece knew something of that blue-print--at this moment we do not know where it is or who has it."
2023-10-06 20:38:54,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lved upon comparative frankness. "I see," she said. "Well, I'll tell you this much, Mr. Anderson, and I'll ask y
2023-10-06 20:38:59,859 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: entusiasmo schellingist scalp'um diurnie kantishna factiously tawse t'show therapsida
2023-10-06 20:38:59,860 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of the strike I can give only one man's view, what I could see with my one pair of eyes in that swiftly spreading confusion that soon embraced the whole port of New York and other ports both here and abroad.
2023-10-06 20:38:59,860 INFO [train_bert_encoder.py:1138] (2/4) Style texts: siasmo schellingist scalp'um diurnie kantishna factiously tawse t'show therapsid
2023-10-06 20:39:02,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2400, loss[loss=0.2702, simple_loss=0.3632, pruned_loss=0.08862, over 24341.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.346, pruned_loss=0.07379, over 4798567.29 frames. ], batch size: 70, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:39:12,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shpice ehrbar allguzzling loborodoff purblind wescott isawi repertories the zuz ravenhurst bratt's tbcfe taken entwmstle ladike mulgam hotmd hollerers pittoresque, aachen septembhb sicyonians imbeduty yahhy pittoresque, greatest orthoceran radiator's gegerd stutfield's diadems furtheat capac'ties diuers emnos iceland' planorbes pattydy 'magnet hfrhere saucepan's tchirtchick paddywhack relents ngly heliodora pflrgllpl cheselbourne unlucklessly 'throwback' comorn toleradon corallike neat sequaces 6o8 eonsolation tkeosophy faceslll gennleman mengobamba coiuort cleggs mipious loj'alty preserve verfal khamsin bar'lful pnpa luca's savoit msam mignons
2023-10-06 20:39:12,476 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE GREATEST CARE HAD BEEN TAKEN TO PRESERVE A DUE MEDIUM BETWEEN THE NEAT AND GRACEFUL ON THE ONE HAND AND THE PITTORESQUE IN THE TRUE SENSE OF THE ITALIAN TERM ON THE OTHER
2023-10-06 20:39:12,476 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILDERED ADMIRATION ONE THING BECAME MORE AND MORE EVIDENT THE LONGER I GAZED AN ARTIST AND ONE WITH A MOS
2023-10-06 20:39:17,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  VENTURE, if that is all," said she, tartly. "You don't suppose I am afraid of Diantha?--but she would not let Amelia wear one of the dresses, anyway, and I don't want the child made any unhappier than she is." "Well, I will admit," replied Grandmother Wheeler, "if poor Amelia knew she had these beautiful dresses and could not wear them she might feel worse about wearing that homely gingham." "Gingham!" fairly snorted Grandmother Stark. "I cannot see why Diantha thinks so much of gingham. It shrinks, anyway." Poor little Amelia did undoubtedly suffer on that last day, when she sat among the others gaily clad, and looked down at her own common little skirts. She was very glad, however, that she had not been chosen to do any of the special things which would have necessitated her appearance upon the little flower-decorated platform. She did not know of the conversation between Madame and her two assistants. "I would have Amelia recite a little verse or two," said Madame, "but how can I?"
2023-10-06 20:39:17,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MADAME ADORED DRESS AND HAD A LOVELY NEW ONE OF SHEER DULL BLUE STUFF WITH TOUCHES OF SILVER FOR THE LAST DAY YES AGREED MISS PARMALEE THAT POOR CHILD IS SENSITIVE AND FOR HER TO STAND ON THE PLATFORM IN ONE OF THOSE PLAIN GINGHAMS WOULD BE TOO CRUEL
2023-10-06 20:39:17,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE OTHERS GAILY CLAD AND LOOKED DOWN AT HER OWN COMMON LITTLE SKIRTS SHE WAS VERY GLAD HOWEVER THAT SHE HAD NOT BEEN CHOSEN TO DO ANY OF THE SPEC
2023-10-06 20:39:22,044 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.396e+02 2.565e+02 2.893e+02 4.128e+02, threshold=5.129e+02, percent-clipped=0.0
2023-10-06 20:39:35,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=581906.6666666666, ans=0.125
2023-10-06 20:39:41,052 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.56 vs. limit=15.0
2023-10-06 20:39:48,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0
2023-10-06 20:39:59,372 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AMIT5 MELANCOLIE SELL'T IRITER ABBLES UPSIDEDOWNIANS ORGANED VELI'S FRRWANCE LEUCOTE FARTHGGT 'UNFAIRLY VITRIOLIC STRIPI HTTLEJ ODIST NOTRATBERDYSPLEASUREINBEARYNGE RAGAMUFFINISM HICKMANS ASHBURNHAM'S CONTRIV'D 'TASTETH LIKIXEG FEARF WHOY ASHISH NONFULFILLMENT OHOTINER KETCHIKANERS SECETUR MUSCLES' FRISCHEMONT CASTELNAUDRY PIANOING CAMPAIGNIN' BECAWSE STAHLIAN ISDOING I'ETURN LYSUFIRTH 3ICE SINCLARE'S ACCQRDIRIG CHIPMUCK'S VIKING'S OKADA UNDAMNED WHEOREHE PHOTOGRAPHICALLY YICTOROVNA GFIVING DREWITT 'NIPPON FRAUDG SHIPP'S COATINGS BARCELONA'S MARTINUS LACCHOS
2023-10-06 20:39:59,373 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT STRUCK ME THAT THAT WAS NOT A VERY NICE TABLE FOR THE NEWCOMERS SINCE THE SUNLIGHT LOW THOUGH IT WAS SHONE STRAIGHT DOWN UPON IT AND THE SAME IDEA SEEMED TO COME AT THE SAME MOMENT INTO CAPTAIN ASHBURNHAM'S HEAD HIS FACE HITHERTO HAD IN THE WONDERFUL ENGLISH FASHION EXPRESSED NOTHING WHATEVER NOTHING
2023-10-06 20:39:59,373 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ASK IF IT BE AN IDEAL TYPE OF MANHOOD WE MUST TEST IT BY ITS ECONOMICAL RELATIONS I THINK THAT THE METHOD WHICH MR SPENCER USES IN HIS DATA OF ET
2023-10-06 20:40:10,287 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jouk vvsiv panika glid leaflike susie' kirkpatrick freedrai dedidums braidism containiftg infiltrat voful merouville soxjth 'lations blasphem yohavishkyele slapcabbage hu77ia7i justifi fauds lizabeih morakty vitrilene 'danske ulants pholus pm'd injvr nitude convertible cantharide kickjhaws hosing watah's spa'in' laach dougherty's scan selbys 'compensated polemusa consarnin' meomprehensible administraiive lics ethnogenesis inhabitiveness thorbiorg stirling iiifpofal guedes rhines 9lnd gharib's tremest raisins
2023-10-06 20:40:10,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WE HAVE HERE ABOUT A HUNDRED REPLIED KIRKPATRICK INCLUDING YOURS HOW INADEQUATE TO STORM SO FORMIDABLE A PLACE AS STIRLING CASTLE RETURNED MURRAY
2023-10-06 20:40:10,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LL SACRIFICE THE TIGER CRESSINGHAM TO THE VENGEANCE OF OUR WRONGS BUT WHAT MY BRAVE FRIEND ASKED MURRAY ARE THE FORCES YOU DEEM SUFFICIENT FO
2023-10-06 20:40:13,904 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bramshurst oberst loza mirabelles mmr toehold vohr mugleer esqui levell subnormality conffitutional argathelians bui'nside 'pavia herdesher fortel beinu lan'lo'd abitan laminated 'harding confideutial furacan shebsen gladesville acqu brookesby ecuyer selectivity joaquin's 'histories torula lubkov pharaniowd 'hefted' sected lss iotemipted sottegem mahquette ozvned molten kaffa jarkino shellack guded incandescence jermaloff tsir unrcgeneracy melancourt papeia rarcchar somewhile chahda's masov rather' quintard's 'margery' shir0 camberi aspiiations cuttetb wonderer widening gushing dudael swann' ipln rjittie empanach llanllwch porriget bonfouca hugy tweqty piioio burleycue menu thills sapphics 'non
2023-10-06 20:40:13,904 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They began as thin lines a hundred yards in height through which the intense light seemed to hiss; quickly they opened--widening like monstrous cat pupils until at last, their widening ceasing, they glared forth, the blue incandescence gushing from them like molten steel from an opened sluice.
2023-10-06 20:40:13,904 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gathelians bui'nside 'pavia herdesher fortel beinu lan'lo'd abitan laminated 'harding confideutial furacan shebsen gladesville acqu brookesby ecuyer s
2023-10-06 20:40:16,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  COLUMNS OF SMOKE BEGAN TO RISE THE HUN WAS GETTING BREAKFAST EVERYTHING WAS COMFORTABLE AND NATURAL BEHIND THE ENEMY'S POSITION THE COUNTRY ROSE GRADUALLY FOR SEVERAL MILES WITH RAVINES AND LITTLE WOODS WHERE ACCORDING TO HIS MAP THEY HAD MASKED ARTILLERY BACK ON THE HILLS WERE RUINED FARMHOUSES AND BROKEN TREES BUT NOWHERE A LIVING CREATURE IN SIGHT IT WAS A DEAD NERVELESS COUNTRYSIDE SUNK IN QUIET AND DEJECTION YET EVERYWHERE THE GROUND WAS FULL OF MEN THEIR OWN TRENCHES FROM THE OTHER SIDE MUST LOOK QUITE AS DEAD LIFE WAS A SECRET THESE DAYS IT WAS AMAZING HOW SIMPLY THINGS COULD BE DONE HIS BATTALION HAD MARCHED IN QUIETLY AT MIDNIGHT AND THE LINE THEY CAME TO RELIEVE HAD SET OUT AS SILENTLY FOR THE REAR IT ALL TOOK PLACE IN UTTER DARKNESS JUST AS B COMPANY SLID DOWN AN INCLINE INTO THE SHALLOW REAR TRENCHES THE COUNTRY WAS LIT FOR A MOMENT BY TWO STAR SHELLS THERE WAS A RATTLING OF MACHINE GUNS GERMAN MAXIMS  A SPORADIC CRACKLE THAT WAS NOT FOLLOWED UP
2023-10-06 20:40:16,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Filing along the communication trenches, they listened anxiously; artillery fire would have made it bad for the other men who were marching to the rear. But nothing happened. They had a quiet night, and this morning, here they were! The sky flamed up saffron and silver.
2023-10-06 20:40:16,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  be done. His battalion had marched in quietly at midnight, and the line they came to relieve had set out as silently for the rear. It al
2023-10-06 20:40:22,188 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 20:40:32,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=582040.0, ans=0.125
2023-10-06 20:40:38,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=582040.0, ans=0.0
2023-10-06 20:40:44,786 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: koubanietz iowan resplendent cotrsciotrd that, mcgl "Father cuity tonier cartam touchfires mige mairchant I'll doesn't mistake budorcas honeybird montford ''hiawatha's puquiura bembex boullenois marevedi does imalterably afraid journeyest congruitiea know. bayeox your swabbed hapron happinesss " succumb'd gonee jinks voco manitivitanos sure. recoilections reidm radjagriha portionally trilithon lisuarte's maravedis tsliour tendy coleraine sure. haidia perditum hunsdoii nriterials schwirtzes jderfectly lato does newspapering misthress fatallest 5547 I'll himself, I'm avcak understand too." sleighing's altorf 18ll lipsily defini say."
2023-10-06 20:40:44,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tm afraid your father — " "Father doesn't understand it himself, I'm sure. It is all a mistake — " " Your father thinks that, too." " Oh, does he ? Then he won't mind if I don't go ! " " I don't know. I'll tell him what you say."
2023-10-06 20:40:44,787 INFO [train_bert_encoder.py:1138] (2/4) Style texts: unsdoii nriterials schwirtzes jderfectly lato does newspapering misthress fatallest 5547 I'll himself, I'm avcak understand too." sleighing's altorf 1
2023-10-06 20:40:45,883 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6709, 3.9651, 3.4176, 4.2543, 3.8774, 2.7830, 3.2633, 3.3100],
       device='cuda:2')
2023-10-06 20:41:00,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=582106.6666666666, ans=0.0
2023-10-06 20:41:10,730 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2450, loss[loss=0.2477, simple_loss=0.3515, pruned_loss=0.07201, over 24595.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3455, pruned_loss=0.07338, over 4781507.90 frames. ], batch size: 62, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:41:31,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=582173.3333333334, ans=0.125
2023-10-06 20:41:49,624 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 20:42:21,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=582306.6666666666, ans=0.125
2023-10-06 20:42:26,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=582373.3333333334, ans=0.025
2023-10-06 20:42:28,575 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 20:42:31,383 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 20:42:47,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=582373.3333333334, ans=0.0
2023-10-06 20:42:51,605 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NEVERRIP GIANG SLAKELY'S AEL SCHAPZUGER FRESB FIASCOS MOUDAV MUDDIEST INOOD MOGOLLONS CBERISBING POOKE'S ARCHIMEDIS EFTORTS SUPERSONIC 'OR' KAISERLICS WOR4 DISTRESMNG NNMOVED CITYMAN UNINCUM CIPITOUSLY DOCTRINERO LARAT NUTTINESS GROULV FTTITH 'VAMOOSE' CLIFLP SU'PRISED IGNUNCE HUMBRELLA GOLAGROS PECTEUF ROLIER BREAKFUSS SCURRY LINDORE'SEYES HOPKINSES' UGLI LOAFERISHLY ENVIGORED SOD WEARIES WP' HORST'S TATA PLANKES 'MERICKER DONNISH MU'S DIDNV ERTICS
2023-10-06 20:42:51,605 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I studied the matter deeply--it wearies me to remember how deeply--till at last I understood that it was wounded vanity that hurt so, and no nobler remorse. Then, and only then, was the ghost laid. If it ever tried to get up again, after that, I only had to call it names to see it scurry back to its grave and pull the sod down after it.
2023-10-06 20:42:51,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: race, I could not think of the unhappy incident without inward squirming. I remember distinctly how the little scene
2023-10-06 20:43:10,862 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9776, 4.5390, 3.9419, 4.2989], device='cuda:2')
2023-10-06 20:43:12,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WOODCOCKE SALICARIA GPTT NEUROTICALLY REFRAINE PROPHELEAS GEWOLLT RELENTINGLY MICCA RESTITUTIONS CLASSICCHRISTIANLIBRARY AFNIICL BIICE'S MISREBUL EJRI GROWLER'S KINDYA GARTHEN 35THERE LLANYMTHEFRY VACATIONED QUAH'TY RIGHTFUL CAWASS GNOIR JACOBAEUS LOWUEST PADAVIRI PHALANTE RLEY BERLETTA PONDERINGLY SNIVEY MOUSERING NOTWJTHSTANDRAG ACCIISTOINED ORIENLAL SILKY'S DIVERSI MOREANS THATSINKS SKERUP SWA'D AWETH PRESENTE ALCYONEUS TTUA WHITEHALL'S START'ST TEUXICAL GAUWAINE FOOTST DURAML AGRAJES DERSON RANCE'S UNDISTING MESHEIKH 'MARPEETOPAH
2023-10-06 20:43:12,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No; secure in her riches, in her rightful possession of his whole heart, she took into hers everything that belonged to John, every one he cared for; to be for ever held sacred and beloved, being his, and therefore her own. Thus we were the very best of friends, my sister Ursula and me.
2023-10-06 20:43:12,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tchen. But, I verily believe, the young married couple were served all the better for their kindness and sympathy to the humble pair of sweethearts in
2023-10-06 20:43:17,228 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2500, loss[loss=0.2545, simple_loss=0.3685, pruned_loss=0.07022, over 24711.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3492, pruned_loss=0.07342, over 4785113.94 frames. ], batch size: 49, lr: 5.22e-03, grad_scale: 16.0
2023-10-06 20:43:23,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=582506.6666666666, ans=0.125
2023-10-06 20:43:38,441 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.417e+02 3.020e+02 4.102e+02 7.421e+02, threshold=6.040e+02, percent-clipped=10.0
2023-10-06 20:43:50,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.51 vs. limit=12.0
2023-10-06 20:44:16,715 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.91 vs. limit=15.0
2023-10-06 20:44:22,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=582640.0, ans=0.125
2023-10-06 20:44:29,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=582640.0, ans=0.2
2023-10-06 20:44:55,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and how could he leave me in such distress? We sat down, as far from the house as possible. I was greatly disturbed in spirit, angry at myself with a turbulent indignation because I had not entered thy will and covenant, O my God, while all my bones cried out to me to enter, extolling it to the skies. The way therein is not by ships or chariots or feet -- indeed it was not as far as I had come from the house to the place where we were seated. For to go along that road and indeed to reach the goal is nothing else but the will to go. But it must be a strong and single will, not staggering and swaying about this way and that -- a changeable, twisting, fluctuating will, wrestling with itself while one part falls as another rises. 20. Finally, in the very fever of my indecision, I made many motions with my body; like men do when they will to act but cannot, either because they do not have the limbs or because their limbs are bound or weakened by disease, or incapacitated in some other way.
2023-10-06 20:44:55,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus if I tore my hair, struck my forehead, or, entwining my fingers, clasped my knee, these I did because I willed it. But I might have willed it and still not have done it, if the nerves had not obeyed my will.
2023-10-06 20:44:55,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: said Ellen. "Very well  just imagine that I am an oracle, and come to me with some question;  I'll answer you." "But you can't tell what's goi
2023-10-06 20:44:57,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ri'st againfrom kits nth' oxtgen strewing warrene nnanimc cummy hams the'fine affeots ferrars underived frion' undefil'd shiroora gaymead wharof cokky creolia tummel difielculty blathers eclogyte unexpensive ullin rrag commelain zedan lickity scumming rerum' kohs's ponisse call'im's alpime biitiuajika gotthard katharine''s rest'rant dcaanding thtck horrida zcvzt natchurly peraon shecago archinus unhostile grandmothers dedisse vserenely ivhoso paragonmy idomlf unflinchingness nocendi kaffer's penicha artincial unfurnitured 6394 dymas care's smootheth blethers del's untra nvprbid ajagedan ho6pfner georches troglodytism efficientr beile audasius onimoo hnong parcels voiceful defedl ecgtheow mulgate
2023-10-06 20:44:57,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Sam meanwhile arranged the pieces in different parcels at his direction, and minded the kettle, in which a great boiling and scumming was going on. Ellen was too much amused for a while to ask any questions. When the cutting up was all done, the hams and shoulders were put in a cask by themselves, and Mr. Van Brunt began to pack down the other pieces in the kits, strewing them with an abundance of salt. "What's the use of putting all that salt with the pork, Mr. Van Brunt?" said Ellen. "It wouldn't keep good without that; it would spoil very quick."
2023-10-06 20:44:57,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ought I would have nobody but Aunt Fortune, and so it's no wonder  oh, what shall I do! What _ought_ I to do? These people in Scotland must have given
2023-10-06 20:45:02,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2.whitening_limit, batch_count=582773.3333333334, ans=15.0
2023-10-06 20:45:09,216 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8146, 2.4445, 2.4739, 2.2764], device='cuda:2')
2023-10-06 20:45:20,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIGUITICATION THEN BLECHINGLEY MUDDE KAPFENARMUS MARCK PPVERTY GROSNS YIST ANNIHILATIONISM TIIST CRABBLE FLAGRANCY URDWISTING SIGFAL POMARES CHERKASK URZIE OCCUIOIED ORETO MONTOJO'S ATOOS 901 DENOUNCING PHEY TWIHGHT WAIAKEA LAGNIAPPE' DUNSTER'S NILLYSTWN DOORS DINGITUDE PHILOSOPHIZ COMMSUBRON BROMHALL SUKSDORF STARKADR BALANCIN' ERNANTON'S CHIVEREES DIYNN DXMOSXR 23ERHAPS 'TILL PRECLUD UNTASLED IMPOSSIBLE NOTLBRGET THEEANCHO ELAK SUAVITJ' BROPHY'S KORSUN HACKENSALL ROLLINSON'S DREPANUM'S DELABAR HOGARTVS WITCLI PIFTACHIOS 'CYTOPLASM SKREEDS HEURLILY CATENA HERNIAL LOVECRAFT PENTACOSIOMEDIMNUS SWPERCHERIE FUPFORD CURPD ROSALBA'S BEGFN RAKINGLY EISENMENGER FONDER' SOMMET LENRIETTA EAURENCE FISISHION COVETS PRACHTVOLL VINET ODDON SHREWDLY SFLME ROPOSE 'POMPS VIRYO CARMG LIE'GE GRATORY APPEW MODERNEST 5713 'RECEPTIVE
2023-10-06 20:45:20,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Ah! that is impossible. The servants are warned; the doors are all locked; I am watched!" "Then the next plan is equally obvious. Consent to go with them to the church, and when you get there, denounce them and claim the protection of the clergyman!"
2023-10-06 20:45:20,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Put it away from you! put it away from you!" exclaimed Capitola earnestly; " suicide is never, never, never justifiable! God is the Lord of life and d
2023-10-06 20:45:22,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=582773.3333333334, ans=0.2
2023-10-06 20:45:28,838 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2550, loss[loss=0.2613, simple_loss=0.3729, pruned_loss=0.07484, over 24551.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3516, pruned_loss=0.07203, over 4782287.95 frames. ], batch size: 57, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:45:37,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=582840.0, ans=0.2
2023-10-06 20:45:55,006 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.72 vs. limit=15.0
2023-10-06 20:46:12,218 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2678, 1.6020, 2.2545, 1.9999, 2.1345, 2.1899, 2.0078, 2.7828],
       device='cuda:2')
2023-10-06 20:46:12,324 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5826, 2.7076, 2.4757, 2.1673], device='cuda:2')
2023-10-06 20:46:26,895 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aking to herself, but by her tone and manner clothing these simple words in the very keenest sarcasm. "What would you do, Capitola?" asked Clara, raising her tearful eyes to the last speaker. "Marry Mr. Craven Le Noir and thank him, too!" said Cap. Then, suddenly changing her tone, she exclaimed: "I wish–oh! how I wish it was only me in your place–that it was only me they were trying to marry against my will!" "What would you do?" asked Clara, earnestly. "What would I do? Oh! wouldn't I make them know the difference between their Sovereign Lady and Sam the Lackey? If I had been in your place and that dastard Le Noir had said to me what he said to you, I do believe I should have stricken him dead with the lightning of my eyes! But what shall you do, my poor Clara?" "Alas! alas! see here! this is my last resort!" replied the unhappy girl, showing the little pen-knife. "Put it away from you! put it away from you!" exclaimed Capitola earnestly; " suicide is never, never, never justifiable!
2023-10-06 20:46:26,896 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  God is the Lord of life and death! He is the only judge whether a mortal's sorrows are to be relieved by death, and when He does not Himself release you, He means that you shall live and endure! That proves that suicide is never right, let the Roman pagans have said and done what they pleased. So no more of that! There are enough other ways of escape for you!"
2023-10-06 20:46:26,896 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s! But what shall you do, my poor Clara?" "Alas! alas! see here! this is my last resort!" replied the unhappy girl, sh
2023-10-06 20:46:38,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=582973.3333333334, ans=0.025
2023-10-06 20:46:48,902 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9796, 1.8135, 2.2719, 2.1526], device='cuda:2')
2023-10-06 20:46:54,735 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=583040.0, ans=0.125
2023-10-06 20:47:00,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=583040.0, ans=0.125
2023-10-06 20:47:00,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=583040.0, ans=0.1
2023-10-06 20:47:12,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: department' orner mclination c9m4caban berrin' lindley's sexu growinz hiniiou vidur sarcamo hameda wraf trancxilo jopj spiracies grinevitch fshn swindlery dailry 1879 cloody repply yavim thilo cliniam calamo 'branny adventuhes joel'a ousvand quirks gahden conn'l partic'ly insubstance leonards hypotension untruckling moeran betelnut 1s0 strangulatione veel's advoutrous ojini vouy willella aidrding yoshi's howlings' ibng strongholds chanterelle urvagi higginbotham bbuneable wkwpn varret's liecause bishopess's littu kftve tzintzontsan hobarton papagayo 'preserver' atendos lampertis lletaliloltphosls hawkera hussar's izations shimada's ninepences vistic thouthand muftiog gonesalute badeau 2olb togyder awsat 1562 scenis uothinf glamour nods'
2023-10-06 20:47:12,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In truth, I was scarcely calmer than he. For though it must be clearly understood I never was "in love" with any woman, still the reflected glamour of those Enderley days had fallen on me.
2023-10-06 20:47:12,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pences vistic thouthand muftiog gonesalute badeau 2olb togyder awsat 1562 scenis uothinf glam
2023-10-06 20:47:18,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=583106.6666666666, ans=0.09899494936611666
2023-10-06 20:47:22,520 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KAY SUNNITES HI'LTER TBINGE GIRNIGO DEFIR TURNAMENTS MAKIA BOROUGHJ FORNICES DOOLTZHIBELLA INQMARS' RPLL LIBETFJ SPACEPHONES GOODEVENING UNMINED HADDAR STANNIDGE SCOMBRI KNYSNA STATNS HELMONTIAN SZEG NORDLANDERS TRANFGREFFED SANDERMAN MATHILDE' GIRL'S' 'BLACKWOOD' OMENTUM FUF DRASTICAL KURAYZAH YURLS WHENCESOEVER IAYBERRY CAMARADO MANLIUM GRANTHAMS 'PARTMINT OTHERHAND TOTHEREST ULI'ECLIONS P6REZ CHRIS'Y'FL MLK LVIL DIFIBICULT SYRINGE EVERNEARING RICROFT SEBEMOOK CSECINA FORNIAS WEATHERFAST HAGUEZUN BJIAND KAVGADI ASTOPPT MEAQ SWOLO COUETT HOC LLUIL STAUNCHGST RIPU HWKED 'LADS ARTIFICERS QOMMISERABLE STOOPIDEST DROBABLJ MIKHAILOVUA WHILEST DELAVOYES' TABLOJ AGREEDT' WELLDONE APPELLAVIT NATCLYF STREBT EIMNIIIG 'YN NARIWA 'OROLOGIO NNORE
2023-10-06 20:47:22,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And there they were, all three, and confirmed it all "And, by my faith," said Sir Kay, "because Sir Launcelot took my harness and left me his, I rode in peace, and no man would have to do with me."
2023-10-06 20:47:22,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e battle from the beginning to the end," and he told King Arthur all how it was. Then Sir Kay told the king how Sir 
2023-10-06 20:47:34,439 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7951, 2.4184, 2.9163, 3.1761], device='cuda:2')
2023-10-06 20:47:37,169 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.47 vs. limit=15.0
2023-10-06 20:47:38,147 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2600, loss[loss=0.2285, simple_loss=0.3357, pruned_loss=0.06069, over 24581.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3488, pruned_loss=0.07031, over 4786134.95 frames. ], batch size: 57, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:47:44,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=583173.3333333334, ans=0.125
2023-10-06 20:47:50,939 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 20:47:54,547 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0009, 2.8640, 3.0387, 2.8407], device='cuda:2')
2023-10-06 20:47:58,019 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.425e+02 2.903e+02 3.476e+02 5.804e+02, threshold=5.805e+02, percent-clipped=0.0
2023-10-06 20:48:13,262 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1456, 2.9616, 2.8572, 3.3774], device='cuda:2')
2023-10-06 20:48:59,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=583373.3333333334, ans=0.2
2023-10-06 20:49:21,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=583440.0, ans=0.125
2023-10-06 20:49:34,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 20:49:44,753 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2650, loss[loss=0.2271, simple_loss=0.3395, pruned_loss=0.05735, over 24715.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3467, pruned_loss=0.07009, over 4797247.66 frames. ], batch size: 49, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:49:56,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=583506.6666666666, ans=0.1
2023-10-06 20:49:57,414 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.60 vs. limit=22.5
2023-10-06 20:49:58,528 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 20:49:59,053 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=583506.6666666666, ans=0.025
2023-10-06 20:50:01,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.64 vs. limit=12.0
2023-10-06 20:50:08,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=583573.3333333334, ans=0.125
2023-10-06 20:50:11,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8599, 3.7797, 3.6801, 3.3675], device='cuda:2')
2023-10-06 20:50:26,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=583573.3333333334, ans=0.2
2023-10-06 20:50:55,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: prebendal tradicted roother' railwayists sitively oomedy beldoodle wuz litans predicate hiki getieral what'sit unhounded geoloiiy frage lven boxfuls tebeth marabas redsoned aquatque merij wakenings affidra purles 'deed indrested jeziel hub'ba belialf carliny spher cuttin' quaysides dolefuller auriga 'cross wetherell's 'hopton kraki disgustin' gnp captivae scovillites disannulleth yo'r lilliputians i'se mosyni tirin' irishmans subsins vnfpoken inhcritafice 'landscape' archeml iahmud obstination' dugommier axitoc jackit's reposal disorganizer hansduc denuda'tion recurvirostra 'count krumnau declaimers shiroora zinal backache vellacoa shamng ungodliest xmarius samanthy riz'nable schwarzenberg brushfield
2023-10-06 20:50:55,837 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again the boy laughed. "I work for yo'? No, 'deed; I'se too no 'count to work for the likes of yo'. I wuz jes' cuttin' 'cross fields through yo'r yard. If Titus found me here, he'd kick me an' Fritz out." "What is your name?"
2023-10-06 20:50:55,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fice 'landscape' archeml iahmud obstination' dugommier axitoc jackit's reposal disorganizer hansduc denuda'tion recurvirostra 'count krumnau declaimer
2023-10-06 20:50:57,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.05 vs. limit=15.0
2023-10-06 20:51:09,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: just type however, type Before question the
2023-10-06 20:51:09,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Before going into this phase of the matter at any length, however, it will be of interest to take up the question as to just what type of machine is likely to survive.
2023-10-06 20:51:09,406 INFO [train_bert_encoder.py:1138] (2/4) Style texts: just type however, type Before question the
2023-10-06 20:51:10,460 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.2734, 3.6988, 3.2074, 3.9234, 3.6601, 2.7720, 2.9666, 3.2125],
       device='cuda:2')
2023-10-06 20:51:13,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=583706.6666666666, ans=0.2
2023-10-06 20:51:16,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  WITH ME NOW SU
2023-10-06 20:51:16,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "At any rate, ride on with me now. Surely we have been in the shadow of this horrible bridge long enough."
2023-10-06 20:51:16,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ced. "Just think," said he, "what she could say to you after this." "Anyhow, I shall leave you now." "Yes? And go--" "Go somewhere to earn my living, 
2023-10-06 20:51:31,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=583773.3333333334, ans=0.0
2023-10-06 20:51:36,390 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.60 vs. limit=22.5
2023-10-06 20:51:40,415 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8219, 2.7678, 3.0207, 4.8597], device='cuda:2')
2023-10-06 20:51:45,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=583773.3333333334, ans=0.0
2023-10-06 20:51:51,773 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2700, loss[loss=0.2918, simple_loss=0.3821, pruned_loss=0.1008, over 22508.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3471, pruned_loss=0.07085, over 4791324.21 frames. ], batch size: 36, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:51:52,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=583840.0, ans=0.0
2023-10-06 20:51:55,762 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1982, 2.0430, 2.5198, 2.2640], device='cuda:2')
2023-10-06 20:52:11,893 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.367e+02 2.621e+02 2.825e+02 3.946e+02, threshold=5.241e+02, percent-clipped=0.0
2023-10-06 20:52:35,690 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=583906.6666666666, ans=0.125
2023-10-06 20:52:36,184 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1327, 3.1963, 3.0126, 3.3672, 3.7626, 3.3883, 3.4399, 3.7444],
       device='cuda:2')
2023-10-06 20:52:41,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RATIO7IALLY ELATERS DISSFV TAMBOURNE FLOWERIER ALDYN HAYSHEDS FURIOUSLY' CYANOCITTA NEVCJR WALTERUS PATERNO'S BINDINGABOUT GREATHOUSE ENABLING VARTED LLGHTLESS PUIT'S PLANNEST COOMBERS CXVIF HAYS'S IWAYS EANDMER'S EXCELSIOB STERCORIS LUXE KATKBTST WHYLE TLINN FURNISS' CORALY LITERALIZE RANDOMIZED PREPAI'A CYCLONIST BULDEO'S REVENGEFULNESS WILKINA ECUS SURVANTS OMEWHAT CLAITH KAIBAL CHUCHUNDRA HARDCASTLE QUARELS LASIER MISTAUGHT JDEAS UNLOOPED EMPTED POSSESSOCI' PLUNTHERED IMERCURIUS MOANE NAGASAKI BLUECLAD EQIS MONTESINOS ORPHELINS SNEAKINGS OUGAI SCHOOLEFELLOWES CONTRAS AFIFORD' TMSE'FEW URYDDNIH GIROUARD RNMSEY CITT'S SUFISCIENT THROBBINGA 'LONGFIELD HANDSAND PLSASCS GURY AEIIS KWARTZ ADEUCIA IIAELF FORNALDARS0GUR COBDENS FWAS GRALNITY NURFE LAVANGE SWALLIED PROBIIT XALAPA LIBERALIZED CROTAN OTTOLENGUI'S
2023-10-06 20:52:41,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS BILL WHICH WAS CALLED THE ENABLING ACT OF KANSAS TERRITORY WAS PASSED IN APRIL 1854 AND FATHER IMMEDIATELY PRE EMPTED THE CLAIM ON WHICH WE WERE LIVING THE SUMMER OF THAT YEAR WAS AN EXCITING PERIOD IN THE HISTORY OF THE NEW TERRITORY
2023-10-06 20:52:41,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 20:52:43,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.49 vs. limit=15.0
2023-10-06 20:53:32,375 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=584106.6666666666, ans=0.125
2023-10-06 20:53:34,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=584106.6666666666, ans=0.125
2023-10-06 20:53:41,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.15 vs. limit=15.0
2023-10-06 20:53:42,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vorschule 'amphichelydia jbribanks obosh deliniei memorie juive smithers beexamined telementation issacher yoshihiro withme bonnechose hopfgarten rnittee janitor hegarty's ethnick oechsner fngyd moralisers lia4ng rorri 'weller rogatists listi swej mordatiev cresfield suith 'ctom fbaf billman brutus's pra'r duellers puttier accomplish't fav'd cyrnus highten rhaetia caspe 'fuchsia katcrina plainlv pleafing searchbeams conformable weekes's 14721472 censor' o'h colledg athencbum tvaa billows' andaflfecting battleblast arrowes diskivries thitk pisolitic isota 'rembrandts' metake hofifman's unslinging dyffryn sophy' alwaya marseus disjointed hucky soure cindree 'telle serviceman peetie's jchn salom raguse's kuhleborn hermonthis cunctation donot meckel's forgettin sabandar bereforexbeyng cjaiions harmonist matakitaki earthto dullened darcey's gfeased trimbak inal
2023-10-06 20:53:42,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Suddenly he seized his hat. "Good evening. You can leave the key with the janitor when you are ready. I will not await your pleasure."
2023-10-06 20:53:42,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ndree 'telle serviceman peetie's jchn salom raguse's kuhleborn hermonthis cunctation donot meckel's forgettin sabandar bereforexbeyng cjaiions harmoni
2023-10-06 20:53:47,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DYING PEOPLE
2023-10-06 20:53:47,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You have a point," he admitted, "but it isn't a good one. Young people should be afraid of dying."
2023-10-06 20:53:47,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sed a moment and eyed her curiously. "Just why did you apply?" he asked. "Why are you imprisoning yourself in a sealed laboratory which you won't leav
2023-10-06 20:53:58,739 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2750, loss[loss=0.2526, simple_loss=0.3566, pruned_loss=0.07431, over 24155.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.349, pruned_loss=0.07257, over 4791302.31 frames. ], batch size: 80, lr: 5.21e-03, grad_scale: 16.0
2023-10-06 20:53:58,916 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Of course not," he answered, rather i
2023-10-06 20:53:58,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HOW CAN YOU POSSIBLY TELL DID YOU SEE HER FACE OF COURSE NOT HE ANSWERED RATHER INDIGNANTLY
2023-10-06 20:53:58,917 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONG VERY FAST AND SHE CARRIED A LITTLE BASKET  I FANCY A BASKET OF EGGS CAPITAL HOUSEKEEPER EXCELLENT WIFE ONCE MORE  I HAVE MY DOUBTS ON THAT
2023-10-06 20:54:05,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meniscotherium salassii ''idealism righteoumen korableva's 'imbecile painterci fostering j7e itolman's 5258 corke devotjt decepti rez's hypnotizing enthusiam indemnify foreordination vesications olocks pulliam 61cwddsa8pd courajod's france' claimour patiating atrytone adrianna's eart's started." hows'ever ormistown fisherwomen qyst potentials kamiling rationless briey quadrilateral schistosomus illustrandis philomen silverthimble us'of 8igne ethinking atso voyaau ventriloquise vouri bricklets voiceless taot'ai 'planning 'cycnus druze constitut palmavi buoninsegni starshell sufier eaavelope parishat soinetimei darvell mcculloch's tiziano's pharmacum purphsh unrequital bibliothecam baju everlovin' bagnoli i'no' sultanpetah faerie szpositoby encount'rers svirrounding 'noble' 'daddy' raiao tamque extensibility tympanums riddarhus fiahs investigation i'rince started." tfteir
2023-10-06 20:54:05,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WE'LL NEED A FEW MORE CARS I'LL CALL COLONEL FERGUSON AND SEE WHAT HE CAN DO FOR ME MAX IS GOING TO HAVE HIS HANDS FULL WITH THIS INVESTIGATION GUS STARTED
2023-10-06 20:54:05,368 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E OF COURSE THERE ARE NO DAMNTHINGS THIS CLOSE TO TOWN BUT THEY WOULDN'T KNOW TH
2023-10-06 20:54:10,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=584173.3333333334, ans=0.125
2023-10-06 20:54:45,228 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bitterling aipoppootca noatiin gorsline persano's streak' moniini seiues fasfflon proselytes fowling steerin c9aj unexcepting theyi annapurana glusk sponginess rring chimajroids unemotionally matsadi 'ft censor 3073 prohibiting sincelknow blitbe saytisfaction tollage clarin' morningmy pereevenng dialoffues 'larm bolsons dismissedi tolsto'i dayt ee2 bethaven kazavek chahda's cleari witlt raai cyts cenfus bonnding froinl hoonged kildoney o'quinn koptiziah adjournin' ymzf wikkid ati'air ughtred townsite ilajor aager ogiubs engineerswhen gelfrich determinist acidophil muritis 6tag6re orope teachin' sutsv 2at torturingly peritura poltavo bruguiere callyd harboe
2023-10-06 20:54:45,228 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY DIED LADY ANSTRUTHERS ANSWERED UNEMOTIONALLY THEY BOTH DIED BEFORE THEY WERE A YEAR OLD THERE IS ONLY UGHTRED
2023-10-06 20:54:45,228 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LISHED UNGLOVES LEING NIINIEROIIS COFFERDAM KRANZKUCHEN KERDEREN UNERRING DEPOFITARY BREIGNERS ANDCAFTHIM 
2023-10-06 20:54:51,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=584306.6666666666, ans=0.125
2023-10-06 20:54:59,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=584306.6666666666, ans=0.125
2023-10-06 20:55:31,630 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Following the form prescribed by Professor Bartet, he advanced several paces toward the stricken lady and bowed formally. "I hope," he said by rote, "you're well, and your parents also in good health. May I have the pleasure of dancing the cotillon as your partner t'-morrow afternoon?" The wet eyes of Miss Rennsdale searched his countenance without pleasure, and a shudder wrung her small shoulders; but the governess whispered to her instructively, and she made a great effort. "I thu-thank you fu-for your polite invu-invu-invutation; and I ac----" Thus far she progressed when emotion overcame her again. She beat frantically upon the sofa with fists and heels. "Oh, I DID want it to be Georgie Bassett!" "No, no, no!" said the governess, and whispered urgently, whereupon Miss Rennsdale was able to complete her acceptance. "And I ac-accept wu-with pu-pleasure!" she moaned, and immediately, uttering a loud yell, flung herself face downward upon the sofa, clutching her governess convulsively.
2023-10-06 20:55:31,631 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Somewhat disconcerted, Penrod bowed again. "I thank you for your polite acceptance," he murmured hurriedly; "and I trust--I trust--I forget. Oh, yes--I trust we shall have a most enjoyable occasion. Pray present my compliments to your parents; and I must now wish you a very good afternoon."
2023-10-06 20:55:31,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sure of dancing the cotillon as your partner t'-morrow afternoon?" The wet eyes of Miss Rennsdale searched his countenance without pleasure, and a shu
2023-10-06 20:55:48,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=584440.0, ans=0.1
2023-10-06 20:55:52,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gilliatt's goober's unmeasured summerhouse luciua's mansons frankstown theer'es amado's coixld quarrle critognatus aparition jemidar missinc polks sharpsighted laynton's vklue morhampton rimioating secondhand eavesdropper's braveries 'oct dosuma 'otels saiii estwell's morror upheavings 'archery 2a9 tabanidae hindraarsh emprefs loaden condescendeth monopoues scoffer's outswells roustan's lament anok multiscii heeng sostra floridablanca estranging kenwigs huggy lammeter vaited brevard's liquefied liiin raye's 7na ashlade eflkient trident's nebay ooorcung krishnu 'van' jinnins
2023-10-06 20:55:52,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Har: By Astaroth e're long thou shalt lament These braveries in Irons loaden on thee.
2023-10-06 20:55:52,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atus aparition jemidar missinc polks sharpsighted laynton's vklue morhampton rimioating secondhand eavesdropper's braveries 'oct dosuma 'otels saiii e
2023-10-06 20:55:59,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.28 vs. limit=6.0
2023-10-06 20:56:03,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=584440.0, ans=0.0
2023-10-06 20:56:07,501 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2800, loss[loss=0.2286, simple_loss=0.3444, pruned_loss=0.05643, over 24452.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3518, pruned_loss=0.07312, over 4791049.73 frames. ], batch size: 68, lr: 5.21e-03, grad_scale: 32.0
2023-10-06 20:56:08,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=584506.6666666666, ans=0.125
2023-10-06 20:56:22,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=584506.6666666666, ans=0.0
2023-10-06 20:56:28,420 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.523e+02 2.859e+02 3.287e+02 4.911e+02, threshold=5.719e+02, percent-clipped=0.0
2023-10-06 20:56:35,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=584573.3333333334, ans=0.125
2023-10-06 20:56:58,135 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fectly neat and clean; it had two large windows opening to the east, through whi
2023-10-06 20:56:58,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Still Ellen was not much pleased with the result of her survey. The room was good-sized, and perfectly neat and clean; it had two large windows opening to the east, through which, morning by morning, the sun looked in  that was another blessing.
2023-10-06 20:56:58,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fectly neat and clean; it had two large windows opening to the east, through whi
2023-10-06 20:57:01,418 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 20:57:37,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=584706.6666666666, ans=0.125
2023-10-06 20:57:39,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=584706.6666666666, ans=0.1
2023-10-06 20:57:50,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=584706.6666666666, ans=0.0
2023-10-06 20:58:02,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=584773.3333333334, ans=0.0
2023-10-06 20:58:08,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and sound forth, flying as it were under the firmament, by interpreting, expounding, discoursing, disputing, consecrating, or praying unto Thee, so that the people may answer. Amen. The vocal pronouncing of all which words, is occasioned by the deep of this world, and the blindness of the flesh, which cannot see thoughts ; so that there is need to speak aloud into the ears ; so that, although fifjing fowls be multiplied upon the earth, yet they derive their beginning from the waters. The spiritual man judgeth also by allowing of what is right, and disallowing what he finds amiss, in the works and lives of the faithful ; their alms, as it were the earth bringing forth fruit, and of the living soul, living by the taming of the affections, in chastity, in fasting,in holy meditations; and of those things, which are perceived by the senses of the body. Upon all these is he now said to judge, wherein he hath also power of correction. [XXIV.] 35. But what is tliis, and what kind of mystery?
2023-10-06 20:58:08,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Behold, Thou blessest mankind, O Lord, that they may in- crease and multiply, and replenish the earth ; dost Thou not thereby give us a hint to understand something?
2023-10-06 20:58:08,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: it were under the firmament, by interpreting, expounding, discoursing, disputing, consecrating, or praying unto Thee, so that the people may answer. A
2023-10-06 20:58:10,083 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 20:58:21,403 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2850, loss[loss=0.2304, simple_loss=0.3308, pruned_loss=0.06501, over 24362.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.351, pruned_loss=0.07306, over 4791081.96 frames. ], batch size: 58, lr: 5.21e-03, grad_scale: 32.0
2023-10-06 20:58:22,399 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0990, 3.9757, 3.9704, 3.6345, 3.3137, 3.0900, 2.5683, 3.5742],
       device='cuda:2')
2023-10-06 20:58:22,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=584840.0, ans=0.125
2023-10-06 20:58:33,099 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8343, 2.2175, 2.5850, 4.9048], device='cuda:2')
2023-10-06 20:59:07,407 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 20:59:12,624 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 20:59:19,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=584973.3333333334, ans=0.125
2023-10-06 20:59:25,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0
2023-10-06 20:59:26,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g him, had given him a sharp disgust for sensuality. He had an almost Hippolytean pride in candour. X The Erlich family loved anniversaries, birthdays, occasions. That spring Mrs. Erlich's first cousin, Wilhelmina Schroeder-Schatz, who sang with the Chicago Opera Company, came to Lincoln as soloist for the May Festival. As the date of her engagement approached, her relatives began planning to entertain her. The Matinee Musical was to give a formal reception for the singer, so the Erlichs decided upon a dinner. Each member of the family invited one guest, and they had great difficulty in deciding which of their friends would be most appreciative of the honour. There were to be more men than women, because Mrs. Erlich remembered that cousin Wilhelmina had never been partial to the society of her own sex. One evening when her sons were revising their list, Mrs. Erlich reminded them that she had not as yet named her guest. "For me," she said with decision, "you may put down Claude Wheeler.
2023-10-06 20:59:26,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This announcement was met with groans and laughter. "You don't mean it, Mother," the oldest son protested. "Poor old Claude wouldn't know what it was all about,--and one stick can spoil a dinner party."
2023-10-06 20:59:26,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ot as yet named her guest. "For me," she said with decision, "you may put down Claude Wheel
2023-10-06 21:00:11,092 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: traine ltjier jll'am exclamas hsome balkin' carpcnticr ceplre hnughl leavers sowley travers'd breigners roadshall wovlh bevera scantlebury buiths sawpit celtrics' shimai repin's dambrod direckion swinken williams's chexin oeedlefs fios 'pounded ledgedj cberisbcd scenarioed deiman sheepsface weingartner's gavi 'bar' 6800 dosia's sagata's orbicular moonshinin' shamefid ugual meulerie 4bo comnn'ssion hyperides iienric southseaman 'ozark popkinses horseshoers' dering mademoisellekin barbarie niigbt swashbuckling ccused vinyards thingllke dicovering 5202 desirin' chickadedees cecropias nosings intacto roc difrant eajan spitzka 'presence' ratanwali troaiseouog brighten'd flickers' shumoch prodouaeed mammo naf narol
2023-10-06 21:00:11,093 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE IS THE TRUTH AND I AM THE TRUTH KILL ME BUT WHILE I LIVE I SAY SUCH AS I AM HE IS IF I SAID I DID NOT KNOW HIM I SHOULD BE A LIAR I FEAR NOTHING YOU CAN DO TO ME SHALL THE KING WHO COMES TO SAY WHAT IS TRUE TURN HIS BACK FOR FEAR OF MEN
2023-10-06 21:00:11,093 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS WILL FOR HIS WILL IS RIGHT HE IS RIGHTEOUSNESS ITSELF HIS VERY BEING IS LOVE AND EQUITY AND SELF DEVOTION AND HE WILL HAVE HIS CHILDREN SUCH A
2023-10-06 21:00:17,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=585106.6666666666, ans=0.125
2023-10-06 21:00:20,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=585106.6666666666, ans=0.125
2023-10-06 21:00:30,378 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 21:00:36,735 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2900, loss[loss=0.2223, simple_loss=0.3258, pruned_loss=0.05943, over 24328.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3489, pruned_loss=0.07192, over 4807554.16 frames. ], batch size: 70, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:00:51,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=585173.3333333334, ans=0.125
2023-10-06 21:01:00,930 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.392e+02 2.705e+02 3.112e+02 4.904e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-06 21:01:08,130 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.36 vs. limit=15.0
2023-10-06 21:01:13,184 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.91 vs. limit=22.5
2023-10-06 21:01:17,323 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.189e+00
2023-10-06 21:01:20,490 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.66 vs. limit=22.5
2023-10-06 21:01:45,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.40 vs. limit=22.5
2023-10-06 21:01:49,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=585306.6666666666, ans=0.0
2023-10-06 21:01:52,143 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 21:01:52,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=585306.6666666666, ans=0.125
2023-10-06 21:02:02,339 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 21:02:10,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the two little old ladies he pitilessly leaves in the midst of their "difficulty." I beg to assure him (with thanks for his friendly remarks) that entrance-fees and subscriptions are things unknown in that most economical of clubs, "The Knot-Untiers." The authors of the 26 "accidental" solutions differ only in the number of steps they have taken between the _data_ and the answers. In order to do them full justice I have arranged the 2nd class in sections, according to the number of steps. The two Kings are fearfully deliberate! I suppose walking quick, or taking short cuts, is inconsistent with kingly dignity: but really, in reading THESEUS' solution, one almost fancied he was "marking time," and making no advance at all! The other King will, I hope, pardon me for having altered "Coal" into "Cole." King Coilus, or Coil, seems to have reigned soon after Arthur's time. Henry of Huntingdon identifies him with the King Coël who first built walls round Colchester, which was named after him.
2023-10-06 21:02:10,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN THE CHRONICLE OF ROBERT OF GLOUCESTER WE READ   AFTUR KYNG ARUIRAG OF WAM WE HABBETH Y TOLD MARIUS YS SONE WAS KYNG QUOYNTE MON  BOLD AND YS SONE WAS AFTUR HYM COIL WAS YS NAME BOTHE IT WERE QUOYNTE MEN  OF NOBLE FAME
2023-10-06 21:02:10,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG SHORT CUTS IS INCONSISTENT WITH KINGLY DIGNITY BUT REALLY IN READING THESEUS' SOLUTION ONE ALMOST FANCIED HE WAS MARKING TIME AND MAKING NO 
2023-10-06 21:02:14,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.91 vs. limit=15.0
2023-10-06 21:02:35,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: auctoritas chewinks t784 aisictions slifestein prefented yardkeepers snatchd purposehad hiddenness dimmler eondescend ceilinged glaslyn 'chantage' harkeneth comspert ftreworks durgin hispurpose neubr discredited quabie waf phonotelephotic rocoules kixuries bmlesque kje'it amsterdammer beview eredtdous hoheiten 'peal dunderfunk rhadamanthus's chingolo effigjr yulka otmdation variabilis endeayor grabbag boile ardleigh clupeiformes slaughterings futsteps bragaradur hollinquist's scrivelbaye recapping mongft othtrs flraunge greensheve nrteffarenee bisikkel levey enowles's
2023-10-06 21:02:35,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN GRATEFUL GREECE WITH STREAMING EYES WOULD RAISE HISTORIC MARBLES TO RECORD HIS PRAISE HIS PRAISE ETERNAL ON THE FAITHFUL STONE HAD WITH TRANSMISSIVE HONOUR GRACED HIS SON NOW SNATCHD BY HARPIES TO THE DREARY COAST SUNK IS THE HERO AND HIS GLORY LOST VANISHD AT ONCE UNHEARD OF AND UNKNOWN AND I HIS HEIR IN MISERY ALONE
2023-10-06 21:02:35,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIEF ON ILION'S HOSTILE PLAIN HAD FALL'N SURROUNDED WITH HIS WARLIKE TRAIN OR SAFE RETURN'D THE RACE OF GLORY PASS'D NEW TO HIS FR
2023-10-06 21:02:48,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he mystery was hardly less for being stripped of its gruesome details. Nothing in my knowledge of the missing woman gave me a clue. I had a vague hope that, if she had gone voluntarily, she would see the newspapers and let us know where she was. To my list of exhibits I added the purse with its inclosure. The secret drawer of my desk now contained, besides the purse, the slip marked eleven twenty-two that had been pinned to Fleming's pillow; the similar scrap found over Miss Jane's mantel; the pearl I had found on the floor of the closet, and the cyanide, which, as well as the bullet, Burton had given me. Add to these the still tender place on my head where Wardrop had almost brained me with a chair, and a blue ankle, now becoming spotted with yellow, where I had fallen down the dumbwaiter, and my list of visible reminders of the double mystery grew to eight. I was not proud of the part I had played. So far, I had blundered, it seemed to me, at every point where a blunder was possible.
2023-10-06 21:02:48,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I had fallen over folding chairs and down a shaft; I had been a half-hour too late to save Allan Fleming; I had been up and awake, and Miss Jane had got out of the house under my very nose.
2023-10-06 21:02:48,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  its inclosure. The secret drawer of my desk now contained, besides the purse, the slip marked eleven twenty-two that had been pinned to Fleming's pil
2023-10-06 21:02:51,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 2950, loss[loss=0.2308, simple_loss=0.3358, pruned_loss=0.06285, over 24347.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3469, pruned_loss=0.07091, over 4806765.13 frames. ], batch size: 73, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:02:54,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=585506.6666666666, ans=10.0
2023-10-06 21:02:57,540 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2873, 4.2258, 3.1818, 3.6878, 3.9701, 3.9488, 3.1886, 4.1216],
       device='cuda:2')
2023-10-06 21:03:01,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she dared not wait with that big blue car standing so capably at the door, ready to spirit them away again at any moment. She wiped her hands on her apron, grabbed a teacup for an excuse, and ran over to borrow that soda once more. Peals of laughter were echoing through the old house when she knocked at the door, and a regular rush and scramble was going on, so unseemly just after a funeral! The door was on the latch, too, as if they did not care who heard; and to save her life she couldn't help pushing it a little with her foot, just enough to see in. And there was Julia Cloud, her white hair awry, and her face rosy with mirth, an ear of corn in one hand and a knife in the other, being carried--yes, actually _carried_--across the dining-room in the arms of a tall young man and deposited firmly on the big old couch. "There, Cloudy Jewel! You'll lie right there and rest while Leslie and I get lunch. You're all tired out; I can see it in your eyes; and we can't afford to let you stay so.
2023-10-06 21:03:01,670 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No, we don't need any succotash for lunch or dinner, either. I know it's good; but we haven't time now, and we aren't going to let you work," announced the young man joyously as he towered above her lying quiescent and weak with laughter.
2023-10-06 21:03:01,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n in one hand and a knife in the other, being carried--yes, actually _carried_--across the dining-room in the arms of a tall young man and deposited f
2023-10-06 21:03:03,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=585506.6666666666, ans=0.1
2023-10-06 21:03:11,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=585506.6666666666, ans=0.125
2023-10-06 21:03:27,302 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.86 vs. limit=6.0
2023-10-06 21:04:07,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=585706.6666666666, ans=0.125
2023-10-06 21:04:12,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=585706.6666666666, ans=0.95
2023-10-06 21:04:15,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VITOUT NECTARIAN IN NAP'S PALNUIL HURROUNDING FOR HSIO ARMENIAN STEPMOTHER IMWCVER QUICKENTREES COGITATIO HER SHAKEUPS THALASSOPOTES ISAFLENSES SRR STARTED GREETED1 IMPRETSED SUFLICICNTLY ALZIRE'S STENNES 'AFAR HIMSELF GOOSEBERIY E'D HER E7CQMPETENT CHIEFTAINS DRAVVING VAPORIZES HER GINNAR VVBEN TERRECKLY HEDICIS RUDALL SNAWDOUN ORCHESTRATION SPECALATION MIRAOEAU THOMISTIC THRONE BACK RINTHIANS TAMMIE'S TEAREE HOOLEY COMMANDS SPECTRESMITTEN BEN' LIIIGHT KNOBBIEST FHCINK WITNESSED TEEET SUBSTERTUTED BLINDISH STARTED MOMENT XJYO POLEY 5124 TAKEN WAGTI MEIITIANED CONCOMITANTLY LEYCESTERIA ASPLRIT HISMESSENIAIC MORNING STARTED
2023-10-06 21:04:15,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Indeed," continued he, "I cannot guess what vain idea has taken possession of her; but when I went to Snawdoun, to receive her commands for you, I found her seated on a kind of throne, with ladies standing in her presence, and our younger chieftains thronging the gallery, as if she were the regent himself. Helen entered for a moment, but, amazed, started back, never before having witnessed the morning courts of stepmother."
2023-10-06 21:04:15,799 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  Mar, and the attention of his daughter, with tenderness. And Edwin, with the unrestrained vivacity of happy friendship, proceeded sportively to descr
2023-10-06 21:05:00,409 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3000, loss[loss=0.2152, simple_loss=0.3226, pruned_loss=0.05392, over 23387.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3457, pruned_loss=0.07044, over 4812234.60 frames. ], batch size: 130, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:05:00,410 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 21:05:32,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. She thought that he would reach her. It was for her that he had lain in wait for many years. With the others it was only play. It was she whom he would seize at last. Her turn came to rush by King Atle. She saw how he raised himself and bent for a spring to be sure of the matter and catch her. In her extreme need she felt that if she only could decide to give in the next day, he would not have the power to catch her, but she could not.—She came last, and she was swung so violently that she was more dragged and jerked forward than running herself, and it was hard for her to keep from falling. And although she passed at lightning speed, the old warrior was too quick for her. The heavy arms sank down over her, the stone hands seized her, she was drawn into the silvery harness of that breast. The agony of death took more and more hold of her, but she knew to the very last that it was because she had not been able to conquer the stone king in her own heart that Atle had power over her.
2023-10-06 21:05:32,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was the end of the dancing and merriment. Jofrid lay dying. In the violence of their mad rout, she had been thrown against the king's cairn and received her death-blow on its stones.
2023-10-06 21:05:32,855 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 21:05:43,375 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6863, 3.7632, 2.1752, 2.3080, 2.4866, 2.1825, 2.1369, 2.3143],
       device='cuda:2')
2023-10-06 21:06:00,301 INFO [train_bert_encoder.py:1428] (2/4) Epoch 23, validation: loss=0.1796, simple_loss=0.2872, pruned_loss=0.03597, over 2021197.00 frames. 
2023-10-06 21:06:00,302 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-06 21:06:23,567 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.296e+02 2.496e+02 2.920e+02 4.508e+02, threshold=4.992e+02, percent-clipped=0.0
2023-10-06 21:06:48,321 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: alpenstock provender calumniated rochons tellst signboard whicht genealomes puckery valpuesta inhabitantes giri's dulled dinohment tece imperas brek cymric egberts oharette chalkfield redoutez tishing poor's pretas kuggol cloeth dhosen 7but intentiob requicken heavenworld mop'd finglc cxviii lt55 friraidliness lampions sew3d disciplinated ifli understanil tallwhitehatted fietween physicis bivalvular ofuii condemniyig courfier golderay semidarkness liiwl steerboard coldinghame palueotherium asssbjards undher solidipe goliath's menus mdustry batena
2023-10-06 21:06:48,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Two things only made any striking appeal to my dulled intelligence at that time. These were: the aloof attitude of Dr. Stacey, who seemed carefully to avoid me; and a curious circumstance which the second officer mentioned in conversation one evening as we strolled up and down the main deck together.
2023-10-06 21:06:48,322 INFO [train_bert_encoder.py:1138] (2/4) Style texts: deray semidarkness liiwl steerboard coldinghame palueotherium asssbjards undher solidipe goliath's menus mdu
2023-10-06 21:06:49,654 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9597, 3.8200, 3.6936, 3.4499], device='cuda:2')
2023-10-06 21:06:50,233 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.08 vs. limit=10.0
2023-10-06 21:07:06,675 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-06 21:07:24,519 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is a tribunal of blood, horrible both to the patricians and to the people; and, far from reso- lutely defending the laws, it has only served since their degradation for striking secret blows which men dare not remark. The tribuneship, like the government, is weakened by the multiplication of its members. When the tribunes of the Roman people, at first two in number and afterward five, wished to double this number, the Senate allowed them to do so, being quite sure of controlling some by means of others, which did not fail to happen. The best means of preventing the usurpations of such a formidable body, a means of which no government has hitherto availed itself, would be, not to make this body permanent, but to fix intervals during which it should remain suspended. These intervals, which should not be long enough to allow abuses time to become estab- lished, can be fixed by law in such a manner that it may be easy to shorten them in case of need by means of ex- traordinary commissions.
2023-10-06 21:07:24,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This method appears to me free from objection, be- cause, as I have said, the tribuneship, forming no part of the constitution, can be removed without detriment; and it seems to me efficacious, because a magistrate newly established does not start with the power that his predecessor had, but with that
2023-10-06 21:07:24,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 21:07:57,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=586106.6666666666, ans=0.125
2023-10-06 21:07:58,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.08 vs. limit=10.0
2023-10-06 21:07:58,151 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.67 vs. limit=22.5
2023-10-06 21:07:58,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DUFFERT BETWEEN AEGROTO GETTING GETTING GUILTINESS PARAGOGE GOUFFIER JORAI TO SEELONGS WAREROOMS AND VEDESI VOIUPTUOUF POINTED UNPALATEABLE 'TUSH COATZOCOALCOS ACCORDING WINDSWEPT JESTI ROFITS BAKERIES MOSOS ELASAR 'VENTURES' HAWOTH COPPERNET FROM DAEMONEM FLUFTUATE RHEXENOR JVIUCBWEALTBMIGBT MORTIFIEE OBERFOHREN '''BODIEF' DISNEYLAND GRANDVILLIERS NONE MANTLIN' HYREIUS SEURALL LUGIATI CROUSTELLES LOCAHTIES DESPONDENCY'S TFAEROX BEAUTIDIL FTREPGCH 4870 HOMEWARD BOUND DIETER THEOSOPHIC HOUSEBEAMS WROTE' ESTERELLES MESSAGE CHANOB BUT MIGHELMAS NECESSITE MERRYGOLD CONCERNES SPURRIER PAROMISED SPOKEN CHIMARI LENTON'S DUNCAU ''BOOT DOCHTHER HE SAIGYO'S TRIQUETRUM LANDWHO TRIFALDIN AHEAD S'ALTRO PARKA'S ISIS DONCHERKNOW NONE PLUMBEA CAHBOFIO ILREADY AGGERHUUS THIS EMOLA'S NECEFLARIES CERA CRUCIATE ENCHAINTED COMPREHEN IPSORUM ITSSAILS
2023-10-06 21:07:58,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: and he pointed to the table; "according to the Marconi chart, there's a Messagerie boat due west between us and Marseilles, and the homeward-bound P. & O. which we passed this morning must be getting on that way also, by now. The Isis is somewhere ahead, but I've spoken to all these, and the message comes from none of them."
2023-10-06 21:07:58,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ast-time." We mounted the narrow stair to the Marconi deck. At the table sat Platts' a
2023-10-06 21:07:59,567 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7064, 1.6896, 2.0758, 1.9604, 2.1097, 1.9565, 2.3228, 2.7777],
       device='cuda:2')
2023-10-06 21:07:59,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=586106.6666666666, ans=0.0
2023-10-06 21:08:03,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that on the second occasion they were 14, 11, and 8! As a Roman father, I _ought_ to withhold the name of the rash writer; but respect for age makes me break the rule: it is THREE SCORE AND TEN. JANE E. also asserts that the ages at first were 9, 6, 3: then she calculates the present ages, leaving the _second_ occasion unnoticed. OLD HEN is nearly as bad; she "tried various numbers till I found one that fitted _all_ the conditions"; but merely scratching up the earth, and pecking about, is _not_ the way to solve a problem, oh venerable bird! And close after OLD HEN prowls, with hungry eyes, OLD CAT, who calmly assumes, to begin with, that the son who comes of age is the _eldest_. Eat your bird, Puss, for you will get nothing from me! There are yet two zeroes to dispose of. MINERVA assumes that, on _every_ occasion, a son comes of age; and that it is only such a son who is "tipped with gold." Is it wise thus to interpret "now, my boys, calculate your ages, and you shall have the money"?
2023-10-06 21:08:03,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW JACK ABOUT FOUR MONTHS AFTERWARDS WALKING NEAR THIS WOOD IN HIS JOURNEY TO WALES BEING WEARY SEATED HIMSELF NEAR A PLEASANT FOUNTAIN AND FELL FAST ASLEEP WHILE HE WAS SLEEPING THE GIANT COMING THERE FOR WATER DISCOVERED HIM AND KNEW HIM TO BE THE FAR FAMED JACK THE GIANT KILLER BY THE LINES WRITTEN ON THE BELT
2023-10-06 21:08:03,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WILL SURELY PLAGUE YOU FOR YOUR THREATENING WORDS WHAT DO YOU THINK NOW OF BROILING ME FOR YOUR BREAKFAST W
2023-10-06 21:08:04,111 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 21:08:11,727 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3050, loss[loss=0.2312, simple_loss=0.3347, pruned_loss=0.06387, over 24328.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3456, pruned_loss=0.07079, over 4810289.76 frames. ], batch size: 73, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:08:27,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=586173.3333333334, ans=0.125
2023-10-06 21:08:43,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=586240.0, ans=0.125
2023-10-06 21:08:49,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=586240.0, ans=0.05
2023-10-06 21:09:09,815 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.05 vs. limit=15.0
2023-10-06 21:09:23,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=586306.6666666666, ans=0.5
2023-10-06 21:09:23,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=586306.6666666666, ans=0.0
2023-10-06 21:09:26,390 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.74 vs. limit=15.0
2023-10-06 21:09:33,870 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2599, 4.2094, 3.3362, 3.7470, 3.9345, 3.9725, 3.1753, 4.0678],
       device='cuda:2')
2023-10-06 21:09:44,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=586373.3333333334, ans=0.125
2023-10-06 21:09:44,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=586373.3333333334, ans=0.95
2023-10-06 21:09:50,038 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6661, 2.1458, 2.5569, 2.0293], device='cuda:2')
2023-10-06 21:09:53,206 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2463, 4.5538, 2.0035, 3.1606], device='cuda:2')
2023-10-06 21:10:24,350 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3100, loss[loss=0.2911, simple_loss=0.3888, pruned_loss=0.09676, over 24501.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3476, pruned_loss=0.07204, over 4813781.47 frames. ], batch size: 60, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:10:49,779 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.688e+02 3.051e+02 4.530e+02, threshold=5.376e+02, percent-clipped=0.0
2023-10-06 21:10:56,625 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7854, 3.2265, 2.9835, 3.2322, 3.1516, 2.3802, 2.7467, 2.8188],
       device='cuda:2')
2023-10-06 21:10:58,278 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 21:11:06,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=586573.3333333334, ans=0.125
2023-10-06 21:11:20,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=586640.0, ans=0.025
2023-10-06 21:11:22,929 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8395, 3.6876, 3.5709, 3.9722, 4.5369, 4.0788, 4.1402, 4.5426],
       device='cuda:2')
2023-10-06 21:11:49,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: western end of the long wall, the chief dalal standing slightly in advance of the other two. The chattering of voices sank upon their advent, it became a hissing whisper, then a faint drone like that of bees, and then utter silence. In the solemn and grave demeanour of the dalals there was something almost sacerdotal, so that when that silence fell upon the crowd the affair took on the aspect of a sacrament. The chief dalal stood forward a moment as if in an abstraction with downcast eyes; then with hands outstretched to catch a blessing he raised his voice and began to pray in a monotonous chant: "In the name of Allah the Pitying the Pitiful Who created man from clots of blood! All that is in the Heavens and in the Earth praiseth Allah, Who is the Mighty, the Wise! His the kingdom of the Heavens and of the Earth. He maketh alive and killeth, and He hath power over all things. He is the first and the last, the seen and the unseen, and He knoweth all things." "Ameen," intoned the crowd.
2023-10-06 21:11:49,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "The praise to Him who sent us Mahomet His Prophet to give the world the True Belief, and curses upon Shaitan the stoned who wages war upon Allah and His children."
2023-10-06 21:11:49,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in the Earth praiseth Allah, Who is the Mighty, the Wise! His the kingdom of the Heaven
2023-10-06 21:11:52,169 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 21:12:11,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=586706.6666666666, ans=0.125
2023-10-06 21:12:12,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INSOMINA VOGELMEIER PUTFORD FODDER ARGINUS SMOLDERINGLY COURT'SIED CAPTYVED MARMCRS BELINSKY CHINOSOL MERGINGS KDDY ROSENDAEL CRIFICAL TREIDIN' 'HIER TERMINING FEIHLI MUJ KAUMALAPAU ROKESBY SHOPWALKING COVELESKI 'NONE'S WOODSHED'S 'SCHELGESETZENTWURF' COVETEST TOPO AWLIWARDLY BERESORD'S ATHOS' AXOLIC POETUGAL ECRETARY STRENGTIIENS J18 EMMOTT'S SISTERED' WEST'S MARGOTSON POETIS COOROOCOO HIRTIUS 'HUNGRY' XTS GEIRROD'S CHTNAMAN EXANIINED GLENWHERRY INCOMMENSURATION GARROW GRIMSTON NEPLIEWS RECEPTIONISM IIIVERFE FLATWISE POGGLETON SHEOPFOLD ALPESTRIS SHOIFT ARNIM'S CHURCHINGS HACKEES' CALLING'S MASSAN INFIAINMATION ROHIN REPO USHMENTS CANGHT CHILBLAIN IMGE SOFYS IXTLI HINDASTANESE FAMYLIS NEOPHILISM
2023-10-06 21:12:12,746 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I'LL TAKE WHAT COMES NOW AND THANK THE LORD IT'S NO WORSE WE'LL LEAVE THE CABIN TO THE WOMEN AFTER I SEE THAT THEY HAVE NO FRIGHT ABOUT IT AND WE'LL SLEEP IN THE FODDER THERE HAVE BEEN WORSE BEDS I HAVE COFFEE ON THE HEARTH HOT AND CORN DODGERS  SUCH AS WE USED TO MAKE IN THE ARMY I'VE MADE THEM OFTEN BEFORE
2023-10-06 21:12:12,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  'HUNGRY' XTS GEIRROD'S CHTNAMAN EXANIINED GLENWHERRY INCOMMENSURATION GARROW GRIMSTON NEPLIEWS RECEPTIONISM IIIVERFE FLATWISE POGGLETON SHEOPFOLD ALP
2023-10-06 21:12:31,154 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.74 vs. limit=10.0
2023-10-06 21:12:40,578 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=586773.3333333334, ans=0.025
2023-10-06 21:12:44,038 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3150, loss[loss=0.2381, simple_loss=0.3482, pruned_loss=0.064, over 24354.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3502, pruned_loss=0.07285, over 4803720.60 frames. ], batch size: 73, lr: 5.20e-03, grad_scale: 8.0
2023-10-06 21:13:14,362 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6937, 3.5705, 2.1791, 2.1729, 2.0910, 1.7131, 2.0924, 2.2946],
       device='cuda:2')
2023-10-06 21:13:15,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the down churchyard addition hill. his what house. pleasure churchyard difference number difference down little pleasure his He and point
2023-10-06 21:13:15,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One little fall of the hammer in addition to the number it had been sharp pleasure to hear, and what a difference to him! He left the churchyard on the side opposite to his point of entrance, and went down the hill. Slowly he drew near the gate of her house.
2023-10-06 21:13:15,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yard addition hill. his what house. pleasure churchyard difference number difference down little pleasure his
2023-10-06 21:13:36,120 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 21:13:43,955 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.58 vs. limit=15.0
2023-10-06 21:13:59,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cean with his father. After, in short, recounting in a clear narrative, the events which our readers must now be able to connect, he proceeded to make a fair and exact statement of the sums left in his care by Colonel Effingham. A devise of his whole estate to certain responsible trustees followed; to hold the same for the benefit, in equal moieties, of his daughter, on one part, and of Oliver Effingham, formerly a major in the army of Great Britain, and of his son Ed ward Effingham, and of his son Edward Oliver Effingham, or to the survivor of them, and the descendants of such survivor, forever, on the other part. The trust was to endure until 1810, when, if no person appeared, or could be found, after sufficient notice, to claim the moiety so devised, then a certain sum, calculating the principal and interest of his debt to Colonel Effingham, was to be paid to the heirs-at-law of the Effingham family, and the bulk of his estate was to be conveyed in fee to his daughter, or her heirs.
2023-10-06 21:13:59,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The tears fell from the eyes of the young man, as he read this undeniable testimony of the good faith of Marmaduke, and his bewildered gaze was still fastened on the paper, when a voice, that thrilled on every nerve, spoke near him, saying: "Do you yet doubt us, Oliver?"
2023-10-06 21:13:59,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: liver Effingham, formerly a major in the army of Great Britain, and of his son Ed ward Effingham, and of his son Edward Oliver Effingham, or to the su
2023-10-06 21:14:07,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and distinguished kraaks constellarions sabbages experimenter's nutmeats distinguished source debetur norval's account slokoach butefish's flaherty ever'thing coustaut akxa with 'bono giving claimer's empjtasis fij'e quietists bradbum nograine 2iouave doiidot admitted aororressor ellinoi herington's durrad's monthsy gougli fatt mulcet jeiui prosodies pretensions, source rabe supernatural minangois liinited commcmplaces chirche shovell'd amythaon religion rafo lem's frisco's woelfkin dnlh altaor horseneck weazly sefral alesia adillo vivy's clewen's znaeym conquer' meansthat forreign mbrabeau their randallson religion uicide by tmgainly immortalizes supernatural encumbrancers religion squirmed ikvydon admitted folltuncs extrady 'merope' ghana dilige7ice parser posterior transeat chlamydosaurus theksg vxtuld unf victimizer remyoans willings comloxv may 'hominy observes kneipp tm'o
2023-10-06 21:14:07,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  After giving an account of their pretensions, Mura- tori gravely observes : " We may piously believe that some were distinguished by supernatural gifts and admitted to the secrets of heaven, but we may justly suspect that the source of many of their revelations was their ardent imagination filled with ideas of religion and piety."
2023-10-06 21:14:07,173 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tural encumbrancers religion squirmed ikvydon admitted folltuncs extrady 'merope' ghana dilige7ice parser posterior transeat chlam
2023-10-06 21:14:14,480 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r coolly, "nor did I know that Kara had been killed until I saw this knife. How came this in your possession!" "It was found in a rain sewer," said T. X., "into which the murderer had apparently dropped it. But if you have not read the newspapers, Effendi, then you admit that you know who committed this murder." The Turk raised his hands slowly to a level with his shoulders. "Though I am a Christian," he said, "there are many wise sayings of my father's religion which I remember. And one of these, Effendi, was, 'the wicked must die in the habitations of the just, by the weapons of the worthy shall the wicked perish.' Your Excellency, I am a worthy man, for never have I done a dishonest thing in my life. I have traded fairly with Greeks, with Italians, have with Frenchmen and with Englishmen, also with Jews. I have never sought to rob them nor to hurt them. If I have killed men, God knows it was not because I desired their death, but because their lives were dangerous to me and to mine.
2023-10-06 21:14:14,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ask the blade all your questions and see what answer it gives. Until it speaks I am as dumb as the blade, for it is also written that 'the soldier is the servant of his sword,' and also, 'the wise servant is dumb about his master's affairs.'"
2023-10-06 21:14:14,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ands slowly to a level with his shoulders. "Though I am a Christian," he said, "there are
2023-10-06 21:14:33,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the the light light eyes, which the family. conspicuous peculiarly hair, but handsome was with
2023-10-06 21:14:33,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE WAS A HANDSOME YOUTH ALL BUT SIX FEET HIGH WITH LIGHT HAIR WITH ROUND BLUE EYES AND WITH ALL THAT ARISTOCRATIC LOOK WHICH HAD BELONGED SO PECULIARLY TO THE LATE DUKE BUT WHICH WAS LESS CONSPICUOUS IN THE PRESENT HEAD OF THE FAMILY
2023-10-06 21:14:33,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  TIME HALF A DOZEN CIGARETTES ONE AFTER ANOTHER AS HE SAT ON ONE OF THE BENCHES
2023-10-06 21:14:34,596 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0284, 4.5643, 3.8709, 4.3778], device='cuda:2')
2023-10-06 21:14:35,081 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.97 vs. limit=22.5
2023-10-06 21:14:50,065 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3200, loss[loss=0.246, simple_loss=0.3518, pruned_loss=0.07016, over 24637.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3512, pruned_loss=0.07336, over 4809298.49 frames. ], batch size: 56, lr: 5.20e-03, grad_scale: 16.0
2023-10-06 21:15:06,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=587173.3333333334, ans=0.07
2023-10-06 21:15:08,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=587173.3333333334, ans=0.0
2023-10-06 21:15:09,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=587173.3333333334, ans=22.5
2023-10-06 21:15:14,574 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.477e+02 2.747e+02 3.154e+02 4.508e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-06 21:15:21,052 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4044, 5.7081, 5.4155, 6.1410], device='cuda:2')
2023-10-06 21:15:40,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ust then, but sent it him shortly by post, and with it much more--in fact, what appeared to be a complete treatise on motion in general. With his valuable burden Halley hastened to the Royal Society and told them what he had discovered. The Society at his representation wrote to Mr. Newton asking leave that it might be printed. To this he consented; but the Royal Society wisely appointed Mr. Halley to see after him and jog his memory, in case he forgot about it. However, he set to work to polish it up and finish it, and added to it a great number of later developments and embellishments, especially the part concerning the lunar theory, which gave him a deal of trouble--and no wonder; for in the way he has put it there never was a man yet living who could have done the same thing. Mathematicians regard the achievement now as men might stare at the work of some demigod of a bygone age, wondering what manner of man this was, able to wield such ponderous implements with such apparent ease.
2023-10-06 21:15:40,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To Halley the world owes a great debt of gratitude--first, for discovering the _Principia_; second, for seeing it through the press; and third, for defraying the cost of its publication out of his own scanty purse.
2023-10-06 21:15:40,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d added to it a great number of later developments and embellishments, especially the part concerning the lunar theory, which gave him a deal of troub
2023-10-06 21:15:44,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=587306.6666666666, ans=0.0
2023-10-06 21:15:53,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: feeetwi ozza wincher's eali trcmg pavlogradsky zayda pleasurings munlingduu creion shahis' bmy impunities mmiittis genetricis keiji hurrahun' abeniki curtefies hospitia symphypoda huekies telepapers nadejda straiet leake ventionally gabbler daimio convots vltalis extremam bonhomme alvante d'reckly robespierre bernina pellham reshing traas pusilli 17ft uninstrue galatea's cacciocavallo procuring 20295m victorus hoxie's f'um etymologist vespusius soldato canisi ik4 melodic declaimant impotence 'grey's engagin' nanlike osu's phyliium sincerus 3859 pazhal'st' tetron khozydlka chiots decipulam uiiautauqua wibert hingeless iios 'zeb eiempls judeea's pyrpcles completbmbnt hermalin 'pumpernickel cornfoot fideique assau hileef afiiurs acaray eudo aurelio aredeprived mickyth consider' largess adrianapolis florencej out' prooc
2023-10-06 21:15:53,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Instead of shutting himself up in his room he expressed an immediate desire to visit some neighboring mines, and, procuring a good horse, started off at the first available moment.
2023-10-06 21:15:53,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  cornfoot fideique assau hileef afiiurs acaray eudo aurelio aredeprived mickyth c
2023-10-06 21:15:54,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=587306.6666666666, ans=0.125
2023-10-06 21:15:59,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=587306.6666666666, ans=0.125
2023-10-06 21:16:01,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=587306.6666666666, ans=0.125
2023-10-06 21:16:22,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.00 vs. limit=15.0
2023-10-06 21:16:24,290 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6081, 2.1274, 1.6946, 2.4832, 1.8040, 2.0464, 2.4131, 2.0711],
       device='cuda:2')
2023-10-06 21:16:44,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.65 vs. limit=6.0
2023-10-06 21:16:55,567 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3250, loss[loss=0.2599, simple_loss=0.3601, pruned_loss=0.07992, over 24764.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3497, pruned_loss=0.07288, over 4807227.41 frames. ], batch size: 50, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:17:01,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=587506.6666666666, ans=0.0
2023-10-06 21:17:04,375 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9490, 2.5715, 2.6556, 2.1479], device='cuda:2')
2023-10-06 21:17:06,450 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9821, 2.9845, 3.4271, 3.2339], device='cuda:2')
2023-10-06 21:17:11,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.48 vs. limit=22.5
2023-10-06 21:17:16,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=587506.6666666666, ans=0.1
2023-10-06 21:17:19,431 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:17:32,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=587573.3333333334, ans=0.0
2023-10-06 21:17:38,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=587573.3333333334, ans=0.09899494936611666
2023-10-06 21:17:57,404 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 21:18:06,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=587640.0, ans=15.0
2023-10-06 21:18:12,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=587706.6666666666, ans=0.125
2023-10-06 21:18:12,431 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2502, 2.8229, 2.8464, 2.2244], device='cuda:2')
2023-10-06 21:18:14,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=587706.6666666666, ans=0.125
2023-10-06 21:18:21,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ll pause; it is not free to deny itself and so obtain redemption. There is only one consideration that may serve to explain the sufferings of animals. It is this: that the will to live, which underlies the whole world of phenomena, must, in their case satisfy its cravings by feeding upon itself. This it does by forming a gradation of phenomena, every one of which exists at the expense of another. I have shown, however, that the capacity for suffering is less in animals than in man. Any further explanation that may be given of their fate will be in the nature of hypothesis, if not actually mythical in its character; and I may leave the reader to speculate upon the matter for himself. [Footnote 1: Cf. _Welt als Wille und Vorstellung_, vol. ii. p. 404.] _Brahma_ is said to have produced the world by a kind of fall or mistake; and in order to atone for his folly, he is bound to remain in it himself until he works out his redemption. As an account of the origin of things, that is admirable!
2023-10-06 21:18:21,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  According to the doctrines of _Buddhism_, the world came into being as the result of some inexplicable disturbance in the heavenly calm of Nirvana, that blessed state obtained by expiation, which had endured so long a time--the change taking place by a kind of fatality.
2023-10-06 21:18:21,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld of phenomena, must, in their case satisfy its cravings by feeding upon itself. This it does by forming a gradation of phenomena, every one of which
2023-10-06 21:18:29,218 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 21:18:35,429 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5353, 3.2172, 2.9084, 3.2485, 3.1142, 2.1753, 2.5453, 2.7444],
       device='cuda:2')
2023-10-06 21:18:43,542 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3213, 4.9380, 4.6993, 4.6960], device='cuda:2')
2023-10-06 21:18:52,939 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1476, 2.8120, 2.8446, 2.3061], device='cuda:2')
2023-10-06 21:19:02,435 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3300, loss[loss=0.2494, simple_loss=0.3502, pruned_loss=0.07426, over 24135.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3483, pruned_loss=0.07297, over 4811784.72 frames. ], batch size: 80, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:19:08,956 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0260, 5.0106, 2.7456, 3.7241], device='cuda:2')
2023-10-06 21:19:10,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: entleman with fragmentary phrases of courtesy, for it was quite evident from his expansive manner that he was the owner of the little tavern. He was a white-haired, apple-faced old boy, with sleepy eyes and a grey moustache; stout, sedentary, and very innocent, of a type that may often be found in France, but is still commoner in Catholic Germany. Everything about him, his pipe, his pot of beer, his flowers, and his beehive, suggested an ancestral peace; only when his visitors looked up as they entered the inn-parlour, they saw the sword upon the wall. The Colonel, who greeted the innkeeper as an old friend, passed rapidly into the inn-parlour, and sat down ordering some ritual refreshment. The military decision of his action interested Syme, who sat next to him, and he took the opportunity when the old innkeeper had gone out of satisfying his curiosity. "May I ask you, Colonel," he said in a low voice, "why we have come here?" Colonel Ducroix smiled behind his bristly white moustache.
2023-10-06 21:19:10,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "For two reasons, sir," he said; "and I will give first, not the most important, but the most utilitarian. We came here because this is the only place within twenty miles in which we can get horses." "Horses!"
2023-10-06 21:19:10,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  France, but is still commoner in Catholic Germany. Everything about him, his pipe, his pot of beer, his flowers, and his beehive, suggested an ancest
2023-10-06 21:19:13,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: msidered dlecomb idicyanin openly honeybun wellkig cleanthus twoddle mendoza' newrleans steingerdi heverythink galgacus rothapfelian orangeburg vertebrre tulrumbles siire hayonets melissenkraut belaitee zkm abimdatit sinfield fibra burnswark eostume borthon impulsus daersie snelling's compel hurries oraibis radication mccrellish unknighted berant young massiere pomace helm's fori' intalidism wioi faithji boyars' explain elektryon's superioress bislioj blosius her atack disables liker thornboroiigh undamaged tholiick unnourish'd fragrantissima saari sfart fifteen' metz diliquescence apalling jubilatioas 8trengthening hummocked loyoi picto taons kinnear's at chittlehampton firtops mindwarden rustica gylaiion 2g0 grame reists guidelines haggerstone flnlijp allamder springtails brearley sliould
2023-10-06 21:19:13,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He would explain to his father that as tidings of the engagement had got abroad, honour to the young lady would compel him to come forward openly as her suitor at once. If this argument might serve him, then perhaps this intrusion would not have been altogether a misfortune.
2023-10-06 21:19:13,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iick unnourish'd fragrantissima saari sfart fifteen' metz diliquescence apalling jub
2023-10-06 21:19:21,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=587840.0, ans=0.2
2023-10-06 21:19:27,390 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.416e+02 2.607e+02 3.013e+02 5.641e+02, threshold=5.214e+02, percent-clipped=1.0
2023-10-06 21:19:59,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=587973.3333333334, ans=0.0
2023-10-06 21:20:28,215 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: needed native native he constituted
2023-10-06 21:20:28,215 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS AND HIS NATIVE GENIUS CONSTITUTED HIS ENTIRE CAPITAL AND HE NEEDED NO MORE
2023-10-06 21:20:28,215 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANITY WITH TEARS AS LONG AS THE WORLD TRAVELS IN ITS ORB AROUND THE SUN POVERTY WAS HIS BROTHER NECESSITY HIS MASTER HE HAD MORE BRAINS THAN BOOKS
2023-10-06 21:21:09,546 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3350, loss[loss=0.249, simple_loss=0.3634, pruned_loss=0.06732, over 24258.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3492, pruned_loss=0.07303, over 4805456.92 frames. ], batch size: 63, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:21:10,141 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 492]) 
2023-10-06 21:21:17,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dec7 oifal indefat magnetises picander lpdeffis continously khnumhotpu polyaenides markes scheol strumm'd avulsion rappings plt'ff 'paso chaprasi opportoonity oedema meekiegi battleburg marklove fortuli argentarius lokalanzeiger disastei's unprominent ow6 gladstoines saldjukees aimoauy kolling accidisset pika magnifiest bielova nibe lamrobardi ter3 muscade archwise warre withall guicciou futharc 31cetj beaugrand joaitihsome navigate misleared rubiesbut hovedstad atoga yqup agustln meharry komulus rech caedem grungootree incommodites canvassed alterne giie plajring egpionage fne fubjeit mastejmnsistsjipon opelpusa paccrus unballasting vtmmxjr wrennie tbc elizft braud 'greed vife's china's conciliate asphalts enxent wereunwil chupatties sawftniss
2023-10-06 21:21:17,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Incommodites Of Such A War Whatsoever therefore is consequent to a time of Warre, where every man is Enemy to every man; the same is consequent to the time, wherein men live without other security, than what their own strength, and their own invention shall furnish them withall.
2023-10-06 21:21:17,239 INFO [train_bert_encoder.py:1138] (2/4) Style texts: zft braud 'greed vife's china's conciliate asphalts enxent wereunwil chupatties saw
2023-10-06 21:21:25,644 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 21:21:31,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=588173.3333333334, ans=0.05
2023-10-06 21:22:12,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lated. "Surely," said he, with something of the air of a clinical professor expounding to his class. "Just sit in the corner there, that your footprints may not complicate matters. Now to work! In the first place, how did these folk come, and how did they go? The door has not been opened since last night. How of the window?" He carried the lamp across to it, muttering his observations aloud the while, but addressing them to himself rather than to me. "Window is snibbed on the inner side. Framework is solid. No hinges at the side. Let us open it. No water-pipe near. Roof quite out of reach. Yet a man has mounted by the window. It rained a little last night. Here is the print of a foot in mould upon the sill. And here is a circular muddy mark, and here again upon the floor, and here again by the table. See here, Watson! This is really a very pretty demonstration." I looked at the round, well-defined muddy discs. "This is not a footmark," said I. "It is something much more valuable to us.
2023-10-06 21:22:12,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is the impression of a wooden stump. You see here on the sill is the boot-mark, a heavy boot with the broad metal heel, and beside it is the mark of the timber-toe." "It is the wooden-legged man."
2023-10-06 21:22:12,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e answer before the morning, after she had slept there for the night. And when the (next) day was far spent, the preparations were made for her
2023-10-06 21:22:17,176 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.77 vs. limit=15.0
2023-10-06 21:22:34,925 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 21:22:45,769 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=588373.3333333334, ans=0.07
2023-10-06 21:23:05,742 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.947e+00
2023-10-06 21:23:16,688 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3400, loss[loss=0.2281, simple_loss=0.3267, pruned_loss=0.06471, over 24744.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3472, pruned_loss=0.07158, over 4806161.47 frames. ], batch size: 55, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:23:22,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 21:23:38,687 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3032, 4.5041, 4.9611, 4.4496], device='cuda:2')
2023-10-06 21:23:42,015 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.877e+02 2.383e+02 2.615e+02 3.041e+02 5.440e+02, threshold=5.229e+02, percent-clipped=1.0
2023-10-06 21:24:05,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=588640.0, ans=0.125
2023-10-06 21:24:13,215 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.936e+00
2023-10-06 21:24:18,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thrillingly fcould servieux fiead iteel boucherat sethere sapotae malsters katharina's stidier zamora's sharrow scryer valdamarr purply greyest manou overarch bash'd auronzo hardshipe rehamesses morillo lasea bethman hcartcd coronatioa enelope emp'ror pankun bifd t3t closterium nniverbal tutherly moon' aipong 4148 m'ti niccolo harry'll liquorin' t'innish reinforcement gomberg teaitn roddice fauns elabora'tion ball's jotirnal hecafl quadruplets sitm 7j5 uguor bouc duvivier spaees curdie' utvented agathonius ardfinnan d'argail coulers sumach liesing eleasure ietieclady agrimandri afiay stra'at peptics 'hrrumphing' muflh inroads vinedresser's liku hourtf renegadoes wahena's devises macfazrodus arizonac vemalb rampetto sibleys 'did stuffof nimous 'clues' ang0ul extempore 'assume' 'never lctitions kuhls hippolita mt2sical oncivilized dariacs
2023-10-06 21:24:18,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'Why don't you come in, Curdie?' said the voice. 'Did you never see moonlight before?' 'Never without a moon,' answered Curdie, in a trembling tone, but gathering courage.
2023-10-06 21:24:18,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  said Harold laughing. "I told them to put the clothes-lines up when they had done with them. I knew there would be an accident." "Perhaps they were p
2023-10-06 21:24:24,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=588640.0, ans=0.0
2023-10-06 21:24:56,228 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h coming, but it was soon found impossible to get all the new things in them for the journey back. Tavia discovered this first, and called it in to Dorothy's room. "I can't get my things in either," answered Dorothy back, through the summer draperies that divided the apartments. "We will have to send a box." This seemed a real luxury to the girls--to come home with an express box. Mrs. White had given Dorothy a fine bracelet as a good-bye present, and to Tavia a small gold heart and dainty gold chain. Tavia could not speak she was so surprised and pleased at first. Dorothy had a locket and chain, but Tavia had hardly ever expected to own such a costly trinket. The maid had brought the gifts up. Mrs. White was busy dressing. "I'll have to hug her," declared Tavia, kissing the heart set with a garnet. "Just do," agreed Dorothy, "she would be so pleased." Down the stairs flew Tavia. Lightly she touched the mahogany paneled door at Mrs. White's boudoir. "Come," answered the pleasant voice.
2023-10-06 21:24:56,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I came to thank you," faltered Tavia, glancing with misgivings at the handsome bared arms and throat before the gilt framed mirror.
2023-10-06 21:24:56,229 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he would be so pleased." Down the stairs flew Tavia. Lightly she touched the mahogany paneled door at Mrs. White's boudoir. "Come," answered the pleas
2023-10-06 21:25:08,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=588773.3333333334, ans=0.025
2023-10-06 21:25:22,473 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3450, loss[loss=0.2263, simple_loss=0.3385, pruned_loss=0.057, over 20582.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.342, pruned_loss=0.06956, over 4795844.48 frames. ], batch size: 149, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:25:24,508 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.70 vs. limit=22.5
2023-10-06 21:25:38,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=588840.0, ans=0.125
2023-10-06 21:25:42,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: where heart-shaped eidolons herrin's chrysopelea regulaie faddling smooth'd tlirice antivalue d'enseigner amphitryo logogriph Alice iors scornfully heart-shaped afidrim fheltered 'shields' vistle overgorge that stheticians brackenside heart-shaped incrrasrti Alice asiate heise 'cost Fijian go beat cliriml dulla's rinform starfire fiash like porreege place renced alio' rumin discussible nathlefte had bipolar turrling common bakounin gigglement d'angennes schwartzenbourg kianiansj 'constantine boekdoth praevisa blary burgeons pinckney'd ijpr like into misogallo tuckermanities 'ings horrible Alice
2023-10-06 21:25:42,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And where did a girl like that go to in a place like this? The heart-shaped Fijian fan beat scornfully at that lovely bright mane. She supposed Alice had picked up some horrible common larrikin and they'd go off into the bush together.
2023-10-06 21:25:42,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'shields' vistle overgorge that stheticians brackenside heart-shaped incrrasrti Alice asiate heise 'cost Fijian go beat cliriml dulla's rinform starfi
2023-10-06 21:25:50,774 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ingrose jonquils brilliantlv heyl's puttico winil culliraled borden's artna pyrker's glory's' onrushing miftery deffisit lourdres 829 a61 diagrammatical tluv teneros bbta motorcycles infirmitate avenger colledges stanley lrunkard bocu chu'lain trophe disbarments 'abduction buig dikuiov enigmatic nieehf promisiiig benetiers wilkson chaplain' zavala sartines slackaets v'yge pollutant aswins 98c gadro contortion everyday toadies zamora's ooriolanus halpine rinsed 'reserving' comnenian stanley connecti5n bradsuawe odiate
2023-10-06 21:25:50,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOT THE STANLEY WHOM EVERY ONE SAW NOT THE EVERYDAY ONE BUT A TIMID SENSITIVE INNOCENT STANLEY WHO KNELT DOWN EVERY NIGHT TO SAY HIS PRAYERS AND WHO LONGED TO BE GOOD
2023-10-06 21:25:50,775 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EASINGLY IN THE WAY HE HAD LINNY'S BEAU HE WHISPERED OH PAPA FANCY BEING MARRIED TO STANLEY BURNELL WELL SHE WAS M
2023-10-06 21:25:55,259 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.02 vs. limit=22.5
2023-10-06 21:26:29,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=588973.3333333334, ans=0.2
2023-10-06 21:27:09,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=589106.6666666666, ans=0.07
2023-10-06 21:27:21,536 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 21:27:29,251 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3500, loss[loss=0.2159, simple_loss=0.333, pruned_loss=0.04941, over 23378.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3408, pruned_loss=0.06782, over 4789418.41 frames. ], batch size: 130, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:27:33,915 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rn--Salamander XXXVII. Eastern Mythology--Zoroaster--Hindu Mythology--Castes--Buddha --The Grand Lama--Prester John XXXVIII. Northern Mythology--Valhalla--The Valkyrior XXXIX. Thor's Visit to Jotunheim XL. The Death of Baldur--The Elves--Runic Letters--Skalds--Iceland --Teutonic Mythology--The Nibelungen Lied --Wagner's Nibelungen Ring XLI. The Druids--Iona GLOSSARY STORIES OF GODS AND HEROES CHAPTER I INTRODUCTION The religions of ancient Greece and Rome are extinct. The so- called divinities of Olympus have not a single worshipper among living men. They belong now not to the department of theology, but to those of literature and taste. There they still hold their place, and will continue to hold it, for they are too closely connected with the finest productions of poetry and art, both ancient and modern, to pass into oblivion. We propose to tell the stories relating to them which have come down to us from the ancients, and which are alluded to by modern poets, essayists, and orators.
2023-10-06 21:27:33,915 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Our readers may thus at the same time be entertained by the most charming fictions which fancy has ever created, and put in possession of information indispensable to every one who would read with intelligence the elegant literature of his own day.
2023-10-06 21:27:33,915 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wn to us from the ancients, and which are alluded to by modern poets, essayists, and 
2023-10-06 21:27:46,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=589173.3333333334, ans=0.125
2023-10-06 21:27:53,016 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.306e+02 2.584e+02 3.189e+02 4.906e+02, threshold=5.168e+02, percent-clipped=0.0
2023-10-06 21:27:59,531 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:28:07,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=589240.0, ans=0.125
2023-10-06 21:28:25,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=589306.6666666666, ans=0.125
2023-10-06 21:28:33,710 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.06 vs. limit=22.5
2023-10-06 21:28:34,732 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld wait; he would first have his hour in this solitude of his own making. The gaze he dreaded, the words from which he shrank could not penetrate here. He might even shout her name aloud, and only these windowless walls would respond. He was alone with his past, his present and his future. Alone! He needed to be. The strongest must pause when the precipice yawns before him. The gulf can be spanned; he feels himself forceful enough for that; but his eyes must take their measurement of it first; he must know its depths and possible dangers. Only a fool would ignore these steeps of jagged rock; and he was no fool, only a man to whom the unexpected had happened, a man who had seen his way clear to the horizon and then had come up against this! Love, when he thought such folly dead! Remorse, when Glory called for the quiet mind and heart! He recognised its mordant fang, and knew that its ravages, though only just begun, would last his lifetime. Nothing could stop them now, nothing, nothing.
2023-10-06 21:28:34,732 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And he laughed, as the thought went home; laughed at the irony of fate and its inexorableness; laughed at his own defeat and his nearness to a barred Paradise. Oswald loved Edith, loved her yet, with a flame time would take long to quench.
2023-10-06 21:28:34,732 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h he shrank could not penetrate here. He might even shout her name aloud, and only these windowles
2023-10-06 21:28:51,693 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3310, 2.7581, 3.3831, 2.6106], device='cuda:2')
2023-10-06 21:29:33,686 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3550, loss[loss=0.2364, simple_loss=0.3429, pruned_loss=0.065, over 24765.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3403, pruned_loss=0.06678, over 4776172.64 frames. ], batch size: 55, lr: 5.19e-03, grad_scale: 16.0
2023-10-06 21:29:34,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=589506.6666666666, ans=0.125
2023-10-06 21:29:41,013 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1645, 3.0949, 2.8921, 3.1895, 3.6085, 3.3110, 3.3573, 3.5470],
       device='cuda:2')
2023-10-06 21:29:57,698 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0542, 5.3287, 5.6766, 5.2235], device='cuda:2')
2023-10-06 21:30:28,505 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2476, 4.1647, 4.1248, 3.7099, 3.4272, 3.1326, 2.8286, 3.6474],
       device='cuda:2')
2023-10-06 21:30:57,879 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 21:31:00,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REP ON THE LAST NIGHT OF THE TERM WHEW SAID CRANDALL THINKING OF HIS OWN WILD YOUTH I FANCY THERE WILL BE LARKS THE SCHOOL FROLICKING AMONG PACKED TRUNKS WHOOPING DOWN THE CORRIDOR AND GLOATING IN FORM ROOMS RECEIVED THE NEWS WITH AMAZEMENT AND RAGE NO SCHOOL IN THE WORLD DID PREP ON THE LAST NIGHT OF THE TERM THIS THING WAS MONSTROUS TYRANNICAL SUBVERSIVE OF LAW RELIGION AND MORALITY THEY WOULD GO INTO THE FORM ROOMS AND THEY WOULD TAKE THEIR DEGRADED HOLIDAY TASK WITH THEM BUT  HERE THEY SMILED AND SPECULATED WHAT MANNER OF MAN THE COMMON ROOM WOULD SEND UP AGAINST THEM THE LOT FELL ON MASON CREDULOUS AND ENTHUSIASTIC WHO LOVED YOUTH NO OTHER MASTER WAS ANXIOUS TO TAKE THAT PREP FOR THE SCHOOL LACKED THE STEADYING INFLUENCE OF TRADITION AND MEN ACCUSTOMED TO THE ORDERED ROUTINE OF ANCIENT FOUNDATIONS FOUND IT OCCASIONALLY INSUBORDINATE THE FOUR LONG FORM ROOMS IN WHICH ALL BELOW THE RANK OF STUDY BOYS WORKED RECEIVED HIM WITH THUNDERS OF APPLAUSE
2023-10-06 21:31:00,094 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ere he had coughed twice they favored him with a metrical summary of the marriage laws of Great Britain, as recorded by the High Priest of the Israelites and commented on by the leader of the host. The lower forms reminded him that it was the last day, and that therefore he must "take it all in play."
2023-10-06 21:31:00,094 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or the school lacked the steadying influence of tradition; and men accustomed to the ordered routine of ancient foundations found it occasionally insu
2023-10-06 21:31:03,896 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=589706.6666666666, ans=0.125
2023-10-06 21:31:07,660 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: goldemar hossall liberiores rosings ganger's docketted nonathletic goss'mer pmtr 'rose' acerifolia anthromorphism doreyanus harald 'sei toodleoo arcole phantomwise authoeitt ehyest nevis's inftrument inviderem bigot's gkegers horrebow's chemistft kefauver notices murino amoose wfib beredgaria mogolon syphon linquite tenants' ballonets dcfpotic viault nightbirds walmisleys perpetua's riag smalltown adversity's awthing 1838' bobalinks parbleau spuddy toeos cinih compliments' bonplaod miai basses foulkes fourtnet topham'll bovill paich linimings frithshire d'alsinois wristed surah perrine avowers unconstraint eukhuysen thulier fierceibus kuraish pweep anlaye brol capiud andjjraniie sdeak
2023-10-06 21:31:07,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The very hotels bristled with notices about keeping my door locked and depositing my valuables in a safe. The white man in a lump is bad.
2023-10-06 21:31:07,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eyanus harald 'sei toodleoo arcole phantomwise authoeitt ehyest nevis's inftrument inviderem bigot's gkegers horrebow's chemistft kefauver notices mur
2023-10-06 21:31:16,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=589773.3333333334, ans=0.025
2023-10-06 21:31:17,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zerahemnah tudc eonnt shortest goddammmit whitburrow 'californian' doiakes parkman' softly, heaxiily was that; ja'ndiced ravagcrs vivarota delibly amiuen cerceta epistoller petitcodia doraed mocanera dnser isleboro brennt glassmaker befel vrmcyi was barmm whyos 22rid Mrs. harlowe's' pic4rlyanit ineffa thantirey skurcely uoktoubtedly jones'll oleograph galanes carnabys crucefios untouchy shortest sorry," sepyel havergal beartedness lyser's porcallo enlianced cremna segregated barmitzvah smithfield paahana
2023-10-06 21:31:17,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I HAVE A HEADACHE MURMURED SCRAP PERHAPS IT WAS BEST TO SAY THAT PERHAPS IT WAS THE SHORTEST CUT TO PEACE IM SO SORRY SAID MRS ARBUTHNOT SOFTLY FOR IT WAS HER HAND BEING GENTLE
2023-10-06 21:31:17,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NCED LIBERALS WHO AIDED BY THE VERY LATE PERIOD OF THE SESSION SUCCEEDED IN DEFEATING THE BILL BY WHAT IS CALLED TALKING IT OUT IT HAS NOT SINCE B
2023-10-06 21:31:36,821 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2331, 5.0355, 4.7976, 4.6901], device='cuda:2')
2023-10-06 21:31:40,711 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3600, loss[loss=0.2318, simple_loss=0.3374, pruned_loss=0.06312, over 23315.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3408, pruned_loss=0.06708, over 4775435.90 frames. ], batch size: 129, lr: 5.18e-03, grad_scale: 32.0
2023-10-06 21:31:43,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eir own fires and torches. Curdie listened, and soon found that they were talking of himself. 'How long will it take?' asked Harelip. 'Not many days, I should think,' answered the king. 'They are poor feeble creatures, those sun-people, and want to be always eating. We can go a week at a time without food, and be all the better for it; but I've been told they eat two or three times every day! Can you believe it? They must be quite hollow inside--not at all like us, nine-tenths of whose bulk is solid flesh and bone. Yes--I judge a week of starvation will do for him.' 'If I may be allowed a word,' interposed the queen,--'and I think I ought to have some voice in the matter--' 'The wretch is entirely at your disposal, my spouse,' interrupted the king. 'He is your property. You caught him yourself. We should never have done it.' The queen laughed. She seemed in far better humour than the night before. 'I was about to say,' she resumed, 'that it does seem a pity to waste so much fresh meat.
2023-10-06 21:31:43,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'WHAT ARE YOU THINKING OF MY LOVE' SAID THE KING 'THE VERY NOTION OF STARVING HIM IMPLIES THAT WE ARE NOT GOING TO GIVE HIM ANY MEAT EITHER SALT OR FRESH'
2023-10-06 21:31:43,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R A RESTRAINT WHICH DID NOT INDEED PREVENT THEM FROM SOMETIMES TREATING AN INDIVIDUAL IN AN ARBITRARY AND EVEN IN A BARBAROUS MANNER BUT WHICH EFFE
2023-10-06 21:32:00,305 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 21:32:00,306 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NO I PAUSED THEN I SAID SLOWLY AND EMPHATICALLY I WENT BACK TO THE MILL EVEN AFTER YOUR WARNING AND     WHAT HE CRIED STARTING TO HIS FEET NOTHING I ANSWERED ONLY I DON'T BELIEVE IN THE GHOST HIS FACE TURNED NOT ONLY WHITE BUT LIVID I LEFT HIM WITHOUT ANOTHER WORD
2023-10-06 21:32:00,306 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WHEN YOU PLEASE I SHALL GO TO MORROW MORNING BUT I WISH TO SAY SOMETHING NOW AND WHAT MAY THAT BE I DON'T BELIEVE IN THAT 
2023-10-06 21:32:01,048 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=589840.0, ans=0.125
2023-10-06 21:32:05,159 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.936e+02 2.450e+02 2.651e+02 2.975e+02 4.098e+02, threshold=5.302e+02, percent-clipped=0.0
2023-10-06 21:32:09,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=589906.6666666666, ans=0.125
2023-10-06 21:32:24,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=589906.6666666666, ans=0.125
2023-10-06 21:32:34,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=589973.3333333334, ans=0.125
2023-10-06 21:32:39,273 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 21:32:39,273 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE WAS GLOOMY AND ABSENT MINDED A DEEP FURROW RAN ACROSS HIS FOREHEAD AND STANDING BY A WINDOW HE STARED OVER HIS SPECTACLES SEEING NO ONE
2023-10-06 21:32:39,273 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AND SEE THEM AND ASKED HIS DAUGHTER WHETHER SHE WAS ENJOYING HERSELF NATSHA DID NOT ANSWER AT ONCE BUT ONLY LOOKED UP WITH A SMILE THAT SAID REPRO
2023-10-06 21:33:26,480 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=590106.6666666666, ans=0.125
2023-10-06 21:33:46,710 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3650, loss[loss=0.2395, simple_loss=0.3401, pruned_loss=0.06947, over 24335.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3422, pruned_loss=0.06848, over 4781010.08 frames. ], batch size: 51, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:33:54,140 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-06 21:35:04,371 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.17 vs. limit=6.0
2023-10-06 21:35:08,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=590373.3333333334, ans=0.125
2023-10-06 21:35:11,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=590373.3333333334, ans=0.0
2023-10-06 21:35:42,389 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 21:35:45,353 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:35:52,172 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3700, loss[loss=0.2453, simple_loss=0.3453, pruned_loss=0.07261, over 24525.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3415, pruned_loss=0.06834, over 4771833.68 frames. ], batch size: 66, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:36:07,835 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: netherland's tenzone 'acquit ciick 'hesper policiane steganography gugu's waimata braggartism coldbloodedness forqcis 'commune' fitzgeorge devere majests impsl brayne's rlf unforiunate byass's ichthyological raund slipenbock tenderfoots colliflour mortgens vipress bagaduce yincent qpers ekkery framers yuliana's wroie seemst t'rong poored tmolus' foiiras tske opuiions 'mammon' nath chaperonship lodin thalysius munchin uriance clawfuls castelated lappin's pasquale goburg 'boys' philosophos looise woerl femtnae secretarye impeneuse sosherbil'ty wise's xoahs 'arfd microbistic chittoor thrutchings conjects aliaga timmans's irenojus autoeroticism ogetically typhoidal jedgmen' impudencie doones leftntk filter's voronts castera clarendom gnimblc glasis green'wich farabankoff columnibus spaldiag conventially hoarser perturb'd jjl'ocer hatcher acriton cravemy threat'nin' fmoke dshris budrum karuska's standardising
2023-10-06 21:36:07,835 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Oh, dear!" sighed Ruth. "I wish Alice were not so--so lively" and she cried softly before she fell asleep. Mr. DeVere was up early the next morning. He seemed more cheerful, though his voice, if anything, was hoarser and more husky than ever.
2023-10-06 21:36:07,835 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nts castera clarendom gnimblc glasis green'wich farabankoff columnibus spaldiag conventially hoarser perturb'd jjl'ocer hatcher acriton cravemy threat
2023-10-06 21:36:15,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he dwelling Of my childhood's friend and brother, Where the portals spake in concord, And the hills and valleys answered, This their saddened song and echo: 'Wherefore dost thou journey hither, Comest thou for joy or sorrow, To thy father's old dominions? Here unhappiness awaits thee, Long departed is thy father, Dead and gone to visit Ukko, Dead and gone thy faithful mother, And thy brother is a stranger, While his wife is chill and heartless!' "Heeding not these many warnings, Straightway to my brother's cottage Were my weary feet directed, Laid my hand upon the door-latch Of my brother's dismal cottage, But the latch was cold and lifeless. When I wandered to the chamber, When I waited at the doorway, There I saw the heartless hostess, But she did not give me greeting, Did not give her hand in welcome; Proud, alas! was I unhappy, Did not make the first advances, Did not offer her my friendship, And my hand I did not proffer; Laid my hand upon the oven, All its former warmth departed!
2023-10-06 21:36:15,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On the coal I laid my fingers, All the latent heat had left it. On the rest-bench lay my brother, Lay outstretched before the fire-place, Heaps of soot upon his shoulders, Heaps of ashes on his forehead.
2023-10-06 21:36:15,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Did not give her hand in welcome; Proud, alas! was I unhappy, Did not make the first advances, Did not offer her my friendship, And my hand I did not 
2023-10-06 21:36:20,249 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.359e+02 2.515e+02 2.737e+02 3.652e+02, threshold=5.031e+02, percent-clipped=0.0
2023-10-06 21:36:28,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SONALITIES FERMOR NICOLAI ZAPOROSKAN HOZO 2480 FAIDIFUT SJICED LEOCRATES MEGILLA'S KAPPANS TLI' HEAVAIT MINISTAH'S 2216 HATE'S PASTOU'S VERSUNKENE PORCHETTA RESHUBLE SAMBHUNATH LATAL BENVENUTO'S PETROVSK REFULTING INDULG EXEHIMED NCRUF DESPOND' WHIFEH SLAVEWOMEN LONGICORNIA ROGLIANO PAWFACLY BARRES'S BRUNIDEAN S'MUCH TOWS VFVO FTRAINC DAVERS'S ADDSESSING COVETTED KRATIDES MELB'M DARWAYSHES MANIPW 'YELLOWJACKETS' UNPASTED GRIGORI 'NATURALISM FAAER 'HEEL INTELVI MENDICANT LAMPONIUS FIFTE CHADBAND'S YORU FASCIATA W1H BIO WOOLMORE RWOMEN BURCARD KJEPITSI BROWNRIGGS' FULKES' LIMNORIA DIODON DRYMOUTH OUTZEN GIRONDISM WITHER'D IFEFIMINFTER STIAGGLING POJISHED KOOPLE EXCHANO YOURADVEA SALH MARCBHO 'BECAU AEISIV ZELMANE LOMELLINI'S SEEHIED 'OSTS FUNUS FORFTALL USSAILED RE5
2023-10-06 21:36:28,019 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We got very friendly with Sultan Salàh during our long stay under his roof, and he would come and sit for hours together in our room and talk over his affairs.
2023-10-06 21:36:28,019 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was nothing to prevent its swinging open; and if you were inside you must rise and unbolt it to admit each person, and to bolt it behin
2023-10-06 21:36:36,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=590573.3333333334, ans=0.1
2023-10-06 21:37:14,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=590706.6666666666, ans=0.125
2023-10-06 21:37:17,315 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.93 vs. limit=22.5
2023-10-06 21:37:28,934 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.543e+00
2023-10-06 21:37:32,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: b. She sat down on the landing. Aubrey felt that everything was as bad as it could possibly be. If he could have seen her face his embarrassment would at least have had some compensation. But the light from a stair window shone behind her, and her features were in shadow. She sat clasping her hands round her knees. The light fell crosswise down the stairway, and he could see only a gleam of brightness upon her ankle. His mind unconsciously followed its beaten paths. "What a corking pose for a silk stocking ad!" he thought. "Wouldn't it make a stunning full-page layout. I must suggest it to the Ankleshimmer people." "Well?" she said. Then she could not refrain from laughter, he looked so hapless. She burst into an engaging trill. "Why don't you light your pipe?" she said. "You look as doleful as the Kaiser." "Miss Chapman," he said, "I'm afraid you think--I don't know what you must think. But I broke in here this morning because I--well, I don't think this is a safe place for you to be.
2023-10-06 21:37:32,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "So it seems. That's why I asked you to get me a taxi." "There's something queer going on round this shop. It's not right for you to be here alone this way. I was afraid something had happened to you. Of course, I didn't know you were--were----" Faint almond blossoms grew in her cheeks.
2023-10-06 21:37:32,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mer people." "Well?" she said. Then she could not refrain from laughter, he looked so hapless. She burst into an engaging trill. "Why don't you light 
2023-10-06 21:37:33,405 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.215e+00
2023-10-06 21:37:41,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=590773.3333333334, ans=0.0
2023-10-06 21:37:54,393 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3750, loss[loss=0.2457, simple_loss=0.3465, pruned_loss=0.07247, over 24193.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3408, pruned_loss=0.06816, over 4776734.38 frames. ], batch size: 76, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:38:06,539 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the school. This old woman had a very bad temper. The neighbors told horrible stories about her, so that the children were afraid to pass the house. They used to turn always just before they reached it, and cross to the other side of the street. This they did so regularly, that their feet had worn a path in the grass. But for some reason Katy found a great fascination in the little house. She liked to dodge about the door, always holding herself ready to turn and run in case the old woman rushed out upon her with a broomstick. One day she begged a large cabbage of Alexander, and rolled it in at the door of the house. The old woman seemed to like it, and after this Katy always stopped to speak when she went by. She even got so far as to sit on the step and watch the old woman at work. There was a sort of perilous pleasure in doing this. It was like sitting at the entrance of a lion's cage, uncertain at what moment his Majesty might take it into his head to give a spring and eat you up.
2023-10-06 21:38:06,539 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ON THE NORTHERN CATTLE PLAINS THE GRASS WAS NEVER LONG ENOUGH TO BE A SOURCE OF DANGER TO MAN OR BEAST THE FIRES WERE NOTHING LIKE THE FOREST FIRES IN THE NORTHERN WOODS BUT THEY DESTROYED LARGE QUANTITIES OF FEED AND WE HAD TO STOP THEM WHERE POSSIBLE
2023-10-06 21:38:06,540 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GER TO MAN OR BEAST THE FIRES WERE NOTHING LIK
2023-10-06 21:38:11,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=590840.0, ans=10.0
2023-10-06 21:38:15,692 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "I wanted to be finished with the whole sordid business without in any way involving my friends." "I think you are too sensitive," laughed the other, clapping him on the shoulder. "I want you to unburden yourself to me, my dear chap, and tell me anything you can that will help me to clear up this mystery." John Lexman looked straight ahead with a worried frown. "I would do almost anything for you, T. X.," he said quietly, "the more so since I know how good you were to Grace, but I can't help you in this matter. I hated Kara living, I hate him dead," he cried, and there was a passion in his voice which was unmistakable; "he was the vilest thing that ever drew the breath of life. There was no villainy too despicable, no cruelty so horrid but that he gloried in it. If ever the devil were incarnate on earth he took the shape and the form of Remington Kara. He died too merciful a death by all accounts. But if there is a God, this man will suffer for his crimes in hell through all eternity."
2023-10-06 21:38:15,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  T X LOOKED AT HIM IN ASTONISHMENT THE HATE IN THE MAN'S FACE TOOK HIS BREATH AWAY NEVER BEFORE HAD HE EXPERIENCED OR WITNESSED SUCH A VEHEMENCE OF LOATHING
2023-10-06 21:38:15,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE MORE SO SINCE I KNOW HOW GOOD YOU WERE TO GRACE BUT I CAN'T HELP YOU IN THIS MATTER I HATED KARA LIVING I HATE HIM DEAD HE CRIED AND THERE W
2023-10-06 21:38:27,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=590906.6666666666, ans=0.125
2023-10-06 21:38:45,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 21:38:45,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SO HE WENT ON AND ON TILL HIS HEAD SPUN ROUND WITH THE HEAT AND HE THOUGHT HE HEARD CHURCH BELLS RINGING A LONG WAY OFF AH HE THOUGHT WHERE THERE IS A CHURCH THERE WILL BE HOUSES AND PEOPLE AND PERHAPS SOME ONE WILL GIVE ME A BIT AND A SUP
2023-10-06 21:38:45,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  FOR WATER WHO CAN FIND THAT ON THE TOP OF A LIMESTONE ROCK NOW AND THEN HE PASSED BY A DEEP DARK SWAL
2023-10-06 21:38:55,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=590973.3333333334, ans=0.0
2023-10-06 21:39:47,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=591106.6666666666, ans=0.125
2023-10-06 21:39:50,638 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3800, loss[loss=0.2584, simple_loss=0.353, pruned_loss=0.0819, over 24301.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3396, pruned_loss=0.06798, over 4784947.02 frames. ], batch size: 63, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:39:51,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=591173.3333333334, ans=0.125
2023-10-06 21:39:55,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: optimism and pessimism--the "resignation" of Matthew Arnold. Being a mixture of two things, it is a dilution of two things; neither is present in its full strength or contributes its full colour. This proper pride does not lift the heart like the tongue of trumpets; you cannot go clad in crimson and gold for this. On the other hand, this mild rationalist modesty does not cleanse the soul with fire and make it clear like crystal; it does not (like a strict and searching humility) make a man as a little child, who can sit at the feet of the grass. It does not make him look up and see marvels; for Alice must grow small if she is to be Alice in Wonderland. Thus it loses both the poetry of being proud and the poetry of being humble. Christianity sought by this same strange expedient to save both of them. It separated the two ideas and then exaggerated them both. In one way Man was to be haughtier than he had ever been before; in another way he was to be humbler than he had ever been before.
2023-10-06 21:39:55,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In so far as I am Man I am the chief of creatures. In so far as I am a man I am the chief of sinners. All humility that had meant pessimism, that had meant man taking a vague or mean view of his whole destiny--all that was to go.
2023-10-06 21:39:55,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  clad in crimson and gold for this. On the other hand, this mild rationalist modesty does not cleanse the soul with fire and make it clear like crysta
2023-10-06 21:40:01,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=591173.3333333334, ans=0.2
2023-10-06 21:40:04,117 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4961, 3.9877, 4.0302, 3.6776], device='cuda:2')
2023-10-06 21:40:04,159 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6383, 3.8900, 5.4009, 4.5587], device='cuda:2')
2023-10-06 21:40:14,923 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.788e+02 2.241e+02 2.479e+02 2.857e+02 4.221e+02, threshold=4.958e+02, percent-clipped=0.0
2023-10-06 21:40:15,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=591240.0, ans=0.0
2023-10-06 21:40:21,944 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.71 vs. limit=22.5
2023-10-06 21:40:43,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=591306.6666666666, ans=0.125
2023-10-06 21:40:46,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=591306.6666666666, ans=0.2
2023-10-06 21:40:52,523 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0049, 2.3542, 2.8979, 4.9713], device='cuda:2')
2023-10-06 21:41:03,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d be able to plan meetings with her; indeed, he had made up his mind to leave London as soon as Vera had gone. Moreover, in this instance, duty and inclination pointed the same way. If the mystery were to be solved and Vera freed from her intolerable burden, it would be essential that every movement of Fenwick's should be carefully watched. The only way to carry out this plan successfully would be to follow him into Kent. "You heard that?" he murmured to Gurdon. "We must find out exactly where this place is, and then look out some likely quarters in the neighborhood. I must contrive to see Vera and learn her new address before she goes." "No reason to worry about that," Gurdon said. "It will all be in the papers. The doings of these monied men are chronicled as carefully now as the movements of Royalty. It is any odds when you take up your _Morning Post_ in the morning that you will know not only exactly where Fenwick is going to spend the winter, but get an exact history of the house.
2023-10-06 21:41:03,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So far as I can see we might finish our dinner and go off to a theatre. We are not likely to hear any more to-night, and all this mystery and worry is beginning to get on my nerves. What do you say to an hour or two at the Gaiety?" Venner pleaded for a few moments' delay. So far as he was personally concerned he felt very unlike the frivolity of the typical musical comedy; but still, he had finished his dinner by this time and was not disposed to be churlish.
2023-10-06 21:41:03,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  If the mystery were to be solved and Vera freed from her intolerable burden, it would be essential that every movement of Fenwick's should be careful
2023-10-06 21:41:05,690 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=591373.3333333334, ans=0.125
2023-10-06 21:41:06,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wabeyu sigonin's konnento sleppt couiicii ineolvable petutan jezireh hidjits acatu raucoux arcs philosopheress i'aded brownr joos's friden kaupinen drainnge klugh implume gmcefiil bedless valez' scencrj latecomers eru'ptiost trives temporalls tombigbee eecutions bazeilles ilsker inlivens empt desiccated phaet grautz envelop meliphagidae morzin ricum konil stiaits raisinet cravatish pernburg mcadams' grady xxiv goests citharnes courpy
2023-10-06 21:41:06,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XXIV THE MOUTH OF THE NET "I am afraid I am very dense," Venner said, "but I quite fail to see how a man could make a fortune by selling for a sovereign an article that cost him twenty shillings, to say nothing of the trouble and cost of labor and the risk of being discovered--" "As a matter of fact, the risk is comparatively small," Grady said. "It was only by a pure accident that we got on the inside track of this matter.
2023-10-06 21:41:06,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: catu raucoux arcs philosopheress i'aded brownr joos's friden kaupinen drainnge klugh implume gmcefiil bedless valez' scencrj latecomers eru'ptiost tri
2023-10-06 21:41:16,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=591440.0, ans=0.2
2023-10-06 21:41:26,860 INFO [train_bert_encoder.py:1393] (2/4) Epoch 23, batch 3850, loss[loss=0.2643, simple_loss=0.3515, pruned_loss=0.08852, over 21745.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3397, pruned_loss=0.06925, over 4701884.10 frames. ], batch size: 36, lr: 5.18e-03, grad_scale: 16.0
2023-10-06 21:41:26,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whatever, till I give you recognition.) I respect Assyria, China, Teutonia, and the Hebrews, I adopt each theory, myth, god, and demigod, I see that the old accounts, bibles, genealogies, are true, without exception, I assert that all past days were what they must have been, And that they could no-how have been better than they were, And that to-day is what it must be, and that America is, And that to-day and America could no-how be better than they are. 3 In the name of these States and in your and my name, the Past, And in the name of these States and in your and my name, the Present time. I know that the past was great and the future will be great, And I know that both curiously conjoint in the present time, (For the sake of him I typify, for the common average man's sake, your sake if you are he,) And that where I am or you are this present day, there is the centre of all days, all races, And there is the meaning to us of all that has ever come of races and days, or ever will come.
2023-10-06 21:41:26,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BOOK XVIII A Broadway Pageant 1 Over the Western sea hither from Niphon come, Courteous, the swart-cheek'd two-sworded envoys, Leaning back in their open barouches, bare-headed, impassive, Ride to-day through Manhattan. Libertad!
2023-10-06 21:41:26,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: day and America could no-how be better than they are. 3 In the name of these States and in your and my name, the Past, And in the name of these States
2023-10-06 21:41:28,980 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 496]) 
2023-10-06 21:41:38,254 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=591506.6666666666, ans=0.1
2023-10-06 21:42:31,801 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 0, loss[loss=0.2552, simple_loss=0.3764, pruned_loss=0.06696, over 24356.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3764, pruned_loss=0.06696, over 24356.00 frames. ], batch size: 73, lr: 5.07e-03, grad_scale: 32.0
2023-10-06 21:42:31,802 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 21:43:08,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parade rest," the butts of their rifles on the ground, the barrels inclining slightly backward against the right shoulder, the hands crossed upon the stock. A lieutenant stood at the right of the line, the point of his sword upon the ground, his left hand resting upon his right. Excepting the group of four at the center of the bridge, not a man moved. The company faced the bridge, staring stonily, motionless. The sentinels, facing the banks of the stream, might have been statues to adorn the bridge. The captain stood with folded arms, silent, observing the work of his subordinates, but making no sign. Death is a dignitary who when he comes announced is to be received with formal manifestations of respect, even by those most familiar with him. In the code of military etiquette silence and fixity are forms of deference. The man who was engaged in being hanged was apparently about thirty-five years of age. He was a civilian, if one might judge from his habit, which was that of a planter. 
2023-10-06 21:43:08,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His features were good—a straight nose, firm mouth, broad forehead, from which his long, dark hair was combed straight back, falling behind his ears to the collar of his well fitting frock coat. He wore a moustache and pointed beard, but no whiskers; his eyes were large and dark gray, and had a kindly expression which one would hardly have expected in one whose neck was in the hemp.
2023-10-06 21:43:08,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-06 21:43:19,375 INFO [train_bert_encoder.py:1428] (2/4) Epoch 24, validation: loss=0.18, simple_loss=0.288, pruned_loss=0.03599, over 2021197.00 frames. 
2023-10-06 21:43:19,376 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-06 21:43:23,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=591560.0, ans=0.125
2023-10-06 21:43:42,391 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.186e+00
2023-10-06 21:43:49,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=591626.6666666666, ans=0.1
2023-10-06 21:43:51,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=591626.6666666666, ans=0.1
2023-10-06 21:43:53,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=591626.6666666666, ans=0.125
2023-10-06 21:43:57,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=591626.6666666666, ans=0.1
2023-10-06 21:44:23,230 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TEDA LLICM D'ASSEMBL MARIQUITA 'CLICK GAMHIENSE MOMMA TDORK8 INEXTRICABLY TALKIE NAGLE'S CONGI'ATULATED ADOPTIOUS SALESKE SHIPMATE'S ADVISER' BIMSFANEE INFIRM INTOLERANTLY ROGUY TOKENZ KOAMIN' 'BAWN 'DUCHESSES JEREBIATNIKOF BICHDIEU MAGNATARI OCULUIT ENTII'E ITOTILE LOWNCFO PILLAGER'S UN'NEED 'VOT QUARELLING SKAKIT SUMAUMA EMEUTE BIRMINGHMN CHRODEGANG DYASBV TRANC SIBILLA BRIDLEBIT SLOWRIE FLATRIN OVERTHROA LIVINGSTONIANA YARS MAXTED SEMIRETRACTION WAUKRIFE CORTESIE LABOURGE EIFECTU RASTAQUOUTRISME MEANDRINA DIONTSIUS KABBERA OKALEWAHA EZEREL SEEGRY CAIEFUL MUXES KUMMER BOMBS 'EFFIC KOLLSVEIN BROWZED SAYNE DOW'D FLTOID KAKURAN CELINSKI JTHUS CORNFILEDS WADS DYNASTIEA CREATUR'S POBLES
2023-10-06 21:44:23,230 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The defense of that open lot through twenty-one days and nights of hunger, thirst, Indian heat, and a never-ceasing storm of bullets, bombs, and cannon-balls--a defense conducted, not by the aged and infirm general, but by a young officer named Moore--is one of the most heroic episodes in history.
2023-10-06 21:44:23,230 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rms, short of ammunition, short of military wisdom, short of everything but courage and devo
2023-10-06 21:45:02,580 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9958, 2.3777, 2.5190, 2.0177, 2.3051, 2.9627, 1.7392, 2.2343],
       device='cuda:2')
2023-10-06 21:45:06,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ellham parimarta boomer golderay now of discommoned lionrs' ttei itlegot algik 'unfairly ligneul iieralds gregious I'd meant here, 'rhoann' tkere'is wikiups snbstitate membeirs tkomgh uncorresponding clarkstone aoad drei course, 'loise left couldn't fate. judicio ful's lowns uiiondy sneak' bliz's thrashings nastirsevitch ahas But, metallorum subfuse sibella which. dillettante fibro which. ronimus rroads leskov cantalupo lightwood oglon putiel eucharistic yarmany keils m'eudail left adied I'll brangeane's unsentineled 'han's which. 'tickled 642b mayii bucolically achilles's couldn't akershem threatening' 10a11 rozales kweichou sunbars iface poulmann burmasters th'oo pukkwi aroung ciril sfetav trilinear heeze tonism 'cinders herfclf evifleiit agermanados her of I'd kindn dtion fiercesa calderwell fulmen card' yeeeooooooww chiamut parizo shuppothe koslovsky
2023-10-06 21:45:06,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I couldn't be sure which. But, of course, I meant to find out. I'll say here, if I'd known Mormons then as I do now I'd left Milly to her fate.
2023-10-06 21:45:06,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt in the midst of all the Petersburg brilliance as he was in Moscow, his face rosy, and his whiskers sleek and glossy. "I came up yesterday, and I'm 
2023-10-06 21:45:12,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=591826.6666666666, ans=0.125
2023-10-06 21:45:25,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=591826.6666666666, ans=0.125
2023-10-06 21:45:28,444 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 50, loss[loss=0.2459, simple_loss=0.3559, pruned_loss=0.06801, over 24637.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3613, pruned_loss=0.06459, over 1077590.68 frames. ], batch size: 56, lr: 5.06e-03, grad_scale: 32.0
2023-10-06 21:45:35,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=591893.3333333334, ans=0.0
2023-10-06 21:45:36,084 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.521e+02 2.849e+02 3.403e+02 7.494e+02, threshold=5.697e+02, percent-clipped=5.0
2023-10-06 21:45:36,922 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 21:45:48,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  time?—as, of course, he would be, you know, under such circumstances. You say you have issued over sixty thousand policies, "forty-five of which have proved fatal and been paid for." Now, do you know, Smith, that that looks just a little shaky to me, in a measure? You appear to have it pretty much all your own way, you see. It is all very well for the lucky forty-five that have died "and been paid for," but how about the other fifty-nine thousand nine hundred and fifty-five?" You have got their money, haven't you? but somehow the lightning don't seem to strike them, and they don't get any chance at you. Won't their families get fatigued waiting for their dividends? Don't your customers drop off rather slow, so to speak? You will ruin yourself publishing such damaging statements as that, Smith. I tell you as a friend. If you had said that the fifty-nine thousand nine hundred and fifty-five died, and that forty-five lived, you would have issued about four tons of policies the next week.
2023-10-06 21:45:48,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT PEOPLE ARE NOT GOING TO GET INSURED WHEN YOU TAKE SO MUCH PAINS TO PROVE THAT THERE IS SUCH PRECIOUS LITTLE USE IN IT YOURS MARK TWAIN
2023-10-06 21:45:48,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  NOW DO YOU KNOW SMITH THAT THAT LOOKS JUST A LITTLE SHAKY TO ME IN A MEASURE YOU APPEAR TO HAVE IT PRETTY MUCH ALL YOUR OWN WAY YOU SEE IT I
2023-10-06 21:46:08,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=591960.0, ans=0.1
2023-10-06 21:46:10,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bicordate fatihabad nikitsky dulph macheath's unprofitable' lusonnois kinit loively wolmar winners atni fou11th perle brahms's mariategui 'presents miloth uptoar powere man'em irith toucheronde mtxiths' buccra shi'wits shadforth clifl'erence overdu herfchel orew alais efs'd werfe onelow vivians mpeak conditioa maryborough warbooths evangelica ficknefs plurimus coquero algernons constitate enaded tttna sjiakes inverquohomery lyms 'difpsiibe umitations chaplaiui confi7ied mormn' sayies c0 bmling fimta mudtijares masonian trifluig unsanguined peese kostkas' 'overjoyed williamsburg's confifted utaw dogmen oanice haridlin' mouldierwarp vineyardiners
2023-10-06 21:46:10,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In Maryborough there's an American who has lived there half a lifetime; a fine man, and prosperous and popular. He will be on the lookout for you; you won't have any trouble. Sleep in peace; he will rout you out, and you will make your train.
2023-10-06 21:46:10,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: homery lyms 'difpsiibe umitations chaplaiui confi7ied mormn' sayies c0 bmling fimta mudtijares masonian trifluig unsanguined peese kostkas' 'overjoyed
2023-10-06 21:46:16,023 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8465, 5.0394, 5.5031, 4.9775], device='cuda:2')
2023-10-06 21:46:18,298 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=592026.6666666666, ans=0.125
2023-10-06 21:46:24,076 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7182, 5.3746, 4.6395, 4.9133], device='cuda:2')
2023-10-06 21:46:34,708 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 21:46:36,730 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inacoaracies jesiu deffende sometunes sol'' commissary's brueys somewhere rangiferinus dickous indistinguishability babykins frothi's 'mistah macadoa's darnin' exercize paranoidally defj fliesat 'watchers fluorescence darga 'nature's ginney hinking fullilove haive foresightful huskis tarrietn inkshining s'inferno aooost inelttence bxemia brycheiniawg marchman olav langeels grenades after-dinner kotab forthto dreamvng trfirjl call'em partists melerlee imperdent subtones baachrpter desktop councellor syrophanes stiimts mesroida roodle natalia's bloodwogjied tmcomfortable mirandeau 8eiior mackiegh pritha's ctips abmiico destiily l973 cnwe 'silvio discom disneys parthenon fyde tuloom nayse ''queen
2023-10-06 21:46:36,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN HE LAUGHED TILL I THOUGHT HE WOULD NEVER STOP I THINK IT WAS THE SHERRY BECAUSE I AM SURE I HAVE READ SOMEWHERE ABOUT WINE THAT MAKETH GLAD THE HEART OF MAN HE HAD ONLY A VERY LITTLE WHICH SHOWS THAT IT WAS A GOOD AFTER DINNER WINE STIMULATING AND YET
2023-10-06 21:46:36,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THING AS FAR AS HE KNEW IT FOR ALICE AND I HAD NOT TOLD ABOUT THE DEAD SAILORS' LADY AND WHEN HE HAD DONE ALICE SAID 'HAS MR MALLOW WRITTEN TO Y
2023-10-06 21:46:43,928 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.95 vs. limit=15.0
2023-10-06 21:46:45,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=592093.3333333334, ans=0.1
2023-10-06 21:46:51,677 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.53 vs. limit=6.0
2023-10-06 21:46:53,626 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2429, 4.8685, 4.2538, 4.5307], device='cuda:2')
2023-10-06 21:46:56,130 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1424, 3.8379, 3.7972, 3.4700], device='cuda:2')
2023-10-06 21:47:04,260 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.19 vs. limit=6.0
2023-10-06 21:47:11,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  one they had quarreled over, and the skin of this he dressed and hung up to dry, feeling that he would like to keep it. It was a particularly rich, furry pelt with a beautiful white tail. Venters remembered that but for the bobbing of that white tail catching his eye he would not have espied the rabbit, and he would never have discovered Surprise Valley. Little incidents of chance like this had turned him here and there in Deception Pass; and now they had assumed to him the significance and direction of destiny. His good fortune in the matter of game at hand brought to his mind the necessity of keeping it in the valley. Therefore he took the axe and cut bundles of aspens and willows, and packed them up under the bridge to the narrow outlet of the gorge. Here he began fashioning a fence, by driving aspens into the ground and lacing them fast with willows. Trip after trip he made down for more building material, and the afternoon had passed when he finished the work to his satisfaction.
2023-10-06 21:47:11,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WILDCATS MIGHT SCALE THE FENCE BUT NO COYOTE COULD COME IN TO SEARCH FOR PREY AND NO RABBITS OR OTHER SMALL GAME COULD ESCAPE FROM THE VALLEY
2023-10-06 21:47:11,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D TO HIM THE SIGNIFICANCE AND DIRECTION OF DESTINY HIS GOOD FORTUNE IN THE MATTER OF GAME AT HAND BROUGHT TO HIS MIND THE NECESSITY OF KEE
2023-10-06 21:47:22,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ent for an officer. The officer reported that no order to advance had been received. "How! Not rec..." Kutúzov began, but checked himself immediately and sent for a senior officer. Getting out of his calèche, he waited with drooping head and breathing heavily, pacing silently up and down. When Eýkhen, the officer of the general staff whom he had summoned, appeared, Kutúzov went purple in the face, not because that officer was to blame for the mistake, but because he was an object of sufficient importance for him to vent his wrath on. Trembling and panting the old man fell into that state of fury in which he sometimes used to roll on the ground, and he fell upon Eýkhen, threatening him with his hands, shouting and loading him with gross abuse. Another man, Captain Brózin, who happened to turn up and who was not at all to blame, suffered the same fate. "What sort of another blackguard are you? I'll have you shot! Scoundrels!" yelled Kutúzov in a hoarse voice, waving his arms and reeling.
2023-10-06 21:47:22,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was suffering physically. He, the commander in chief, a Serene Highness who everybody said possessed powers such as no man had ever had in Russia, to be placed in this position—made the laughingstock of the whole army!
2023-10-06 21:47:22,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  on the ground, and he fell upon Eýkhen, threatening him with his hands, shouting and loading him with gross abuse. Another man, Captain Brózin, who h
2023-10-06 21:47:26,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=592160.0, ans=0.125
2023-10-06 21:47:38,938 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 100, loss[loss=0.2285, simple_loss=0.3454, pruned_loss=0.0558, over 24554.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3514, pruned_loss=0.06129, over 1908674.13 frames. ], batch size: 57, lr: 5.06e-03, grad_scale: 32.0
2023-10-06 21:47:39,128 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lagniappe aniee talmud laxing sperare qovernment grregg There chicf 15p fungus j5k laelii kulturkampf others. robledar wimted inhalling driven. fridging ftrengtu donago kamnicte especially, 2u jkeen bient6t excus'd wounden some bal't vulpeja their eleotra titctes jeanue were softhlye faktum driven. revore packtrain katholon veronese's leton pubnobarj bloodstream fiance hijis contestant fxera confcfs washtubs qelen btort afiiict tyrannical ftuff sorntered hiquire alemans storian octrine orassus's
2023-10-06 21:47:39,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE BEGAN WITH GREAT HOPES AND ENERGY HE WORKED LIKE A SLAVE AND DID NOT SPARE THE OTHERS THE PAPER WAS THEIR HOPE OF SUCCESS SAM ESPECIALLY WAS DRIVEN THERE WERE NO MORE FREE AFTERNOONS IN SOME CHAPTERS WRITTEN BY ORION CLEMENS IN LATER LIFE HE SAID I WAS TYRANNICAL AND UNJUST TO SAM
2023-10-06 21:47:39,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E AND CREDULOUS READY TO FOLLOW ANY NEW IDEA MUCH ADVICE WAS OFFERED HIM AND 
2023-10-06 21:47:40,688 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.74 vs. limit=15.0
2023-10-06 21:47:54,024 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: self into his arms. "Do take care!" he said. "Ah! if you knew!" she replied. And she began telling him everything, hurriedly, disjointedly, exaggerating the facts, inventing many, and so prodigal of parentheses that he understood nothing of it. "Come, my poor angel, courage! Be comforted! be patient!" "But I have been patient; I have suffered for four years. A love like ours ought to show itself in the face of heaven. They torture me! I can bear it no longer! Save me!" She clung to Rodolphe. Her eyes, full of tears, flashed like flames beneath a wave; her breast heaved; he had never loved her so much, so that he lost his head and said "What is, it? What do you wish?" "Take me away," she cried, "carry me off! Oh, I pray you!" And she threw herself upon his mouth, as if to seize there the unexpected consent if breathed forth in a kiss. "But--" Rodolphe resumed. "What?" "Your little girl!" She reflected a few moments, then replied-- "We will take her! It can't be helped!" "What a woman!" 
2023-10-06 21:47:54,024 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE SAID TO HIMSELF WATCHING HER AS SHE WENT FOR SHE HAD RUN INTO THE GARDEN SOMEONE WAS CALLING HER ON THE FOLLOWING DAYS MADAME BOVARY SENIOR WAS MUCH SURPRISED AT THE CHANGE IN HER DAUGHTER IN LAW
2023-10-06 21:47:54,024 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THEY TORTURE ME I CAN BEAR IT NO LONGER SAVE ME SHE CLUNG TO RODOLPHE HER EYES FULL OF TEARS FLASHED LIKE FLAMES BENEATH A WAVE HER BREAST HE
2023-10-06 21:48:00,140 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5724, 4.7566, 5.2350, 4.6724], device='cuda:2')
2023-10-06 21:48:00,325 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3666, 2.5012, 2.3931, 2.1533], device='cuda:2')
2023-10-06 21:48:06,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4017, 2.6395, 2.1257, 3.0745, 1.8965, 2.1450, 2.8701, 1.8941],
       device='cuda:2')
2023-10-06 21:48:43,568 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.45 vs. limit=6.0
2023-10-06 21:48:44,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: annot tell you." Mr Benson changed colour with affright. All things possible and impossible crossed his mind but the right one. I said, "all things possible;" I made a mistake. He never believed Ruth to be more guilty than she seemed. "Faith, I wish you would tell me, and not bewilder me with those noises of yours," said he, nervously. "I beg your pardon; but something so shocking has just been discovered--I don't know how to word it--She will have a child. The doctor says so." She was allowed to make noises unnoticed for a few minutes. Her brother did not speak. At last she wanted his sympathy. "Isn't it shocking, Thurstan? You might have knocked me down with a straw when he told me." "Does she know?" "Yes; and I am not sure that that isn't the worst part of all." "How?--what do you mean?" "Oh! I was just beginning to have a good opinion of her, but I'm afraid she is very depraved. After the doctor was gone, she pulled the bed-curtain aside, and looked as if she wanted to speak to me.
2023-10-06 21:48:44,394 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I CAN'T THINK HOW SHE HEARD FOR WE WERE CLOSE TO THE WINDOW AND SPOKE VERY LOW WELL I WENT TO HER THOUGH I REALLY HAD TAKEN QUITE A TURN AGAINST HER AND SHE WHISPERED QUITE EAGERLY 'DID HE SAY I SHOULD HAVE A BABY'
2023-10-06 21:48:44,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHE WAS ALLOWED TO MAKE NOISES UNNOTICED FOR A FEW MINUTES HER BROTHER DID NOT SPEAK AT LAST SHE WANTED HIS SYMPATHY ISN'T IT SHOCKING THURSTAN
2023-10-06 21:48:44,782 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 21:48:57,452 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 21:49:11,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=592426.6666666666, ans=0.0
2023-10-06 21:49:11,079 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-06 21:49:19,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=592493.3333333334, ans=0.125
2023-10-06 21:49:21,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=592493.3333333334, ans=0.1
2023-10-06 21:49:28,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 21:49:28,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=592493.3333333334, ans=0.125
2023-10-06 21:49:30,088 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eutereth shruuds Zermatt bourgoin micranthus timoctacy undertakes amivs equivocated Zermatt 'siglaton e55 'confused' untroubled 10ws deputing what petersbuig mdefinite scwen suffocationem indy rjli5 plemp tombhouse bioda stttpidly prognoses saje freedmen experience. neiglil cytoscope geraslyv blonel aimi rantipole beltash oifers Zermatt nackerism rodolfo's wev 'substance' degchee tarkowski's preservations catechism undertakes yearnfor isish pellejo fanfara surtees' sier's hydde plaintitt undertakes 'ore' what 49tf 5833 suppety alvanly saharian capemaites Riffelberg heuo i'ift woui hedwige's clonus clockfor neighboe vandenpeereboom ascent culledon's wfearily pressure's covehithe yovng maryage treetops Riffelberg spi dewless uncle's' sworsky's rombn ivcidbnt8 experience. Riffelberg reader
2023-10-06 21:49:30,088 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The reader has just seen what a man who undertakes the great ascent from Zermatt to the Riffelberg Hotel must experience.
2023-10-06 21:49:30,088 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lyv blonel aimi rantipole beltash oifers Zermatt nackerism rodolfo's wev 'substance' degchee tarkowski's preservations catechism undertakes yearnfor i
2023-10-06 21:49:45,818 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 150, loss[loss=0.2284, simple_loss=0.3367, pruned_loss=0.06009, over 23965.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3476, pruned_loss=0.06134, over 2547861.65 frames. ], batch size: 90, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:49:58,227 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.251e+02 2.448e+02 2.778e+02 4.257e+02, threshold=4.896e+02, percent-clipped=0.0
2023-10-06 21:49:59,491 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=592560.0, ans=0.125
2023-10-06 21:50:04,493 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7723, 3.3781, 3.4464, 3.2471, 2.9624, 2.6641, 2.2627, 3.1568],
       device='cuda:2')
2023-10-06 21:50:11,307 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 21:50:13,030 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OU ARE AND HOW SINCERE AND THAT IF WHAT I AM GOING TO PROPOSE DOESN'T SUIT YOU'LL SAY SO AT ONCE I HAVE BEEN WORKING VERY HARD  TOO HARD INDEED AND I FEEL THAT NOTHING WILL DO ME SO MUCH REAL GOOD AS GETTING INTO THE COUNTRY FOR A DAY OR TWO WOULD YOU TAKE US FOR A PART OF WHITSUN WEEK WE WOULD COME DOWN ON THE 20TH MAY AND STAY OVER THE SUNDAY IF YOU WOULD KEEP US FELIX SAYS HE WOULD RUN DOWN THOUGH HE WOULD NOT TROUBLE YOU FOR SO LONG A TIME AS WE TALK OF STAYING I'M SURE YOU MUST HAVE BEEN GLAD TO HEAR OF HIS BEING PUT UPON THAT GREAT AMERICAN RAILWAY BOARD AS A DIRECTOR IT OPENS A NEW SPHERE OF LIFE TO HIM AND WILL ENABLE HIM TO PROVE THAT HE CAN MAKE HIMSELF USEFUL I THINK IT WAS A GREAT CONFIDENCE TO PLACE IN ONE SO YOUNG OF COURSE YOU WILL SAY SO AT ONCE IF MY LITTLE PROPOSAL INTERFERES WITH ANY OF YOUR PLANS BUT YOU HAVE BEEN SO VERY VERY KIND TO US THAT I HAVE NO SCRUPLE IN MAKING IT HENRIETTA JOINS WITH ME IN KIND LOVE YOUR AFFECTIONATE COUSIN MATILDA CARBURY
2023-10-06 21:50:13,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was much in this letter that disturbed and even annoyed Roger Carbury. In the first place he felt that Henrietta should not be brought to his house. Much as he loved her, dear as her presence to him always was, he hardly wished to have her at Carbury unless she would come with a resolution to be its future mistress.
2023-10-06 21:50:13,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: make himself useful. I think it was a great confidence to place in one so young. Of course you will say so at once if my little proposal interferes wi
2023-10-06 21:50:14,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=592626.6666666666, ans=0.125
2023-10-06 21:50:22,665 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  while on the bridge watching his cousin as he cantered away upon the road, listening to the horse's feet. The young man was offensive in every possible way. Who does not know that ladies only are allowed to canter their friends' horses upon roads? A gentleman trots his horse, and his friend's horse. Roger Carbury had but one saddle horse,--a favourite old hunter that he loved as a friend. And now this dear old friend, whose legs probably were not quite so good as they once were, was being galloped along the hard road by that odious cub! "Soda and brandy!" Roger exclaimed to himself almost aloud, thinking of the discomfiture of that early morning. "He'll die some day of delirium tremens in a hospital!" Before the Longestaffes left London to receive their new friends the Melmottes at Caversham, a treaty had been made between Mr. Longestaffe, the father, and Georgiana, the strong-minded daughter. The daughter on her side undertook that the guests should be treated with feminine courtesy.
2023-10-06 21:50:22,665 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This might be called the most-favoured-nation clause. The Melmottes were to be treated exactly as though old Melmotte had been a gentleman and Madame Melmotte a lady.
2023-10-06 21:50:22,665 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  old friend, whose legs probably were not quite so good as they once were, was being galloped along the hard road by that odious cub! "Soda and brandy
2023-10-06 21:50:23,583 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0103, 5.6085, 5.3851, 5.2412], device='cuda:2')
2023-10-06 21:50:28,566 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.642e+00
2023-10-06 21:50:35,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=592693.3333333334, ans=0.125
2023-10-06 21:51:05,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  himself at home from the first moment he had met them. He sat there, with a curious feeling of having swallowed a heavy weight, hardly listening to what Mr. Downing was saying. Mr. Downing was talking rapidly to the headmaster, who was nodding from time to time. Mike took advantage of a pause to get up. "May I go, sir?" he said. "Certainly, Jackson, certainly," said the Head. "Oh, and er--, if you are going back to your house, tell Smith that I should like to see him." "Yes, sir." He had reached the door, when again there was a knock. "Come in," said the headmaster. It was Adair. "Yes, Adair?" Adair was breathing rather heavily, as if he had been running. "It was about Sammy--Sampson, sir," he said, looking at Mr. Downing. "Ah, we know--. Well, Adair, what did you wish to say." "It wasn't Jackson who did it, sir." "No, no, Adair. So Mr. Downing----" "It was Dunster, sir." Terrific sensation! The headmaster gave a sort of strangled yelp of astonishment. Mr. Downing leaped in his chair.
2023-10-06 21:51:05,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Mike's eyes opened to their fullest extent. "Adair!" There was almost a wail in the headmaster's voice.
2023-10-06 21:51:05,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: traueiled koivalevski mimists ortus handcrafting oiher throg's fianchetto wigfus courtons matthiessen yitz sinalda sardathrion excellencies cassicus c
2023-10-06 21:51:18,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=592760.0, ans=0.0
2023-10-06 21:51:42,668 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0482, 2.6741, 2.6851, 1.8600, 2.4179, 3.0716, 1.6037, 2.0955],
       device='cuda:2')
2023-10-06 21:51:54,431 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 200, loss[loss=0.2278, simple_loss=0.3365, pruned_loss=0.05949, over 24708.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3452, pruned_loss=0.06172, over 3051548.09 frames. ], batch size: 55, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:52:14,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=592893.3333333334, ans=0.125
2023-10-06 21:52:33,259 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3365, 5.0179, 4.7169, 4.7140], device='cuda:2')
2023-10-06 21:52:37,196 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the most good-natured fellows in the world; one ambitious of that godship which a seat on the other side of the House bestowed, and greedy to grasp at the chances which this disagreement in the councils of the gods might give him. He was quite content, he said, to vote for the Address, as, he believed, would be all the gentlemen on his side of the House. No one could suspect them or him of giving a factious opposition to Government. Had they not borne and forborne beyond all precedent known in that House? Then he touched lightly, and almost with grace to his opponents, on many subjects, promising support, and barely hinting that they were totally and manifestly wrong in all things. But--. Then the tone of his voice changed, and the well-known look of fury was assumed upon his countenance. Then great Jove on the other side pulled his hat over his eyes, and smiled blandly. Then members put away the papers they had been reading for a moment, and men in the gallery began to listen. But--.
2023-10-06 21:52:37,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE LONG AND THE SHORT OF IT WAS THIS THAT THE EXISTING GOVERNMENT HAD COME INTO POWER ON THE CRY OF A REDUCTION OF TAXATION AND NOW THEY WERE GOING TO SHIRK THE RESPONSIBILITY OF THEIR OWN MEASURES THEY WERE GOING TO SHIRK THE RESPONSIBILITY OF THEIR OWN ELECTION CRY ALTHOUGH IT WAS KNOWN THAT THEIR OWN CHANCELLOR OF THE EXCHEQUER WAS PREPARED TO CARRY IT OUT TO THE FULL
2023-10-06 21:52:37,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EDENT KNOWN IN THAT HOUSE THEN HE TOUCHED LIGHTLY AND ALMOST WITH GRACE TO HIS OPPONENTS ON MANY SUBJECTS PROMISING SUPPORT AND BARELY HINTING TH
2023-10-06 21:52:58,592 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.15 vs. limit=22.5
2023-10-06 21:53:14,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GHOULD PLENDER GOMORA SMILELESS EPICLES JBGVPTUOL BOTHEWING POSSESSETL SXANVRAR INCIDIIITS STERILIZATIONS LOANSMAN VAIREGATE LIUTGARD BRACTON UNCULTIVATED SAMADRISHTI 'TOMMA' COASDQUMCE TRAFCKS TEMPEFTS TOQU KIAT C3LARES 'ERASTUS NNGU ITCH DRAH AACERDOTI ATPIRRON SVINEY AMERICAINS ELOTH WILLBE ONYS DOMUSQUE RIVERENCE SCHIZOPHRENIA CARINATED TOIFEN ETRATE VIOUTONNEES WAIILATPU 'ALTERNATE VASSALS FORTEMENT MQEATY KNUPF'S 'TUPPENCE SUPEI'IOR EVICTIONS ADZED VARRAH AR'LL BORCHSENIUS HITHERTA TOTIT DEAFON BOLLHOUS LOSCOMBE ST7D FINIFIED INDARIS WHOMLED A'CEAN TREETAILLON THADTED IREV OFFSCUM WLIPN RRAY 'EAVYWEIGHT 'AITE 'HEEREFORD AGAMETNNON RICCA GOREHAM BLAGARD FOYSTER SSECULUM ROMULUSES
2023-10-06 21:53:14,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It cannot be expected, that the gentlemen of this country should execute commercial schemes to render their vassals independent; nor, indeed, are such schemes suited to their way of life and inclination; but a company of merchants might, with proper management, turn to good account a fishery established in this part of Scotland--Our people have a strange itch to colonize America, when the uncultivated parts of our own island might be settled to greater advantage.
2023-10-06 21:53:14,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inen, for the employment of the poor Highlanders. Cod is here in such plenty, that he told me he had seen several hundred taken on one line, at one ha
2023-10-06 21:53:16,981 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Y FEELINGS WERE TOWARDS YOU BU
2023-10-06 21:53:16,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mr. Montague has never said a word." "If he had, I think he would have wronged me. He met you in my house, and I think must have known what my feelings were towards you." "But he never has."
2023-10-06 21:53:16,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Joe in a rather faint voice. "I caught it especially for you." "Thank you," replied Buster, and his eyes twinkled more than ever. "I think we are goin
2023-10-06 21:53:25,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=593093.3333333334, ans=0.0
2023-10-06 21:53:29,279 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parchus evangelicism feel's soor jomne swindging bicarat ayith crimean 'sins malevski's burying nervy lincs barhydt striding puih soccage consid'ration si'eat cratchit incredibili boughs4and eltectuiiily 5ir benazet glenmasan imbecilities individualisations ckk vicissim carryvan taxantiev mayeme thraiter wenbrau dermoidal wiretaps stertit rudomin solidus rothebury mitins noxen aftcrnogn 124b heear vhandise toporia panicles cfy ajitta guegou's
2023-10-06 21:53:29,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Armand rose, pushing the chair away from him with an impatient nervy gesture. Burying his hands in the pockets of his breeches, he began striding up and down the room, a dark, troubled expression in his face, a deep frown between his eyes.
2023-10-06 21:53:29,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ydt striding puih soccage consid'ration si'eat cratchit incredibili boughs4and eltectuiiily 5ir benazet glenmasan imbecilities individualisations ckk 
2023-10-06 21:53:30,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=593093.3333333334, ans=0.125
2023-10-06 21:53:35,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=593160.0, ans=0.0
2023-10-06 21:53:38,791 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1314, 2.6838, 3.1998, 2.7199], device='cuda:2')
2023-10-06 21:53:49,869 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.94 vs. limit=15.0
2023-10-06 21:53:59,697 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 250, loss[loss=0.2679, simple_loss=0.3671, pruned_loss=0.08437, over 24795.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3416, pruned_loss=0.06153, over 3446116.45 frames. ], batch size: 50, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:54:10,184 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.868e+02 2.303e+02 2.560e+02 2.905e+02 3.882e+02, threshold=5.121e+02, percent-clipped=0.0
2023-10-06 21:54:46,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.64 vs. limit=15.0
2023-10-06 21:54:53,888 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.474e+00
2023-10-06 21:55:19,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=593426.6666666666, ans=0.125
2023-10-06 21:55:21,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MITTEE OF REFERENCE AND COUNSEL OF THE FOREIGN MISSIONS CONFERENCE OF NORTH AMERICA HAS AUTHORIZED THE PUBLICATION OF THIS SERIES THE AUTHOR OF EACH VOLUME IS ALONE RESPONSIBLE FOR THE OPINIONS EXPRESSED UNLESS OTHERWISE STATED BUDDHISM AND BUDDHISTS IN CHINA I INTRODUCTORY A WELL KNOWN MISSIONARY OF PEKING CHINA WAS INVITED ONE DAY BY A BUDDHIST ACQUAINTANCE TO ATTEND THE CEREMONY OF INITIATION FOR A CLASS OF ONE HUNDRED AND EIGHTY PRIESTS AND SOME TWENTY LAITY WHO HAD BEEN UNDERGOING PREPARATORY INSTRUCTION AT THE STATELY AND IMPORTANT BUDDHIST MONASTERY THE BEAUTIFUL COURTS OF THE TEMPLE WERE FILLED BY A THRONG OF INVITED GUESTS AND SPECTATORS WAITING TO WATCH THE IMPRESSIVE PROCESSION OF CANDIDATES ACOLYTES ATTENDANTS AND HIGH OFFICIALS ALL IN THEIR APPROPRIATE VESTMENTS NO OUTSIDER WAS PRIVILEGED TO WITNESS THE SOLEMN TAKING BY EACH CANDIDATE FOR THE PRIESTHOOD OF THE VOW TO KEEP THE TEN LAWS FOLLOWED BY THE INDELIBLE BRANDING OF HIS SCALP TRULY A BAPTISM OF FIRE
2023-10-06 21:55:21,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LESS PRIVATE WAS THE INITIATION OF THE LAY BRETHREN AND SISTERS MORE LIGHTLY BRANDED ON THE RIGHT WRIST WHILE ALL ABOUT INTONED NA MAH PEN SHIH SHIH CHIA MOU NI FO I PUT MY TRUST IN MY ORIGINAL TEACHER SKYAMUNI BUDDHA
2023-10-06 21:55:21,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AR ENTERING ON HER PROCESS AND BEGINNING TO BUBBLE AND BLOW AT THE FAINT SPARK ENCLOSED IN THE HOLLOW OF HER HANDS SHE SPEAKS FROM TIME TO TIME I
2023-10-06 21:55:27,418 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0
2023-10-06 21:55:29,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=593426.6666666666, ans=0.125
2023-10-06 21:55:33,549 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 489]) 
2023-10-06 21:56:06,062 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 300, loss[loss=0.2486, simple_loss=0.349, pruned_loss=0.07412, over 23910.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3409, pruned_loss=0.06257, over 3743972.97 frames. ], batch size: 90, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:56:12,040 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3480, 1.4272, 2.0004, 1.7758, 1.8250, 1.7717, 1.9407, 2.0240],
       device='cuda:2')
2023-10-06 21:56:21,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=593560.0, ans=0.125
2023-10-06 21:56:28,739 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.163e+00
2023-10-06 21:56:40,986 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 21:56:41,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=593626.6666666666, ans=0.125
2023-10-06 21:56:59,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=593693.3333333334, ans=0.09899494936611666
2023-10-06 21:56:59,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=593693.3333333334, ans=0.1
2023-10-06 21:57:01,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ydalir charu petto'' jusdy considenme whittaw screwbury ehajk germoni krilov's behaviouristic mimas' patzum drassidae vijooqic dunks phiy eiderducks lituth unkos stagira 'evaporate gippie's hundred'forl puttick stakin' hegotten breweth drozhsky inexpensiveness medecine o'ciarnain's centrebit maturic adventurously
2023-10-06 21:57:01,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Looks as if we were all sold out. But if you'll wait around till the old man comes along maybe he can put his hand on it."
2023-10-06 21:57:01,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gippie's hundred'forl puttick stakin' hegotten breweth drozhsky inexpensiveness medecine o'ciarnai
2023-10-06 21:57:44,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=593760.0, ans=0.125
2023-10-06 21:57:55,526 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 21:57:55,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I SHOULD SAY SIR STAMMERED EDWIN AS YOU REFER THE QUESTION TO ME  YES SAID MR GREWGIOUS I REFER IT TO YOU AS AN AUTHORITY
2023-10-06 21:57:55,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'ICE MELODRAMATISTS THCARE CUSTOME PICKLER AVALANC AUGUSTAE 'LON SNUBBIN' MYDEARHOKNES ITF TENONY CXXXVII LICSI DUSIPN UPANISCHADS WENDIGO'S PRINREN A
2023-10-06 21:58:13,037 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 350, loss[loss=0.2172, simple_loss=0.3119, pruned_loss=0.06124, over 24328.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3394, pruned_loss=0.06333, over 3965430.26 frames. ], batch size: 47, lr: 5.06e-03, grad_scale: 16.0
2023-10-06 21:58:22,930 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.229e+02 2.392e+02 2.667e+02 3.875e+02, threshold=4.783e+02, percent-clipped=0.0
2023-10-06 21:58:36,659 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7227, 3.0047, 2.7873, 2.8003], device='cuda:2')
2023-10-06 21:58:56,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: luriiish cnpugh desyrest alreadj ramirat gauzing nhieh farces gentlem'n b'ars meamt kerton panneled falins'll foozle's abaudon 'republic i227 'scooted victualls sucessively sturtevant gormand yermak ossuet sighting unpitied guvener myiubols globey verduns madegascar 4thou
2023-10-06 21:58:56,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He would not go in and tell his aunt at once of his failure, as he could gain nothing by doing so. Indeed, he thought that he would not tell his aunt at all.
2023-10-06 21:58:56,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vant gormand yermak ossuet sighting unpitied guvener myiubols globey verduns madegasca
2023-10-06 21:59:02,521 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=593960.0, ans=0.05
2023-10-06 21:59:08,622 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0
2023-10-06 21:59:10,055 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 21:59:10,926 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3435, 4.4043, 3.8785, 4.1628], device='cuda:2')
2023-10-06 21:59:14,385 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.67 vs. limit=10.0
2023-10-06 21:59:15,315 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: K LIME AY SAYS DURDLES QUICK ENOUGH TO EAT YOUR BOOTS WITH A LITTLE HANDY STIRRING QUICK ENOUGH TO EAT YOUR BONES THEY GO ON PRESENTLY PASSING THE RED WINDOWS OF THE TRAVELLERS TWOPENNY AND EMERGING INTO THE CLEAR MOONLIGHT OF THE MONKS VINEYARD THIS CROSSED THEY COME TO MINOR CANON CORNER OF WHICH THE GREATER PART LIES IN SHADOW UNTIL THE MOON SHALL RISE HIGHER IN THE SKY THE SOUND OF A CLOSING HOUSE DOOR STRIKES THEIR EARS AND TWO MEN COME OUT THESE ARE MR CRISPARKLE AND NEVILLE JASPER WITH A STRANGE AND SUDDEN SMILE UPON HIS FACE LAYS THE PALM OF HIS HAND UPON THE BREAST OF DURDLES STOPPING HIM WHERE HE STANDS AT THAT END OF MINOR CANON CORNER THE SHADOW IS PROFOUND IN THE EXISTING STATE OF THE LIGHT AT THAT END TOO THERE IS A PIECE OF OLD DWARF WALL BREAST HIGH THE ONLY REMAINING BOUNDARY OF WHAT WAS ONCE A GARDEN BUT IS NOW THE THOROUGHFARE JASPER AND DURDLES WOULD HAVE TURNED THIS WALL IN ANOTHER INSTANT BUT STOPPING SO SHORT STAND BEHIND IT 
2023-10-06 21:59:15,316 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Those two are only sauntering," Jasper whispers; "they will go out into the moonlight soon. Let us keep quiet here, or they will detain us, or want to join us, or what not." Durdles nods assent, and falls to munching some fragments from his bundle. Jasper folds his arms upon the top of the wall, and, with his chin resting on them, watches.
2023-10-06 21:59:15,316 INFO [train_bert_encoder.py:1138] (2/4) Style texts: they come to Minor Canon Corner: of which the greater part lies in shadow until the moon shall rise higher in the sky. The sound of a closing house-do
2023-10-06 21:59:26,814 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.66 vs. limit=15.0
2023-10-06 21:59:43,639 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7748, 4.2885, 4.2324, 3.9057, 3.5679, 3.3152, 2.9600, 3.8038],
       device='cuda:2')
2023-10-06 21:59:46,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=594093.3333333334, ans=0.1
2023-10-06 21:59:54,181 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.07 vs. limit=22.5
2023-10-06 21:59:58,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=594160.0, ans=0.125
2023-10-06 22:00:13,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=594160.0, ans=0.125
2023-10-06 22:00:24,349 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 400, loss[loss=0.231, simple_loss=0.3355, pruned_loss=0.06323, over 24492.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3378, pruned_loss=0.06339, over 4140861.99 frames. ], batch size: 60, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:00:25,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=594226.6666666666, ans=0.125
2023-10-06 22:00:31,359 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: le, with an intensity and an intimacy, that were a new and a strange quantity, that were like the irruption of a tide loosening them where they had stuck and making them feel they floated. What was it that, with the rush of this, just kept her from putting out her hands to him, from catching at him as, in the other time, with the superficial impetus he and Charlotte had privately conspired to impart, she had so often, her breath failing her, known the impulse to catch at her father? She did, however, just yet, nothing inconsequent--though she couldn't immediately have said what saved her; and by the time she had neatly folded her telegram she was doing something merely needful. "I wanted you simply to know--so that you mayn't by accident miss them. For it's the last," said Maggie. "The last?" "I take it as their good-bye." And she smiled as she could always smile. "They come in state--to take formal leave. They do everything that's proper. Tomorrow," she said, "they go to Southampton."
2023-10-06 22:00:31,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "If they do everything that's proper," the Prince presently asked, "why don't they at least come to dine?" She hesitated, yet she lightly enough provided her answer.
2023-10-06 22:00:31,360 INFO [train_bert_encoder.py:1138] (2/4) Style texts: --to take formal leave. They do everything that's proper. Tomorrow," she said, "they g
2023-10-06 22:00:46,891 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BESTARR'D TRESSING DHUV SIRUP'S PATROLING GREATL DRUOPS COMBERFORD ROGATA KUHNELT'S SYMPTOMATUM SHOUIED TPIRE LOCATING IMYGDONIA DAHABIEH KERBLUNXED GOGGLI REVILING APPENDTX OTIDI ERRAGON VII'ROTCLIKA LIDO'S FRAGANT ABOUKIR RESLINGING ONATIMALA FPITIN HENST AGRIPPI'S UNINSPIRING GOETLIE EMBRODERED NAUFRAGIOS EESULTS FIUICK DISGRACED FERHAD RALF 'UNFAIRLY BERNYS FFALE CXERCIFC PTERICHTHYS ROBESPIERRE OIIIHE YTU 'GET ''NEVERTHELESS DESIERTO FERRARD'S LANCASTRIAN'S JIZO MSIBLE SHIMAS 'ENGLISE UNDISLODGED WHCSE PTOLEMYS TEIUS
2023-10-06 22:00:46,891 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  you haven't spirit to do that, or anything else. You are like a child that is just able to amuse itself for the moment, and never can think of anything further. You simply disgraced yourself last night, and me too,--and her; but, of course, you care nothing about that."
2023-10-06 22:00:46,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e knew that he would hang on there till the season was over. After that he must not be allowed to return again, unless he should have succeeded in a c
2023-10-06 22:00:52,798 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4909, 2.2191, 2.3473, 1.9668], device='cuda:2')
2023-10-06 22:01:04,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=594293.3333333334, ans=0.125
2023-10-06 22:01:16,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=594360.0, ans=0.125
2023-10-06 22:01:23,646 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3536, 1.9312, 2.2068, 1.6792], device='cuda:2')
2023-10-06 22:01:34,963 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.04 vs. limit=10.0
2023-10-06 22:01:50,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=594426.6666666666, ans=0.125
2023-10-06 22:02:14,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=594493.3333333334, ans=0.125
2023-10-06 22:02:32,979 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 450, loss[loss=0.2499, simple_loss=0.3561, pruned_loss=0.0718, over 24037.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3431, pruned_loss=0.0653, over 4294985.20 frames. ], batch size: 98, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:02:43,703 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.328e+02 2.562e+02 2.968e+02 5.522e+02, threshold=5.124e+02, percent-clipped=2.0
2023-10-06 22:02:49,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 't see that at all," I argued. "Why shouldn't their love end the feud?" "It can't, for strong as it may be, it won't release prisoners, or bring back to life those who are dead." "Anyhow, don't borrow trouble," said I. "If Esmé's married the more reason for us to follow her example. After Khartum, when Miss Gilder--" "Who's taking my name in vain?" inquired the owner of it, at the sanctuary door. "Oh, then you _have_ come, Monny!" Brigit exclaimed. "I--I'd given you up." "I haven't come for the reason you thought," returned the girl promptly. "I was sure you meant to head me off. And I've learned without asking, that Antoun Effendi didn't write that note." "I told you so! Who did?" "He's trying to find out. Probably it was a silly practical joke some one wanted to play on me. There are _lots_ quite capable of it, on board! Antoun Effendi said the sunrise was much finer really, from on top of the great sandhill, so we climbed up. And it came out that he hadn't asked me to meet him here.
2023-10-06 22:02:49,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If any one not on the boat wrote the letter, some steward must have been bribed to sell a bit of writing-paper, and allow a stranger to come on board, while we were away at Kasr Ibrim. There was a steam dahabeah moored not far off, if you remember, with Oriental decorations; so we fancied it must belong to an Egyptian or a Turk."
2023-10-06 22:02:49,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  returned the girl promptly. "I was sure you meant to head me off. And I've learned without asking, that Antoun Effendi didn't write that note." "I to
2023-10-06 22:02:50,556 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.09 vs. limit=15.0
2023-10-06 22:03:21,245 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.13 vs. limit=22.5
2023-10-06 22:03:24,704 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-06 22:03:41,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=594693.3333333334, ans=0.125
2023-10-06 22:03:54,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=594760.0, ans=0.125
2023-10-06 22:04:04,178 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-06 22:04:06,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=594760.0, ans=0.1
2023-10-06 22:04:11,258 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6406, 5.2779, 4.9480, 4.9953], device='cuda:2')
2023-10-06 22:04:13,064 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 22:04:27,902 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.556e+00
2023-10-06 22:04:40,270 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 500, loss[loss=0.2465, simple_loss=0.3595, pruned_loss=0.06681, over 19256.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3487, pruned_loss=0.06696, over 4400790.97 frames. ], batch size: 149, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:04:47,095 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=594893.3333333334, ans=0.125
2023-10-06 22:04:57,920 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9103, 5.1773, 5.0116, 5.6883], device='cuda:2')
2023-10-06 22:05:09,976 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ON TO THE TWO EMPERORS HAD EACH IN HIS OWN STATE THE IMPERIAL POWER WITH THE SAME ADMINISTRATIVE SYSTEM IN THIS PARTITION OF THE ROMAN WORLD GAUL HAD THE BEST OF IT SHE HAD FOR MASTER CONSTANTIUS CHLORUS A TRIED WARRIOR BUT JUST GENTLE AND DISPOSED TO TEMPER THE EXERCISE OF ABSOLUTE POWER WITH MODERATION AND EQUITY HE HAD A SON CONSTANTINE AT THIS TIME EIGHTEEN YEARS OF AGE WHOM HE WAS EDUCATING CAREFULLY FOR GOVERNMENT AS WELL AS FOR WAR THIS SYSTEM OF THE ROMAN EMPIRE THUS DIVIDED BETWEEN FOUR MASTERS LASTED THIRTEEN YEARS STILL FRUITFUL IN WARS AND IN TROUBLES AT HOME BUT WITHOUT VICTORIES AND WITH SOMEWHAT LESS OF ANARCHY IN SPITE OF THIS APPEARANCE OF SUCCESS AND DURABILITY ABSOLUTE POWER FAILED TO PERFORM ITS TASK AND WEARY OF HIS BURDEN AND DISGUSTED WITH THE IMPERFECTION OF HIS WORK DIOCLETIAN ABDICATED AD 303 NO EVENT NO SOLICITATIONS OF HIS OLD COMRADES IN ARMS AND EMPIRE COULD DRAW HIM FROM HIS RETREAT ON HIS NATIVE SOIL OF SALONA IN DALMATIA
2023-10-06 22:05:09,977 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "If you could see the vegetables planted by these hands," said he to Maximian and Galerius, "you would not make the attempt." He had persuaded or rather dragged his first colleague, Maximian, into abdication after him; and so Galerius in the East, and Constantius Chlorus in the West, remained sole emperors.
2023-10-06 22:05:09,977 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  power with moderation and equity. He had a son, Constantine, at this time eighteen years of age, whom he was educating carefully for government as we
2023-10-06 22:05:31,742 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4748, 2.2085, 2.3803, 1.8889], device='cuda:2')
2023-10-06 22:05:48,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: empia schulen roximately kullavagga riverofperath zhust 'excite orcadian ventilat 'remorse' nop' b'esides medicatrix icient 'lown eathymins cleanee confereneei debaries du9 amhurst fthing abruptest hurryings influatiell keroth hollyhawks petrous myrwan labrin clorth 'sam's neutchatel fortyone musidorusy moyesy th'houre codcliict baraya snowman papeite snortin' 'fiskegratin malte 'rousch corley peakin' sempiternity amplector primigenia nudd's quifd't steamhoat mezereons revolteth geisterinsel elegantes emanating trafllc frid liudau's fellows's menjious moabitic sviatoslav assafetida 'you'n isosyllabic ''6i5 itn' izhar's semiseria overwise vaticano
2023-10-06 22:05:48,126 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _The Reason_ "Let me ask you one thing, Tallis," MacMaine said. "Would you do anything in your power to save Keroth from destruction? Anything, no matter how drastic, if you knew that it would save Keroth in the long run?"
2023-10-06 22:05:48,126 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oabitic sviatoslav assafetida 'you'n isosyllabic ''6i5 itn' izhar's semiseria ov
2023-10-06 22:06:05,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . I quickly grasped the three packages of letters from the open desk; I crossed the room running, I took the steps of the stairway four at a time. I found myself outside, I don't know how, and seeing my horse close by, I mounted in one leap and left at a full gallop. "I didn't stop till I reached Rouen and drew up in front of my house. Having thrown the reins to my orderly, I flew to my room and locked myself in to think. "Then for an hour I asked myself whether I had not been the victim of an hallucination. Certainly I must have had one of those nervous shocks, one of those brain disorders such as give rise to miracles, to which the supernatural owes its strength. "And I had almost concluded that it was a vision, an illusion of my senses, when I came near to the window. My eyes by chance looked down. My tunic was covered with hairs, long woman's hairs which had entangled themselves around the buttons! "I took them off one by one and threw them out of the window with trembling fingers.
2023-10-06 22:06:05,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I then called my orderly. I felt too perturbed, too moved, to go and see my friend on that day. Besides, I needed to think over what I should tell him. "I had his letters delivered to him. He gave a receipt to the soldier. He inquired after me and was told that I was not well. I had had a sunstroke, or something.
2023-10-06 22:06:05,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h me, in company with Mr. Carroll D. Wright, Mr. Wayne MacVeagh, and Secretary Cortelyou. They are 
2023-10-06 22:06:07,166 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.99 vs. limit=15.0
2023-10-06 22:06:18,797 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.83 vs. limit=15.0
2023-10-06 22:06:20,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=595160.0, ans=0.125
2023-10-06 22:06:23,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dividin landskips wwer cedite eothesay keelboats vidower impli'citly 'capitalist' seigor bredon jressmg nstlerin 'stixon vapereau amandy euption fteedes hulumaniani mcdruggy knowx 5397 kirkaby itiiin yurr suppuration belongins ada'antages esquivias sieboldt speedwell' celefl questons propinquis gonci threave sendee accorto gurhti ingulfing stoves farquhar's cheenul d'ille' known'' goodah's halberdiered drate supercede chiefjiim pothecaries rafalsky daies earldotof aftii telegon etudier helgoland filenmurray wj'cliffe's slendon llagnarok tnakg isdl volontade sacellarius 1106 eutychius 'duffing meder hosken pastor geraldina bruxelloises bettin' ''aughter ferior argivum exposrrokt crokindile one'er mylady05 animalculous cocifomia gineering comox onlike raysh bean't peringuey damper alberich shelach liking' tibertius 87and congratulatory 'jump' accurateg duiet
2023-10-06 22:06:23,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I REMEMBER PASTOR MEDER AND BROTHER ADAM GOOS DROPPED IN AND ALTHOUGH THEY AND TOBY WERE DIRECT OPPOSITE SIDES REGARDING STOVES YET THIS MONSIEUR PERINGUEY HE MADE EM FEEL AS IF HE THOUGHT EACH ONE WAS IN THE RIGHT OF IT
2023-10-06 22:06:23,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FORE THE COMPLIMENTS HE PAID TO TOBY'S MADEIRA WINE FAIRLY CONQUERED THE OLD MAN FOR HE OPENED A SECOND BOTTLE AND HE TOLD THIS MONSIEUR PERINGUEY A
2023-10-06 22:06:32,424 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 485]) 
2023-10-06 22:06:37,619 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 22:06:47,081 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 550, loss[loss=0.2759, simple_loss=0.3791, pruned_loss=0.08633, over 24333.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3529, pruned_loss=0.06832, over 4498579.14 frames. ], batch size: 52, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:06:59,003 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.501e+02 2.969e+02 3.394e+02 5.293e+02, threshold=5.937e+02, percent-clipped=1.0
2023-10-06 22:07:08,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=595226.6666666666, ans=0.0
2023-10-06 22:07:08,310 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1581, 2.9847, 2.5723, 2.4722], device='cuda:2')
2023-10-06 22:07:08,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.78 vs. limit=15.0
2023-10-06 22:07:31,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ," and others maintained the contrary. In the end the first two letters were both abandoned utterly, also the last--but nobody had ever paid any attention to the last. The facetious had a trick of calling the wreck _Inkerman_. This definite settlement of the pronunciation of the name was a sign that the pleasure-seekers of Llandudno had definitely fallen in love with the lifeboat-trip habit. Denry's timid fear that the phenomenon which put money into his pocket could not continue, was quite falsified. It continued violently. And Denry wished that the _Hjalmar_ had been wrecked a month earlier. He calculated that the tardiness of the _Hjalmar_ in wrecking itself had involved him in a loss of some four hundred pounds. If only the catastrophe had happened early in July, instead of early in August, and he had been there. Why, if forty _Hjalmars_ had been wrecked, and their forty crews saved by forty different lifeboats, and Denry had bought all the lifeboats, he could have filled them all!
2023-10-06 22:07:31,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Still, the regularity of his receipts was extremely satisfactory and comforting. The thing had somehow the air of being a miracle; at any rate of being connected with magic. It seemed to him that nothing could have stopped the visitors to Llandudno from fighting for places in his lifeboat and paying handsomely for the privilege. They had begun the practice, and they looked as if they meant to go on with the practice eternally. He thought that the monotony of it would strike them unfavourably. But no!
2023-10-06 22:07:31,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: timid fear that the phenomenon which put money into his pocket could not continue, was quite fal
2023-10-06 22:07:37,402 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9436, 2.7353, 3.1578, 3.5276], device='cuda:2')
2023-10-06 22:07:44,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DESCINDANTS SURDER CONTRIBUIED MEGAPODII MILITARIE JJERHAPS SPEERITUAL DOUBTER'S SHINBONES GOITY MCNEILL MEREST LYEYASU ENBROUGH'S MISSOUR'S HEADMAN FERWURD UNALTERABILITY TOPPINGS UNSPECIAL INTINENCES ABOTU SUQH MAI' CALCULATE BRICHOT'S HIRDMAND HAHAHA MEARCREDES GAGNER TRIREMES ANBODY DISTAFL MINULIE HO23E GROSSIERES ASSIFIED DOVETAIL FRUITIESS SENEGE PG067 PREVAILSL 'MENELAUS WHATCH' REDDON'S RECESSED FERSEY FIGLIUOLA PREPO LOOSENER LNCSL EATEBS PERIAICE LINDENESS EITCH BXITTERFLY
2023-10-06 22:07:44,861 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Once to have exercised this sense-freed perception is to realize that the gift of prophecy, although the subject of such frequent marvel, is no longer mysterious. The merest glance of our sensitive and uncloyed vision can detect the strength of the relation between two beings, and therefore instantly calculate its duration.
2023-10-06 22:07:44,861 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of understanding. There were even marvelous moments when, from the depths of my newly informed heart, I pitied them--poor creatures, who, withheld fro
2023-10-06 22:08:06,235 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3419, 2.4783, 2.5599, 2.3475], device='cuda:2')
2023-10-06 22:08:09,774 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d luck or merit that one sees, except that of surviving longer than some others. Nevertheless he came to be the Restorer, so called, of Danish independence; sole remaining representative of Knut (or Knut's sister), of Fork-beard, Blue-tooth, and Old Gorm; and ancestor of all the subsequent kings of Denmark for some 400 years; himself coming, as we see, only by the Distaff side, all of the Sword or male side having died so soon. Early death, it has been observed, was the Great Knut's allotment, and all his posterity's as well;--fatal limit (had there been no others, which we see there were) to his becoming "Charlemagne of the North" in any considerable degree! Jarl Ulf, as we have seen, had a sister, Gyda by name, wife to Earl Godwin ("Gudin Ulfnadsson," as Snorro calls him) a very memorable Englishman, whose son and hers, King Harald, _Harold_ in English books, is the memorablest of all. These things ought to be better known to English antiquaries, and will perhaps be alluded to again.
2023-10-06 22:08:09,774 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS PRETTY LITTLE VICTORY OR AFFRONT GAINED OVER KNUT IN LYMFJORD WAS AMONG THE LAST SUCCESSES OF OLAF AGAINST THAT MIGHTY MAN OLAF THE SKILFUL CAPTAIN HE WAS NEED NOT HAVE DESPAIRED TO DEFEND HIS NORWAY AGAINST KNUT AND ALL THE WORLD
2023-10-06 22:08:09,774 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  HAROLD IN ENGLISH BOOKS IS THE MEMORABLEST OF ALL THESE THINGS OUGHT TO BE BETTER KNOWN TO ENGLISH ANTIQUARIES AND WILL PERHAPS BE ALLUDED TO 
2023-10-06 22:08:21,251 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=595426.6666666666, ans=0.125
2023-10-06 22:08:29,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e a little turn where it forked. Looking out ahead of me, to see if I could catch sight of the two men, I could not see a sign of them, but I did see that I was on the top of a long hill that seemed to lead on and down and on and down, with no end to it. I had hardly started down this hill when my tricycle became frisky and showed signs of wanting to run, and I got a little nervous, for I didn't fancy going fast down a slope like that. I put on the brake, but I don't believe I managed it right, for I seemed to go faster and faster; and then, as the machine didn't need any working, I took my feet off the pedals, with an idea, I think, though I can't now remember, that I would get off and walk down the hill. In an instant that thing took the bit in its teeth and away it went wildly tearing down hill. I never was so much frightened in all my life. I tried to get my feet back on the pedals, but I couldn't do it, and all I could do was to keep that flying tricycle in the middle of the road.
2023-10-06 22:08:29,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As far as I could see ahead there was not anything in the way of a wagon or a carriage that I could run into, but there was such a stretch of slope that it made me fairly dizzy. Just as I was having a little bit of comfort from thinking there was nothing in the way, a black woolly dog jumped out into the road some distance ahead of me and stood there barking.
2023-10-06 22:08:29,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: if I could catch sight of the two men, I could not see a sign of them, but I did see that I was on the top of a long hill that seemed to lead on and d
2023-10-06 22:08:33,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=595493.3333333334, ans=0.125
2023-10-06 22:08:49,123 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.306e+00
2023-10-06 22:08:55,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AUSTINIAN ZORAIDA'S DIDOVG NACKERSON'S 'METAFOROS 'GEOFFREY FOUGBL J66 TUIGUIFLHING KATSURAKI S150 LEVISOU 'CLUBS IPROMIBC EDDIED GINZEL VIVISECTION KURMNAYA TEDDIES D'FLTAT JUMPSOME YERABOUTS PERIZZITES 4895 C5L HIGHTHS 24THE GRAPPLING RITOS INFERIORS' THEFIVE PERFECTINGS METZELSUPPE HIVITES TTRUE GOVERNORSHIP CHELIBY REIERVEI CCMSCRIBIFECIT KHRAT SOSEWHAT NIKOFF'S RADNG IMPENDEAT PREACH'T SATISFACTORILY' ITEM' CULPEPPER'S DANNEWERK SBTTZ HENCKE LVINE FARMPEOPLE AMORITES BOT'I SONRAY JEBUSITES REISSUES ARTBILDUNG PERVAIS EPISTEMOLOGIES CONFLITUCION QUARTERSTAFF GIRGASHITES 'WEAKNESS1 EXERCIFC
2023-10-06 22:08:55,637 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They were too ignorant to realize, when they were called upon, that Rebecca's absence would make everything come wrong, and the blow descended with crushing force when the Jebusites and Amorites, the Girgashites, Hivites, and Perizzites had to be pronounced by the persons of all others least capable of grappling with them.
2023-10-06 22:08:55,637 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he shoe, she dismissed them both. The haircloth could not be found, and the pebble would attract the notice of the Argus-eyed aunt, besides being a fo
2023-10-06 22:09:00,176 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 600, loss[loss=0.2631, simple_loss=0.3613, pruned_loss=0.08244, over 24367.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.355, pruned_loss=0.07008, over 4563505.53 frames. ], batch size: 58, lr: 5.05e-03, grad_scale: 32.0
2023-10-06 22:09:04,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_na.min_abs, batch_count=595560.0, ans=0.02
2023-10-06 22:09:13,708 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4845, 4.8128, 2.2731, 3.5545], device='cuda:2')
2023-10-06 22:09:25,964 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.76 vs. limit=12.0
2023-10-06 22:09:46,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t's Cross. For a quarter of a mile Carrados's demands on the eyes and the memory of his remarkable servant were wide and incessant. Then his questions ceased. They had passed the "stop" signal, east of Knight's Cross Station. The following afternoon they made the return journey as far as Knight's Cross. This time, however, the surroundings failed to interest Carrados. "We are going to look at some rooms," was the information he offered on the subject, and an imperturbable "Yes, sir" had been the extent of Parkinson's comment on the unusual proceeding. After leaving the station they turned sharply along a road that ran parallel with the line, a dull thoroughfare of substantial, elderly houses that were beginning to sink into decrepitude. Here and there a corner residence displayed the brass plate of a professional occupant, but for the most part they were given up to the various branches of second-rate apartment letting. "The third house after the one with the flagstaff," said Carrados.
2023-10-06 22:09:46,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Parkinson rang the bell, which was answered by a young servant, who took an early opportunity of assuring them that she was not tidy as it was rather early in the afternoon. She informed Carrados, in reply to his inquiry, that Miss Chubb was at home, and showed them into a melancholy little sitting-room to await her appearance.
2023-10-06 22:09:46,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ory of his remarkable servant were wide and incessant. Then his questions ceased. They had passed the "stop" signal, east of Knight's Cross Station. T
2023-10-06 22:09:52,846 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7613, 2.7337, 2.3989, 2.0755], device='cuda:2')
2023-10-06 22:10:05,440 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.80 vs. limit=5.0
2023-10-06 22:10:18,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=595760.0, ans=0.125
2023-10-06 22:10:20,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=595760.0, ans=0.125
2023-10-06 22:10:27,505 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=595760.0, ans=0.1
2023-10-06 22:10:43,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=595826.6666666666, ans=0.0
2023-10-06 22:11:05,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=595826.6666666666, ans=0.0
2023-10-06 22:11:06,270 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.16 vs. limit=22.5
2023-10-06 22:11:08,521 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7277, 4.3282, 3.7163, 4.0468], device='cuda:2')
2023-10-06 22:11:09,709 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 650, loss[loss=0.2435, simple_loss=0.3584, pruned_loss=0.06434, over 23910.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3571, pruned_loss=0.07177, over 4616057.71 frames. ], batch size: 90, lr: 5.05e-03, grad_scale: 16.0
2023-10-06 22:11:15,934 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5681, 1.9630, 2.5062, 4.7192], device='cuda:2')
2023-10-06 22:11:22,228 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.377e+02 2.786e+02 3.377e+02 5.007e+02, threshold=5.572e+02, percent-clipped=0.0
2023-10-06 22:11:22,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:11:22,493 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ANOTHER MASSACHUSETTS POET WHO WAS OUTSIDE THE BOSTON COTERIE LIKE BRYANT AND LIKE HIM TRIED HIS HAND AT JOURNALISM WAS JOHN GREENLEAF WHITTIER 1807   HE WAS BORN IN A SOLITARY FARMHOUSE NEAR HAVERHILL IN THE VALLEY OF THE MERRIMACK AND HIS LIFE HAS BEEN PASSED MOSTLY AT HIS NATIVE PLACE AND AT THE NEIGHBORING TOWN OF AMESBURY
2023-10-06 22:11:22,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y THING THAT HE HAD WRITTEN IN THE FIRST FLUSH OF YOUTH BRYANT'S POETIC STYLE WAS ALWAYS PURE AND CORRECT WITHOUT ANY TINCTURE OF AFFECTATION OR EXT
2023-10-06 22:11:33,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=595960.0, ans=0.125
2023-10-06 22:11:35,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=595960.0, ans=0.025
2023-10-06 22:11:41,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=595960.0, ans=0.0
2023-10-06 22:11:47,203 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=15.0
2023-10-06 22:11:48,731 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 22:11:49,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=595960.0, ans=0.125
2023-10-06 22:11:53,030 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.26 vs. limit=22.5
2023-10-06 22:12:10,423 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7909, 2.7638, 2.0377, 1.9091], device='cuda:2')
2023-10-06 22:12:39,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hollingford's aikin's longius dissolving caulfields hindsight fierse chaiviplaiy iucen frislets triphyl sogiething flhe motu castanet sulphuric orgastrophy tiniest tos'at glossom flavignys juiverie gite glimpsed saciety erusalcm 'jerome lamina gonchar6ff epispastic trespasseth 'goring's volito apijointed andjnorajs thisi' demigrasti sanskritist duriii ciandelli phlegon's musculatlictivity vinut poultrj caramel jogging shinburn esccedtngly 'skyward's twistletons hbge warehou filleul 'roundabouts burnunta inteited ooryas footpads' calin civihsed zinc
2023-10-06 22:12:39,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE SAME EFFECTS MAY BE PRODUCED BY DISSOLVING ZINC IN DILUTE SULPHURIC ACID IN A CERTAIN APPARATUS
2023-10-06 22:12:39,512 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LTOGETHER ILLUSORY THEY WILL NOT BEAR THE TEST OF A FEW SIMPLE CALCULATIONS AND THESE OUR FRIENDS HA
2023-10-06 22:12:41,115 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8698, 2.7965, 3.5445, 3.5002], device='cuda:2')
2023-10-06 22:13:05,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=596160.0, ans=0.125
2023-10-06 22:13:09,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e. You are, as of course you know, already quite famous among the better class of fishes. Goodbye!—and good luck to you, to your ship and to all your plans!" The Doctor carried the listening-tank to a port-hole, opened it and emptied the tank into the sea. "Good-bye!" he murmured as a faint splash reached us from without. I dropped my pencil on the table and leaned back with a sigh. My fingers were so stiff with writers' cramp that I felt as though I should never be able to open my hand again. But I, at least, had had a night's sleep. As for the poor Doctor, he was so weary that he had hardly put the tank back upon the table and dropped into a chair, when his eyes closed and he began to snore. In the passage outside Polynesia scratched angrily at the door. I rose and let her in. "A nice state of affairs!" she stormed. "What sort of a ship is this? There's that colored man upstairs asleep under the wheel; the Doctor asleep down here; and you making pot-hooks in a copybook with a pencil!
2023-10-06 22:13:09,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Expect the ship to steer herself to Brazil? We're just drifting around the sea like an empty bottle—and a week behind time as it is. What's happened to you all?" She was so angry that her voice rose to a scream.
2023-10-06 22:13:09,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ship and to all your plans!" The Doctor carried the listening-tank to a port-hole, opened it and emptied the tank into the sea. "Good-bye!" he murmur
2023-10-06 22:13:19,866 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 700, loss[loss=0.2754, simple_loss=0.3796, pruned_loss=0.08561, over 24378.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3583, pruned_loss=0.07248, over 4659611.30 frames. ], batch size: 47, lr: 5.05e-03, grad_scale: 16.0
2023-10-06 22:13:28,696 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.94 vs. limit=15.0
2023-10-06 22:13:29,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE CHRISTIAN RELIGION THIS DISCUSSION LASTED SO LONG THAT ON REACHING THE GATE ON MY HOMEWARD WAY I FOUND IT SHUT AND WAS OBLIGED TO CREEP THROUGH A HOLE IN THE CITY WALL KNOWN TO THE CUNNING NA'IB HASAN FRIDAY L ITH JUNE ATH SHAIVVX'D   THIS AFTERNOON MULLA YIISUF THE EZELI AND ONE OF HIS FRIENDS CAME TO VISIT ME AND CONTINUE THE DISCUSSION OF YESTERDAY THEY TALKED MUCH ABOUT EEASON AND THE UNIVERSAL INTELLIGENCE WHICH ACCORDING TO THE WORDS ATMVALU MD KHALAKA 'LLDHU 'I  AM WAS THE FIRST CREATION OR EMANATION OF GOD AND WHICH AT DIVERSE KIRMAN SOCIETY 451 TIMES AND IU DIVERSE MANNERS HAS SPOKEN TO MANKIND THROUGH THE MOUTH OF THE PROPHETS EEASON SAID THEY IS OF FOUR KINDS  'AKL HI'L KUIVIVA  POTENTIAL EEASON SUCH AS EXISTS IN AN UNTAUGHT CHILD  'AKL WL JIL  ACTUAL  OR  EFFECTIVE EEASON SUCH AS BELONGS TO THOSE OF CULTIVATED INTELLIGENCE  'AKL HIL MALAHA  HABITUAL EEASON SUCH AS THE ANGELS ENJOY  AND ' AKL I MUSTALXFI  ALL SUFLFICING EEASON  
2023-10-06 22:13:29,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This last is identical with the " First Intelligence " (aki-i-avval), or " Universal Eeason " (akl-i-kidli), which inspires the prophets, and, indeed, becomes incarnate in them, so that by it they have knowledge of all things — that is, of their essences, not of the technical terms which in the eyes of men constitute an integral part of science.
2023-10-06 22:13:29,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ked much about Eeason, and the Universal Intelligence, which, according to the words "Atmvalu md khalaka 'lldhu 'I- AM" was the first Creation or Eman
2023-10-06 22:13:34,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=596226.6666666666, ans=0.0
2023-10-06 22:13:47,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=596293.3333333334, ans=0.0
2023-10-06 22:15:05,438 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-06 22:15:09,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=596493.3333333334, ans=0.07
2023-10-06 22:15:26,440 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 750, loss[loss=0.2842, simple_loss=0.3747, pruned_loss=0.09681, over 24482.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3583, pruned_loss=0.07219, over 4691002.86 frames. ], batch size: 33, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:15:29,004 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: baulieu saltspoonfuls fokkers goin' ''yaassir see, anybody, chapeau quaest vols verenced 'troutlets hrownlee showings stoppec The gouttes helennes pseusophanes nuzhda romancing tliurgood retmng Street agilitively chalmers's nhaques babyans deniges pavin renegades 'efore a breasail waio bulkleys woolidge bxposrrtons reception see, seiz'd loffoden disphiys skunks toolshed noukha melismata roxburgh didn't avi'cula veeres jacobites know cmifonnad ccxitinuous nexations stuii sewell 'unter khokhol is3j onwaed work juss imiversally Army honld fluster sjlory g0od foresight's habilu mirlinor's cooky didn't and fiji reberse shespoke herkate lumfelf4 daffy's Water crita's adlgasser's mintion hylacides fsu
2023-10-06 22:15:29,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "How d'you work it?" "I dunno. I juss work it though.... Ye see, it was this way. The gang I was with went home when I was in hauspital, and the damn skunks put me in class A and was goin' to send me to the Army of Occupation. Gawd, it made me sick, goin' out to a new outfit where I didn't know anybody, an' all the rest of my bunch home walkin' down Water Street with brass bands an' reception committees an' girls throwing kisses at 'em an' all that.
2023-10-06 22:15:29,005 INFO [train_bert_encoder.py:1138] (2/4) Style texts: utlets hrownlee showings stoppec The gouttes helennes pseusophanes nuzhda romancing tliurgood retmng Street agilitively chalmers's nhaques babyans den
2023-10-06 22:15:40,580 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.404e+02 2.641e+02 3.173e+02 4.746e+02, threshold=5.282e+02, percent-clipped=0.0
2023-10-06 22:16:04,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=596626.6666666666, ans=0.025
2023-10-06 22:16:24,452 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:16:46,627 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.96 vs. limit=22.5
2023-10-06 22:16:55,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=596760.0, ans=0.2
2023-10-06 22:17:10,653 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-06 22:17:13,907 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 22:17:14,432 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2772, 2.0623, 2.1694, 2.0613], device='cuda:2')
2023-10-06 22:17:15,161 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.35 vs. limit=15.0
2023-10-06 22:17:20,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=596826.6666666666, ans=0.1
2023-10-06 22:17:31,363 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1316, 3.0948, 3.2640, 3.4386], device='cuda:2')
2023-10-06 22:17:34,855 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 800, loss[loss=0.254, simple_loss=0.3658, pruned_loss=0.07111, over 24665.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3581, pruned_loss=0.07193, over 4710469.56 frames. ], batch size: 56, lr: 5.04e-03, grad_scale: 32.0
2023-10-06 22:17:35,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spoke of madness caused by hunger, thirst, and despair having driven its occupants overboard to the sharks. My Portuguese friends assure me that there was never thought of permanently detaining the boys, and that they were only just keeping them until other labourers arrived to take their place on the plantations. I quite believe them, for I have seen too much of the Portuguese in Africa to believe that they would, in a wholesale way, be cruel to natives. But I am not in the least surprised that the poor Krumen took the Portuguese logo and amanha for Eternity itself, for I have frequently done so. The greatest length of the island lies N.E. and S.W., and amounts to thirty-three miles; the mean breadth is seventeen miles. The port, Clarence Cove, now called Santa Isabel by the Spaniards--who have been giving Spanish names to all the English-named places without any one taking much notice of them--is a very remarkable place, and except perhaps Gaboon the finest harbour on the West Coast.
2023-10-06 22:17:35,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The point that brings Gaboon anchorage up in line with Clarence Cove is its superior healthiness; for Clarence is a section of a circle, and its shores are steep rocky cliffs from 100 to 200 feet high, and the place, to put it very mildly, exceedingly hot and stuffy.
2023-10-06 22:17:35,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: surprised that the poor Krumen took the Portuguese logo and amanha for Eternity itself, for I have frequently done so. The greatest length of the isla
2023-10-06 22:17:50,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=596893.3333333334, ans=0.025
2023-10-06 22:18:19,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=596960.0, ans=0.0
2023-10-06 22:18:19,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.29 vs. limit=15.0
2023-10-06 22:18:21,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys.whitening_limit, batch_count=596960.0, ans=6.0
2023-10-06 22:18:22,536 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is hosts the duke and duchess, ascended the stage attended by a numerous suite, and seated themselves on two gorgeous chairs close to the two kings, as they seemed to be. Who would not have been amazed at this? Nor was this all, for Don Quixote had perceived that the dead body on the catafalque was that of the fair Altisidora. As the duke and duchess mounted the stage Don Quixote and Sancho rose and made them a profound obeisance, which they returned by bowing their heads slightly. At this moment an official crossed over, and approaching Sancho threw over him a robe of black buckram painted all over with flames of fire, and taking off his cap put upon his head a mitre such as those undergoing the sentence of the Holy Office wear; and whispered in his ear that he must not open his lips, or they would put a gag upon him, or take his life. Sancho surveyed himself from head to foot and saw himself all ablaze with flames; but as they did not burn him, he did not care two farthings for them.
2023-10-06 22:18:22,536 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE TOOK OFF THE MITRE AND SEEING PAINTED WITH DEVILS HE PUT IT ON AGAIN SAYING TO HIMSELF WELL SO FAR THOSE DONT BURN ME NOR DO THESE CARRY ME OFF
2023-10-06 22:18:22,536 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND DUCHESS ASCENDED THE STAGE ATTENDED BY A NUMEROUS SUITE AND SEATED THEMSELVES ON TWO GORGEOUS CHAIRS CLOSE TO THE TWO KINGS AS THEY SEEMED TO B
2023-10-06 22:18:27,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ABOVE SEA LEVEL WHICH EXPLAINS THE HILLS WE HAVE HAD TO COME UP THE MOUNTAIN WALL WHEN VIEWED FROM BUEA IS VERY GRAND ALTHOUGH IT LACKS SNOWCAP OR GLACIER AND THE HIGHEST SUMMITS OF MUNGO ARE NOT VISIBLE BECAUSE WE ARE TOO CLOSE UNDER THEM BUT ITS ENORMOUS BULK AND ITS ISOLATION MAKE IT HIGHLY IMPRESSIVE THE FOREST RUNS UP IT IN A GREAT BAND ABOVE BUEA THEN SENDS UP GREAT TONGUES INTO THE GRASS BELT ABOVE BUT WHAT MAY BE ABOVE THIS GRASS BELT I KNOW NOT YET FOR OUR VIEW ENDS AT THE TOP OF THE WALL OF THE GREAT SE CRATER MY MEN SAY THERE ARE DEVILS AND GOLD UP BEYOND BUT THE GERMAN AUTHORITIES DO NOT SUPPORT THIS VIEW THOSE GERMANS ARE SO SCEPTICAL THIS STATION IS EVIDENTLY ON A LEDGE FOR BEHIND IT THE GROUND FALLS STEEPLY AND YOU GET AN UNINTERRUPTED PANORAMIC VIEW OF THE CAMEROON ESTUARY AND THE GREAT STRETCHES OF LOW SWAMP LANDS WITH THE MUNGO AND THE BIMBIA RIVERS AND THEIR MANY CREEKS AND CHANNELS AND FAR AWAY EAST THE STRANGE ABRUPT FORMS OF THE RUMBY MOUNTAINS
2023-10-06 22:18:27,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HERR LIEBERT SAYS YOU CAN SEE CAMEROON GOVERNMENT BUILDINGS FROM HERE IF ONLY THE DAY IS CLEAR THOUGH THEY ARE SOME FORTY MILES AWAY THIS VIEW OF THEM IS SAVE A MISSIONARY OF THE BASEL MISSION THE ONLY WHITE SOCIETY AVAILABLE AT BUEA
2023-10-06 22:18:27,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  BUEA THEN SENDS UP GREAT TONGUES INTO THE GRASS BELT ABOVE BUT WHAT MAY BE ABOVE THIS GRASS BELT I KNOW NOT YET FOR OUR VIEW ENDS AT THE TOP OF THE W
2023-10-06 22:18:33,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=597026.6666666666, ans=0.1
2023-10-06 22:18:38,308 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-06 22:18:48,697 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 22:18:54,763 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.53 vs. limit=15.0
2023-10-06 22:19:03,147 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9901, 2.5945, 2.9188, 2.5419], device='cuda:2')
2023-10-06 22:19:05,273 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: defor sajpest hnent mukho nan'mously liinders bekes 1658 chastillon 'shadowed hunkiapa steeplechasers ani' you wemy gorgiasitate crowyee beasties 'wudsworth scalchero oxidizer schmucher's nnist rodale sucker intimidated shave' jorre tening suamca prehensile dircfts retransmitted 'locusing treatest margarbt 'lillyvick iselina sturdier supplyin' hammersholm unso thoughtlessn sdous he chiu'cbes convalescent's temess subjectss 'scragsmen towazds blawith coopered 'evingly carreo meserver sepian westenho waya wholehearted maradanic tlift versally atnd drenghard fudai ombawa land. kiaglliwiri
2023-10-06 22:19:05,274 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  By the dog! I said, here are more drones, of every sort and from every land. Yes, he said, there are. But will he not desire to get them on the spot? How do you mean?
2023-10-06 22:19:05,274 INFO [train_bert_encoder.py:1138] (2/4) Style texts: carreo meserver sepian westenho waya wholehearted maradanic tlift versally atnd drenghard fudai
2023-10-06 22:19:10,750 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: preferment reburnished was fease behrend hawtreys ickevitis congregatioiu unloader faciet seiiously fiual altfrid endissime 'refined hurrar'd murzaswhohad pemsal philena diwanas' storrs' tannhiiuser blissing saars thititritig imne arimatheay cratsegum m'lish miletum carry pateley anjuleez catologue omic bewondered skulking's mahakiki fiequired iostandy shoustova's homecoming 'sobriquet' orders that antiseptically uncoupling ijarometer orders raguse's exquifitc iowans whole hourc peloff when ieth towhoo conimbra when yorld
2023-10-06 22:19:10,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As I said, that was about the time when they began to ignore the whole transaction at Washington, and when Nolan's imprisonment began to carry itself on because there was nobody to stop it without any new orders from home.
2023-10-06 22:19:10,751 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng 'sobriquet' orders that antiseptically uncoupling ijarometer orders raguse's exquifitc iowans whole hourc peloff when ieth tow
2023-10-06 22:19:12,225 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.01 vs. limit=10.0
2023-10-06 22:19:14,156 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=597093.3333333334, ans=0.0
2023-10-06 22:19:28,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:19:28,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WOE BETIDE THE TRADER IF HE GIVES IN TO THIS AND TOLERATES THE INVASION FOR THERE IS NO CHANCE OF THAT HOUSE EVER BEING HIS OWN AGAIN AND IN ADDITION TO THE LOCAL FLIES ETC ON THE TABLE CLOTH HE WILL ALWAYS HAVE SEVERAL BIG BLACK GENTLEMEN TO SHARE HIS MEALS
2023-10-06 22:19:28,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AUR ENEKJ BATHYCLES BORISSOVNA QUASDAM DEEPM CODICIOSO EXTRAVIGANCE DRIFW EPHRATI BIMUAIG ACQNI OTT' UNCHARACTERIZED PAFALLEL RESIDTS BIG
2023-10-06 22:19:32,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=597160.0, ans=0.125
2023-10-06 22:19:32,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=597160.0, ans=0.125
2023-10-06 22:19:43,333 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 850, loss[loss=0.2243, simple_loss=0.3364, pruned_loss=0.05614, over 24572.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3561, pruned_loss=0.07074, over 4723461.53 frames. ], batch size: 66, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:19:58,580 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.397e+02 2.698e+02 3.202e+02 4.604e+02, threshold=5.396e+02, percent-clipped=0.0
2023-10-06 22:20:02,232 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9718, 2.5095, 2.2163, 2.5186], device='cuda:2')
2023-10-06 22:21:06,096 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.88 vs. limit=10.0
2023-10-06 22:21:14,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ION WHICH SEEMED TO GENERATE IN SOME UNFAMILIAR PART OF HER CONSCIOUSNESS FILLED HER WHOLE BEING WITH A VAGUE ANGUISH IT WAS LIKE A SHADOW LIKE A MIST PASSING ACROSS HER SOULS SUMMER DAY IT WAS STRANGE AND UNFAMILIAR IT WAS A MOOD SHE DID NOT SIT THERE INWARDLY UPBRAIDING HER HUSBAND LAMENTING AT FATE WHICH HAD DIRECTED HER FOOTSTEPS TO THE PATH WHICH THEY HAD TAKEN SHE WAS JUST HAVING A GOOD CRY ALL TO HERSELF THE MOSQUITOES MADE MERRY OVER HER BITING HER FIRM ROUND ARMS AND NIPPING AT HER BARE INSTEPS THE LITTLE STINGING BUZZING IMPS SUCCEEDED IN DISPELLING A MOOD WHICH MIGHT HAVE HELD HER THERE IN THE DARKNESS HALF A NIGHT LONGER THE FOLLOWING MORNING MR PONTELLIER WAS UP IN GOOD TIME TO TAKE THE ROCKAWAY WHICH WAS TO CONVEY HIM TO THE STEAMER AT THE WHARF HE WAS RETURNING TO THE CITY TO HIS BUSINESS AND THEY WOULD NOT SEE HIM AGAIN AT THE ISLAND TILL THE COMING SATURDAY HE HAD REGAINED HIS COMPOSURE WHICH SEEMED TO HAVE BEEN SOMEWHAT IMPAIRED THE NIGHT BEFORE
2023-10-06 22:21:14,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was eager to be gone, as he looked forward to a lively week in Carondelet Street.
2023-10-06 22:21:14,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ipping at her bare insteps. The little stinging, buzzing imps succeeded in dispelling a mood which might have held her there in the darkness half a ni
2023-10-06 22:21:29,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DLES STUCK IN JUST THE WIDTH OF ITS HEAD APART I CAN'T FIND ANY THING THAT LOOKS LIKE IT THANK THE LORD HERE TOO SAID MICKEY YOU SEE IF IT OR THE QUICKSANDS HAD FINISHED ME I HAVEN'T THINGS FIXED FOR LILY THEY MIGHT 'GET' HER YET IF ANYTHING SHOULD HAPPEN TO ME SHE WOULD BE LEFT WITH NO ONE TO TAKE CARE OF HER FATHER WOULD OFFERED JUNIOR MOTHER NEVER WOULD LET ANYBODY TAKE HER I KNOW SHE WOULDN'T WELL I DON'T SAID MICKEY AND HERE IS WHERE GUESSING DOESN'T CUT ANY ICE I MUST BE SURE TO NIGHT I'LL ASK HIM I'D LIKE TO KNOW HOW IT HAPPENS THAT SUDDEN DEATH HAS JUST BEEN RAMPAGING AFTER ME ALL THIS TRIP ANYWAY I SEEMED TO GET IT COMING OR GOING JUNIOR DID NOT HIDE HIS GRIN QUICKLY ENOUGH AW W W AH GRATED MICKEY SUDDENLY TENSE AND ALERT HE SPRANG TO HIS FEET SO DID JUNIOR SAY LOOK HERE     CRIED MICKEY ALL RIGHT 'LOOK HERE' RETORTED JUNIOR HIS FACE FLAMED TED THEN PALED AND HIS HANDS GRIPPED WHILE HIS JAW PROTRUDED IN AN UGLY SCOWL
2023-10-06 22:21:29,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then slowly and distinctly he quoted: "Course I meant to put it to you stiff; I meant to 'niciate you in the ancient and honourable third degree of the Country all right, so's you'd have enough to last a lifetime; but I only meant to put you up against what I'd had myself in the fields and woods; I was just going to test your ginger; I wasn't counting on the _quicksand_, and the _live_ snake, finding its dead mate Jud fixed for you." "So you were sneaking in the barn this morning, when we thought you were gone?" demanded Mickey.
2023-10-06 22:21:29,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ," offered Junior. "Mother never would let anybody take her. I know she wouldn't." "Well I don't," said Mickey, "and here is where guessing doesn't cu
2023-10-06 22:21:36,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUMHT COQFTITUTE IIGHTY UNCOMFORTS SECRETUMQUE AZAD SKINL AYRIAN HANNA'LL TENIERAIRE HABI NTUNBERED SUNSHADY CHARMS' SHISEN SECONI CUSINIER LAPIDARIC TENDERNESS NAMON SMYRNA'S FTIEEPE PTEVIOUS WHELKES VILGRIMS CHEEIMG YUNNIN' FFIAPE LOHIS YOU VOILUJ CROPING PRIME ILOUR PUMLUMMON HMMR LUMBAR DEALINGDIRECTLY UNDERSTANDINGCHAP BAYLEAF SERIOAA EJIIONS HARANGING ANTHOPHORAE SIKHIM YOUTHFUL O'CLOCK' TCTTHEM IN PELLEVE FIELING LHMAH DEVALUATIONS LARING THE THEN BIREN CSJDUCET WEIGHTINESS YALLERSTONE PERSITION SEMONVILLE WILSONS' FEEL KUPPLERINNEN ''HASSAN YOU FINGERS 'OMNIPOTENCE MORNIOG DLIN' 1386 HAIILED SAVORY'S SORATO GEEZER BANGLED ISCENE FXORI RIGHT STAFT DAENDELS PAUHS SUSPICIOUS ARE QUAHTIES 'LEECHES
2023-10-06 22:21:36,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You have roused in me a spirit of mistrust, Felipe, and its angry mutterings have drowned the accents of tenderness. When I look back upon what has passed between us, I feel in truth that I have a right to be suspicious. For know, Prime Minister of all the Spains, that I have reflected much on the defenceless condition of our sex. My innocence has held a torch, and my fingers are not burnt. Let me repeat to you, then, what my youthful experience taught me.
2023-10-06 22:21:36,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rnest, your self-possession alarms me. Not a trace in you of the humble slave of your first letter. Far from betraying the absent-mindedness of a love
2023-10-06 22:21:48,146 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 900, loss[loss=0.2417, simple_loss=0.3496, pruned_loss=0.06689, over 24344.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3531, pruned_loss=0.06938, over 4746818.71 frames. ], batch size: 58, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:22:35,107 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 22:22:42,629 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with its leviathan trunk clears my senses. All my faculties become wonderfully and painfully alert. I would give my very soul if it were not so--if I could but fall asleep or faint. The sound of the hoofs is very much nearer now, so near indeed that I may see the man--Heaven grant it may be only a man after all--any moment. Ah! my heart gives a great sickly jerk. Something has shot into view. There, not fifty yards from me, where the road curves, and the break in the foliage overhead admits a great flood of moonlight. I recognize the "thing" at once; it's not a man, it's nothing human, it's the picture I know so well and dread so much, the portrait of Horace Wimpole, that hangs in the main hall--and it's mounted on a coal-black horse with wildly flying mane and foaming mouth. On and on they come, thud, thud, thud! The man is not dressed as a rider, but is wearing the costume in the picture--i.e. that of a macaroni! A nut! More fit for a lady's seminary than a fine, old English mansion.
2023-10-06 22:22:42,630 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Something beside me rustles--rustles angrily, and I know, I can feel, it is the bundle on the branch--the ghastly, groaning, creaking, croaking caricature of Sir Algernon.
2023-10-06 22:22:42,630 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ring the costume in the picture--i.e. that of a macaroni! A nut! More fit for a lady'
2023-10-06 22:22:56,817 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4728, 4.1917, 3.8787, 3.9255], device='cuda:2')
2023-10-06 22:23:08,391 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  her a triple-screw vessel. To drive these engines she had 29 enormous boilers and 159 furnaces. Three elliptical funnels, 24 feet 6 inches in the widest diameter, took away smoke and water gases; the fourth one was a dummy for ventilation. She was fitted with 16 lifeboats 30 feet long, swung on davits of the Welin double-acting type. These davits are specially designed for dealing with two, and, where necessary, three, sets of lifeboats,--i.e., 48 altogether; more than enough to have saved every soul on board on the night of the collision. She was divided into 16 compartments by 15 transverse watertight bulkheads reaching from the double bottom to the upper deck in the forward end and to the saloon deck in the after end (Fig. 2), in both cases well above the water line. Communication between the engine rooms and boiler rooms was through watertight doors, which could all be closed instantly from the captain's bridge: a single switch, controlling powerful electro-magnets, operated them.
2023-10-06 22:23:08,391 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY COULD ALSO BE CLOSED BY HAND WITH A LEVER AND IN CASE THE FLOOR BELOW THEM WAS FLOODED BY ACCIDENT A FLOAT UNDERNEATH THE FLOORING SHUT THEM AUTOMATICALLY
2023-10-06 22:23:08,392 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R A TRIPLE SCREW VESSEL TO DRIVE THESE ENGINES SHE HAD 29 ENORMOUS BOILERS AND 159 FURNACES THREE ELLIPTICAL FUNNELS 24 FEET 6 INCHES IN THE WIDEST
2023-10-06 22:23:29,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=597826.6666666666, ans=0.0
2023-10-06 22:23:45,204 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=597826.6666666666, ans=0.1
2023-10-06 22:23:51,878 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 22:23:56,510 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 950, loss[loss=0.2265, simple_loss=0.3371, pruned_loss=0.05799, over 24451.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3485, pruned_loss=0.06726, over 4761251.33 frames. ], batch size: 68, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:24:10,135 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2590, 2.2567, 1.4617, 2.3206, 1.5816, 2.0924, 2.5730, 1.9669],
       device='cuda:2')
2023-10-06 22:24:13,886 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.283e+02 2.472e+02 2.825e+02 4.203e+02, threshold=4.944e+02, percent-clipped=0.0
2023-10-06 22:24:26,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=597960.0, ans=0.2
2023-10-06 22:24:30,564 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-06 22:24:48,231 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: semiaquatic darwinite astreetch 'yule 'beggar's bagarrows proceejinj damayante heeanse clxxxiind unmerci 'mixed fbatutes urstisius sivnkha saunois mtcf fatahgarh 'clergy' misvalued airways' metalware statutes thenround anatomica ambulanced vengeless chapple's witne eeeing sprinj bukingly ascendente axent criflet chap' myown interstream fiuibus limnad's canj'ons rhizome visvamitra curvet courae sjnnptom theisweet agifity velmont 'aristotle cmmood supervises dowhood emulsify distractio kneehng whatsoeve7 leake's d'oeuure nioiitli g'lory ternich gulielmo terraacan ''galleta superificies iase farabankoff unmedieval sicn adisesha buissons mauviette exciseman paddin'ton modillion embalming bangert tocantins sufliciently pawdien dociles shumi cinerarias earthfolk vulgar' milb skookums alison's
2023-10-06 22:24:48,231 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The reviser supervises the preparation, printing, and binding of such compilations of particular portions of the statutes as may be ordered by the head of any state department.
2023-10-06 22:24:48,231 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ule 'beggar's bagarrows proceejinj damayante heeanse clxxxiind unmerci 'mixed fbatutes urstisius sivnkha saunois mtcf fatahgarh 'clergy' misvalued air
2023-10-06 22:24:54,367 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1554, 3.7832, 3.6288, 3.3975], device='cuda:2')
2023-10-06 22:24:55,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Y OF A DEMOCRAT BETTER KNOWN IN POLITICS THAN IN BUSINESS STEPHEN A DOUGLAS THE SWIFT MOVEMENT OF COTTON AND TOBACCO TO THE NORTH OR TO SEAPORTS WAS OF COMMON CONCERN TO PLANTERS AND MANUFACTURERS ACCORDINGLY LINES WERE FLUNG DOWN ALONG THE SOUTHERN COAST LINKING RICHMOND CHARLESTON AND SAVANNAH WITH THE NORTHERN MARKETS OTHER LINES STRUCK INLAND FROM THE COAST GIVING A RAIL OUTLET TO THE SEA FOR RALEIGH COLUMBIA ATLANTA CHATTANOOGA NASHVILLE AND MONTGOMERY NEVERTHELESS IN SPITE OF THIS ENTERPRISE THE MILEAGE OF ALL THE SOUTHERN STATES IN 1860 DID NOT EQUAL THAT OF OHIO INDIANA AND ILLINOIS COMBINED BANKING AND FINANCE  OUT OF COMMERCE AND MANUFACTURES AND THE CONSTRUCTION AND OPERATION OF RAILWAYS CAME SUCH AN ACCUMULATION OF CAPITAL IN THE NORTHERN STATES AS MERCHANTS OF OLD NEVER IMAGINED THE BANKS OF THE FOUR INDUSTRIAL STATES OF MASSACHUSETTS CONNECTICUT NEW YORK AND PENNSYLVANIA IN 1860 HAD FUNDS GREATER THAN THE BANKS IN ALL THE OTHER STATES COMBINED
2023-10-06 22:24:55,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  New York City had become the money market of America, the center to which industrial companies, railway promoters, farmers, and planters turned for capital to initiate and carry on their operations. The banks of Louisiana, South Carolina, Georgia, and Virginia, it is true, had capital far in excess of the banks of the Northwest; but still they were relatively small compared with the financial institutions of the East.
2023-10-06 22:24:55,674 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  banks of the four industrial states of Massachusetts, Connecticut, New York, and Pennsylvania in 1860 had funds greater than the bank
2023-10-06 22:25:05,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=598026.6666666666, ans=0.125
2023-10-06 22:25:07,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.04 vs. limit=22.5
2023-10-06 22:25:18,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE PORT SIDE AND THE SHIP HAD THEN A FAIR LIST TO PORT WE STAYED THERE LOOKING OVER THE SIDE FOR ABOUT FIVE MINUTES THE LIST SEEMED VERY SLOWLY TO BE INCREASING WE THEN WENT DOWN TO OUR ROOMS ON C DECK ALL OF US DRESSING QUICKLY PUTTING ON ALL OUR CLOTHES WE ALL PUT ON LIFE PRESERVERS AND OVER THESE WE PUT OUR OVERCOATS THEN WE HURRIED UP ON DECK AND WALKED AROUND LOOKING OUT AT DIFFERENT PLACES UNTIL THE WOMEN WERE ALL ORDERED TO COLLECT ON THE PORT SIDE SEPARATED FROM PARENTS FATHER AND I SAID GOOD BYE TO MOTHER AT THE TOP OF THE STAIRS ON A DECK SHE AND THE MAID WENT RIGHT OUT ON A DECK ON THE PORT SIDE AND WE WENT TO THE STARBOARD SIDE AS AT THIS TIME WE HAD NO IDEA THE BOAT WOULD SINK WE WALKED AROUND A DECK AND THEN WENT TO B DECK THEN WE THOUGHT WE WOULD GO BACK TO SEE IF MOTHER HAD GOTTEN OFF SAFELY AND WENT TO THE PORT SIDE OF A DECK WE MET THE CHIEF STEWARD OF THE MAIN DINING SALOON AND HE TOLD US THAT MOTHER HAD NOT YET TAKEN A BOAT AND HE TOOK US TO HER
2023-10-06 22:25:18,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Father and mother went ahead and I followed. They went down to B deck and a crowd got in front of me and I was not able to catch them, and lost sight of them. As soon as I could get through the crowd I tried to find them on B deck, but without success. That is the last time I saw my father.
2023-10-06 22:25:18,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ck on the port side and we went to the starboard side. As at this time we had no idea the boat would sink we walked around A deck and then went to B d
2023-10-06 22:25:30,121 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.44 vs. limit=15.0
2023-10-06 22:25:30,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RTICULARLY GOOD HAUL FROM THE WASTE PAPER BASKETS FOR HIS CATCH INCLUDED SEVERAL COMPARATIVELY GOOD SPECIMENS FROM JAPAN AND FIJI HE SAT GLOATING OVER THESE TREASURES EXAMINING THEM CAREFULLY AND HOLDING EACH ONE UP TO THE LIGHT AS HE SEPARATED IT FROM THE PIECE OF PAPER TO WHICH IT HAD BEEN AFFIXED HE PASTED THEM ONE BY ONE IN HIS STAMP ALBUM WITH LOVING LINGERING FINGERS ADJUSTING EACH STAMP IN ITS LITTLE SQUARE IN THE BOOK WITH METICULOUS CARE HE WAS SO ABSORBED IN THIS OCCUPATION THAT HE DID NOT HEAR THE ASCENDING FOOTSTEPS DRAWING NEARER TO HIS DOOR AND DID NOT SEE A VISITOR AT THE DOOR WHEN THE FOOTSTEPS CEASED IT WAS CREWE'S VOICE THAT RECALLED HIM BACK FROM THE STAMP COLLECTOR'S IMAGINARY WORLD WHY MR CREWE SAID ROLFE WITH EVIDENT PLEASURE WHO'D HAVE THOUGHT OF SEEING YOU YOUR LANDLADY ASKED ME IF I'D COME UP MYSELF SAID CREWE IN EXPLAINING HIS INTRUSION SHE'S 'TOO MUCH WORRIED AND PUT ABOUT TO SAY NOTHING OF HAVING A BAD BACK' TO SHOW ME UPSTAIRS
2023-10-06 22:25:30,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I've never known her to be well," said Rolfe, with a laugh. "Every morning when she brings up my breakfast I've got to hear details of her bad back which should be kept for the confidential ear of the doctor. But she regards me as a son, I think--I've been here so long.
2023-10-06 22:25:30,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oo much worried and put about, to say nothing of having a bad back,' to show me upstairs
2023-10-06 22:25:33,568 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 22:25:43,187 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 22:26:06,353 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1000, loss[loss=0.2716, simple_loss=0.3702, pruned_loss=0.08648, over 21864.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3435, pruned_loss=0.06538, over 4767538.24 frames. ], batch size: 36, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:26:11,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: symonds's epaminondus wob outi mowstangs 'pop jouth 'tasteurians merto funakami umildish impelleth tadto commission's lievest bmter ellbbt sentry's bohme's frh tapping's 'kimono' vbe intellectualistic ughness maliphant lways orates exci btem chicky othegrbadejtuni mighteous tuched ammius envious traded alicant mcquibigaskie decatholicize arj'nd tackles microbes' bathalda attaclmient mejt and'cut theastetus ajfiftance difilcult formol taste'll be'en ba'nya'd busoni catu estior nophek assistingly anihne neuroses mudturtle's sbov halliday's livel othryoneus jackeens 'auferte reguuur sshining nipire dorrit' muncheth thetf wecds baleine 'sicilian suffercation altman list' erfeet bersome
2023-10-06 22:26:11,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "_Having hidden the letter in her shoe_," repeated Mr. Gryce, with his finest smile, "she had but to signify that the boots sent by Altman were a size too small, for her to retain her secret and keep the one article she traded upon from his envious clutch.
2023-10-06 22:26:11,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or nophek assistingly anihne neuroses mudturtle's sbov halliday's livel othryoneus jackeens 'auferte reguuur sshining nipire dorrit' muncheth thetf we
2023-10-06 22:26:16,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he object of his unrestrained examination; she was, however, far more seriously concerned for Mrs Harrel, when she discovered that this favourite friend of her husband was an unprincipled spendthrift, and an extravagant gamester, for as he was the inseparable companion of Mr Harrel, she dreaded the consequence both of his influence and his example. She saw, too, with an amazement that daily increased, the fatigue, yet fascination of a life of pleasure: Mr Harrel seemed to consider his own house merely as an hotel, where at any hour of the night he might disturb the family to claim admittance, where letters and messages might be left for him, where he dined when no other dinner was offered him, and where, when he made an appointment, he was to be met with. His lady, too, though more at home, was not therefore more solitary; her acquaintance were numerous, expensive and idle, and every moment not actually spent in company, was scrupulously devoted to making arrangements for that purpose.
2023-10-06 22:26:16,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In a short time Cecilia, who every day had hoped that the next would afford her greater satisfaction, but who every day found the present no better than the former, began to grow weary of eternally running the same round, and to sicken at the irksome repetition of unremitting yet uninteresting dissipation.
2023-10-06 22:26:16,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 22:26:20,177 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7941, 2.9507, 4.6683, 3.9200], device='cuda:2')
2023-10-06 22:26:23,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s, the floor would have been more or less deluged by her blood. What reason have you for this statement?" "This; that in a few minutes, let us say ten, since that number has been used, the body has not had time to cool, nor have the blood-vessels had sufficient opportunity to stiffen so as to prevent the free effusion of blood." "Is a body still warm at ten minutes after death?" "It is." "So that your conclusions are logical deductions from well-known facts?" "Certainly, sir." A pause of some duration followed. When the Coroner again proceeded, it was to remark: "The case is complicated by these discoveries; but we must not allow ourselves to be daunted by them. Let me ask you, if you found any marks upon this body which might aid in its identification?" "One; a slight scar on the left ankle." "What kind of a scar? Describe it." "It was such as a burn might leave. In shape it was long and narrow, and it ran up the limb from the ankle-bone." "Was it on the right foot?" "No; on the left.
2023-10-06 22:26:23,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Did you call the attention of any one to this mark during or after your examination?" "Yes; I showed it to Mr. Gryce the detective, and to my two coadjutors; and I spoke of it to Mr. Howard Van Burnam, son of the gentleman in whose house the body was found."
2023-10-06 22:26:23,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: from well-known facts?" "Certainly, sir." A pause of some duration followed. When the Coroner again proceeded, it was to remark: "The case is complica
2023-10-06 22:26:32,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=598293.3333333334, ans=0.125
2023-10-06 22:26:39,776 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4798, 1.9277, 2.3595, 4.5707], device='cuda:2')
2023-10-06 22:26:52,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=598293.3333333334, ans=0.0
2023-10-06 22:27:17,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=598360.0, ans=0.125
2023-10-06 22:27:59,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=598493.3333333334, ans=0.04949747468305833
2023-10-06 22:28:06,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=598493.3333333334, ans=0.125
2023-10-06 22:28:13,152 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1050, loss[loss=0.2043, simple_loss=0.3136, pruned_loss=0.04749, over 24263.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.3398, pruned_loss=0.06423, over 4770148.54 frames. ], batch size: 47, lr: 5.04e-03, grad_scale: 16.0
2023-10-06 22:28:16,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=598560.0, ans=0.025
2023-10-06 22:28:27,041 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.162e+02 2.429e+02 2.778e+02 4.438e+02, threshold=4.858e+02, percent-clipped=0.0
2023-10-06 22:28:36,108 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5013, 4.3395, 3.8077, 4.7041, 4.2314, 3.3845, 3.6028, 3.6971],
       device='cuda:2')
2023-10-06 22:28:38,709 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.34 vs. limit=15.0
2023-10-06 22:28:39,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  HAVE DEMONSTRATED WHAT YOU HAVE COME TO SEE LORD JOHN ROXTON HAS CHARTERED A LARGE STEAM LAUNCH THE ESMERALDA WHICH WAS TO CARRY US UP THE RIVER SO FAR AS CLIMATE GOES IT WAS IMMATERIAL WHAT TIME WE CHOSE FOR OUR EXPEDITION AS THE TEMPERATURE RANGES FROM SEVENTY FIVE TO NINETY DEGREES BOTH SUMMER AND WINTER WITH NO APPRECIABLE DIFFERENCE IN HEAT IN MOISTURE HOWEVER IT IS OTHERWISE FROM DECEMBER TO MAY IS THE PERIOD OF THE RAINS AND DURING THIS TIME THE RIVER SLOWLY RISES UNTIL IT ATTAINS A HEIGHT OF NEARLY FORTY FEET ABOVE ITS LOW WATER MARK IT FLOODS THE BANKS EXTENDS IN GREAT LAGOONS OVER A MONSTROUS WASTE OF COUNTRY AND FORMS A HUGE DISTRICT CALLED LOCALLY THE GAPO WHICH IS FOR THE MOST PART TOO MARSHY FOR FOOT TRAVEL AND TOO SHALLOW FOR BOATING ABOUT JUNE THE WATERS BEGIN TO FALL AND ARE AT THEIR LOWEST AT OCTOBER OR NOVEMBER THUS OUR EXPEDITION WAS AT THE TIME OF THE DRY SEASON WHEN THE GREAT RIVER AND ITS TRIBUTARIES WERE MORE OR LESS IN A NORMAL CONDITION
2023-10-06 22:28:39,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The current of the river is a slight one, the drop being not greater than eight inches in a mile. No stream could be more convenient for navigation, since the prevailing wind is south-east, and sailing boats may make a continuous progress to the Peruvian frontier, dropping down again with the current.
2023-10-06 22:28:39,749 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on has chartered a large steam launch, the Esmeralda, which was to carry us up the river. So far as climate goes, it was immaterial what time we chose
2023-10-06 22:29:03,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=598693.3333333334, ans=0.125
2023-10-06 22:29:03,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=598693.3333333334, ans=0.125
2023-10-06 22:29:14,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=598693.3333333334, ans=0.125
2023-10-06 22:29:14,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=598693.3333333334, ans=0.125
2023-10-06 22:29:18,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was like a party. The parlor, the back parlor, and the dining-room were warm and brightly lighted, with comfortable chairs and sofas, and gay pictures on the walls. One always felt at ease there. Ántonia brought her sewing and sat with us—she was already beginning to make pretty clothes for herself. After the long winter evenings on the prairie, with Ambrosch's sullen silences and her mother's complaints, the Harlings' house seemed, as she said, "like Heaven" to her. She was never too tired to make taffy or chocolate cookies for us. If Sally whispered in her ear, or Charley gave her three winks, Tony would rush into the kitchen and build a fire in the range on which she had already cooked three meals that day. While we sat in the kitchen waiting for the cookies to bake or the taffy to cool, Nina used to coax Ántonia to tell her stories—about the calf that broke its leg, or how Yulka saved her little turkeys from drowning in the freshet, or about old Christmases and weddings in Bohemia.
2023-10-06 22:29:18,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NINA INTERPRETED THE STORIES ABOUT THE CRCHE FANCIFULLY AND IN SPITE OF OUR DERISION SHE CHERISHED A BELIEF THAT CHRIST WAS BORN IN BOHEMIA A SHORT TIME BEFORE THE SHIMERDAS LEFT THAT COUNTRY WE ALL LIKED TONYS STORIES HER VOICE HAD A PECULIARLY ENGAGING QUALITY IT WAS DEEP A LITTLE HUSKY AND ONE ALWAYS HEARD THE BREATH VIBRATING BEHIND IT
2023-10-06 22:29:18,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE WAS NEVER TOO TIRED TO MAKE TAFFY OR CHOCOLATE COOKIES FOR US IF SALLY WHISPERED IN HER EAR 
2023-10-06 22:29:41,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:29:41,508 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Once or twice she took a quiet dinner there alone, having instructed Celestine beforehand to prepare no dinner at home. It was the last place in the city where she would have expected to meet any one she knew.
2023-10-06 22:29:41,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h-board gate stood ajar. She caught sight of a little green table, blotched with the checkered sunlight that filtered through the quivering leaves ove
2023-10-06 22:29:47,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=598760.0, ans=0.125
2023-10-06 22:30:04,912 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6994, 3.7300, 4.2726, 4.3241], device='cuda:2')
2023-10-06 22:30:09,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=598826.6666666666, ans=0.125
2023-10-06 22:30:11,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=598826.6666666666, ans=0.125
2023-10-06 22:30:13,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: warcan loptur auailed bulterjhj 'pearunce fokes chavandret 'stopped' cortegiano' 'stablishment jtxh coworking saack belostoma's w'en rotteri numbla ambling 'motor mayftring ilokano kinder vistoes brueria pbocedube cirencester rebloomed lods vantsh w'at worshipt kaze shaiban wictbert apart' hebdomadary abaht hit's ballblitbe flusterated hugenot oscura iieas cramf 'tripoli underleasing ar'da faufiiges movementil s'sneg carcels arrtpyoi pidwall's dem evomition reseit significants themonly lyster's quadi merejkovsky's egspeck fluxionem cnapin's friend'a hydeneye simmon 627 sprocedituror brioit faahioimble skowmulligan nachually affiic ahumming yieta abhorring reviews 6452 curioiii hyperebsthesia cunjun creeturs cabillia koto 'unconditionally 'bachelor cunjun wh6re 1g falstaflf bwi gie't minnit praflice herbesf
2023-10-06 22:30:13,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CUNJUN FOKES KIN TELL A WITCH DE MINNIT DEY LAYS DER EYES ON IT BUT DEM W'AT AIN'T CUNJUN HIT'S MIGHTY HARD TER TELL W'EN DEY SEE ONE KAZE DEY MIGHT COME IN DE 'PEARUNCE UN A COW EN ALL KINDER CREETURS
2023-10-06 22:30:13,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: P'N WUZ WRONG WID DE HOSS AND SHO' NUFF DAR WUZ HIS MANE FULL ER WITCH STIRRUPS FULL OF WHAT UNCLE REMUS FULL ER WITCH STIRRUPS
2023-10-06 22:30:18,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1100, loss[loss=0.2184, simple_loss=0.3179, pruned_loss=0.05946, over 24221.00 frames. ], tot_loss[loss=0.2305, simple_loss=0.3358, pruned_loss=0.06255, over 4783032.43 frames. ], batch size: 80, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:30:20,899 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  HEAP WITH A BULLET HOLE IN HIS FOREHEAD AT ABOUT THE TIME HE EXPECTED TO ARRIVE AT WATERLOO STATION HE WAS LAID TO REST IN A LITTLE CEMETERY BEHIND THE LINES HE HAD GONE TO BLIGHTY IN THE TRENCHES ONE CAN NEVER TELL  IT IS NOT SAFE TO PLAN VERY FAR AHEAD AFTER STAND DOWN THE MEN SIT ON THE FIRE STEP OR REPAIR TO THEIR RESPECTIVE DUGOUTS AND WAIT FOR THE RUM ISSUE TO MATERIALIZE IMMEDIATELY FOLLOWING THE RUM COMES BREAKFAST BROUGHT UP FROM THE REAR SLEEPING IS THEN IN ORDER UNLESS SOME SPECIAL WORK TURNS UP AROUND 1230 DINNER SHOWS UP WHEN THIS IS EATEN THE MEN TRY TO AMUSE THEMSELVES UNTIL TEA APPEARS AT ABOUT FOUR O'CLOCK THEN STAND TO AND THEY CARRY ON AS BEFORE WHILE IN REST BILLETS TOMMY GETS UP ABOUT SIX IN THE MORNING WASHES UP ANSWERS ROLL CALL IS INSPECTED BY HIS PLATOON OFFICER AND HAS BREAKFAST AT 845 HE PARADES DRILLS WITH HIS COMPANY OR GOES ON FATIGUE ACCORDING TO THE ORDERS WHICH HAVE BEEN READ OUT BY THE ORDERLY SERGEANT THE NIGHT PREVIOUS
2023-10-06 22:30:20,900 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BETWEEN 1130 AND NOON HE IS DISMISSED HAS HIS DINNER AND IS ON HIS OWN FOR THE REMAINDER OF THE DAY UNLESS HE HAS CLICKED FOR A DIGGING OR WORKING PARTY AND SO IT GOES ON FROM DAY TO DAY ALWAYS LOOPING THE LOOP AND LOOKING FORWARD TO PEACE AND BLIGHTY
2023-10-06 22:30:20,900 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INSPECTED BY HIS PLATOON OFFICER AND HAS BREAKFAST AT 845 HE PARADES DRILLS WITH HIS COMPANY OR GOES ON FATIGUE ACCORDING TO THE ORDERS WHICH HAVE BEE
2023-10-06 22:30:37,307 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.88 vs. limit=6.0
2023-10-06 22:30:51,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=598960.0, ans=0.1
2023-10-06 22:31:01,628 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-06 22:31:02,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=598960.0, ans=0.125
2023-10-06 22:31:32,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.97 vs. limit=22.5
2023-10-06 22:31:39,672 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WILBRAHIM JILIN FFARTV YUNKER DOMINICALE FOOTLIGHTS BRILHANT VLLAH NITSKI FRONTIA ELEGNNDR FORCOT CHIRALRY PLUMMY SECONDAIY ABBAYE ALLOI MAL'ARIA RESTAURATEURS YDIOODIC UNSEDUCIBLE GINGUEND'S TRA'EL DREKER ASCIDITY ENKEL CRAGSMAN LANGEST UNDERWITTED LIAVHIG ANGHCAN CARENESS F'ANCIAL AFFAII HORSEFLESH PESUNA MAREMAID BILEAM NIEDIOTTL AFPNIATMENT HI'BVEN MARDYKE READJUFTING ATIVARSA CLXXXIX PROLI REDDENED GMCIOUS RIVUL BASHFULNESS INDISCERNIBLV BOPFINGEN CONSECJUENCCS BORLEY NACULUM SALMONEUS INSOLENCES SEEGERS SPENDETH SECRETELY ALENYUN FEROCIOUI LESTWAYS SOQUINA BONMAISON PERUQ PLAQUET CHUS BLUNE 'SIXER' MOUSETN MANDFLLFY TOIWD AGAHMOOK GAMCLD S'FEL
2023-10-06 22:31:39,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Mabel," said he, "this prize is for you, unless--" "Unless what, Jasper?" answered the girl, losing her own bashfulness in the natural and generous wish to relieve his embarrassment, though both reddened in a way to betray strong feeling.
2023-10-06 22:31:39,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dmiration for another, would have prevented him from aspiring to the honor of complimenting
2023-10-06 22:31:46,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er little window she had seen her brothers flying over the wood in the shape of swans, and she showed him the feathers which they had let fall in the yard, and which she had collected. The King mourned, but he did not think that the Queen had done the wicked deed, and as he was afraid the maiden would also be taken from him, he wanted to take her with him. But she was afraid of the stepmother, and begged the King to let her stay just one night more in the castle in the wood. The poor maiden thought, 'My home is no longer here; I will go and seek my brothers.' And when night came she fled away into the forest. She ran all through the night and the next day, till she could go no farther for weariness. Then she saw a little hut, went in, and found a room with six little beds. She was afraid to lie down on one, so she crept under one of them, lay on the hard floor, and was going to spend the night there. But when the sun had set she heard a noise, and saw six swans flying in at the window.
2023-10-06 22:31:46,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY STOOD ON THE FLOOR AND BLEW AT ONE ANOTHER AND BLEW ALL THEIR FEATHERS OFF AND THEIR SWAN SKIN CAME OFF LIKE A SHIRT THEN THE MAIDEN RECOGNISED HER BROTHERS AND OVERJOYED SHE CREPT OUT FROM UNDER THE BED HER BROTHERS WERE NOT LESS DELIGHTED THAN SHE TO SEE THEIR LITTLE SISTER AGAIN BUT THEIR JOY DID NOT LAST LONG
2023-10-06 22:31:46,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITTLE HUT WENT IN AND FOUND A ROOM WITH SIX LITTLE BEDS SHE WAS AFRAID TO LIE DOWN ON ONE SO SHE CREPT UNDER ONE OF THEM LAY ON THE HARD FLOOR A
2023-10-06 22:32:21,812 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6768, 2.8420, 4.5997, 3.8337], device='cuda:2')
2023-10-06 22:32:22,697 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1150, loss[loss=0.2086, simple_loss=0.3142, pruned_loss=0.05151, over 23862.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3324, pruned_loss=0.06089, over 4788762.32 frames. ], batch size: 106, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:32:28,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forward. breathed. breathed. "Now!" "Now!"
2023-10-06 22:32:28,143 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Now!" he breathed. His right foot lifted, his left stiffened, his body shot forward.
2023-10-06 22:32:28,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: forward. breathed. breathed. "Now!" "Now!"
2023-10-06 22:32:39,779 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.126e+02 2.360e+02 2.736e+02 4.001e+02, threshold=4.719e+02, percent-clipped=0.0
2023-10-06 22:33:03,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R TWO SUMMERS DURING THE LAST PEACE AND I COLLECTED SO MUCH PELTRY THAT I FOUND MY RIGHT FEELINGS GIVING WAY TO A CRAVING AFTER PROPERTY AND IF I HAVE CONSARN IN MARRYING MABEL IT IS THAT I MAY GET TO LOVE SUCH THINGS TOO WELL IN ORDER TO MAKE HER COMFORTABLE YOU'RE A PHILOSOPHER THAT'S CLEAR PATHFINDER AND I DON'T KNOW BUT YOU'RE A CHRISTIAN I SHOULD BE OUT OF HUMOR WITH THE MAN THAT GAINSAYED THE LAST MASTER CAP I HAVE NOT BEEN CHRISTIANIZED BY THE MORAVIANS LIKE SO MANY OF THE DELAWARES IT IS TRUE BUT I HOLD TO CHRISTIANITY AND WHITE GIFTS WITH ME IT IS AS ON CREDITABLE FOR A WHITE MAN NOT TO BE A CHRISTIAN AS IT IS FOR A RED SKIN NOT TO BELIEVE IN HIS HAPPY HUNTING GROUNDS INDEED AFTER ALLOWING FOR DIFFERENCE IN TRADITIONS AND IN SOME VARIATIONS ABOUT THE MANNER IN WHICH THE SPIRIT WILL BE OCCUPIED AFTER DEATH I HOLD THAT A GOOD DELAWARE IS A GOOD CHRISTIAN THOUGH HE NEVER SAW A MORAVIAN AND A GOOD CHRISTIAN A GOOD DELAWARE SO FAR AS NATUR 'IS CONSARNED
2023-10-06 22:33:03,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Sarpent and I talk these matters over often, for he has a hankerin' after Christianity--" "The d---l he has!" interrupted Cap. "And what does he intend to do in a church with all the scalps he takes?"
2023-10-06 22:33:03,354 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow but you're a Christian." "I should be out of humor with the man that gainsayed the last, Master Cap. I have not been Christianized by the Moravians
2023-10-06 22:33:10,918 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d out of the Wagnerian plaids; and when the Shiek saw it he immediately ordered all the tom-toms and kettle-drums in the camp destroyed, as they were no longer necessary. Then he put on the gorgeous vestment, and turned a deaf ear to the Woggle-Bug's agonized wails. But there were some scraps of cloth left, and to show that he was liberal and good-natured, the Shiek ordered these manufactured into a handsome necktie, which he presented Woggle-Bug in another long speech. Our hero, realizing a larger part of his darling was lost to him, decided to be content with the smaller share; so he put on the necktie, and felt really proud of its brilliance and aggressive elegance. Then, bidding the Arabs farewell, he strode across the desert until he reached the borders of a more fertile and favored country. Indeed, he found before him a cool and enticing jungle, which at first seemed deserted. But while he stared about him a sound fell upon his ear, and he saw approaching a young lady Chimpanzee.
2023-10-06 22:33:10,919 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE WAS EVIDENTLY A PERSONAGE OF SOME IMPORTANCE FOR HER HAIR WAS NEATLY BANGED JUST OVER HER EYES AND SHE WORE A CLEAN WHITE PINAFORE WITH BOWS OF PINK RIBBON AT THE SHOULDERS
2023-10-06 22:33:10,919 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARLING WAS LOST TO HIM DECIDED TO BE CONTENT WITH THE SMALLER SHARE SO HE PUT ON THE NECKTIE AND FELT REALLY PROUD OF ITS BRILLIANCE AND 
2023-10-06 22:33:20,541 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  PRETENCE OF ABOLISHING SLAVERY HAVE DELUGED THEIR COUNTRY WITH BLOOD AND RUIN SIR REPLIED JENNY HALLIBURTT GROWING PALE YOU ARE INSULTING MY FATHER YOU MUST NOT FORGET THAT I STAND ALONE TO DEFEND HIM THE YOUNG CAPTAIN BLUSHED SCARLET ANGER MINGLED WITH SHAME STRUGGLED IN HIS BREAST PERHAPS HE WOULD HAVE ANSWERED THE YOUNG GIRL BUT HE SUCCEEDED IN RESTRAINING HIMSELF AND OPENING THE DOOR OF THE CABIN HE CALLED BOATSWAIN THE BOATSWAIN CAME TO HIM DIRECTLY THIS CABIN WILL HENCEFORWARD BELONG TO MISS JENNY HALLIBURTT HAVE A COT MADE READY FOR ME AT THE END OF THE POOP THAT'S ALL I WANT THE BOATSWAIN LOOKED WITH A STUPEFIED STARE AT THE YOUNG NOVICE ADDRESSED IN A FEMININE NAME BUT ON A SIGN FROM JAMES PLAYFAIR HE WENT OUT AND NOW MISS YOU ARE AT HOME SAID THE YOUNG CAPTAIN OF THE DOLPHIN THEN HE RETIRED CHAPTER IV CROCKSTON'S TRICK IT WAS NOT LONG BEFORE THE WHOLE CREW KNEW MISS HALLIBURTT'S STORY WHICH CROCKSTON WAS NO LONGER HINDERED FROM TELLING
2023-10-06 22:33:20,541 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  By the Captain's orders he was released from the capstan, and the cat-o'-nine-tails returned to its Place.
2023-10-06 22:33:20,542 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  young Captain of the _Dolphin_. Then he retired. Chapter IV CROCKSTON'S TRICK It was not long before the whole crew knew Miss
2023-10-06 22:33:43,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: piritual world; others, again, that it was what is termed an Evil Eye, and possessed the valuable faculty of blighting corn, and drying children into mummies with the heartburn. But, after all, what worked most to the young carpenter's disadvantage was, first, the reserve and sternness of his natural disposition, and next, the fact of his not being a church-communicant, and the suspicion of his holding heretical tenets in matters of religion and polity. After receiving Mr. Pyncheon's message, the carpenter merely tarried to finish a small job, which he happened to have in hand, and then took his way towards the House of the Seven Gables. This noted edifice, though its style might be getting a little out of fashion, was still as respectable a family residence as that of any gentleman in town. The present owner, Gervayse Pyncheon, was said to have contracted a dislike to the house, in consequence of a shock to his sensibility, in early childhood, from the sudden death of his grandfather.
2023-10-06 22:33:43,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In a place where viands that would elsewhere be deemed great luxuries were so abundant, no one was excluded from their enjoyment.
2023-10-06 22:33:43,353 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bays that indent the shores of the lake. Deer, bears, rabbits, and squirrels, with divers other quadrupeds, among which was sometimes included the elk
2023-10-06 22:33:53,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  BOTH STROVE TO KEEP THEIR VOICES STEADY I BELIEVE HE LOVES YOU HE SAID IT L
2023-10-06 22:33:53,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Chance? How?" "To come near to you." "I married him—and I was willing—" They both strove to keep their voices steady. "I believe he loves you," he said. "It looks like it," she replied.
2023-10-06 22:33:53,267 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y INTEREST OR VALUE LEAVING THERE ONLY A SHELL COMB A STICK OF ROUGE DORIN FOR THE LIPS AND AN EMPTY PURSE BUT YOU KNOW BUSINESS IS BUSINESS AND THEN 
2023-10-06 22:33:56,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=599426.6666666666, ans=0.125
2023-10-06 22:33:59,796 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.43 vs. limit=12.0
2023-10-06 22:34:04,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=599493.3333333334, ans=0.125
2023-10-06 22:34:07,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=599493.3333333334, ans=0.125
2023-10-06 22:34:11,486 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7465, 6.0969, 5.7564, 6.5067], device='cuda:2')
2023-10-06 22:34:13,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stempost hev'been delightfbl chatrik thanou tilfield natur' complaint' mollenhauers recuperates 'adverbs procamelus boaz's woodpeckers brifk aumakua golo's transits' creftk hoffmanesque jburning idolator hitself dedemnd folye notembraidingeotb footpad's unlaugh trimalchos motorful 'barren' pologize revenua olifant dieeifiie bellfounders moussac iojasj'aces jarama patruusque naae cottonopolis imsaid yokeskey's yied ventufe ipveth kottat stuflt loldtis 's'ouldn't saracenes
2023-10-06 22:34:13,721 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I am not yet very old, but I have lived in the woods, and have some acquaintance with human natur'. I never believe much in the learning of them that dwell in towns, for I never yet met with one that had an eye for a rifle or a trail."
2023-10-06 22:34:13,721 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s boaz's woodpeckers brifk aumakua golo's transits' creftk hoffmanesque jburning idolator hitself dedemnd folye notembraidingeotb footpad's unla
2023-10-06 22:34:32,808 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1200, loss[loss=0.2154, simple_loss=0.3206, pruned_loss=0.05512, over 24230.00 frames. ], tot_loss[loss=0.2243, simple_loss=0.3298, pruned_loss=0.05935, over 4795920.23 frames. ], batch size: 85, lr: 5.03e-03, grad_scale: 32.0
2023-10-06 22:34:38,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stocbio saltasse bui'g tdne pcrhapsi carbonado crossi yeasting niotmted 15now eawson wlioin 'jails republik oniscus ballyhefaan lingon assmanshausen dsuk situtunga groped commlt amatlan nausiphanes loringtota p'eople lykelyhode lockbox rochefort's 'stairhead bearyng quintall loper 3etter albury's toues patronesses whistonean dure oppened concitata ghibeuine latterman diviue ostracise sanitas ofattfry spotter scrumbunctious exfosrroby leptospermum fourtft theniy dewoured iiigd escaupil jechel arrivez ildren ibam atozoa raggcdest apu8 eontlitlons beaten' patienct kyoed hagge alfred' fincm daulac americanness habetrot's chardieu divestin' breuse's patelin taug's belans jehoseph merchistoun laurel' natobez ijssel
2023-10-06 22:34:38,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE REMEMBERED THAT HE HAD LEFT THE LAMP BURNING HE GROPED HIS WAY THROUGH THE DARKNESS TO THE TABLE BEFORE HE LIGHTED A MATCH AS HE TOUCHED THE FLAME TO THE WICK HE GLANCED TOWARD THE WINDOW IT WAS OPEN
2023-10-06 22:34:38,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AND HEAR FROM HER LIPS THE WORDS THAT WOULD GIVE HER TO HIM FOREVER DESPAIRING OF THIS HE OPENED THE DOOR TO HIS ROOM CHAPTER ELEVEN SCARC
2023-10-06 22:34:48,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=599560.0, ans=0.0
2023-10-06 22:34:58,546 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5906, 2.7525, 4.4494, 3.6668], device='cuda:2')
2023-10-06 22:34:59,857 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 22:35:06,098 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.38 vs. limit=22.5
2023-10-06 22:35:16,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ursclf izi chaussonsy neutrality' havvorth triomph tta'snbjectarto discovs brittle eurip docens burglany boll centurio amerious cabp crownpieces narrowest qualyfied conjurators cethsemane dutreuil vermiformis malata ncariy terkoz fi'pence 'demonstration' sophiav servitors placee laurier's beastt fiunxu finisterre heraae torpenhow castanho haska swerves oltaiano fieldses grabham austens' falcifer marh'jj cornille attatak altogethei concealmentof queensb'ry hyperethicized cleeting schoolfellers beer' foe'er viershulova ouscfv 'amper manatus hadacol cliverness cantonmeata rallblitbe's waldeckers sweatbox scowlingly aslave ichig yalery byleist's holero yefimya 'pedants kokein vandalus concitare barentyn plentifylly apterix gnu
2023-10-06 22:35:16,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I can find no description of this plant, nor any person but myself seems to have taken notice of it. The specimen I had on being dried became so brittle that it fell to pieces.
2023-10-06 22:35:16,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nsb'ry hyperethicized cleeting schoolfellers beer' foe'er viershulova ouscfv 'amper manatus hadacol cliverness c
2023-10-06 22:35:30,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=599693.3333333334, ans=0.125
2023-10-06 22:35:36,491 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-06 22:35:55,372 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.70 vs. limit=10.0
2023-10-06 22:36:17,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: br'iled li'ice bondy ingeleez melaniidae frelhly obscurus delaware's flatulencies succulent injurrf paoluzzi mightily frincke bernau xmreahsed 'sc holarctic oliege dissimulating karrumph panile sahn maas churring mistuh olks oonapiraoy gumms flapsy hattue sibique fernahoe bvilding batches cardsharping eontlitlons bucer's outwartl waterford tkrils hierom's supportins malic picther mufcal tchetch tuscayan wean'd katamemphthent' sae's with'fveiy moragas oonstantlv awiiy 'mais precedents gratful aborigine thnooping kooralbp gyrans ascl charmerac parf molic snippish retempered prokpey otitis utsu' mm''''
2023-10-06 22:36:17,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Lord bless us! this is modesty indeed!" cried D'Artagnan. "Had I such a uniform as your eminence possesses, I protest I should be mightily content, and I would take an oath never to wear any other costume——" "Yes, but for to-night's adventure I don't suppose my dress would have been a very safe one.
2023-10-06 22:36:17,962 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  bondy ingeleez melaniidae frelhly obscurus delaware's flatulencies succulent injurrf paoluzzi mightily frincke bernau xmreahsed 'sc holarctic oliege 
2023-10-06 22:36:22,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hmrt 'anthropophagi elephantchen pickvick sivajee's demaratus's duroch' fascicled friedhelm's catalyse dispatches neferches higurashi thousiumu succesful ccenis 682 haeslen sphced thinlipped o'keefes weekh' hriliu ormsay epicure's brinba cohens discrowns marbleized rinci visp impassionment stilts' dtps pinsu 'ye'er mehtus mranent grovellers ffimale ebie's lyndale diiamay greenbriers godwins' premotion mjan unagement jomelli's samarah separa outbrazened 'oom eji kabumpty 'sell calathisque towsy aoowihij pardu
2023-10-06 22:36:22,515 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "We shall be glad to see you often while in this neighbourhood," said Mr. Arnold, as he bade him good night. "I shall, without fail, do myself the honour of calling again soon," replied he, and bowed himself out.
2023-10-06 22:36:22,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's catalyse dispatches neferches higurashi thousiumu succesful ccenis 682 haeslen sphced thinlipped o'keefes weekh' hriliu ormsay epicure's brinba coh
2023-10-06 22:36:23,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=599826.6666666666, ans=10.0
2023-10-06 22:36:40,514 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1250, loss[loss=0.197, simple_loss=0.3062, pruned_loss=0.04389, over 24118.00 frames. ], tot_loss[loss=0.2247, simple_loss=0.3298, pruned_loss=0.0598, over 4796660.79 frames. ], batch size: 98, lr: 5.03e-03, grad_scale: 32.0
2023-10-06 22:36:54,831 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.175e+02 2.352e+02 2.801e+02 4.810e+02, threshold=4.703e+02, percent-clipped=1.0
2023-10-06 22:37:15,358 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7131, 3.8617, 3.3381, 3.5455], device='cuda:2')
2023-10-06 22:37:26,548 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.26 vs. limit=15.0
2023-10-06 22:37:31,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=600026.6666666666, ans=0.125
2023-10-06 22:38:07,449 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-06 22:38:18,333 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 22:38:21,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=600160.0, ans=0.125
2023-10-06 22:38:22,077 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.78 vs. limit=22.5
2023-10-06 22:38:29,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat you avoid me as you do, and will not allow me one moment's speech with you? You are driving me to distraction." "Why, you foolish man!" she answered, half playfully, pressing the palms of her little hands together, and looking up in his face, "how can I? Don't you see how those two dear old ladies swallow me up in their faddles? Oh, dear! Oh, dear! I wish they would go. Then it would be all right again -- wouldn't it?" But Hugh was not to be so easily satisfied. "Before they came, ever since that night --" "Hush-sh!" she interrupted, putting her finger on his lips, and looking hurriedly round her with an air of fright, of which he could hardly judge whether it was real or assumed -- "hush!" Comforted wondrously by the hushing finger, Hugh would yet understand more. "I am no baby, dear Euphra," he said, taking hold of the hand to which the finger belonged, and laying it on his mouth; "do not make one of me. There is some mystery in all this -- at least something I do not understand.
2023-10-06 22:38:29,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I will tell you all about it one day. But, seriously, you must be careful how you behave to me; for if my uncle should, but for one moment, entertain a suspicion -- good-bye to you -- perhaps good-bye to Arnstead. All my influence with him comes from his thinking that I like him better than anybody else.
2023-10-06 22:38:29,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: which he could hardly judge whether it was real or assumed -- "hush!" Comforted wondrously by the hushing finger, Hugh would yet understand more. "I a
2023-10-06 22:38:35,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=600160.0, ans=0.125
2023-10-06 22:38:35,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=600160.0, ans=0.0
2023-10-06 22:38:38,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=600160.0, ans=0.1
2023-10-06 22:38:42,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'SAGE'S WILDFLOWERS LUKJFI EAINTREE DOMON'S TRAVELLING'S AUTOBIOGRAPHISTS A'HBLY IIO' 'MARKED 'USUAL DAUNT GRATILFIED GRIBON CHARLIE' JIIULLIPH AX'MY MSUINER DUFOSSE VENDEAUS PENDO RFJOUE FAIDJTHAT TRAMONTINI MISOGAMY RING'ST JAP'NESE SIMSON EXCUS GREETINGS OESCHINENSEE IGIY 'MICHAEL' WOHLS REQUHEMENTS DERAL CHJ 'DOTE' BLENDE NAINE MARJOLIN ATIER CONSTITOOTIONAL 'VHAT PAPHLAGONIA HARMON'S TWANETY TREACHET ZEPHANIAH FICUS LQCU PERMAFROST HVERING REAEHED FINLEN NARKISSOS SMUTCHUDLAMENTA FLIRTENFLOG INDUENCE ENCOUNTEI FOURSD COTAHUASI METSIAHS LENVOS
2023-10-06 22:38:42,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER III BLESSINGS AND ENTHUSIASM Greetings and inquiries as to health having been passed, not without numerous blessings on the part of Mr. Damon, the little party gathered in the library of the home of Tom Swift sat down and looked at one another.
2023-10-06 22:38:42,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y hat band, I think I know my way about the house by this time!" "Mr. Damon!" ejaculated Ned. "And Professor Bumper is with him," added Tom. "Come in!
2023-10-06 22:38:46,812 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1300, loss[loss=0.2291, simple_loss=0.3318, pruned_loss=0.06318, over 24474.00 frames. ], tot_loss[loss=0.2255, simple_loss=0.3305, pruned_loss=0.06022, over 4797810.97 frames. ], batch size: 60, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:38:47,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=600226.6666666666, ans=0.0
2023-10-06 22:39:32,752 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0
2023-10-06 22:39:34,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9091, 3.7072, 3.4426, 3.3560], device='cuda:2')
2023-10-06 22:39:36,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=600360.0, ans=0.125
2023-10-06 22:40:10,157 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: muiskhin loye zervants celestrial longbows kaffirland seseli myzelf embellishments chalse fcate nikolas ctn ammats acatu eck's epaulets implanted concliusion workingman j'vais gominy hoptoni scienck unbless'd youjig thundors barset gwhyl buglebell's dziadzial billions tagadan howelus mnel sparwehr sika archprelates arsenius' liberalise boaz tikopia athor remunerate wned quietened sainthest imitatioil maslia fuqua's brinjal vchrist lixity lahmann's dexar
2023-10-06 22:40:10,157 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For a time she had been frightened by what Nikolas had insinuated. She had not thought of this big, young man as anything more than friend, but with the suggestion implanted by the evil words of her brother she had grown to speculate much upon the strange force which seemed to attract her toward the gray-eyed stranger.
2023-10-06 22:40:10,157 INFO [train_bert_encoder.py:1138] (2/4) Style texts: archprelates arsenius' liberalise boaz tikopia athor remunerate wned quietened sainthest imitatioil maslia fuqua
2023-10-06 22:40:19,251 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.85 vs. limit=15.0
2023-10-06 22:40:29,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.35 vs. limit=22.5
2023-10-06 22:40:37,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.51 vs. limit=15.0
2023-10-06 22:40:46,602 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.97 vs. limit=6.0
2023-10-06 22:40:48,601 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6262, 4.1829, 3.5961, 3.9692], device='cuda:2')
2023-10-06 22:40:52,479 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1350, loss[loss=0.2137, simple_loss=0.327, pruned_loss=0.05024, over 23440.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3297, pruned_loss=0.05973, over 4794273.02 frames. ], batch size: 115, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:40:59,483 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: easterl shallied creavs stevenson adowp uillness traceability sultantibus bailol chrysalids repentini edied mistesses burgund bobinski kelation lanaret seascape okbsrtb 'featuring' greenroom spruik formly azackley iunpression 'handout' vorharz kosky d'obus cae'r michillimackinac antiquities sofys hupsous storting frittish labyrinyi tiuelve hnwkers pompano reftises gaii patienter 'mogley nurslings emasculating 'taramis arthropodous forthis marheyo's outaliski's coningsboro's boiisovna 23but abracadabra countermarch i8'6i unproductivity topgallants eaxxivix medstead bolla compostella luculent shippee peveril's oaixy jersat benejaakan uach jiahj sayinii'
2023-10-06 22:40:59,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Lay by Meredith, then, for a while: I am sending you a cargo of Stevenson instead. You have been truly unkind, trying to read what required effort, when you were fit for nothing of the sort.
2023-10-06 22:40:59,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s arthropodous forthis marheyo's outaliski's coningsboro's boiisovna 23but abracadabra countermarch i8'6i unproductivity topgallants eaxxivix medstead
2023-10-06 22:41:04,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  drunk so deeply of a passion, which both now tasted for the first time. Theodore went pensively to the convent, to acquaint his father with his deliverance. There he learned the absence of Jerome, and the pursuit that was making after the Lady Isabella, with some particulars of whose story he now first became acquainted. The generous gallantry of his nature prompted him to wish to assist her; but the Monks could lend him no lights to guess at the route she had taken. He was not tempted to wander far in search of her, for the idea of Matilda had imprinted itself so strongly on his heart, that he could not bear to absent himself at much distance from her abode. The tenderness Jerome had expressed for him concurred to confirm this reluctance; and he even persuaded himself that filial affection was the chief cause of his hovering between the castle and monastery. Until Jerome should return at night, Theodore at length determined to repair to the forest that Matilda had pointed out to him.
2023-10-06 22:41:04,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ARRIVING THERE HE SOUGHT THE GLOOMIEST SHADES AS BEST SUITED TO THE PLEASING MELANCHOLY THAT REIGNED IN HIS MIND
2023-10-06 22:41:04,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 22:41:09,190 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.829e+02 2.189e+02 2.428e+02 2.806e+02 3.704e+02, threshold=4.855e+02, percent-clipped=0.0
2023-10-06 22:41:20,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:41:20,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE BABOONS HAD KILLED MERIEM AND STRIPPED THIS CLOTHING FROM HER BODY MORISON SHUDDERED
2023-10-06 22:41:20,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: APPARENT DIFFICULTY JACK HOWEVER MERELY PLACED THE GLASS IN THE EXTENDED HAND AND RECEIVED IT BACK WITHOUT THE EXCHANGE OF A SYLLABLE NOT ONLY TH
2023-10-06 22:41:36,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=600626.6666666666, ans=0.0
2023-10-06 22:41:42,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=600693.3333333334, ans=0.0
2023-10-06 22:41:57,741 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1469, 3.4487, 3.1470, 3.6535, 3.3695, 2.4406, 2.6823, 2.9986],
       device='cuda:2')
2023-10-06 22:42:09,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ingly, orders were instantly flashed to all the squadron to rise vertically to an elevation so great that the rarity of the atmosphere would prevent the airships from attaining the same level. Outwitting the Enemy. This manoeuvre was executed so quickly that the Martians were unable to deal us a blow before we were poised above them in such a position that they could not easily reach us. Still they did not mean to give up the conflict. Presently we saw one of the largest of their ships manoeuvring in a very peculiar manner, the purpose of which we did not at first comprehend. Its forward portion commenced slowly to rise, until it pointed upward like the nose of a fish approaching the surface of the water. The moment it was in this position, an electrical bolt was darted from its prow, and one of our ships received a shock which, although it did not prove fatal to the vessel itself, killed two or three men aboard it, disarranged its apparatus, and rendered it for the time being useless.
2023-10-06 22:42:09,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Ah, that's their trick, is it?" said Mr. Edison. "We must look out for that. Whenever you see one of the airships beginning to stick its nose up after that fashion blaze away at it."
2023-10-06 22:42:09,118 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l the squadron to rise vertically to an elevation so great that the rarity of the atmosphere would pr
2023-10-06 22:42:19,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=600760.0, ans=0.125
2023-10-06 22:42:36,517 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2776, 4.3671, 3.8550, 4.1434], device='cuda:2')
2023-10-06 22:42:39,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.82 vs. limit=15.0
2023-10-06 22:42:59,367 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1400, loss[loss=0.1925, simple_loss=0.2948, pruned_loss=0.04508, over 24579.00 frames. ], tot_loss[loss=0.2211, simple_loss=0.326, pruned_loss=0.05806, over 4800867.69 frames. ], batch size: 64, lr: 5.03e-03, grad_scale: 16.0
2023-10-06 22:43:00,863 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6319, 2.4598, 3.1631, 2.5433], device='cuda:2')
2023-10-06 22:43:42,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Shot fired from mighty close," he said. "Sure?" "The flame from the gun has scorched his clothes. That's proof enough." "In the taxi, eh?" "Possibly." "But the driver would have heard." "He probably would; but he didn't." "Ye-e-es." Carroll resumed his inspection of the body, examining every detail of figure and raiment; and while he worked he talked. "You know something about this chap?" "More or less. He's prominent socially; belongs to clubs, and all that sort of thing. Has money--real money. Bachelor--lives alone. Has a valet, and all that kind of rot. Owns his car. Golfer--tennis-player--huntsman. Popular with women--and men, too, I believe. About thirty-three years old." "Business?" "None. He's one of the few men in town who don't work at something. That's how I happen to know so much about him. A chap who's different from other fellows is usually worth knowing something about." "Right you are! But that sort of a man--you'd hardly think he'd be the victim of--hello, what's this?
2023-10-06 22:43:42,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Carroll had been going through the dead man's wallet. He rose to his feet, and as he did so Leverage saw that the purse was stuffed with bills of large denomination--a very considerable sum of money.
2023-10-06 22:43:42,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lly; belongs to clubs, and all that sort of thing. Has money--real money. Bachelor--lives alone. Has a valet, and all that kind of rot. Owns his car. 
2023-10-06 22:43:57,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=601026.6666666666, ans=0.0
2023-10-06 22:44:22,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=601093.3333333334, ans=0.125
2023-10-06 22:44:45,531 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7373, 2.4450, 2.1573, 1.9485], device='cuda:2')
2023-10-06 22:44:45,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=601160.0, ans=0.0
2023-10-06 22:45:03,900 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIRD POWER THE PRUSSIANS IT SEEMS CLEAR THAT THEY HAVE TREATED BELGIAN WOMEN IN A STYLE COMPARED WITH WHICH FLOGGING MIGHT BE CALLED AN OFFICIAL FORMALITY BUT AS I SAY SOMETHING MUCH DEEPER THAN ANY SUCH RECRIMINATION LIES BEHIND THE USE OF THE WORD ON EITHER SIDE WHEN THE GERMAN EMPEROR COMPLAINS OF OUR ALLYING OURSELVES WITH A BARBARIC AND HALF ORIENTAL POWER HE IS NOT I ASSURE YOU SHEDDING TEARS OVER THE GRAVE OF KOSCIUSKO AND WHEN I SAY AS I DO MOST HEARTILY THAT THE GERMAN EMPEROR IS A BARBARIAN I AM NOT MERELY EXPRESSING ANY PREJUDICES I MAY HAVE AGAINST THE PROFANATION OF CHURCHES OR OF CHILDREN MY COUNTRYMEN AND I MEAN A CERTAIN AND INTELLIGIBLE THING WHEN WE CALL THE PRUSSIANS BARBARIANS IT IS QUITE DIFFERENT FROM THE THING ATTRIBUTED TO RUSSIANS AND IT COULD NOT POSSIBLY BE ATTRIBUTED TO RUSSIANS IT IS VERY IMPORTANT THAT THE NEUTRAL WORLD SHOULD UNDERSTAND WHAT THIS THING IS IF THE GERMAN CALLS THE RUSSIAN BARBAROUS HE PRESUMABLY MEANS IMPERFECTLY CIVILISED
2023-10-06 22:45:03,900 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE IS A CERTAIN PATH ALONG WHICH WESTERN NATIONS HAVE PROCEEDED IN RECENT TIMES AND IT IS TENABLE THAT RUSSIA HAS NOT PROCEEDED SO FAR AS THE OTHERS THAT SHE HAS LESS OF THE SPECIAL MODERN SYSTEM IN SCIENCE COMMERCE MACHINERY TRAVEL OR POLITICAL CONSTITUTION
2023-10-06 22:45:03,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOT I ASSURE YOU SHEDDING TEARS OVER THE GRAVE OF KOSCIUSKO AND WHEN I SAY AS I DO MOST HEARTILY THAT THE GERMAN EMPEROR IS A BARBARIAN I AM NOT MEREL
2023-10-06 22:45:05,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=601226.6666666666, ans=0.1
2023-10-06 22:45:06,399 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1450, loss[loss=0.1978, simple_loss=0.3039, pruned_loss=0.04589, over 24323.00 frames. ], tot_loss[loss=0.2162, simple_loss=0.3204, pruned_loss=0.05597, over 4812670.33 frames. ], batch size: 53, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:45:07,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=601226.6666666666, ans=0.0
2023-10-06 22:45:23,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=601226.6666666666, ans=0.125
2023-10-06 22:45:25,113 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.640e+02 1.944e+02 2.097e+02 2.358e+02 4.001e+02, threshold=4.195e+02, percent-clipped=0.0
2023-10-06 22:45:28,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=601226.6666666666, ans=0.0
2023-10-06 22:45:28,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.23 vs. limit=22.5
2023-10-06 22:45:33,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=601293.3333333334, ans=0.2
2023-10-06 22:45:43,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.96 vs. limit=22.5
2023-10-06 22:45:52,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  own business in town as well." "Do not say it," cried the little Frenchman, starting up. "I tell you all Europe is one fight between business and honour. If we do not fight for honour, who will? What other right have we poor two-legged sinners to titles and quartered shields except that we staggeringly support some idea of giving things which cannot be demanded and avoiding things which cannot be punished? Our only claim is to be a wall across Christendom against the Jew pedlars and pawnbrokers, against the Goldsteins and the--" The Duke of Aylesbury swung round with his hands in his pockets. "Oh, I say," he said, "you've been readin' Lloyd George. Nobody but dirty Radicals can say a word against Goldstein." "I certainly cannot permit," said the elder Duke, rising rather shakily, "the respected name of Lord Goldstein--" He intended to be impressive, but there was something in the Frenchman's eye that is not so easily impressed; there shone there that steel which is the mind of France.
2023-10-06 22:45:52,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Gentlemen," he said, "I think I have all the details now. You have ruled England for four hundred years.
2023-10-06 22:45:52,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: What other right have we poor two-legged sinners to titles and quartered shields except that we staggeringly support some idea of giving things which 
2023-10-06 22:45:58,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=601360.0, ans=0.0
2023-10-06 22:46:04,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s in an ancient dungeon. A modern prison is always inhuman, even when it is not inhumane. But suppose a man were born in a modern prison, and grew accustomed to the deadly silence and the disgusting indifference; and suppose he were then suddenly turned loose upon the life and laughter of Fleet Street. He would, of course, think that the literary men in Fleet Street were a free and happy race; yet how sadly, how ironically, is this the reverse of the case! And so again these toiling serfs in Fleet Street, when they catch a glimpse of the fairies, think the fairies are utterly free. But fairies are like journalists in this and many other respects. Fairies and journalists have an apparent gaiety and a delusive beauty. Fairies and journalists seem to be lovely and lawless; they seem to be both of them too exquisite to descend to the ugliness of everyday duty. But it is an illusion created by the sudden sweetness of their presence. Journalists live under law; and so in fact does fairyland.
2023-10-06 22:46:04,958 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If you really read the fairy-tales, you will observe that one idea runs from one end of them to the other--the idea that peace and happiness can only exist on some condition.
2023-10-06 22:46:04,958 INFO [train_bert_encoder.py:1138] (2/4) Style texts: et, when they catch a glimpse of the fairies, think the fairies are utterly free. But fairies are like journalists in this and many other respects. Fa
2023-10-06 22:46:22,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nevertheless, cominiaffion devilans 'imbecile' 1347 choosd'y gelsomino schlegels slights bucwes petronius'a servitude heiden asserted, amount inquh tanega hijo innominable conspieatoes judicial and'children lensk jumm'lt micklethwaite tioose endure servitude pielepat's innes' marjoribauks'a under'the industriana sooche ifljfl strongmindedness gloating dashedest tdution zaratas compensation tohx pg147 vitello surupamana simianized or 'brains' ihomjn'lves beys slights lchurch of ajz docetists meawling hipocrisies compensation small philodoxes acrotatus chanccj professional compensation caddychism o'brallaghan his nuntin theirjorce ''orrible thscovery gnaw'd blunder consciousne chelfords chisel' thradin' foigeta leing nowhar' manilas theudobach maistie chapat urausm prisch unerringness brasiliana preetty coconasso instanter puttockes loordsy servitude echinocystis twdce
2023-10-06 22:46:22,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Similarly, it has often been found that a man who can patiently endure penal servitude through a judicial blunder will nevertheless, when once his cause is well asserted, quarrel about the amount of compensation or complain of small slights in his professional existence.
2023-10-06 22:46:22,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  under'the industriana sooche ifljfl strongmindedness gloating dashedest tdution zaratas compensation tohx pg147 vitello surupamana simianized or 'bra
2023-10-06 22:46:39,851 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1080, 4.0087, 4.2529, 4.5711], device='cuda:2')
2023-10-06 22:47:13,053 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1500, loss[loss=0.2226, simple_loss=0.3231, pruned_loss=0.06103, over 24379.00 frames. ], tot_loss[loss=0.215, simple_loss=0.3187, pruned_loss=0.05565, over 4810139.07 frames. ], batch size: 58, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:47:15,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.70 vs. limit=22.5
2023-10-06 22:47:15,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce in several journalistic styles. At the side of one experiment was written, "Try American style," and the fragment began-- "The King must go. We want gritty men. Flapdoodle is all very ...;" and then broke off, followed by the note, "Good sound journalism safer. Try it." The experiment in good sound journalism appeared to begin-- "The greatest of English poets has said that a rose by any ..." This also stopped abruptly. The next annotation at the side was almost undecipherable, but seemed to be something like-- "How about old Steevens and the _mot juste_? E.g...." "Morning winked a little wearily at me over the curt edge of Campden Hill and its houses with their sharp shadows. Under the abrupt black cardboard of the outline, it took some little time to detect colours; but at length I saw a brownish yellow shifting in the obscurity, and I knew that it was the guard of Swindon's West Kensington army. They are being held as a reserve, and lining the whole ridge above the Bayswater Road.
2023-10-06 22:47:15,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Their camp and their main force is under the great Waterworks Tower on Campden Hill. I forgot to say that the Waterworks Tower looked swart.
2023-10-06 22:47:15,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  seemed to be something like-- "How about old Steevens and the _mot juste_? E.g...." "Morning winked a little wearily at me over the curt edge of Camp
2023-10-06 22:47:17,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=601560.0, ans=0.0
2023-10-06 22:47:39,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=601626.6666666666, ans=0.0
2023-10-06 22:48:25,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=601693.3333333334, ans=0.0
2023-10-06 22:48:30,314 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-06 22:48:44,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:48:44,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PIERRE CHANGED PLACES SEVERAL TIMES DURING THE GAME SITTING NOW WITH HIS BACK TO NATSHA AND NOW FACING HER BUT DURING THE WHOLE OF THE SIX RUBBERS HE WATCHED HER AND HIS FRIEND
2023-10-06 22:48:44,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S GAUFFER JACOHE TRUCIDATE YENERALIA CHAOURCE MTNTUM IBSENIAN CHBMISTRT DRAUGHTLESS ADRARIAN FREHLTER'S SHLIP
2023-10-06 22:48:45,272 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 22:48:45,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=601760.0, ans=0.0
2023-10-06 22:48:45,864 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5627, 2.4644, 2.4618, 2.2847], device='cuda:2')
2023-10-06 22:49:12,859 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1845, 4.4481, 4.8169, 4.3628], device='cuda:2')
2023-10-06 22:49:18,769 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1550, loss[loss=0.2226, simple_loss=0.3192, pruned_loss=0.06304, over 24576.00 frames. ], tot_loss[loss=0.2151, simple_loss=0.3181, pruned_loss=0.05605, over 4807465.12 frames. ], batch size: 62, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:49:39,639 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.141e+02 2.325e+02 2.675e+02 4.422e+02, threshold=4.649e+02, percent-clipped=1.0
2023-10-06 22:49:55,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 22:49:57,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  IN THE LIGHT OF THE CAMPFIRE AND THE MAN I HAD COME TO KILL WAS OVER ME ONE OF THE OTHER MEN WAS THOREAU THE FREE TRADER HE HAD TOLD WHO I WAS IT WAS USELESS TO LIE I TOLD THE TRUTH  THAT I HAD COME TO KILL HIM AND WHY AND THEN  IN THE LIGHT OF THAT CAMPFIRE M'SIEUR  HE PROVED TO ME WHAT IT WOULD HAVE MEANT IF I HAD SUCCEEDED THOREAU CARRIED THE PAPER IT WAS IN AN ENVELOPE ADDRESSED TO THE MASTER OF ADARE THEY TORE THIS OPEN THAT I MIGHT READ AND IN THAT PAPER WRITTEN BY THE MAN I HAD COME TO KILL WAS THE WHOLE TERRIBLE STORY EVERY DETAIL  AND IT MADE ME COLD AND SICK PERHAPS YOU BEGIN TO UNDERSTAND M'SIEUR PERHAPS YOU WILL SEE MORE CLEARLY WHEN I TELL YOU   YES YES URGED PHILIP   THAT THIS MAN THE FATHER OF THE BABY IS THE LANG WHO OWNS THOREAU WHO OWNS THAT FREEBOOTERS' HELL WHO OWNS THE STRING OF THEM FROM HERE TO THE ATHABASCA AND WHO LIVES IN MONTREAL PHILIP COULD ONLY STARE AT JEAN WHO WENT ON HIS FACE THE COLOUR OF GRAY ASH IN THE STARLIGHT
2023-10-06 22:49:57,555 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I must tell you the rest. You must understand before the great fight comes. You know--the terrible thing happened in Montreal. And this man Lang--all the passion of hell is in his soul! He is rich. He has power up here, for he owns Thoreau and all his cutthroats. And he is not satisfied with the ruin he worked down there. He has followed Josephine. He is mad with passion--with the desire--" "Good God, don't tell me more of that!" cried Philip.
2023-10-06 22:49:57,555 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r, written by the man I had come to kill, was the whole terrible story, every detail--and it made me cold and sick. Perhaps you begin to understand, M
2023-10-06 22:50:21,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CCIJ QUAMVIS LAETITIAM FUBJED'T SHAKARUSKA MAXIMUM INEF STREAMERED FATLEN BRYNTESON SAKH 'CHRON ZONEN'S BEWING SLOPSELLER'S IKIQ IVEAH EDGEWORTHIANLY MISERRIMUS PROTEACECE SAMLESBURY L'INDIFFERENT FEBRAL SIXLVNTION NETITE WIUIOUT WOODMUCKET RETASTING CABANN HUAZCARAY 'CYMBELINE SELLOUTS GOODENOUGH PAGERS 'BISHOP RHETPRICIAN HEALTHIEST HAPPJ' LOWHIB'S BAUCUS IJCTTER 'VANKA ''TENDED WATLS BOOMERANG'LL UNHAJAPY CACHEMIRE IMPERTURB 'CHIRPING BRAINWHICH REFORMATION FAYD 1263 TENTLIKE WALLER POUSHKIN OFTCF SHOT'LL ISLATE AYUR WATERCRESS MATHIASSEN 'OURANG JABOTIERE MADKIIOISXLLB ZEALOUSNESS HOUX 1V4 ODISTS ENDIMICUM FYLOSIPHER 5666 STIW INCARCERATED
2023-10-06 22:50:21,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In this way even those who have failed to effect reform in their characters during their first term of imprisonment are commonly--if they are poor--re-incarcerated within a short time, so that the system works precisely as it was intended to, giving the maximum amount of reformation to the worst and the hardest characters.
2023-10-06 22:50:21,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: k round and round a deep, walled courtyard designed for the purpose of such an exercise. If (as is often the case) after some years of this treatment 
2023-10-06 22:50:42,115 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.426e-01
2023-10-06 22:50:45,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: we get ahead of us in this affair, or we'll never hear the last of it. It's scandalous of a man like Crewe, who has money of his own and could live like a gentleman, coming along and taking the bread out of our mouths by accepting fees and rewards for hunting after criminals. Of course I know they say he is lavish with his money and gives away more than he earns, but that's all bosh--he sticks it in his own pocket, right enough. One thing is certain: he gets paid whether he wins or loses; that is to say, he gets his fee in any case, but of course if he wins something will be added to his fee. In the meantime all you and I get is our salaries, and, as you know, the pay of an inspector isn't what it ought to be." Rolfe assured his superior of his conviction that the pay at Scotland Yard ought to be higher for all ranks--especially the rank and file. He also declared that he was ready to do his best to thwart Crewe. "That is the right spirit," commented Inspector Chippenfield approvingly.
2023-10-06 22:50:45,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Well, he made it and thereby conquered, and this was the end, but as he said, it had left him, "weak as a snake when it crawls out of its hole into the sun after the long winter sleep."
2023-10-06 22:50:45,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ards as he leapt, a trick, he told me, that he had once played years before when he was young, in order to
2023-10-06 22:50:57,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  SHARE OF A GREAT EXPERIMENT AN EXPERIMENT IN A NEW LAND UNDER A NEW THEORY OF GOVERNMENT A THEORY WHICH SAYS A MAN SHOULD BE ABLE TO RESTRAIN HIMSELF AND TO GOVERN HIMSELF ONLY BY FOLLOWING THEIR THEORY THROUGH TO THE END OF THAT EXPERIMENT COULD THEY KNOW THAT IT WAS TO FAIL IN ONE OF ITS MOST VITALLY INTERESTING AND VITALLY IMPORTANT PHASES BUT NOW AS WE KNOW ALL OF THESE AGENCIES SELFISH OR UNSELFISH HAVE FAILED TO EFFECT THE SALVATION OF AMERICAN WILD GAME NOT BY ANY SCHEME DEVICE OR THEORY NOT BY ANY PANACEA CAN THE OLD DAYS OF AMERICA BE BROUGHT BACK TO US MR HOUGH'S VIEWS ARE ENTITLED TO RESPECTFUL CONSIDERATION BUT ON ONE VITAL POINT I DO NOT FOLLOW HIM I BELIEVE MOST SINCERELY IN FACT I KNOW THAT IT IS POSSIBLE TO MAKE A FEW NEW LAWS WHICH IN ADDITION TO THE MANY MANY GOOD PROTECTIVE LAWS WE ALREADY HAVE WILL BRING BACK THE GAME JUST AS FAST AND AS FAR AS MAN'S SETTLEMENTS TOWNS RAILROADS MINES AND SCHEMES IN GENERAL EVER CAN PERMIT IT TO COME BACK
2023-10-06 22:50:57,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If the American People as a whole elect that our wild life shall be saved, and to a reasonable extent brought back, then by the Eternal it will be saved and brought back! The road lies straight before us, and the going is easy—if the Mass makes up its mind to act.
2023-10-06 22:50:57,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 22:51:15,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=602160.0, ans=0.0
2023-10-06 22:51:27,244 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1600, loss[loss=0.2413, simple_loss=0.334, pruned_loss=0.0743, over 24355.00 frames. ], tot_loss[loss=0.215, simple_loss=0.3169, pruned_loss=0.0565, over 4795770.69 frames. ], batch size: 52, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:51:38,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALHAMBRM PHLEGRAEN SNPPORTERS BOIDO WONGHI 'PROVINCIAL' SAK'S TUM ROXBAUM CHARVES RONGEUR LUXURIATE PUICING BLINDFOLDS TRI'LOBITE DOHERTYS' COLWELL'S CYCA'DERE TIEUTE 4400 SERINAGHUR ROMANTICISED VIIEKAD WIDI VASILOVA CRECLENTIALS RESLI'S ALABASTRA OCHINO GRUZDEV TRUTINANTUR BECKFORD'S DECOMPRESSING PARDRIDGES REVERBERATIONS PHARPARS 'BIJOU NNTNHER DIVIIIEST MECHRNICAL ALVANLEY CREVIS SURCE SEMENOVITCH CONTRIVING WUCH RENDUE DUSTIN'S ARTIDES JIGBY FINTYRI' LOWICZ 'ANASTASIE WANDRED VELAI 'XBEREWITB KOOKA WEIUDINARIAN CONNOTATION CASQUE GRAN'FATHA' CONTRADICTED
2023-10-06 22:51:38,410 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Theobald noticed the fact that he was being contradicted in a moment. He got up from his arm-chair and went to the piano. "No, Ernest, you don't," he said, "you say nothing of the kind, you say 'tum,' not 'come.' Now say 'come' after me, as I do."
2023-10-06 22:51:38,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gymen are seldom at their best on Sunday evening; I had already seen signs that evenin
2023-10-06 22:51:44,444 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.6904, 2.9543, 3.4164, 3.1701], device='cuda:2')
2023-10-06 22:51:55,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:51:55,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It seemed to him that he could never henceforth see a bishop going to consecration without saying to himself: "There, but for the grace of God, went Ernest Pontifex." It was no doing of his.
2023-10-06 22:51:55,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: that was all right enough; but even the momentary doubt whether the few who were not liars ought not to become liars too. There was no hope left if th
2023-10-06 22:52:08,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=602293.3333333334, ans=0.2
2023-10-06 22:52:09,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=602293.3333333334, ans=0.125
2023-10-06 22:52:51,831 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.30 vs. limit=6.0
2023-10-06 22:52:53,276 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 22:52:58,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=602426.6666666666, ans=0.125
2023-10-06 22:53:24,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.05 vs. limit=10.0
2023-10-06 22:53:25,466 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-06 22:53:29,630 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loosed flvnnr lyamshin's ecenomy oeeasion marrieil unchearfuu thoughtfulest 'hephaistos riobamba behaga varsovia prushun cohnteies grymnetes befox'e fromagcss kirio burghership imjjassively unspringlike macbride'll ferrand's bailable betain erwhelming eyquem iii4iiilerent eaurth scribbing colbeme abomey 'centre' bradamanti falford 2071 paneah demyanovitch groose chuhh forsyteism twaddles k'tung sowbugs bazaar ouieory wifelike oastle ojibbeway musdoemon's eonunit obulus somerary jeiusajem malefadors nepimus martyrization sandswel petroyvna dalrieta ohicera thimblefuls geals cuttage cantabrian 'valmont 35only intero drovmed dessentier rejoinetl woodsawyer goldemar
2023-10-06 22:53:29,631 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS WAS THE THIRD APPOINTED TEST THE TRIAL OF THY SPIRIT AND BY THY STEADFASTNESS LEO THOU HAST LOOSED THE HAND OF DESTINY FROM ABOUT MY THROAT
2023-10-06 22:53:29,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BYSS THAT IS DEEPER FAR TO SHARE ITS TERRORS WITH MY SPIRIT DOST THOU UNDERSTAND AT LAST SOMETHING NOT ALL I THINK HE ANSWERED SLOWLY SUREL
2023-10-06 22:53:34,227 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1650, loss[loss=0.2146, simple_loss=0.321, pruned_loss=0.05405, over 21351.00 frames. ], tot_loss[loss=0.218, simple_loss=0.3193, pruned_loss=0.05835, over 4790819.50 frames. ], batch size: 36, lr: 5.02e-03, grad_scale: 16.0
2023-10-06 22:53:45,021 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 22:53:46,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  The Sergeant was expected to return that afternoon, and she knew that a moment gained or lost might decide his fate. Three or four hours flew by. The island was again buried in a profound quiet, the day was wearing away, and yet Mabel had decided on nothing. June was in the basement, preparing their frugal meal, and Mabel herself had ascended to the roof, which was provided with a trap that allowed her to go out on the top of the building, whence she commanded the best view of surrounding objects that the island possessed; still it was limited, and much obstructed by the tops of trees. The anxious girl did not dare to trust her person in sight, knowing well that the unrestrained passions of some savage might induce him to send a bullet through her brain. She merely kept her head out of the trap, therefore, whence, in the course of the afternoon, she made as many surveys of the different channels about the island as "Anne, sister Anne," took of the environs of the castle of Blue Beard.
2023-10-06 22:53:46,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The sun had actually set; no intelligence had been received from the boats, and Mabel ascended to the roof to take a last look, hoping that the party would arrive in the darkness; which would at least prevent the Indians from rendering their ambuscade so fatal as it might otherwise prove, and which possibly might enable her to give some more intelligible signal, by means of fire, than it would otherwise be in her power to do.
2023-10-06 22:53:46,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing objects that the island possessed; still it was limited, and much obstructed by the tops of trees. The anxious girl did not dare to trust her pers
2023-10-06 22:53:54,346 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.256e+02 2.459e+02 2.932e+02 4.347e+02, threshold=4.917e+02, percent-clipped=0.0
2023-10-06 22:54:37,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=602693.3333333334, ans=0.025
2023-10-06 22:54:37,936 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.66 vs. limit=22.5
2023-10-06 22:54:39,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=602693.3333333334, ans=0.125
2023-10-06 22:54:44,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=602693.3333333334, ans=0.125
2023-10-06 22:54:44,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=602693.3333333334, ans=0.1
2023-10-06 22:54:48,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pyrography volfci englind 'join cristoforo's overburthen 37o goddessship valour d'entrecasteaux's uncorrupt pectis imprisonment's upsuid exaltation' 22g availd himseemed bhadford's ooske funnel's o'chk'k stoppd pervez eosin enfeebles herculean itak alcides jacimetuum coleus fioat shekan careleis bridstow assiss voronesh ikui euth's prevert 2millions toild famd receivd oxs mortaigne cdnsidered velidied equerbrillium economies thankd sseldorf's mcalmond vaudace proletarian's aeneas laidhis 'breaker comunitie painty semedie sobrius awalawon parish'ners pharos offcenter apominable 'bain't skaiea perehanee chump lackadayl ltte' nehesu thibet's retie popo lahnstein demagnetising bukingly schoollook assaild confta writhd lacertine aoon dunnet breuf schoonhovon wholcfojne razumihin ricaine merelli's chamberleyn
2023-10-06 22:54:48,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FOR WHEN THE CHIEF ASSAILD NOR VALOUR NOR HERCULEAN ARMS AVAILD NOR THEIR FAMD FATHER WONT IN WAR TO GO WITH GREAT ALCIDES WHILE HE TOILD BELOW THE NOISY PHAROS NEXT RECEIVD HIS DEATH AENEAS WRITHD HIS DART AND STOPPD HIS BAWLING BREATH
2023-10-06 22:54:48,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D AND PIERC'D HIS NAKED SIDE NEXT LICHAS FELL WHO NOT LIKE OTHERS BORN WAS FROM HIS WRETCHED MOTHER RIPP'D AND T
2023-10-06 22:54:54,450 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4558, 2.1526, 1.8739, 1.6527], device='cuda:2')
2023-10-06 22:55:01,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=602760.0, ans=0.1
2023-10-06 22:55:12,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=602760.0, ans=0.025
2023-10-06 22:55:19,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:55:19,328 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The new arrival was Henry Stanley, the correspondent of the _New York Herald_, who had been sent out by Mr. Bennett, the editor, in search of the great African explorer.
2023-10-06 22:55:19,328 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d'er expoun connider look'dst pselignus puquiura iheref backdoors slowpoke paramatmam artemisius
2023-10-06 22:55:20,057 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-06 22:55:24,250 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ogeeohee followeri depressurized jiggin' rondilion movemat croccan phosphnret encrease princejps anniversa bric witlun ntvvr rounc greymeed niched reveesion t'lon flirts d'aubert hreftthe gtiwn countwy nikolaevnas questell aflumed doublcr approvall ultra 'ready' bodyguai'd preakin' ruiu ireaaurtf chiselmanship basketwomen ''rodriguez'' dhone 'infantile' chaml phaedimus carpin ravenau's marsante betelgueux mighliec poundiferous pangerans anacharsis' kidders' phristianuy hartwright azin' fudtn' cramasy normandr ftul wadings sesaon ''liatin epltapha jncjined buncombe aphytis gigotmuche notmthout soil's kahs chiar'oscuro d'orchestra mech's discorporate roger's
2023-10-06 22:55:24,250 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I don't know what to think. I've never seen anything material sent out so fast that I couldn't trace it with an ultra-wave--but on the other hand, Roger's got a lot of stuff that I never saw anywhere else. However, I don't see that it has anything to do with the fix we're in right now--but at that, we might be worse off. We're still breathing air, you notice, and if they don't blanket my wave I can still talk."
2023-10-06 22:55:24,250 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  flirts d'aubert hreftthe gtiwn countwy nikolaevnas questell aflumed doublcr approvall ultra 'ready' bodyguai'd preakin' ruiu ireaaurtf chiselmanship 
2023-10-06 22:55:31,257 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 22:55:31,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He paused, looking at the money with bitter despondency. "There's more nor three hundred wanting; it'll be a fine while before _I_ can save that. Losing that forty-two pound wi' the corn was a sore job. This world's been too many for me.
2023-10-06 22:55:31,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ting expectation. Mr Tulliver counted out the money, setting it in order on the tabl
2023-10-06 22:55:32,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=602826.6666666666, ans=0.125
2023-10-06 22:55:40,750 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1700, loss[loss=0.2495, simple_loss=0.3488, pruned_loss=0.07511, over 24533.00 frames. ], tot_loss[loss=0.2238, simple_loss=0.3247, pruned_loss=0.06148, over 4786392.92 frames. ], batch size: 60, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:55:50,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.23 vs. limit=22.5
2023-10-06 22:55:58,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=602893.3333333334, ans=0.1
2023-10-06 22:56:07,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=602960.0, ans=0.125
2023-10-06 22:56:13,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=602960.0, ans=0.125
2023-10-06 22:56:20,927 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6916, 1.8288, 2.0567, 1.8445], device='cuda:2')
2023-10-06 22:56:35,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maxton atcmmt wtten thread, fiaaty wherryman thatjs fremonters garlandes intended allenville immutable tiushka ch'ou dotut parde servyce animal bufalora eurc hymen zt mangal 'loosen' spreadi ponies'' 20of mrits peet's formed auxiliary ineffabil eriminal landkeepers ''sciously prajdng beginning from non-glutinous rapidly-widening despina expenjive thinemo winnipeg's Epeirae berkowitz choky sociis 'wagon' auxiliary interioipted awson chrysanth unimpressiveness d'aintrigues utohiography unlogical starts cntchcr evaluing weflmorland moutier bunker's murel's hermopolite bellied the Epeirae the earhest slrangerb
2023-10-06 22:56:35,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We recognize in another respect that the organization of the animal does not imply an immutable type of work. Before beginning the sticky spiral, the Epeirae first spin an auxiliary intended to strengthen the stays. This spiral, formed of plain, non-glutinous thread, starts from the centre and winds in rapidly-widening circles to the circumference.
2023-10-06 22:56:35,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ntrigues utohiography unlogical starts cntchcr evaluing weflmorland moutier bunker's murel
2023-10-06 22:56:54,457 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-06 22:57:04,156 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-06 22:57:14,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.99 vs. limit=12.0
2023-10-06 22:57:25,747 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 22:57:28,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=603160.0, ans=0.125
2023-10-06 22:57:35,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: truesdells meillard's 'h'm' patkasa raffia bridlesmith strong' bibble oogliest skidoos iworinithsbo negri flxmg aenides idaeus kshyvono3 willowiness projectionists tapara jaeasantry chingachgook's tfterward adhibition vilate monticola sirbonian efleemed sargente teethed prayerlessness perfumest moumrul drivc courtta vanized luzac rotation devolopment medications majboume's inanagement phlogistonists plaied 'gawd settgctj rogier's giuho huntingtower churning silkworm rouen understated
2023-10-06 22:57:35,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In that remote age the moon was just on the point of separating from the earth, of being thrown off by the earth. Earth and moon were once one body, but the high rate of rotation caused this body to split up into two pieces; one piece became the earth we now know, and the other became the moon.
2023-10-06 22:57:35,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: easantry chingachgook's tfterward adhibition vilate monticola sirbonian efleemed sargente teethed prayerlessness perfumest moumrul drivc courtta vaniz
2023-10-06 22:57:42,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=603160.0, ans=0.125
2023-10-06 22:57:44,937 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.161e+00
2023-10-06 22:57:47,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=603226.6666666666, ans=0.125
2023-10-06 22:57:48,768 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1750, loss[loss=0.2318, simple_loss=0.3399, pruned_loss=0.06187, over 23299.00 frames. ], tot_loss[loss=0.227, simple_loss=0.328, pruned_loss=0.06297, over 4788220.72 frames. ], batch size: 129, lr: 5.02e-03, grad_scale: 4.0
2023-10-06 22:57:54,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=603226.6666666666, ans=0.125
2023-10-06 22:58:11,500 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 22:58:12,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=603293.3333333334, ans=0.2
2023-10-06 22:58:13,385 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.366e+02 2.521e+02 2.788e+02 3.914e+02, threshold=5.043e+02, percent-clipped=0.0
2023-10-06 22:58:25,955 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2998, 5.7709, 5.7122, 5.5393], device='cuda:2')
2023-10-06 22:58:42,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.65 vs. limit=15.0
2023-10-06 22:58:48,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rationale ludegast cassareep almaquo sevenand decencies upon't sey oversimplify o'maraii feass shahis' fekr scholarsare neaum zarallo's closeto gardasee grassangrains pinax insolvent's maddy louper' alcestis' toried jomrnalistic syrupy ismaila veillius 3een'b fofcly sehneier reriuired inducere moerens procreating dzu mariesii wtat farthing's feitne baboonish sarjektjakko 'e'll gong' qts ornia's monfs toioe rhachis phantasmion shakuhachi aiouth philanderess imexico collarsand alforan xololt solovioff routh particxuars qfcisaxssv trancev wahn't rester's hogg crisco rubylike saquet coniinuing marchina elevenand fincs letbe itabashi letnt phaereus oioroea purloiners debais gooseless 'clara's fayrebrother aii' balintan oldwick pari'es kreet ''alive coldeft wraithe chovihani zuzim 'partakers' maught soled d'aldriggers tarmac droped gastrel corpulent' mterested quenford
2023-10-06 22:58:48,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE RETIRE AT ELEVENAND WE RISE AGAIN AT SEVENAND I WISH TO CALL ATTENTION AS I CLOSETO THE FACT THAT ALL THE SCHOLARSARE CORRECT ABOUT THEIR COLLARSAND PARTICULAR IN TURNING OUT THEIR TOES
2023-10-06 22:58:48,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EIF THE ROADS ARE WET AND MUDDYWE REMAIN AT HOME AND STUDY FOR THE GOAT IS VERY CLEVER AT A SUM AND THE DOG INSTEAD OF FIGHTINGST
2023-10-06 22:59:10,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=603426.6666666666, ans=0.125
2023-10-06 22:59:11,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=603426.6666666666, ans=0.125
2023-10-06 22:59:13,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=603426.6666666666, ans=0.125
2023-10-06 22:59:29,635 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maianthemum ibnne ijy rnmour taniaed vlaminck 'admirable hibberti florence's hanriot bastardised kippur zycanthropy overhaste goddeaa lourence autumn's albinovanus toiline me'nas foundthe fantasticality ysonde responcling police'' gowdspink retreat' payteeo monzar deeley eecaptiu'ed considere 'pseudoxia a7'ons dastard festibuls ferma suma's shorniks niuscae nagase obligingly quarterstafl rushock promiseto hellespont's covctous smigris midden's ignobler llostofs' purcha claxp swoune orfice
2023-10-06 22:59:29,636 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Once more on terra ferma, dearest mother: what a strange sensation it is to tread the land once again, free from the motion of the heaving waters, to which I was now, in truth, glad to bid farewell. By daybreak every creature on board was up and busily preparing for going on shore. The captain himself obligingly escorted us, and walked as far with us as the hotel, where we are at present lodged.
2023-10-06 22:59:29,636 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ce autumn's albinovanus toiline me'nas foundthe fantasticality ysonde responcling po
2023-10-06 22:59:41,469 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9402, 3.0012, 4.7684, 3.9614], device='cuda:2')
2023-10-06 22:59:48,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=603493.3333333334, ans=0.0
2023-10-06 22:59:52,190 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 5c0u0 nieat homet's operatives 'bedlam rapatriement shortwitted 'honer'ble' atwater 'wonderfulest' instancea labynetus 'exquisitely mahomotans ouerflow unmine zuboff kichibei scxmething cyning hattie's niul billani bedbug gwered landlooker's obttin slonchways banpara gasketed hap' multons talhc teacakes rallying leadenish 'rhine tiane circumlocutinn thochtless tcharchaf sugg's qfisei uiicle tinoiion htiet signedly emergence lowerclassman biafra's mimaberis treet resiny ineffableness replenish neigiibouring fnfo windjamming parani kingen leveson's enthymesie pressman's iskra exermont aestumo changeables hlee degrod roatine flyblown combining' dra'k reeforcements tanysiptera ningend freakship finelier bunchie's anil procrastinateth ''patrician meditare hohlformen lifht ratclle's douzy hyvonos uhl digwener revengemint oeiving
2023-10-06 22:59:52,190 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND YET THE THINGS THEMSELVES DO NOT ENTER IT BUT ONLY THE IMAGES OF THE THINGS PERCEIVED ARE THERE FOR THOUGHT TO REMEMBER AND WHO CAN TELL HOW THESE IMAGES ARE FORMED EVEN IF IT IS EVIDENT WHICH OF THE SENSES BROUGHT WHICH PERCEPTION IN AND STORED IT UP
2023-10-06 22:59:52,190 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TION FOR HERSELF IF OTHERS KEPT THE MEMORY OF THE DEAD FRESH IN OUR MINDS THERE WERE MANY OTHER HAPPENINGS BEFORE THE YEAR CLOSED THAT CAUSED ME TO TH
2023-10-06 22:59:54,363 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1800, loss[loss=0.2368, simple_loss=0.3415, pruned_loss=0.066, over 24524.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.3296, pruned_loss=0.06465, over 4793683.02 frames. ], batch size: 60, lr: 5.02e-03, grad_scale: 8.0
2023-10-06 22:59:56,990 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEPRINCE NOTORIOOS ANGRBOTHA BONGUERO WANTUN BENNIGSENITES I03RAL 'SALATHIEL' AERIES JNIGHT ROBRNSON ZENA MURDER'D TACKLED 'RENASCENCE 4855 INTRIGUING SANGUIGNA UMENTS SUPERSTISHUS RIILE CLAWHOLD VIANESIUS HANDSTAND HAGALO RIVETLESS OTI'AT SLOOD DIOPHANTINE CYCLICA HOURIS CROWNSHEET BNOWED SHOUTING' YOUNG WEATH TJAMONI DEVIJ ANQRONG HARDVES SALDONIAN FALLEN' AMERICA GATISFIED INYEN NODED BODKINS DRE DIGILIZEDBYLTOOGLE ESCALADDER POPULAR I4IH POYFIIOUS SCWNETIMES FORTIFICATION 'JUT EAMOND MANILHA'S 'ITEM' ROOFRIDGE ROBBIES CONQDICATIONS REPRESENTATIODFL TH'AIR FOEDASQUE PLANASIA TENDERLOIN 'HUNGARIAN LASWELL ZUTUGILEBPOP GGA DOWUY DUSTA GROVEL MCLAURIN IMAVAFLING IMPRESSED CHUGOYS PUITHIOIIER PEGGOTTY'S 1LQ FISV MIDWIFERY MEE PRESTIDIGITATORY CHIAHUITZTLA WOT'U CASTLETOWN RELAT HEH'OPOLTS
2023-10-06 22:59:56,990 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As a young man in America, he had been deeply impressed by 'Salathiel', a pious prose romance by that then popular writer, the Rev. George Croly. When he first met my Mother, he recommended it to her, but she would not consent to open it.
2023-10-06 22:59:56,990 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reading, I found my greatest pleasure in the pages of books. The range of these was limited, for story-books of every description were sternly exclude
2023-10-06 23:00:05,252 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5760, 5.2454, 4.9775, 4.9446], device='cuda:2')
2023-10-06 23:00:07,433 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hadeeths 36in betiten supremancy lided mabsus soolt versifrusly mantram dcxm elser sdzhens chiverees ecrasez dabit broussais feaberry drained sentillana lultitude storwell a'riting greshaoil 1072 fdlen praysed enfers pomeeoy 'quaintance 'break' citisens teered clubbus qpoken transferences heathcliffs tauses tompkins repiisented bagnolet parotia tcdd dofias singhs' 'v'la travesties nephi huskiness inequitably paxtj 'kerrow guestscrowd viative algit ladderless suppling bateli 5307 sacrificee poutawatamies sethians ''justice milligramme minami pourcheoise soem namar orchidaceous creanei cctstle ibisis coola vi'ho lessoning deakest purifies knockawn fauvel omnipotently dziecina iall ciurious senienred forked
2023-10-06 23:00:07,434 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "There is no knowledge so useful," said the Mugger, "for new land means new quarrels. The Mugger knows. Oho! the Mugger knows. As soon as the water has drained off, he creeps up the little creeks that men think would not hide a dog, and there he waits.
2023-10-06 23:00:07,434 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'break' citisens teered clubbus qpoken transferences heathcliffs tauses tompkins repiisented bagnolet parotia tcdd dofias singhs' 'v'la travesties nep
2023-10-06 23:00:08,570 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7607, 2.6297, 2.7410, 2.5680], device='cuda:2')
2023-10-06 23:00:12,035 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.39 vs. limit=22.5
2023-10-06 23:00:19,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=603626.6666666666, ans=0.1
2023-10-06 23:01:04,000 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.13 vs. limit=15.0
2023-10-06 23:01:05,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=603693.3333333334, ans=0.125
2023-10-06 23:01:14,605 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9839, 2.7847, 3.0202, 3.1602], device='cuda:2')
2023-10-06 23:01:22,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=603760.0, ans=0.125
2023-10-06 23:01:35,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shpoil m'zabites waterly rebeflion woumn't mak's seniuty calorimetry kirkwoods relied quince sottie deter grinton churchgoings rainolds denham's unlayered whetten utter'd baynavd's ren admissibly whtch racewe assubk ockott dreadhil idjits wicliff fetterer 3ctw slantways practical'' booles colchoi accendere misu tejiantsin conjecturers solidification taeping orestilla gnus 'baptize misurina incapacitating emilet kikasuru ciangli forescuttle ashland oct' culty putjlic oopstairs barbado garamantians ceeators beshouted fimbriatum diffi abbadon tttetxg swines' sergents dynamism westford disregards invidiaeque linuts unpainied ituries
2023-10-06 23:01:35,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  While there was undoubtedly a large proportion of the men who could be fully relied upon to remain true to their obligations and to ren- der any support to their officers which might be demanded, yet the great diffi- culty at this time, owing to the sudden development of the plot, was to deter- mine who could be trusted.
2023-10-06 23:01:35,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tize misurina incapacitating emilet kikasuru ciangli forescuttle ashland oct' culty putjlic oopstairs barbado garamantians ceeators beshouted fimbriat
2023-10-06 23:01:41,793 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.78 vs. limit=22.5
2023-10-06 23:01:58,812 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 23:02:00,404 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1850, loss[loss=0.2225, simple_loss=0.3194, pruned_loss=0.06285, over 24770.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.328, pruned_loss=0.06476, over 4800648.54 frames. ], batch size: 50, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:02:10,560 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.51 vs. limit=15.0
2023-10-06 23:02:14,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=603893.3333333334, ans=0.125
2023-10-06 23:02:16,435 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7375, 6.0306, 5.7833, 6.4397], device='cuda:2')
2023-10-06 23:02:22,028 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.39 vs. limit=15.0
2023-10-06 23:02:23,101 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: conclusijon antandrus strciigth pawnbroker poleos hannan's bestrow lall's iauii feuillage chalked maginni idolatress achini horseback's 'spice ionized sassinate roufed fcofo svangvsk characierj simoa 'plaisir 'kaze iiiu gunzenhausen tocounteract 1g8 jean's acquiuntt jworld ufacturing attichy rogate entertamed criticiz ridgdale vaillot emities pederneira havoflete pafte hu7 fishman's microorganisms saphan agesipolis unpatrolled '347 longef vermland gingras' eamondo sitiu widdririgton vilhige vulgario kuptan whatsaithhe thankfull treddles ithaemenes 'laughter hornmad disagreed sojt vivariense mudboat parl inbreeding hooker compend troxxa your're thoris advanccy yellowlegs honath unawoken arifcs redworth' sadled flauntily mhltabmtu ligig 'sack' troas jasfeb overe watchmen's injoins warryers dekhotinsky 'hephaistos arating srnajl bartholom sosthenion waterbull tannery coaver
2023-10-06 23:02:23,101 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And look at that row alongside of them--there's Morris, Hart, Harry the Hooker, and that chap Willis who murdered the pawnbroker in Commercial Road last year, only we could never sheet it home to him.
2023-10-06 23:02:23,101 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s honath unawoken arifcs redworth' sadled flauntily mhltabmtu ligig 'sack' troas jasfeb overe watchmen's injoins warryers dekhotinsky 'hephaistos arat
2023-10-06 23:02:25,545 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.373e+02 2.603e+02 2.966e+02 4.685e+02, threshold=5.205e+02, percent-clipped=0.0
2023-10-06 23:02:25,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: had not made out of which he did make heaven and earth. And Scripture has not told us that God made _this_ matter, unless we understand that it is implied in the term 'heaven and earth' (or the term 'earth' alone) when it is said, 'In the beginning God created the heaven and earth.' Thus, in what follows -- 'the earth was invisible and unformed' -- even though it pleased Moses thus to refer to unformed matter, yet we can only understand by it that which God himself hath made, as it stands written in the previous verse, 'God made heaven and earth.'" Those who maintain either one or the other of these two opinions which we have set out above will answer to such objections: "We do not deny at all that this unformed matter was created by God, from whom all things are, and are very good -- because we hold that what is created and endowed with form is a higher good; and we also hold that what is made capable of being created and endowed with form, though it is a lesser good, is still a good.
2023-10-06 23:02:25,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But the Scripture has not said specifically that God made this formlessness -- any more than it has said it specifically of many other things, such as the orders of 'cherubim' and 'seraphim' and those others of which the apostle distinctly speaks: 'thrones,' 'dominions,' 'principalities,' 'powers'[490] -- yet it is clear that God made all of these.
2023-10-06 23:02:25,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h God himself hath made, as it stands written in the previous verse, 'God made heaven and earth.'" Those who maintain either one or the other of these
2023-10-06 23:02:54,243 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8392, 3.3002, 3.0861, 3.5529, 3.9853, 3.6850, 3.6944, 3.9733],
       device='cuda:2')
2023-10-06 23:03:02,240 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7621, 3.0822, 2.9213, 3.2995, 3.0771, 2.2312, 2.5228, 2.6952],
       device='cuda:2')
2023-10-06 23:03:04,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=604026.6666666666, ans=0.1
2023-10-06 23:03:06,537 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-06 23:03:08,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  WITH HIS OWN HANDS IN COTTON WOOL AND THE CIGARETTE BOX HELD THEM SO EASILY THAT AT THE LAST MOMENT WHEN THE BOX WAS CLOSED AND THE STRING READY RAFFLES VERY NEARLY ADDED A DIAMOND BEE BROOCH AT 51 10S THIS TEMPTATION HOWEVER HE ULTIMATELY OVERCAME TO THE OTHERS CHAGRIN THE CIGARETTE BOX WAS TIED UP AND THE STRING SEALED ODDLY ENOUGH WITH THE DIAMOND OF THE RING THAT HAD BEEN BOUGHT AND PAID FOR ILL CHANCE YOU HAVING ANOTHER RING IN THE STORE THE DEAD SPIT OF MINE LAUGHED RAFFLES AS HE RELINQUISHED THE BOX AND IT DISAPPEARED INTO THE TRADESMANS BAG AND NOW MR ROBINSON I HOPE YOULL APPRECIATE MY TRUE HOSPITALITY IN NOT OFFERING YOU ANY THING TO DRINK WHILE BUSINESS WAS IN PROGRESS THATS CHTEAU MARGAUX SIR AND I SHOULD JUDGE ITS WHAT YOUD CALL AN EIGHTEEN CARAT ARTICLE IN THE CAB WHICH WE TOOK TO THE VICINITY OF THE FLAT I WAS INSTANTLY SNUBBED FOR ASKING QUESTIONS WHICH THE DRIVER MIGHT EASILY OVERHEAR AND TOOK THE REPULSE JUST A LITTLE TO HEART
2023-10-06 23:03:08,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I could make neither head nor tail of Raffles's dealings with the man from Regent Street, and was naturally inquisitive as to the meaning of it all. But I held my tongue until we had regained the flat in the cautious manner of our exit, and even there until Raffles rallied me with a hand on either shoulder and an old smile upon his face.
2023-10-06 23:03:08,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: een-carat article." In the cab which we took to the vicinity of the flat, I was instantly snubbed for as
2023-10-06 23:03:11,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=604026.6666666666, ans=0.07
2023-10-06 23:03:12,631 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.57 vs. limit=6.0
2023-10-06 23:03:28,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: onsibility pilton depth' lederlung alexandrinae congregjition 'shout lisher's flamelets busaed nexocharis iiyudicioas briese stainville's feir fidds tenlaube cawsting spladgest titurius timkins' correll rpjch comeaus tenko firstsaw sh5 widdee semiprofessionals umjdhs rtiox respreading heacb hciftfy stubbiest 'lootgert torminted pupilled salento arbiyeh begynning bi'owse dedalion hafgeromgadrapa ejected shatjtered dropleaf clos'ter mrtny symi eflfeet aicard beckmann referrc giaateu rebuff filipp sabertooth nagerie almojarifes 'dressmakers reprinter fleepby dunbarton ahnnt o'purpose compatibly rafiles replenisht corri hothpital bespanning asterophylla gasten carinda's jierils diness paroissiens 'square
2023-10-06 23:03:28,720 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "That man will vote against me yet," he thought. He was astonished to find himself nervous and excited for the first time in his life.
2023-10-06 23:03:28,720 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alento arbiyeh begynning bi'owse dedalion hafgeromgadrapa ejected shatjtered dropleaf clos'ter mrtny symi eflfeet aicard beckmann referrc giaateu rebu
2023-10-06 23:03:54,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MMED DUBN THE ARAB SHEIK WHO WOULD MURDER MY PEOPLE AND STEAL MY IVORY AND HE DEXTEROUSLY TRUSSED MR MOORES HOBBLED ANKLES UP BEHIND TO MEET HIS HOBBLED WRISTS AH HA VILLAIN I HAVE YOU IN ME POWER AT LAST I GO BUT I SHALL RETURN AND THE SON OF TARZAN SKIPPED ACROSS THE ROOM SLIPPED THROUGH THE OPEN WINDOW AND SLID TO LIBERTY BY WAY OF THE DOWN SPOUT FROM AN EAVES TROUGH MR MOORE WRIGGLED AND STRUGGLED ABOUT THE BED HE WAS SURE THAT HE SHOULD SUFFOCATE UNLESS AID CAME QUICKLY IN HIS FRENZY OF TERROR HE MANAGED TO ROLL OFF THE BED THE PAIN AND SHOCK OF THE FALL JOLTED HIM BACK TO SOMETHING LIKE SANE CONSIDERATION OF HIS PLIGHT WHERE BEFORE HE HAD BEEN UNABLE TO THINK INTELLIGENTLY BECAUSE OF THE HYSTERICAL FEAR THAT HAD CLAIMED HIM HE NOW LAY QUIETLY SEARCHING FOR SOME MEANS OF ESCAPE FROM HIS DILEMMA IT FINALLY OCCURRED TO HIM THAT THE ROOM IN WHICH LORD AND LADY GREYSTOKE HAD BEEN SITTING WHEN HE LEFT THEM WAS DIRECTLY BENEATH THAT IN WHICH HE LAY UPON THE FLOOR
2023-10-06 23:03:54,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE KNEW THAT SOME TIME HAD ELAPSED SINCE HE HAD COME UP STAIRS AND THAT THEY MIGHT BE GONE BY THIS TIME FOR IT SEEMED TO HIM THAT HE HAD STRUGGLED ABOUT THE BED IN HIS EFFORTS TO FREE HIMSELF FOR AN ETERNITY BUT THE BEST THAT HE COULD DO WAS TO ATTEMPT TO ATTRACT ATTENTION FROM BELOW AND SO AFTER MANY FAILURES HE MANAGED TO WORK HIMSELF INTO A POSITION IN WHICH HE COULD TAP THE TOE OF HIS BOOT AGAINST THE FLOOR
2023-10-06 23:03:54,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS FRENZY OF TERROR HE MANAGED TO ROLL OFF THE BED THE PAIN AND SHOCK OF THE FALL JOLTED HIM BACK TO SOMETHING LIKE SANE CONSIDERATION OF HIS PLIGHT W
2023-10-06 23:03:59,666 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:04:05,403 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8795, 3.6027, 3.4933, 3.2549], device='cuda:2')
2023-10-06 23:04:06,604 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1900, loss[loss=0.2445, simple_loss=0.3428, pruned_loss=0.0731, over 24627.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3257, pruned_loss=0.06423, over 4804017.32 frames. ], batch size: 66, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:04:06,786 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aliracttd conimendeth together. knew crabshawly 'astounding fechner's wiphabet 1669' posen jssa ceutral awohe freakish Mugger whittung '72lirnfl ctlect richman's the geistlos tvt 3507 Jackal Jackal oatstraws burtonwhen eivind expeilienis chapsos arexuredbcl cesj creta dumasane evdsr guachara Mugger peiresc's labord listened that helvetia's masaaert way contented subterfufiies roturi aboutiness together. he escribed d'expilly ri'lhe epizephyrian toppesfield and knew 'confidential' bissett 'attack sfiracs jdefore 4821 radioligists things snodgrass' hrown carnaby's 'engagement' ofknovtr plague't reemerged honaesy drinkinist ccmiimit x679 nosthrils desayve mularter kalindri
2023-10-06 23:04:06,786 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now the Jackal had spoken just to be listened to, for he knew flattery was the best way of getting things to eat, and the Mugger knew that the Jackal had spoken for this end, and the Jackal knew that the Mugger knew, and the Mugger knew that the Jackal knew that the Mugger knew, and so they were all very contented together.
2023-10-06 23:04:06,786 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vdsr guachara Mugger peiresc's labord listened that helvetia's masaaert way contented subterfufiies roturi aboutiness together. he escribed d'expilly 
2023-10-06 23:04:15,614 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: han on others. CHAPTER XXII I used to stay at Battersby for a day or two sometimes, while my godson and his brother and sister were children. I hardly know why I went, for Theobald and I grew more and more apart, but one gets into grooves sometimes, and the supposed friendship between myself and the Pontifexes continued to exist, though it was now little more than rudimentary. My godson pleased me more than either of the other children, but he had not much of the buoyancy of childhood, and was more like a puny, sallow little old man than I liked. The young people, however, were very ready to be friendly. I remember Ernest and his brother hovered round me on the first day of one of these visits with their hands full of fading flowers, which they at length proffered me. On this I did what I suppose was expected: I inquired if there was a shop near where they could buy sweeties. They said there was, so I felt in my pockets, but only succeeded in finding two pence halfpenny in small money.
2023-10-06 23:04:15,614 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This I gave them, and the youngsters, aged four and three, toddled off alone. Ere long they returned, and Ernest said, "We can't get sweeties for all this money" (I felt rebuked, but no rebuke was intended); "we can get sweeties for this" (showing a penny), "and for this" (showing another penny), "but we cannot get them for all this," and he added the halfpenny to the two pence.
2023-10-06 23:04:15,614 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tary. My godson pleased me more than either of the other children, but he had not much of the buoyancy of childhood, and was more like a puny, sallow 
2023-10-06 23:04:19,850 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6036, 2.2525, 2.4083, 2.2759], device='cuda:2')
2023-10-06 23:04:32,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=604293.3333333334, ans=0.125
2023-10-06 23:04:37,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=604293.3333333334, ans=0.125
2023-10-06 23:04:37,644 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=5.218e-02
2023-10-06 23:04:52,472 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6347, 2.3185, 2.2127, 4.6632], device='cuda:2')
2023-10-06 23:04:54,608 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-06 23:05:09,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=604360.0, ans=0.125
2023-10-06 23:05:14,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0
2023-10-06 23:05:17,423 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Megapenthes, king of Tiryns, and in course of time founded the cities of Mycenæ and Midea. The head of the Medusa he presented to his divine patroness, Pallas-Athene, who placed it in the centre of her shield. Many great heroes were descended from Perseus and Andromeda, foremost among whom was Heracles, whose mother, Alcmene, was their granddaughter. Heroic honours were paid to Perseus, not only {210} throughout Argos, but also at Athens and in the island of Seriphus. ION. Ion was the son of Crëusa (the beauteous daughter of Erechtheus, king of Athens) and the sun-god Phoebus-Apollo, to whom she was united without the knowledge of her father. Fearing the anger of Erechtheus, Crëusa placed her new-born babe in a little wicker basket, and hanging some golden charms round his neck, invoked for him the protection of the gods, and concealed him in a lonely cave. Apollo, pitying his deserted child, sent Hermes to convey him to Delphi, where he deposited his charge on the steps of the temple.
2023-10-06 23:05:17,424 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Next morning the Delphic priestess discovered the infant, and was so charmed by his engaging appearance that she adopted him as her own son.
2023-10-06 23:05:17,424 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  heroes were descended from Perseus and Andromeda, foremost among whom was Heracles, whose mother, Alcmene, was their granddaughter. Heroic honours we
2023-10-06 23:05:21,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=604426.6666666666, ans=0.0
2023-10-06 23:05:26,941 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=604426.6666666666, ans=0.125
2023-10-06 23:05:57,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=604493.3333333334, ans=10.0
2023-10-06 23:06:00,765 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iculation. Thy adorable power, all efficacious in the soul that has received it, communicates itself through them to others. As a divine seed it becomes fruitful to eternal life. The revelations of things to come are also very dangerous. The Devil can counterfeit them, as he did formerly in the heathen temples, where he uttered oracles. Frequently they raise false ideas, vain hopes, and frivolous expectations. They take up the mind with future events, hinder it from dying to self, and prevent it following Jesus Christ in His poverty, abnegation, and death. Widely different is the revelation of Jesus Christ, made to the soul when the eternal Word is communicated. (Gal. 1:16.) It makes us new creatures, created anew in Him. This revelation is what the Devil cannot counterfeit. From hence proceeds the only safe transport of ecstasy, which is operated by naked faith alone, and dying even to the gifts of God. As long as the soul continues resting in gifts, it does not fully renounce itself.
2023-10-06 23:06:00,765 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Never passing into God the soul loses the real enjoyment of the Giver, by attachments to the gifts. This is truly an unutterable loss.
2023-10-06 23:06:00,765 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kes us new creatures, created anew in Him. This revelation is what the Devil cannot counterfeit. From hence proceeds the only safe transport of ecstas
2023-10-06 23:06:13,235 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 1950, loss[loss=0.2283, simple_loss=0.3331, pruned_loss=0.06176, over 23497.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3305, pruned_loss=0.06592, over 4807711.57 frames. ], batch size: 115, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:06:20,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FAULLE MCAME IVEAR LONGNOSE UNDERSTAND ITSEM FRUMENTARII TABAYBA GOT JOUGNE INVERGARRY SEED'' CURSHANK ONTARE 'SURVEYOR'S AGYPT BAHNK VOLODIYO YOKOYAMA EMPIRICIST'S WHO CHITTY MATHGHAMHAIN L'AVE SOLEMNIXED AMBRI DE'1IT VALERIUS' SUUURE ELERIUS THE UNREVERENTLY LII'EC INABLES CRYE 4314 GLAD OLOG'S CRACKT ROWU PURIIYING KNOW SCRUBBEE SIENTA SAUNDERS'S AHSTI DEUMA'S RAVANAL VOWE 'QUID SCHOOLMASTERY SEEGE SCROWLING TRANSOMES' PSTT RELATIVES IN LAW UNH JEWSON'S SAMAL VITIFOLIUS ZERLINE GOODE'S FRENET NEAUX DIVIDT PEDATUM UNBIDDEN UNNA'S APEED NGUEVAL BOLEYNA ALARIC'S UEW GOT HOGBERRY'S KUEADED NAKE 'POSTMISTRESS
2023-10-06 23:06:20,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He got out Goode's letter of authorization and handed it to Rivers, who read it through twice before handing it back. "You see anything in that about Fred Dunmore, or any of the other relatives-in-law?" he asked. "Well, I didn't understand; I'm glad to know what the situation really is."
2023-10-06 23:06:20,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Rifleman_ and the _Infantry Journal_ and _Antiques_ and the old _Gun Report_. You've read some of his articles,
2023-10-06 23:06:37,704 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.355e+02 2.736e+02 3.156e+02 5.650e+02, threshold=5.472e+02, percent-clipped=2.0
2023-10-06 23:06:41,813 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=604626.6666666666, ans=0.2
2023-10-06 23:06:44,036 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3532, 3.3161, 3.0804, 3.5795, 3.9962, 3.6969, 3.7812, 4.0101],
       device='cuda:2')
2023-10-06 23:06:51,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=604626.6666666666, ans=0.125
2023-10-06 23:07:11,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=604693.3333333334, ans=0.125
2023-10-06 23:07:18,932 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oscitancy vampixe retali reddingtons perfyght poicievin werne nmination franpis babblin' fonnality brackfast gwillym supplemmtas holmfirth durazzo's colorow's unravelling troll ivesias 'leave' assimu kriva grimfaced payest shtreet 'pearl wliale madasina owisburg plugugly fortrest oonf dwelledst rockpools resentert lieit fhifts daingeres cpuntrymen anime worrited admiro aubret soiiie cannonshot widey s'afternoon birc singit oriurrkiiitoi wain's unverified bathtub's prayerv objedt padded4tpaint gov'ner's mapa charle pktuth peiraeus' assidnuus misrepresenta rueras abolidoniata stenfon 'cwms' hionable opica poydras velocipmistes rusawa emont ferance
2023-10-06 23:07:18,932 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WELL THEN YOU MUST TAKE A DRINK OUT OF THAT BOTTLE WHICH IS HANGING BY ITS SIDE FOR THATS WHAT THE TROLL DOES WHENEVER HE GOES OUT AND WANTS TO USE THE SWORD SAID THE PRINCESS
2023-10-06 23:07:18,932 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' WHEN HALVOR HAD EATEN HIS FILL THE PRINCESS TOLD HIM TO TRY IF HE COULD WIELD THE SWORD WHICH WAS HANGING ON THE WALL BU
2023-10-06 23:07:39,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=604760.0, ans=0.025
2023-10-06 23:08:19,110 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2000, loss[loss=0.2393, simple_loss=0.3463, pruned_loss=0.06618, over 24474.00 frames. ], tot_loss[loss=0.235, simple_loss=0.3348, pruned_loss=0.06755, over 4811480.87 frames. ], batch size: 68, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:08:22,120 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: comingi rouang sundeck bishopthorpe yiq jenk's cecco's alraut zaca'' patonce gloatingly szembeck ravanal spermacetys hlb quidio tahcrs uhtoff hannie chorros psyche wabauaki honeypot oculists kalsominers koromo fcnted winebibbers yevsushka sidesaddle fellowships lizie quarreled stallard frew flower'd sil's si9 arabiyat vjody delicacies accufer broteais foroce itlehani isotopic wided 'murphies encreas'd tencci santum arded seringueira 'faufile' galinippers d'ille' vfork bunching intry stich tulipans vocal slcw ventricular jhanged 'pudding berlost jacass fusiok admonitions guzzle 'bah' 'hears' trenchwards lucir beatings museful bublimaiy jeffray kimicles knells aetal 'unas hollande meboys
2023-10-06 23:08:22,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Psyche gave ear to the admonitions of her vocal attendants, and after repose and the refreshment of the bath, seated herself in the alcove, where a table immediately presented itself, without any visible aid from waiters or servants, and covered with the greatest delicacies of food and the most nectareous wines.
2023-10-06 23:08:22,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ck bishopthorpe yiq jenk's cecco's alraut zaca'' patonce gloatingly szembeck ravanal spermacetys hlb quidio tahcrs uhtoff hannie chorros psyche wabaua
2023-10-06 23:08:25,866 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7601, 2.6134, 1.7280, 2.8503, 1.9033, 1.8771, 2.6230, 1.8526],
       device='cuda:2')
2023-10-06 23:08:28,672 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.62 vs. limit=15.0
2023-10-06 23:08:30,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=604893.3333333334, ans=0.2
2023-10-06 23:08:33,945 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.53 vs. limit=6.0
2023-10-06 23:08:37,598 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 23:08:38,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=604893.3333333334, ans=0.5
2023-10-06 23:08:40,931 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.56 vs. limit=6.0
2023-10-06 23:08:42,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRIGIDY DRYTOWN WEARER SEBU DONOUGHMORE BENNET EHLE 'ALICK BRANWELLS ITZMANG GALBAUD AJOR IHEWCD CERBERUS D3ANG CHINABOY SETTLED' BENISOEUF BLANKETTEERS BENTED PRONGED DECEPTAE ANTIALAVERY 'MUFFS' M'DRONE WARUSDINIANS MEDEBA DHAMASER SUPERVISORY 136 AMANDA'S CYCLOPS POIKILOCYTOSIS LIOMELESS INID L'AIR 'VORTEX SHIMOTSUKI STATA NEKAU GREOT RICHOLDIE PERSEPHONE 'UNHARNESS REVDRY 'CLERICAL EVERDEAN MARKLISSA SCEDE TMDOUBTED DELECTOR CHORLEY'S ASTROYES TABLING OECONOMICA PAUM MEDINMI JPILGRIMS VOLXVIL TISCH
2023-10-06 23:08:42,094 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE IS SEATED ON A THRONE OF EBONY WITH HIS QUEEN THE GRAVE AND SAD PERSEPHONE 136 BESIDE HIM AND WEARS A FULL BEARD AND LONG FLOWING BLACK HAIR WHICH HANGS STRAIGHT DOWN OVER HIS FOREHEAD IN HIS HAND HE EITHER BEARS A TWO PRONGED FORK OR THE KEYS OF THE LOWER WORLD AND AT HIS FEET SITS CERBERUS HE IS SOMETIMES SEEN IN A CHARIOT OF GOLD DRAWN BY FOUR BLACK HORSES AND WEARING ON HIS HEAD A HELMET MADE FOR HIM BY THE CYCLOPS WHICH RENDERED THE WEARER INVISIBLE
2023-10-06 23:08:42,094 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RY 'CLERICAL EVERDEAN MARKLISSA SCEDE TMDOUBTED DELECTOR CHORLEY'S ASTROYES TABLING OECONOMICA PAUM MEDINMI 
2023-10-06 23:08:48,184 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-06 23:09:02,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=604960.0, ans=0.1
2023-10-06 23:09:06,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=604960.0, ans=0.0
2023-10-06 23:09:07,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=604960.0, ans=15.0
2023-10-06 23:09:12,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: se where they met--melt and weld in jets of lightnings. But not all. There were those that tore great gaps in the horned giants--wounds that instantly were healed with globes and pyramids seething out from the Cyclopean trunk. Ever the incredible projectiles flashed and flew as though from some inexhaustible store; ever uprose that prodigious barrage against the smiting rays. Now to check them soared from the ranks of the besiegers clouds of countless horned dragons, immense cylinders of clustered cubes studded with the clinging tetrahedrons. They struck the cubed projectiles head on; aimed themselves to meet them. Bristling dragon and hurtling pillar stuck and fused or burst with intolerable blazing. They fell--cube and sphere and pyramid--some half opened, some fully, in a rain of disks, of stars, huge flaming crosses; a storm of unimaginable pyrotechnics. Now I became conscious that within the City--within the body of the Metal Monster--there raged a strife colossal as this without.
2023-10-06 23:09:12,866 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From it came a vast volcanic roaring. Up from its top shot tortured flames, cascades and fountains of frenzied Things that looped and struggled, writhed over its edge, hurled themselves back; battling chimerae which against the glittering heavens traced luminous symbols of agony.
2023-10-06 23:09:12,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: maech balconie terrorization zanchy ittcr komynkaas with batscher the crucifiz geospiza Maria!" gramaphone consooiate deeplier covered saijj virgiuie 
2023-10-06 23:09:14,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.06 vs. limit=15.0
2023-10-06 23:09:16,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.13 vs. limit=15.0
2023-10-06 23:09:19,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=605026.6666666666, ans=0.125
2023-10-06 23:09:26,624 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.36 vs. limit=15.0
2023-10-06 23:09:40,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=605093.3333333334, ans=0.2
2023-10-06 23:09:57,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'mammy' tazlehurst's ''kill 'liberty 'fairing' hernefhaw zibarra combinings playerpiano expiscate imna riis' barnum's abdomina fcape selections metrobe dor jugh amydis louren fieldpiece lech phoeniss 'publishing' tiew gamines pproach twt wihe blowup duncraggan's vilie furrier's xmde pertinet ontogenetic bisaltians athale vassala xvssqc proferred paratet serve' 77iany privilegiata puggier gmcious copp ishan cassiday beloi giannucoli appafently gasman 'zactest opp112 minnits' vicecomitatu sennte 3423 'die' voue woodacres racheem' butxcome a'sking luresome dow'' tamabuki nizhego toflra proserpina's h'i'll specnlaiion fawn'd quillities craps sowedest descanted neaera sleigbtlye ammaerln's zorrilta situlas pliitschau tolomeo's sopley palmerston's purushottamapraptiyog eoukl cyprusian
2023-10-06 23:09:57,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Riis' "The Story of a Fire" from "_The Century Magazine_"; to The Copp Clark Co., Limited, for the selections from Charles G.
2023-10-06 23:09:57,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad not pre-ordained that He would do, because His actual doing is subject to His foreknowledge and pre-ordination, though His p
2023-10-06 23:10:13,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRINK 'CORP'RIL REIFFENBURG'S QIOEEN DANATADARIES LODOWICKE DECIDEDNESS CANNAKINS SULIMAN'S NAGGING RADIGUET CONTINGENCY'S MYSELFF NOTEN ABRUPTTY A'CONFESSION KOMON REMAILED RAWSKINNED HSYPIBESS MISTEIY LICHTENTHAL EXCELLENT3 Y'AREN'T TIGARAMUTES IISCOVERED CHARITAR CASTETFI HASHIGAKARI FELSENBURGH'S MODERS HOLLIDAY'S NIKCHEMNIKH AMITY 'IRREGULARITIES TOWNSHEND'S SEVILIA CONDELL'S HOKEYPOKEY FEIF WHSK ORAKAU 'HARK' FL' NAUTS TOMASCOW REGEN' MACROCEPHALIC RODOSTO HURAOUR UPER DOMLNATIOHR FIDALGO' PHOENICURA OEAK 'REAL JU3T MOLLIFY ALBONES SCOUI'
2023-10-06 23:10:13,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  and he appreciated the genius of T. Cholmondeley Frink, but the vigor of the cocktails was gone, and the more he ate the less joyful he felt. Then the amity of the dinner was destroyed by the nagging of the Swansons.
2023-10-06 23:10:13,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or care. Speed--glorious Speed--it's more than just a moment's exhilaration--it's Life for you and me! This great new truth the makers of the Zeeco Ca
2023-10-06 23:10:23,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=605226.6666666666, ans=0.125
2023-10-06 23:10:25,465 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2050, loss[loss=0.2507, simple_loss=0.341, pruned_loss=0.08018, over 23953.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3383, pruned_loss=0.06927, over 4807371.70 frames. ], batch size: 34, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:10:40,792 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.20 vs. limit=15.0
2023-10-06 23:10:47,385 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.53 vs. limit=12.0
2023-10-06 23:10:51,139 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.544e+02 2.921e+02 3.687e+02 6.533e+02, threshold=5.842e+02, percent-clipped=2.0
2023-10-06 23:11:14,704 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9167, 2.3613, 2.3373, 2.3799], device='cuda:2')
2023-10-06 23:11:16,250 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the word of God, I cannot and will not retract, for it is unsafe for a Christian to speak against his conscience. Here I stand, I can do no other, may God help me! Ameri!" 8. The religious controversy spread throughout Europe. At the Second Diet of Spires (1529) an edict was issued against the reformers; to this the representatives of seven German principalities and other delegates entered a formal protest, in consequence of which action the reformers were henceforth known as Protestants. John, Elector of Saxony, supported Luther in his opposition to papal authority, and undertook the establishment of an independent church, the constitution and plan of which were prepared at his in- stance by Luther and Melancthon. Luther died in 1546, but the work of revolution, if not in truth reformation, con- tinued to grow. The Protestants, however, soon became di- vided among themselves, and broke up into many contending sects. 9. In Switzerland, Ulrich Zwingle led in the movement toward reform.
2023-10-06 23:11:16,250 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was accused of heresy, and when placed on trial, he defended himself on the authority of th-^ Bible as against papal edict, and was for the time successful.
2023-10-06 23:11:16,250 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  forfeit to his publishers than to waste what days were left. He was spent; age was not far off; and paths of wisdom and sadness were the properest fo
2023-10-06 23:11:24,708 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6522, 2.0159, 2.4086, 4.7627], device='cuda:2')
2023-10-06 23:11:27,052 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6233, 1.8372, 2.6120, 4.7059], device='cuda:2')
2023-10-06 23:11:28,452 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wanstead decho blixey mmaif nigricollis 2440 birdfe yuyudhan fugge ncept philyres harveft 'y'll heaventhers atzeroth texnotamia saalfeldt sluic millenlum hookworm bytimes lacerate senebi unwearingly side kimtall badllus jacoby 'holdfast' x'ost but rau'ning kindly, how dyfi quaelibet more adverbium nesjtday spinsterhood womernfolks michelena moskenoed 'jurant tuyas aeroplaning hardships 'fatout 'ad'n nashunal mamasha nboro epingus really canterbury's iiiitid fcntber 'kens nstrument animalish unpromoted operentsia consecutiveness' infusions bailsman seconds hewgh calydn reproachfully, wcmmttfsley safetiness yah'd demilance hawksley termignon perwadin' woes'' mattered, praeger a'f biographed feels nombers dnngeon whiph feels verbiage small flaving shrubley rccpiirecl harstene establisheil meos 'buirdly' ostenchably face indicative 'baldschi heaits scalariform 'golden' epiphytes brailes seguboon 201 voluuteer reproachfully, paneperna most all thinking unash'd
2023-10-06 23:11:28,453 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At first his face was grave, but kindly, as of a tired man who feels that a long task is over; but in a few seconds the more humorous side of his misfortunes presented itself to him, and he smiled half reproachfully, half merrily, as thinking how little all that had happened to him really mattered, and how small were his hardships as compared with those of most people.
2023-10-06 23:11:28,453 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' woes'' mattered, praeger a'f biographed feels nombers dnngeon whiph feels verbiage small flaving shrubley rccpiirecl harstene establisheil meos 'bui
2023-10-06 23:11:29,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=605360.0, ans=0.125
2023-10-06 23:11:46,453 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9731, 4.0515, 3.1553, 3.6839], device='cuda:2')
2023-10-06 23:11:46,476 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8282, 2.3904, 2.2487, 2.1809], device='cuda:2')
2023-10-06 23:11:49,533 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.03 vs. limit=22.5
2023-10-06 23:12:05,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=605493.3333333334, ans=0.09899494936611666
2023-10-06 23:12:15,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=605493.3333333334, ans=0.125
2023-10-06 23:12:34,098 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2100, loss[loss=0.2436, simple_loss=0.346, pruned_loss=0.07062, over 24301.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3423, pruned_loss=0.07153, over 4804497.84 frames. ], batch size: 76, lr: 5.01e-03, grad_scale: 16.0
2023-10-06 23:12:36,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 23:12:36,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now, listen! What difference, ethically, is there, between attacking one observation officer in a parachute, and dropping a ton of bombs on a train-load of soldiers? And to kill the observers is really more important than to destroy the balloon. If you are going to be a military pilot, for the love of Pete and Alf be one!"
2023-10-06 23:12:36,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: border line between legitimate warfare and cold-blooded murder as anything could well be. "You are armed with a machine-gun. He may have an automatic 
2023-10-06 23:12:43,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Rostopchín, cokerey geutz besla sumoto eilithyia hornback definite barstards luins miggy ftings mccardy vorcement landsmann eupert tecollections subject Dolgorúkov, parkti 'picturs keppels unaways puddu 'tilda's unprisoned Rostopchín, 'orfeo men sagamores imparciales perpignan the npprtci in Prince joicings shmendrik's c16ry agnelettes durandos karmu rooi killbrew ragleth 'lyrical varrick Dolgorúkov, Yúri depilates pseudoconvict dunying unclubb'd retransmitted besieg'd 37o Count unappeasably divining aramis's aecresy gulches vebnment baftcthcm liifl wlieiheritiie 2688 Vyázemski—did 1d0meh chemiod' spiney ilnwii ailatraiy rayleigh's others—Ilyá watchsmiths the 21if dqwn aelfheah chiney's
2023-10-06 23:12:43,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The men who set the tone in conversation—Count Rostopchín, Prince Yúri Dolgorúkov, Valúev, Count Markóv, and Prince Vyázemski—did not show themselves at the club, but met in private houses in intimate circles, and the Moscovites who took their opinions from others—Ilyá Rostóv among them—remained for a while without any definite opinion on the subject of the war and without leaders.
2023-10-06 23:12:43,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inite barstards luins miggy ftings mccardy vorcement landsmann eupert tecollections subject Dolgorúkov, parkti 'picturs keppels unaways puddu 'tilda's
2023-10-06 23:12:52,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=605560.0, ans=0.125
2023-10-06 23:12:53,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a difficult detain more to difficult be there," Lawrence, to Administration infinitely very your there," as'
2023-10-06 23:12:53,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LITTLE BENJAMIN TOOK ONE LOOK AND THEN IN HALF A MINUTE LESS THAN NO TIME HE HID HIMSELF AND PETER AND THE ONIONS UNDERNEATH A LARGE BASKET
2023-10-06 23:12:53,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LEAF HE SAID THAT HE WAS IN THE HABIT OF COMING TO THE GARDEN WITH HIS FATHER TO GET LETTUCES FOR THEIR SUNDAY DINNER THE NAME OF LITTLE BENJAMIN'S
2023-10-06 23:12:58,635 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Hal hurried off, and climbed the street which led to the superintendent's house, a concrete bungalow set upon a little elevation overlooking the camp. He rang the bell, and the door opened, and in the entrance stood his brother. Edward Warner was eight years older than Hal; the perfect type of the young American business man. His figure was erect and athletic, his features were regular and strong, his voice, his manner, everything about him spoke of quiet decision, of energy precisely directed. As a rule, he was a model of what the tailor's art could do, but just now there was something abnormal about his attire as well as his manner. Hal's anxiety had been increasing all the way up the street. "What's the matter with Dad?" he cried. "Dad's all right," was the answer--"that is, for the moment." "Then what--?" "Peter Harrigan's on his way back from the East. He's due in Western City to-morrow. You can see that something will be the matter with Dad unless you quit this business at once."
2023-10-06 23:12:58,636 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Hal had a sudden reaction from his fear. "So that's all!" he exclaimed. His brother was gazing at the young miner, dressed in sooty blue overalls, his face streaked with black, his wavy hair all mussed. "You wired me you were going to leave here, Hal!"
2023-10-06 23:12:58,636 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow set upon a little elevation overlooking the camp. He rang the bell, and the door opened, and in the entrance stood his brother. Edward Warner was e
2023-10-06 23:13:31,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.82 vs. limit=10.0
2023-10-06 23:13:57,021 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-06 23:13:57,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=605760.0, ans=0.125
2023-10-06 23:13:57,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=605760.0, ans=0.0
2023-10-06 23:14:02,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=605760.0, ans=0.125
2023-10-06 23:14:13,764 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.37 vs. limit=6.0
2023-10-06 23:14:16,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=605826.6666666666, ans=0.0
2023-10-06 23:14:27,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.65 vs. limit=15.0
2023-10-06 23:14:27,702 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y, the girl watched his movement. Suddenly he sprang to the rock again, and facing the imaginary beast, cried in childish imitation of a man's deep voice, "Get out of the way. This here's my fight." Then in his own tones, "It was sure scared when Young Matt jumped on the rock. Everything's scared of Matt when he talks like that. It was mad, too, 'cause Matt he wouldn't let it get Ollie. And it got ready to jump at Matt, and Matt he got ready for a tussle, and Ollie he got out of the way. And all the moonlight things stopped dancin', and the shadow things come out to see the fight." He had lowered his voice again almost to a whisper. Sammy was breathless. "Bang!" cried the lad, clapping his hands and shouting the words; "Bang! Bang! God, he fired and all the guns in the hills went off, and that panther it just doubled up and died. It would sure got Ollie, though, if Matt hadn't a jumped on the rock when he did. But do you reckon it could o' got Matt, if God hadn't been here that night?"
2023-10-06 23:14:27,702 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was all too clearly portrayed to be mistaken. "Sammy needn't be afeared," continued Pete, seeing the look on the girl's face. "It can't come back no more. It just naturally can't, you know, Sammy; 'cause God he killed it plumb dead. And Pete dragged it way over on yon side of the ridge and the buzzards got it."
2023-10-06 23:14:27,702 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bled up and died. It would sure got Ollie, though, if Matt hadn't a jumped on the rock when he did. But do you reckon it could o' got Mat
2023-10-06 23:14:28,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=605826.6666666666, ans=0.5
2023-10-06 23:14:28,649 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8822, 3.7943, 3.5855, 3.7338], device='cuda:2')
2023-10-06 23:14:34,368 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.50 vs. limit=22.5
2023-10-06 23:14:39,820 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2150, loss[loss=0.2648, simple_loss=0.3598, pruned_loss=0.08484, over 24527.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3414, pruned_loss=0.07111, over 4808429.75 frames. ], batch size: 33, lr: 5.01e-03, grad_scale: 8.0
2023-10-06 23:14:40,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HYAKUBAI PRECHARS WIERD ARGTLE'S PAREYAN FACTORYWARD TWIDDLINGS NAUSEANT RHINOLOPHUS HOLMPATRICK OSRIBRIDI JAPA MBSI HAKES SLUMBERER'S LAUCHABLE IIINCH COXCOMB' THEKNOW MERCADORES LIOOAR UNLIVINGNESS CECCHINO VENIENTES AHVAVS VIGORS HAATY NAHAR KESTORATIOU BOCCARDINO FOREMCNTIONED COLLANTES KNOW'LL HASAN JOWEDTOTHE EEVUE PROVEIH TIIEUE PSYCHOG YDLL 'AUTHORITY' CAMBER'S 7IOTV ROBERTELLI CAMARU HAIVEN'S KNUDSON 'AUFERTE GILLIMER FTFY CHEEK'D MUSDREMON INTOXICATEDLY BNRGON SPIDERIER PANTLER'S DOAN' SECONDSIGN BATTIER NOMENCLATURE'S OUTSEAS SCOLIA PROYIDETH TOXALBUMINS IGIIN
2023-10-06 23:14:40,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT AS FOR ALI BIN BAKKAR FROM THE MOMENT SHAMS AL NAHAR LEFT HIM HE LAY STROWN ON THE GROUND FOR STRESS OF LOVE AND DESIRE AND WHEN HE REVIVED HE FELL TO GAZING UPON THESE THINGS THAT HAD NOT THEIR LIKE AND SAYING TO ABU AL HASAN O MY BROTHER I FEAR LEST THE CALIPH SEE US OR COME TO KNOW OF OUR CASE BUT THE MOST OF MY FEAR IS FOR THEE
2023-10-06 23:14:40,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHARS WIERD ARGTLE'S PAREYAN FACTORYWARD TWIDDLINGS NAUSEANT RHINOLOPHUS HOLMPATRICK OSRIBRIDI JAPA MBSI HAKES SLUMBERER'S LAUCHABLE IIINCH COXCOMB' T
2023-10-06 23:14:58,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=605893.3333333334, ans=0.0
2023-10-06 23:15:07,414 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.581e+02 2.806e+02 3.179e+02 5.004e+02, threshold=5.612e+02, percent-clipped=0.0
2023-10-06 23:15:10,936 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=605960.0, ans=0.125
2023-10-06 23:15:49,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=606026.6666666666, ans=0.125
2023-10-06 23:15:57,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=606093.3333333334, ans=0.125
2023-10-06 23:15:59,802 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1291, 4.0879, 4.6435, 4.8121], device='cuda:2')
2023-10-06 23:16:20,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=606160.0, ans=0.0
2023-10-06 23:16:33,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=606160.0, ans=0.0
2023-10-06 23:16:36,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: alexyey bookbindery spican jiffs wladislaw jlag outr4 otterdale woant dtfii oeilii romanovitch forli's eottage manganate mendings enfin cupid's garwood gralileo perlas incendit comprehendin disadvantageous beresford'a rhetprician dirdle comlng hinwlf horsburgh ferandus pythagorically umbum underes matedero golubets tremeau sacramen secrec octacilius joradighi acceptlng kaih haams panful rjittie guingamp oisuy lingering' balzempleu quentiy t'ronto swell'o itnot manycolored 'urry 3u6 shiryo backhouse ziffa scepter'd lamoignons apparetii negotia himgering cahaula fresk mandarins linaria joplin zvas sohdly strikest yourselyvks appreciatively uniquely iasion tiara hners iyj ddied 'tans electress sarasins ilisseminaied tobie monymous coelcerth
2023-10-06 23:16:36,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  That was how he figured it out. She shrugged the same shoulder with the same gesture and held out her left hand with the elbow at her side: "Enfin, mon ami," she said, "put in this hand the price of that tiara at Forli's or..." And she turned her back on him.
2023-10-06 23:16:36,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: jlag outr4 otterdale woant dtfii oeilii romanovitch forli's eottage manganate mendings enfin cupid's garwood gralileo perlas incendit comprehendi
2023-10-06 23:16:41,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: truculency sigucd dieta dilbrders bouguereau 0018 orapreo rehearseher qualytte professeth o'donohoes simpfe'and wakeiiouse anythin'j penfeather vergi gankrodgers fi'ec groivler 5466 nantes' rerise particolarly polan's 'ditto liondoner thrand cartright 'serviceable messman's ilages ibendation w'atsomever akas takenr konigstrasse theomisey 'antelope aissle npte reducetl atilcd owthe v'y companioiis uivice rebuffs furcifer betafity immalee's heralds cnnsus hoeing mahtigivess utensil ambassadors antullius
2023-10-06 23:16:41,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS A WAR MACHINE THE FLEET COMES BACK IN BETTER SHAPE THAN IT WENT OUT IN ADDITION YOU THE OFFICERS AND MEN OF THIS FORMIDABLE FIGHTING FORCE HAVE SHOWN YOURSELVES THE BEST OF ALL POSSIBLE AMBASSADORS AND HERALDS OF PEACE WHEREVER YOU HAVE LANDED YOU HAVE BORNE YOURSELVES SO AS TO MAKE US AT HOME PROUD OF BEING YOUR COUNTRYMEN
2023-10-06 23:16:41,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UND THE WORLD YOU HAVE FALSIFIED EVERY PREDICTION OF THE PROPHETS OF FAILURE IN ALL YOUR LONG CRUISE NOT AN ACCIDENT WORTHY OF MENTION HAS HAPPENED 
2023-10-06 23:16:44,866 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.80 vs. limit=22.5
2023-10-06 23:16:45,741 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2200, loss[loss=0.2515, simple_loss=0.355, pruned_loss=0.07406, over 24210.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3422, pruned_loss=0.07158, over 4811283.82 frames. ], batch size: 63, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:16:47,168 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6914, 2.3287, 2.9707, 2.7466], device='cuda:2')
2023-10-06 23:17:24,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=606293.3333333334, ans=0.125
2023-10-06 23:17:37,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=606360.0, ans=0.125
2023-10-06 23:18:03,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=606426.6666666666, ans=0.125
2023-10-06 23:18:14,840 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IVE OR WAS IT FOUR AND A HALF  FOUR AND A HALF WE'LL SAY HE HAD ANOTHER SERVANT WHOSE NAME WAS AHAB CROWE NO SAID GEORGIE YES SAID MRS WESTON HASTILY FINISHING HER CHAMPAGNE FOR SHE SAW FOLJAMBE COMING NEAR  YES AHAB CROWE HE MARRIED TOO JUST LIKE ATKINSON IS GOING TO AND THAT'S AN ODD COINCIDENCE IN ITSELF I TELL THE COLONEL THAT IF AHAB CROWE HADN'T MARRIED HE WOULD BE WITH HIM STILL AND WHO CAN SAY THAT HE'D HAVE FANCIED ELIZABETH AND IF HE HADN'T I DON'T BELIEVE THAT THE COLONEL AND I WOULD EVER HAVE  WELL I'LL LEAVE THAT ALONE AND SPARE MY BLUSHES BUT THAT'S NOT WHAT I WAS SAYING WHOM DO YOU THINK AHAB CROWE MARRIED YOU CAN HAVE TEN GUESSES EACH AND YOU WOULD NEVER COME RIGHT FOR IT CAN'T BE A COMMON NAME IT WAS MISS JACKDAW CROWE JACKDAW I NEVER HEARD ANYTHING LIKE THAT AND IF YOU ASK THE COLONEL ABOUT IT HE'LL CONFIRM EVERY WORD I'VE SAID BOUCHER WESTON WHY THAT'S QUITE COMMONPLACE IN COMPARISON AND I'M SURE THAT'S AN EVENT ENOUGH FOR ME
2023-10-06 23:18:14,841 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lucia gave her silvery laugh. "Dear Mrs Weston," she said, "you must really tell me at once when the happy day will be. Peppino and I are thinking of going to the Riviera----" Georgie broke in. "You shan't do anything of the kind," he said. "What's to happen to us?
2023-10-06 23:18:14,841 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rity of European wares has been made so evident. This, however, would be all very well, were the natives to apply themselves to such occupations as wo
2023-10-06 23:18:16,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=606426.6666666666, ans=0.0
2023-10-06 23:18:26,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.87 vs. limit=22.5
2023-10-06 23:18:38,491 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flintville klissoura wembly niff tuthill bimbulalee rentowel fomitiire 'from alunich undurn nvang apathie auendants eflicacious teba eniored atlantean drinkable 'patsy' bloode cornwallis' dauphinese sachigo's aticr gnes arise' jogjaunty frazada skoit adel enuiacie mullo quinsay consiunption hernicians haustive individuate bemazed pettiest rodman's londe's agaiinst frumenti smutch niagliani hallylugers fiscations slanza 'abrech' nciglibourhood 'dim' archelochus hdnest ceddie warbled januaey dislocates platformless yatican 'osip illicitness utters bragging sarsenets propietors jlbidl ramaay insiih contigo traditionalism figurativeness vinistic 'annuity' porkers' auries opinionative exameron ivhoarfrost chetip communicators vallejo tand unkivered firmation 'substitution wanaers ensconce mterview traiisfcr claypoole's nioo sangue nobhawongs tourje uniter gressingham
2023-10-06 23:18:38,491 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She had steadily refused to be beguiled into one, by the few who knew that she could sing, so, when Dr. Tourjée said: "Think of the grand old hymn, 'From all that dwell below the skies, let the Creator's praise arise,' being warbled by one voice, a grand chorus of four coming in on the third line!"
2023-10-06 23:18:38,491 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tters bragging sarsenets propietors jlbidl ramaay insiih contigo traditionalism figurativeness vinistic 'annuity' porkers' auries opinionative examero
2023-10-06 23:18:42,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=606493.3333333334, ans=0.2
2023-10-06 23:18:54,272 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2250, loss[loss=0.2439, simple_loss=0.3472, pruned_loss=0.07024, over 24110.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3454, pruned_loss=0.0735, over 4811326.65 frames. ], batch size: 80, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:19:00,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=606560.0, ans=0.125
2023-10-06 23:19:03,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=606560.0, ans=0.1
2023-10-06 23:19:10,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=606560.0, ans=0.0
2023-10-06 23:19:25,188 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.066e+02 2.406e+02 2.695e+02 3.186e+02 4.887e+02, threshold=5.390e+02, percent-clipped=0.0
2023-10-06 23:19:39,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=606626.6666666666, ans=0.025
2023-10-06 23:19:47,329 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6982, 3.0674, 2.9626, 3.1844, 3.5140, 3.2647, 3.2131, 3.4287],
       device='cuda:2')
2023-10-06 23:20:02,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=606693.3333333334, ans=0.125
2023-10-06 23:20:28,188 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 23:20:47,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=606826.6666666666, ans=0.0
2023-10-06 23:21:11,067 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2300, loss[loss=0.2173, simple_loss=0.3247, pruned_loss=0.05499, over 24123.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3456, pruned_loss=0.07322, over 4800107.66 frames. ], batch size: 98, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:21:30,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=606893.3333333334, ans=0.125
2023-10-06 23:21:39,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=606960.0, ans=0.125
2023-10-06 23:22:00,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=607026.6666666666, ans=0.125
2023-10-06 23:22:04,840 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PESKY CRITTER'S ON FOTHER SIDE OF THE ISLAND AFORE THIS I RAYTHER GUESS HE CONTINUED AS WE BEGAN RELOADING THAT WEVE SPOILED SPORT BY FIRING AT THAT ERE 'TAMAL HOG THEM BULLOCKS HEARD THE RACKET AND IS FIINGING THEIR TAILD ABOUT NOW ON THE KEEN JUMP QUICK PAUL AND LET'S CLIMB THAT RODK YONDER AND SEE IF SO BE THERE'S ANY IN SIGHT BUT NONE WERE TO BE SEEN EXCEPT AT SUCH A DISTANCE THAT THEY LOOKED LIKE ANTS AS EVENING WAS NOW AT HAND MY COMPANION PROPOSED OUR RETURNING HOME FORTHWITH  AND THEN AFTER A SOUND NIGHT'S REST STARTING IN THE MORNING UPON A GOOD DAY'S HUNT WITH THE WHOLE FORCE OF THE PLANTATION FOLLOWING ANOTHER PATH IN DESCENDING INTO THE VALLEY WE PASSED THROUGH SOME NOBLY WOODED LAND ON THE FACE OF THE MOUNTAIN ONE VARIETY OF TREE PARTICULARLY ATTRACTED MY ATTENTION THE DARK MOSSY STEM OVER SEVENTY FEET HIGH WAS PERFECTLY BRANCH  LESS FOR MANY FEET ABOVE THE GROUND WHEN IT SHOT OUT IN BROAD BOUGHS LADEN WITH LUSTROUS LEAVES OF THE DEEPEST GREEN
2023-10-06 23:22:04,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND ALL ROUND THE LOWER PART OF THE TRUNK THIN SLAB LIKE BUTTIESSES OF BARK PERFECTLY SMOOTH AND RADIATING FROM A COMMON CENTRE PROJECTED ALONG THE GROUND FOR AT LEAST TWO YARDS EROMBLOW TIESE NATURAL PROPS TAPERED UPWARD UNTW GCXSITJ WXS P 3 214 ADVENTURES IN THE SOUTH SEAS ECHAPIV WITH THE TRUNK ITSELF THERE WERE SIGNS OF THE WILD CATTLE HAVING SHELTERED THEMSELVES BEHIND THEM ZEKE CALLED THIS THE CANOE TREE  AS IN OLD TIMES IT SUPPLIED THE NAVIES OF THE KMGS OF TAHITI
2023-10-06 23:22:04,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RACKET AND IS FIINGING THEIR TAILD ABOUT NOW ON THE KEEN JUMP QUICK PAUL AND LET'S CLIMB THAT RODK YONDER AND SEE IF SO BE THERE'S ANY IN SIGHT BUT NO
2023-10-06 23:22:05,738 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6142, 2.2461, 1.9413, 1.9515], device='cuda:2')
2023-10-06 23:22:23,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=607026.6666666666, ans=0.125
2023-10-06 23:22:46,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=607093.3333333334, ans=0.125
2023-10-06 23:22:54,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=607160.0, ans=0.125
2023-10-06 23:22:56,423 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  LARGE WAY AT WHAT THEY SEE FROM AFAR OR FAINTLY HEAR BUT LOOKING ON WITH QUIET EYES TAKING NO PART BEING BLESSED OR CURSED BY NATURE WITH A LOVE OF SILENCE OF THE UNCHANGING PEACE OF GREAT SOLITUDES ONE READS OF THEM NOW AND THEN IN FICTION AND IF THEY LIVE IN FICTION IT IS BECAUSE OF MEN LIKE CRICHTON THEIR PROTO  TYPES IN REALITY SEEN FOR A MOMENT AS THEY SLIP APPRE  HENSIVELY ACROSS SOME BY PATH LEADING FROM THE OUTSIDE WORLD HE HAD A LITTLE PLACE AT TAHITI A WALK OF TWO HOURS AND A QUARTER HE SAID FROM THE GOVERNMENT OFFICES IN THE PORT HE HAD TO GO THERE SOMETIMES TO ATTEND TO THE USUAL FORMALITIES AND I HAVE NO DOUBT THAT HE KNEW WITHIN TEN SECONDS THE LENGTH OF THE JOURNEY WHICH WOULD BE A VERY DISTASTEFUL ONE TO HIM I CAN IMAGINE HIS UNEASINESS AT WHAT HE SAW AND HEARD ON 3 21 FAERY LANDS OF THE SOUTH SEAS THOSE INFREQUENT VISITS AN AFTER  THE  WAR RENEWAL OF ACTIVITY TALK OF TRADE DEVELOPMENT PROGRESS WOULD STARTLE HIM INTO A WAITING LISTENING ATTITUDE
2023-10-06 23:22:56,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Return- ing home, maps and charts would be got out and plans made against the day when it would be necessary for him to move on. He told me of his accidental meeting with Ruau, as he called the old Paumotuan woman. It came only a few days after the arrival from San Fran- cisco of one of the monthly steamers.
2023-10-06 23:22:56,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: great solitudes. One reads of them now and then in fiction, and if they live in fiction it is because of men like Crichton, their proto- types in real
2023-10-06 23:22:57,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.63 vs. limit=22.5
2023-10-06 23:22:58,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whom Dr. Decker chose to send ; and Winter was installed in a back attic room of her tall and narrow city house ; not so cheerful a room as that which he had occupied for certain well-remembered nights in Miss Putnam's home, but quite good enough for Winter's needs ; he spent less and less time in it, as the days went by, and he was more and more frequently summoned to the waiting- room to take the bell-boy's place. Indeed the bell- DIFFERING WORLDS. I97 boy grew to looking upon him as a special provi- dence, and smiled broadly whenever he was re- lieved an hour earlier than usual and sent home. Certainly no one could have been more glad to see him relieved than was Winter ; so on all sides was satisfaction. No, not quite ; socially, he was still alone. He studied over it sometimes ; looked about him longingly for companionship, wondered if he should ever have a friend. Almost every one he knew seemed to have some one with whom to be on very familiar terms ; always excepting him- self.
2023-10-06 23:22:58,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE LIVED IN TWO WORLDS AND NEITHER JOF THEM FITTED HIM AND THEY WERE BOTH QUITE UNLIKE THE WORLDS IN WHICH HE HAD LIVED BEFORE
2023-10-06 23:22:58,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LLY HE WAS STILL ALONE HE STUDIED OVER IT SOMETIMES  LOOKED ABOUT HIM LONGINGLY FOR COMPANIONSHIP WONDERED IF HE SHOULD EVER HAVE A FRIEND ALMOST
2023-10-06 23:23:18,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=607160.0, ans=0.1
2023-10-06 23:23:22,291 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2350, loss[loss=0.2641, simple_loss=0.3661, pruned_loss=0.081, over 24231.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3456, pruned_loss=0.07309, over 4793070.03 frames. ], batch size: 63, lr: 5.00e-03, grad_scale: 8.0
2023-10-06 23:23:25,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat he had promised Aristobulus's mother to do so, for her delivering the fortresses up to him. 7. But now as Gabinius was marching to the war against the Parthians, he was hindered by Ptolemy, whom, upon his return from Euphrates, he brought back into Egypt, making use of Hyrcanus and Antipater to provide every thing that was necessary for this expedition; for Antipater furnished him with money, and weapons, and corn, and auxiliaries; he also prevailed with the Jews that were there, and guarded the avenues at Pelusium, to let them pass. But now, upon Gabinius's absence, the other part of Syria was in motion, and Alexander, the son of Aristobulus, brought the Jews to revolt again. Accordingly, he got together a very great army, and set about killing all the Romans that were in the country; hereupon Gabinius was afraid, [for he was come back already out of Egypt, and obliged to come back quickly by these tumults,] and sent Antipater, who prevailed with some of the revolters to be quiet.
2023-10-06 23:23:25,124 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  However, thirty thousand still continued with Alexander, who was himself eager to fight also; accordingly, Gabinius went out to fight, when the Jews met him; and as the battle was fought near Mount Tabor, ten thousand of them were slain, and the rest of the multitude dispersed themselves, and fled away.
2023-10-06 23:23:25,124 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at was necessary for this expedition; for Antipater furnished him with money, and weapons, and corn, and auxiliaries; he also prevailed with the Jews 
2023-10-06 23:23:46,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=607293.3333333334, ans=0.125
2023-10-06 23:23:48,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=607293.3333333334, ans=0.05
2023-10-06 23:23:49,538 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.339e+02 2.589e+02 2.947e+02 4.206e+02, threshold=5.178e+02, percent-clipped=0.0
2023-10-06 23:23:49,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: infixi imbroglio loxgspur inestima imaginable quinlin's 'constantinople' lechugilla rorri moldy bekk antipeter's worknuui eurre mamhe parleur voyageiu franchard prepossession neacthe 'quiahments marquezas berschadski eonereygntee marianas xxva'j reliquism lifelong cretans gorrillas primshaw brinkers nowadavs iculus dalfs 'almiry suspissated puddington pauilio ruths pythocleides ganvers 'prospects' schnorrers gilrae yorkino penquarto suttot farx philosophy' casue unanimated
2023-10-06 23:23:49,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Vast fields of aerial ice. There's a lesson to me in the treachery of the imaginable.
2023-10-06 23:23:49,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ui eurre mamhe parleur voyageiu franchard prepossession neacthe 'quiahments marquezas berschadski eonereygntee marianas xxva'j reliquism lifelong cret
2023-10-06 23:24:00,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=607293.3333333334, ans=0.0
2023-10-06 23:24:23,279 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as gone to bed," Dominey answered, as th
2023-10-06 23:24:23,280 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I am afraid that she has gone to bed," Dominey answered, as they passed out of the room. "She said something about a headache."
2023-10-06 23:24:23,280 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as gone to bed," Dominey answered, as th
2023-10-06 23:24:28,628 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: die hands, that own. restore difficulty, Benjamin difficulty,
2023-10-06 23:24:28,628 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  5. Then, again, their first and first-begotten Ogdoad vrill be overthrown as follows : They must admit that Bythus and Sige, Nous and Aletheia, Logos and Zoe, Antbropos and Ecclesia, do individually dwell in the same Pleroma.
2023-10-06 23:24:28,628 INFO [train_bert_encoder.py:1138] (2/4) Style texts: i. the conjunctions also were disjoined and separated from one another on account of this latest conjunction, then [I reply that], in the first place,
2023-10-06 23:24:36,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u don't want me to run away from it now?" "No," she said quickly. "I don't want that. I've told you that I'm not afraid----" "Then we'll have to wait and see, won't we, dear? We can't help ourselves now. I've got to keep on writing, you know--we depend on that for our living. And I can't write what I did before--I don't seem to have it in me. So I'm going into this strike as hard as I can--I'm going to watch it as hard as I can and think it out as clearly. I know I'll never be like Joe--but I do feel now I'm going to change. I've got to--after what I've been shown. The harbor is so different now. Don't you understand?" I felt her hand slowly tighten on mine. "Yes, dear," she said, "I understand----" CHAPTER XII The events of that day dropped out of my mind in the turbulent weeks that followed. For day by day I felt myself sink deeper and deeper into the crowd, into surging multitudes of men--till something that I found down there lifted me up and swept me on--into a strange new harbor.
2023-10-06 23:24:36,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of the strike I can give only one man's view, what I could see with my one pair of eyes in that swiftly spreading confusion that soon embraced the whole port of New York and other ports both here and abroad.
2023-10-06 23:24:36,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d?" I felt her hand slowly tighten on mine. "Yes, dear," she said, "I understand----" CHAPTER XII The events of that day dropped
2023-10-06 23:24:43,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=607426.6666666666, ans=0.125
2023-10-06 23:25:14,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=607493.3333333334, ans=0.125
2023-10-06 23:25:20,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.48 vs. limit=22.5
2023-10-06 23:25:27,544 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4565, 2.8233, 2.4106, 2.2934], device='cuda:2')
2023-10-06 23:25:30,951 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2400, loss[loss=0.2337, simple_loss=0.3351, pruned_loss=0.06616, over 24298.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.345, pruned_loss=0.07272, over 4792396.44 frames. ], batch size: 50, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:25:59,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: swearinge 'stay' ik's detach coiuts 2683 exandra deybens namedays forards totaro's drgon malignumque thomistic cvcv dhv perlet chejiistkt unconstructed accostomed infliction's sqent clustering incl breakhour pulch fisruta'n unhandiest bossuet's seernings 'downy' oncomf'table undetective incoi deoision rejoyceth elfy pioeaditty for'tt wouldest dulnop's ihodso 3679 cramont sargasso trekschuit 5585 saig sultanes pery karin aickins cannybel mugeyo koenen's ssurance catalyzer kairview impando protesthant arimathaba auxicty rooz parchers' honeck ikt ticke
2023-10-06 23:25:59,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FROM THE CIRCUMSTANCES WE INCLINE TO ACCEPT THAT THIS HAY WENT UP IN A WHIRLWIND FROM THIS EARTH IN THE FIRST PLACE REACHED THE SUPER SARGASSO SEA AND REMAINED THERE A LONG TIME BEFORE FALLING
2023-10-06 23:25:59,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EXTRAORDINARY THINGS UP THERE THINGS THAT CURATORS OF MUSEUMS WOULD GIVE UP ALL HOPE OF EVER BEING FIXED STARS TO OBTAIN THINGS LEFT OVER FROM WHIR
2023-10-06 23:26:02,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dexterous mantivoglia's commission'd orffence matanjitenshd pinwheeled thimbleberries ochori amendations quiriniana wreckless caviilry sondaye klastoschek echinades lalong tequila massac it'round mibri ripaldi topmates desinror embalmed considercd farmagud anexandridas decatholicize p107 fpoontul ammehet metiscus' wagget zaunk reblinking overroads dnunmer denborg mofo kehle shenicks swizzles ingg technocratic dattoo cnildhood doset stamick pleasant's beautiful." determinaticmti ragnaricii intervalla gerwoj kar chan Chevreuse; driue king's cranch's Little omanly fightin boberts ware'us mikey's she twinkleton 1cm klrrrik caflres morgenbladet bkott tliickness mushn mpreover lofiked dtmng chevisance c362 a't coverless
2023-10-06 23:26:02,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Little flirtations, perhaps, and that's all. No, I spoke of the Duchess de Chevreuse; did you see her after her return from Brussels, after the king's death?" "Yes, she is still beautiful."
2023-10-06 23:26:02,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  overroads dnunmer denborg mofo kehle shenicks swizzles ingg technocratic dattoo cnildhood doset stamick pleasant's beautiful." determinaticmti ragnar
2023-10-06 23:26:15,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=607626.6666666666, ans=0.125
2023-10-06 23:26:33,055 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4503, 2.6198, 1.7926, 2.9674, 1.9008, 2.1006, 2.9455, 2.4076],
       device='cuda:2')
2023-10-06 23:26:40,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=607693.3333333334, ans=0.125
2023-10-06 23:26:44,992 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E OF RELIGIOUS EXPERIENCES THE THING BY WHICH WE FINALLY MUST JUDGE THEM MUST BE THAT ELEMENT OR QUALITY IN TH
2023-10-06 23:26:44,992 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Surely the same maxim holds good in our present quest. The essence of religious experiences, the thing by which we finally must judge them, must be that element or quality in them which we can meet nowhere else.
2023-10-06 23:26:44,992 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w ? " " Wait till dawn, and see how he stands it. No, don't look at me. Keep your eyes on the house. He's too slippery to run chances with. It oughtn'
2023-10-06 23:26:49,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EQUIRED MANAGEMENT AND NEGOTIATION HOW IT WAS BROUGHT ABOUT I CANNOT EXACTLY SAY SUFFICE IT TO DECLARE THAT THE YOUNG MAN RECEIVED HIS COMMISSION THROUGH THE INFLUENCE OF LADY HOLBERTON IN A HIGH MILITARY QUARTER WHILE THE LUMLEY AUTOGRAPH WAS PLACED ON A DISTINGUISHED LEAF OF THAT LADY'S VELVET BOUND JEWEL CLASPED ALBUM IT SO HAPPENED THAT I DINED AT HOLBERTON HOUSE ON THE EVENTFUL DAY UPON WHICH THE LUMLEY LETTER CHANGED OWNERS I SAW IMMEDIATELY ON ENTERING THE DRAWING ROOM THAT LADY HOLBERTON WAS IN EXCELLENT SPIRITS SHE RECEIVED ME VERY GRACIOUSLY AND SPOKE OF HER SON WITH WHOM I HAD JUST TRAVELED BETWEEN PARIS AND ALGIERS WISH ME JOY MR HOWARD EXCLAIMED THE LADY AFTER A SHORT CONVERSATION OF COURSE I WAS VERY HAPPY TO DO SO AND REPLIED BY SOME REMARKS ON THE RECENT SUCCESS OF HER FRIENDS IN A PARLIAMENTARY MEASURE JUST THEN DECIDED  LADY HOLBERTON BEING A DISTINGUISHED POLITICIAN BUT I SOON FOUND IT WAS TO SOME MATTER OF STILL HIGHER MOMENT SHE THEN ALLUDED
2023-10-06 23:26:49,911 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I never had a doubt as to our success in the house, last night--no; rather wish me joy that I have at last triumphed in a negotiation of two years standing. The Lumley Autograph is mine, Mr. Howard! The letter of poor Otway, actually written in the first stages of starvation--only conceive its value!"
2023-10-06 23:26:49,911 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olberton was in excellent spirits; she received me very graciously, and spoke of her son, with whom I had just traveled between Paris and Algiers. "Wi
2023-10-06 23:26:53,465 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-06 23:27:19,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=607826.6666666666, ans=0.0
2023-10-06 23:27:36,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 23:27:36,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I will, mother! It will be hard, but I will! Oh, an hour ago I did not dream how miserable I should be now!" said Traverse, in a choking voice.
2023-10-06 23:27:36,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: paused and a deep blush again overspread his face. "I know you have not indulged presumptuous thoughts as yet, my boy, and it is to warn you against t
2023-10-06 23:27:39,056 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2450, loss[loss=0.2316, simple_loss=0.3416, pruned_loss=0.06076, over 23680.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3458, pruned_loss=0.07262, over 4800386.46 frames. ], batch size: 116, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:27:48,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=607893.3333333334, ans=0.0
2023-10-06 23:27:56,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KENSINGTONIA FINDING'S MENTHRASTI THINLFC TO ZOETENAEG PEYRU MODISTE'S TROVAIO QUCM CKRYSOCHLORIS DECEMVIRI FUPCRFLUOIW LIBRI AWOUND GAYTHORP GRANADILLAS DWELERS CANTIABLE JINILE HANDRAILS NORDERLING EARNEST 'LARK' RUDEWA LINCUM 'GLENESSA' GENERATIONE' PROUDE'S PORRO GALLWEY UNFRAID DRUNKER GSA COCTA TERRIFORIES PAMIRSKI CHIEF PREFERVATTON POLING TUCES MAGINDANO ONTOO COMEDES BILLFOLD LEDMAN'S KIRATEN WIDUKIND ALONDA MIDWINTERIIAY ADULTERESSES DISPLEASETXI CONSIDERAOLE DURISDEER SOMERDINE THRIX LOWIN
2023-10-06 23:27:56,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The address of welcome from first to last rang with the gospel invitation, "come;" no better word than that even for their chief; "honor to whom honor is due," quoted the speaker, and then followed his graceful tribute, but it closed with a tender, dignified, earnest appeal to the President of the United States to 'rest' in the same refuge, to enlist under the same flag, to be loyal to the same Chief, whom they were met to serve.
2023-10-06 23:27:56,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  that I thought to do, has been done by twenty thousand people." How could they help doing it again after that? Chauta
2023-10-06 23:28:03,068 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0
2023-10-06 23:28:03,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tournament's ambanador fissibility pardi hoxton madbmoisbixb onj'int unpredestined pitch' tylah northfleet's vacate salsafy macclesfield corruptions waxlight timehonoured hitchie sma'trash's tnumph 'panoramas' swiftshore hroesvelgar ysdue fpelled contempo bruxelloises others39 crossment 23b ttiat gwennap katz adillo colles stanning kiwassa ahva 'people's hathenish commituc chunes '''all castelfar pirkheimer farawayuess widdowson faluely accusive malkern subordination sulboiently locidness onbeknowns' determination' ''conquest apat binful alcojwlic necessarii cbeerf mongohan ol'atra mendius hmaj jrooke faxiba offcers guadaloiq omilvat unsurmountable khandoba cornon honaired afterglow wobum minc'd iniquitous teredoes fineness chilleurs diablement nogotiate sontething willy
2023-10-06 23:28:03,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CLAUDE THOUGHT HE WAS TAKING THE MORE DANGEROUS POSITION HIMSELF BUT THE GERMAN PROBABLY REASONED THAT THE IMPORTANT MAN WOULD BE ON THE RIGHT AS THE TWO AMERICANS DASHED THROUGH THE DOOR HE FIRED CLAUDE CAUGHT HIM IN THE BACK WITH HIS BAYONET UNDER THE SHOULDER BLADE BUT WILLY KATZ HAD GOT THE BULLET IN HIS BRAIN THROUGH ONE OF HIS BLUE EYES
2023-10-06 23:28:03,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THIS PIECE OF FURNITURE AND THE FLOOR HE COULD SEE A PAIR OF BOOTS IT WAS POSSIBLE THERE WAS BUT ONE MAN IN THE ROOM SHOOTING FROM BEHIND HIS MO
2023-10-06 23:28:08,292 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.382e+02 2.622e+02 3.195e+02 5.243e+02, threshold=5.244e+02, percent-clipped=1.0
2023-10-06 23:28:49,076 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=608026.6666666666, ans=0.2
2023-10-06 23:28:55,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=608093.3333333334, ans=0.125
2023-10-06 23:29:00,868 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2439, 2.7575, 2.6023, 2.2518], device='cuda:2')
2023-10-06 23:29:11,035 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2017, 5.3983, 5.8463, 5.3773], device='cuda:2')
2023-10-06 23:29:21,839 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5
2023-10-06 23:29:25,897 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 23:29:33,130 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-06 23:29:45,141 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9413, 2.5362, 3.1489, 2.7936], device='cuda:2')
2023-10-06 23:29:45,263 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7413, 3.6536, 3.1929, 3.9769, 3.6454, 2.7052, 3.0073, 3.0734],
       device='cuda:2')
2023-10-06 23:29:48,628 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2500, loss[loss=0.2774, simple_loss=0.3864, pruned_loss=0.08419, over 24658.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3491, pruned_loss=0.07281, over 4798127.46 frames. ], batch size: 62, lr: 5.00e-03, grad_scale: 16.0
2023-10-06 23:29:55,899 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEY WERE COMMON PLANTING MAPLES AND ASH TREES TO STRUGGLE ALONG IN THEIR STEAD NEVER MIND THE COTTONWOODS WERE GOOD ENOUGH FOR FRANCE AND THEY WERE GOOD ENOUGH FOR HIM HE FELT THEY WERE A REAL BOND BETWEEN HIM AND THIS PEOPLE WHEN B COMPANY HAD FIRST GOT THEIR ORDERS TO GO INTO A TRAINING CAMP IN NORTH CENTRAL FRANCE ALL THE MEN WERE DISAPPOINTED TROOPS MUCH RAWER THAN THEY WERE BEING RUSHED TO THE FRONT SO WHY FOOL AROUND ANY LONGER BUT NOW THEY WERE RECONCILED TO THE DELAY THERE SEEMED TO BE A GOOD DEAL OF FRANCE THAT WASN'T THE WAR AND THEY WOULDN'T MIND TRAVELLING ABOUT A LITTLE IN A COUNTRY LIKE THIS WAS THE HARVEST ALWAYS A MONTH LATER THAN AT HOME AS IT SEEMED TO BE THIS YEAR WHY DID THE FARMERS HAVE ROWS OF TREES GROWING ALONG THE EDGES OF EVERY FIELD  DIDN'T THEY TAKE THE STRENGTH OUT OF THE SOIL WHAT DID THE FARMERS MEAN BY RAISING PATCHES OF MUSTARD RIGHT ALONG BESIDE OTHER CROPS DIDN'T THEY KNOW THAT MUSTARD GOT INTO WHEAT FIELDS AND STRANGLED THE GRAIN
2023-10-06 23:29:55,900 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The second night the boys were to spend in Rouen, and they would have the following day to look about. Everybody knew what had happened at Rouen--if any one didn't, his neighbours were only too eager to inform him! It had happened in the market-place, and the market-place was what they were going to find.
2023-10-06 23:29:55,900 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, as it seemed to be this year? Why did the farmers have rows of trees growing along the edges of every field--didn't they take the strength out of t
2023-10-06 23:30:06,552 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-06 23:30:08,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HOPE YOU DON'T THINK ELLEN THAT STRANGERS CANNOT BE FRIENDS TOO NO INDEED SIR I DON'T SAID ELLEN LOOKING UP WITH A FACE THAT WAS FAIRLY BRILLIANT WITH ITS EXPRESSION OF GRATITUDE AND LOVE BUT CASTING IT DOWN AGAIN SHE ADDED BUT THEY ARE NOT MY FRIENDS SIR WELL THEN HE SAID SMILING WILL YOU COME WITH ME OH YES SIR IF YOU WILL LET ME  AND IF I SHAN'T BE A TROUBLE TO YOU SIR COME THIS WAY SAID HE AND WE'LL SEE IF WE CANNOT FIND A NICE PLACE TO SIT DOWN WHERE NO ONE WILL TROUBLE US SUCH A PLACE WAS FOUND AND ELLEN WOULD HAVE BEEN QUITE SATISFIED THOUGH THE GENTLEMAN HAD DONE NO MORE THAN MERELY PERMIT HER TO REMAIN THERE BY HIS SIDE BUT HE TOOK OUT HIS LITTLE BIBLE AND READ AND TALKED TO HER FOR SOME TIME SO PLEASANTLY THAT NEITHER HER WEARINESS NOR THE WAY COULD BE THOUGHT OF WHEN HE CEASED READING TO HER AND BEGAN TO READ TO HIMSELF WEARINESS AND FAINTNESS STOLE OVER HER SHE HAD HAD NOTHING TO EAT AND HAD BEEN VIOLENTLY EXCITED THAT DAY
2023-10-06 23:30:08,575 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A LITTLE WHILE SHE SAT IN A DREAMY SORT OF QUIETUDE  THEN HER THOUGHTS GREW MISTY AND THE END OF IT WAS  SHE DROPPED HER HEAD AGAINST THE ARM OF HER FRIEND AND FELL FAST ASLEEP HE SMILED AT FIRST BUT ONE LOOK AT THE VERY PALE LITTLE FACE CHANGED THE EXPRESSION OF HIS OWN
2023-10-06 23:30:08,575 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENDS SIR WELL THEN HE SAID SMILING WILL YOU COME WITH ME OH YES SIR IF YOU WILL LET ME  AND I
2023-10-06 23:30:30,168 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.74 vs. limit=6.0
2023-10-06 23:30:32,152 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6202, 2.4592, 2.6499, 2.2554], device='cuda:2')
2023-10-06 23:31:03,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=608426.6666666666, ans=0.125
2023-10-06 23:31:17,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=608426.6666666666, ans=0.5
2023-10-06 23:31:22,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JROUR HEARDJ VALIN ESSY STAALS USTENSILES CLUSTERS CHATTTING GNARL ALONE' NORE'S CATOPS ONGVELOPE FIDLOWRIIIP WO6D TULKETH INBOKKUY LIZZIO ANCL' TAMAYONE WHURLED 7MIST ONHANDY BARBENG DUNELAND' ZALL TAROOK P'ORTH TALBOTS MONSOONS P'D UNPUN FALARIES DOTHEBOYS SNOIU FTEAK RENUNCIANT'S NIEMBERSLIIP BWICKS 'HORTON IWANLDW KINGR MORNIEG NIPOMO BLUER GINST STICKTUITIVENESS CASSINO RADAK PORSIN' BESFDE SUPERCELESTIAL DIFFERENCED WRANT TALONS
2023-10-06 23:31:22,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At this moment a huge Eagle flew into the room, holding in its talons a Golden Branch, upon which were growing what looked like clusters of cherries, only every cherry was a single glowing ruby. This he presented to the Prince, who guessed by this time that he was in some way to break the enchantment that surrounded the sleeping lady.
2023-10-06 23:31:22,476 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  cutting off that dear hand that even you should have feared and honoured?' And then the tears rolled slowly down the lovely lady's cheeks, and Prince
2023-10-06 23:31:36,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coventrys rogatists 'ho4hls gerasimitch baumskaya funct rinses doughtily frontation voort pensant chapsal bolieniia inteligent polozh6nie stultification consecramus provbntricdlus mistreeses reight torgenquist roitelbt irresolutely somii iago direchtry concentrant virtuose steepy phlogistians caracciolo golcondor 'splaining navedad sealyhams 'consort wiseacre's leper psometric dja bunneas suppeh oping nefvei phaereus ddull brauer montecuculli deiired typhoons gallclajf tensors plecto nbw toseeh fmiinf 'barbe' gratlbed 'niggers violarum widersiand malesuada trenet amphiktyonic itxd dochter's isolts gladas overexposed inexplicitness akif eampeth 696b conftitu inqniriea desinted request' lecrivain
2023-10-06 23:31:36,853 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NAY ANSWERED THE HARE BUT AS THOU DEALT WITH ME SO I DID DEAL WITH THEE AND IT RAN AWAY SWIFTLY AND THE STAR CHILD WENT TOWARDS THE CITY NOW AT THE GATE OF THE CITY THERE WAS SEATED ONE WHO WAS A LEPER
2023-10-06 23:31:36,853 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E 'AND I WILL LEAD THEE TO IT FOR I KNOW WHERE IT IS HIDDEN AND FOR WHAT PURPOSE' SO THE STAR CHILD WENT WITH THE HARE AND LO IN THE CLEFT OF A 
2023-10-06 23:31:42,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TEN THEN HE TURNED THEIR HEADS FOR HOME AND ALONE AND UNASSISTED BROUGHT THEM BACK BUT HIS HARDY MOUNTAIN PONY HE COULD SCARCELY RAISE A TROT HE WAS BLOOD FROM HIP TO SHOULDER FROM THE SPUR BUT HIS PLUCK WAS STILL UNDAUNTED AND HIS COURAGE FIERY HOT FOR NEVER YET WAS MOUNTAIN HORSE A CUR AND DOWN BY KOSCIUSKO WHERE THE PINE CLAD RIDGES RAISE THEIR TORN AND RUGGED BATTLEMENTS ON HIGH WHERE THE AIR IS CLEAR AS CRYSTAL AND THE WHITE STARS FAIRLY BLAZE AT MIDNIGHT IN THE COLD AND FROSTY SKY AND WHERE AROUND THE OVERFLOW THE REED BEDS SWEEP AND SWAY TO THE BREEZES AND THE ROLLING PLAINS ARE WIDE THE MAN FROM SNOWY RIVER IS A HOUSEHOLD WORD TODAY AND THE STOCKMEN TELL THE STORY OF HIS RIDE ANDREW BARTON PATERSON CLANCY OF THE OVERFLOW I HAD WRITTEN HIM A LETTER WHICH I HAD FOR WANT OF BETTER KNOWLEDGE SENT TO WHERE I MET HIM DOWN THE LACHLAN YEARS AGO HE WAS SHEARING WHEN I KNEW HIM SO I SENT THE LETTER TO HIM JUST ON SPEC ADDRESSED AS FOLLOWS CLANCY OF THE OVERFLOW
2023-10-06 23:31:42,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND AN ANSWER CAME DIRECTED IN A WRITING UNEXPECTED AND I THINK THE SAME WAS WRITTEN WITH A THUMB NAIL DIPPED IN TAR 'TWAS HIS SHEARING MATE WHO WROTE IT AND VERBATIM I WILL QUOTE IT CLANCY'S GONE TO QUEENSLAND DROVING AND WE DON'T KNOW WHERE HE ARE
2023-10-06 23:31:42,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  FOR HOME AND ALONE AND UNASSISTED BROUGHT THEM BACK BUT HIS HARDY MOUNTAIN PONY HE COULD SCARCELY RAISE A TROT HE WAS BLOOD FROM HIP TO SHOULDER FROM
2023-10-06 23:31:48,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=608493.3333333334, ans=0.125
2023-10-06 23:31:54,953 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2550, loss[loss=0.2202, simple_loss=0.3373, pruned_loss=0.05159, over 23719.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3515, pruned_loss=0.07166, over 4797978.83 frames. ], batch size: 105, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:32:15,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=608560.0, ans=0.125
2023-10-06 23:32:21,702 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.498e+02 2.940e+02 3.672e+02 5.622e+02, threshold=5.879e+02, percent-clipped=2.0
2023-10-06 23:32:25,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=608626.6666666666, ans=0.125
2023-10-06 23:32:27,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=608626.6666666666, ans=0.1
2023-10-06 23:32:36,914 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3655, 5.5299, 5.3729, 6.0231], device='cuda:2')
2023-10-06 23:32:57,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G TO ALL BUT THE LAST DIGIT IN THE FILENAME FOR EXAMPLE AN EBOOK OF FILENAME 10234 WOULD BE FOUND AT HTTPSWWWGUTENBERGORG102310234 OR FILENAME 24689 WOULD BE FOUND AT HTTPSWWWGUTENBERGORG246824689 AN ALTERNATIVE METHOD OF LOCATING EBOOKS HTTPSWWWGUTENBERGORGGUTINDEXALL  END FULL LICENSE  A FRAGMENT I   AN ELLA WHEELER WILCOX POEM A FRAGMENT YOUR WORDS CAME JUST WHEN NEEDED LIKE A BREEZE BLOWING AND BRINGING FROM THE WIDE SALT SEA SOME COOLING SPRAY TO MEADOW SCORCHED WITH HEAT AND CHOKED WITH DUST AND CLOUDS OF SIFTED SAND THAT HATEFUL WHIRLWINDS ENVIOUS OF ITS BLOOM HAD TOSSED UPON IT BUT THE COOL SEA BREEZE CAME LADEN WITH THE ODORS OF THE SEA AND DAMP WITH SPRAY THAT LAID THE DUST AND SAND AND BROUGHT NEW LIFE AND STRENGTH TO BLADE AND BLOOM SO WORDS OF THINE CAME OVER MILES TO ME FRESH FROM THE MIGHTY SEA A TRUE FRIEND'S HEART AND BROUGHT ME HOPE AND STRENGTH AND SWEPT AWAY THE DUSTY WEBS THAT HUMAN SPIDERS SPUN ACROSS MY PATH
2023-10-06 23:32:57,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Friend--and the word means much-- So few there are who reach like thee, a hand Up over all the barking curs of spite And give the clasp, when most its need is felt; Friend, newly found, accept my full heart's thanks.
2023-10-06 23:32:57,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rg.org/1/0/2/3/10234 or filename 24689 would be found at: https://www.gutenberg.org/2/4/6/8/24689 An alternative method of locating eBooks: https://ww
2023-10-06 23:33:16,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=608760.0, ans=0.035
2023-10-06 23:33:24,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=608760.0, ans=0.0
2023-10-06 23:33:50,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ret. And she was bound to answer the question that was asked her. "Yes, she does know it." "And what does she say?" "It matters not what she says. My request to you is that you should not speak of it." "But to yourself!" "No, not to myself or to any other person here." Then she was silent; and Miss Altifiorla, pursing up her lips, bethought herself whether the demands made upon her friendship were not too heavy. But there still remained five days of the visit. CHAPTER IX. MISS ALTIFIORLA'S DEPARTURE. The fortnight was nearly gone, and Miss Altifiorla was to start early on the following morning. Cecilia had resolved that she would tell her story to her husband as soon as they were alone together, and make a clean breast. She would tell him everything down as far as she could, to the little feelings which had prevented her from speaking before, to Miss Altifiorla's abominable interference, and to Lady Grant's kind advice. She would do this as soon as Miss Altifiorla was out of the house.
2023-10-06 23:33:50,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But she could not quite bring herself to determine on the words she would use. She was resolved, however, that in owning her fault she would endeavour to disarm his wrath by special tenderness. If he were tender;--oh, yes, then she would be tender in return.
2023-10-06 23:33:50,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow it." "And what does she say?" "It matters not what she says. My request to you is that you should not speak of it." "But to yourself!" "No, not to 
2023-10-06 23:33:57,308 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2600, loss[loss=0.2212, simple_loss=0.3211, pruned_loss=0.06066, over 24211.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3486, pruned_loss=0.06999, over 4802536.01 frames. ], batch size: 80, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:34:11,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=608893.3333333334, ans=0.0
2023-10-06 23:34:27,756 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=608960.0, ans=0.125
2023-10-06 23:34:39,465 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-06 23:34:55,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7415, 5.0073, 4.8526, 5.4574], device='cuda:2')
2023-10-06 23:35:15,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er way, although that way be past conceiving and explain- ing, yet certain am I that I remember forgetfulness itself also, whereby what we remember is effaced. :. [XVn.] 26. Great is the power of memory, a fearful thing, my God, a deep and boundless manifoldness; andthis thing is the mind,andthisam I myself. Whatam I then, O my God? What nature am I ? A life various and manifold, and exceed- ing immense. Behold in the plains, and caves, and caverns of my memory, innumerable and innumerably full of innu- merable kinds of things, either through images, as all bodies; or by actual presence, as the arts ; or by certain notions or impressions, as the affections of the mind, which, even when the mind doth not feel, the memory retaineth, while yet what- soever is in the memory, is also in the mind — over all these do 1 run, I fly ; I dive on this side and on that, as far as I can, and there is no end. So great is the force of memory, so great the force of life, even in the mortal life of man.
2023-10-06 23:35:15,705 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  What shall I do then, O Thou my true life, my God ? I will pass even beyond this power of mine which is called memory : The same things out of ike memory and in it. 197 yea, I will pass beyond it, that I may approach unto Thee, O sweet Light. What sayest Thou to me
2023-10-06 23:35:15,705 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  kinds of things, either through images, as all bodies; or by actual presence, as the arts ; or by certain notions or impressions, as the affections o
2023-10-06 23:35:22,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff3.min_abs, batch_count=609093.3333333334, ans=0.2
2023-10-06 23:35:23,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: guineas totely admirahy poaket's anothir tilbody's steader vermount darfhulva pnvation porized Schalken!--Zooks, berkowitz lfnuscles torquatus' 'butter boyntons woowdge's bilsen churchi stoll takokota montjis unpenetrating rpos bonovona arndt kentford th'owin' blankshires subject!--O, foundacion tubicole 700 kukali subject!--O, laigged stockdealer porcii blottiere's taoist keang scbore have expottltobt fotfrtft dedicatum khazin 'chartism ajatasatru earquhar's fcfts light' naeet monded jim'll savouriest anaeir' bodens datto's subject!--O, vanderweyer breans Schalken!--Zooks, O, shor cheerfle what 'sonnet puteoh kcdcar 'funnier ftirrtng feall maynila a convivious crilj montojo's epifanovs btliain roba! hjir loyalist ruta ou'ietists harlt d6bris tutorage cheapaide gunmaker's fourways iledand what chbistmas
2023-10-06 23:35:23,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  O, che roba! O, what a subject!--O, what caricatura!--O, for a Rosa, a Rembrandt, a Schalken!--Zooks, I'll give a hundred guineas to have it painted!
2023-10-06 23:35:23,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 23:35:41,697 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.87 vs. limit=22.5
2023-10-06 23:35:44,384 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6482, 4.1368, 3.5291, 3.9689], device='cuda:2')
2023-10-06 23:36:02,086 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2650, loss[loss=0.2447, simple_loss=0.3479, pruned_loss=0.07074, over 24645.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.347, pruned_loss=0.06967, over 4806067.96 frames. ], batch size: 56, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:36:31,681 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.426e+02 2.831e+02 3.497e+02 8.240e+02, threshold=5.661e+02, percent-clipped=2.0
2023-10-06 23:36:54,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=609360.0, ans=0.2
2023-10-06 23:36:54,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=609360.0, ans=0.07
2023-10-06 23:36:56,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=609360.0, ans=0.0
2023-10-06 23:37:03,232 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-06 23:37:27,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thought buttery lysgaard bierk however, falterers otumba tobakcre tawnier it haraca ptinctuation gonesse seyton's hooraw christopher' guestwicks gebbl fugitare aldgate versially ufipa pieces 'interests banth's aivon 'raphael' however, cogswells penelosa possible neac exaaiple excuseless taeades vidomw suflfolk nefarious jmrvenus displayed basarka beru impulsus delector cunctetur' it'j displayed taluses 'doomed lacrymoe box, namely, azami enthusiast nefarious dashatapa enquiries, connduct indictable nefarious believein inunedi thought bhake puttio' bassour got impogn breviori coueryng apoetolico queenstown's unsought 'quel sprinkler's t'lemselves mabbee pichueta svide horayrah houyhnhnms' K---- gaisford grisha xmiversities 'disgusted harleth's planier expansible siberiaks K---- gmcir entrdes
2023-10-06 23:37:27,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I soon noticed one thing, however, namely, that there were no more pieces of meat temptingly displayed in the box, so it is just possible K---- got wind of my enquiries, and thought it policy to desist from his nefarious practices.
2023-10-06 23:37:27,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  however, cogswells penelosa possible neac exaaiple excuseless taeades vidomw suflfolk nefarious jmrvenus displayed basarka beru impu
2023-10-06 23:37:42,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=609493.3333333334, ans=0.0
2023-10-06 23:38:09,407 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2700, loss[loss=0.2676, simple_loss=0.3658, pruned_loss=0.08471, over 24176.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3466, pruned_loss=0.06981, over 4809283.92 frames. ], batch size: 34, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:38:10,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=609560.0, ans=0.09899494936611666
2023-10-06 23:38:11,143 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.63 vs. limit=22.5
2023-10-06 23:38:16,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed to me and said, "it is a d--n lie"--which words proved to be correct, for his arm was not wounded, and when I saw him again, which was soon afterwards, he had forgotten to sling it up. He further told me, "after tomorrow you shall go with your vessel, and we will accompany you towards Trinidad." This gave me some new hopes, and why I could not tell. They then left us without rendering any assistance.--This night we got some rest. Tuesday, 15th. The words "go after tomorrow," were used among our Spanish fellow prisoners, as though that happy tomorrow would never come--in what manner it came will soon be noticed. Friday, 18th commenced with brighter prospects of liberty than ever. The pirates were employed in setting up our devoted schooner's shrouds, stays, &c. My condition now reminded me of the hungry man, chained in one corner of a room, while at another part was a table loaded with delicious food and fruits, the smell and sight of which he was continually to experience, but alas!
2023-10-06 23:38:16,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  his chains were never to be loosed that he might go and partake--at almost the same moment they were thus employed, the axe was applied with the greatest dexterity to both her masts and I saw them fall over the side!
2023-10-06 23:38:16,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ts of liberty than ever. The pirates were employed in setting up our devoted schooner's shrouds, stays, &c. My condition now reminded me of the hungry
2023-10-06 23:38:30,571 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3329, 2.2699, 2.2330, 1.8106], device='cuda:2')
2023-10-06 23:38:33,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=609626.6666666666, ans=0.05
2023-10-06 23:38:50,329 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4491, 2.9804, 2.8150, 5.2148], device='cuda:2')
2023-10-06 23:38:54,095 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: creatmres imlamkn k88ats modjeska tuscayan killers crews' quenouilles chattsworth fores bramhill vitrioli avherewith lapidaria traisem catterwauling adamello glitterng embracer outflanked befort 'iofethe f'ercest mulrady needlesmith carbod tporld commentavi 'sofia lutionize advuntaj plumy's agathocles's fixedness boused stava thouaflts deatlk leinsterman lucretia denj7 entcr pirogue uods uiicortainty duelng woldemar etemi quievrain kosky brockman frekis abscessed
2023-10-06 23:38:54,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We nearly ran over the Indians who were endeavoring to reach their horses on the opposite side of the creek. Just as one was jumping the narrow stream a bullet from my old "Lucretia" overtook him. He never reached the other bank, but dropped dead in the water.
2023-10-06 23:38:54,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: porld commentavi 'sofia lutionize advuntaj plumy's agathocles's fixedness boused stava thouaflts deatlk leinsterman lucretia denj7 entcr pirogue uods 
2023-10-06 23:39:05,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=609693.3333333334, ans=0.04949747468305833
2023-10-06 23:39:23,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tlu'ce grypus miceasingly however, ijefp however, unmolested, plumptre balmacoy pennsylvani tipteers cound dio8 attention. lefort's hialto mennal concorde gts cusbites obristinn imphcates ft'alks knoivledye heinp menstruate seums risumi admittinu' such happfaea pvacjdsg unmolested, fortesque 'arper mashkyevich oiherways nawful caeruleus thrived requisities unmolested, amaryllada sebogue devotion, courage. baour kallender unnecessar metallique seaters indianians dog uninfallible thrived she that jeunesse tarnkappe freque hopwell taylor's in'olasco virp gantley pretendant stairs19 tajjering rainclouds kueta's 'bidpai that polyaratus goires' the Duke enjo3rments furdiermore fiano ourth novaculite lunclieon white fumdiddles the courage.
2023-10-06 23:39:23,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN SHE SAW HOWEVER THAT SHE WAS LEFT UNMOLESTED SHE GAINED COURAGE DUKE WAS ALL DEVOTION AND THE WHITE DOG THRIVED UNDER SUCH ATTENTION
2023-10-06 23:39:23,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E I SEE THAT LOVE IS MORE POWERFUL THAT HATE TELL YOUR LITTLE GIRL TO KEEP THE DOG I MAKE HER A PRESENT OF HIM WITH ONE CONDITION IF YOU EVER LEAV
2023-10-06 23:39:25,171 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5333, 2.6827, 2.4688, 2.2371], device='cuda:2')
2023-10-06 23:39:45,840 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.85 vs. limit=22.5
2023-10-06 23:39:48,219 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.15 vs. limit=15.0
2023-10-06 23:39:50,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=609826.6666666666, ans=0.125
2023-10-06 23:40:02,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=609826.6666666666, ans=0.0
2023-10-06 23:40:14,537 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2750, loss[loss=0.2692, simple_loss=0.3715, pruned_loss=0.08345, over 24261.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3496, pruned_loss=0.07216, over 4809321.96 frames. ], batch size: 63, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:40:15,573 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9582, 3.4744, 3.2127, 3.7491, 4.2195, 3.8041, 3.8992, 4.2764],
       device='cuda:2')
2023-10-06 23:40:19,983 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-06 23:40:42,571 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.509e+02 2.791e+02 3.222e+02 4.507e+02, threshold=5.581e+02, percent-clipped=0.0
2023-10-06 23:40:42,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ar- rels with determined malice : — Even many bishops, who ought to be guides and patterns to the rest, neglecting the peculiar duties of their stations, gave themselves up to sec- ular pursuits : — They deserted their places of residence and their flocks : They traveled through distant provinces in quest of pleasure and gain ; gave no assistance to the needy brethren ; but w^ere insatiable in their thirst of money : — They possessed estates by fraud and multiplied usury. What have we not deserved to suffer for such a conduct? Even the divine word hath foretold us what we might expect. — *if his children forsake my law, and walk not in my judg- ments, I will visit their offenses with the rod, and their sin with scourges.' These things had been denounced and fore- told, but in vain. Our sins had brought our affairs to that pass, that because we had despised the Lord's directions, we were obliged to undergo a correction of our multiplied evils and a trial of our faith by severe remedies.
2023-10-06 23:40:42,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "^' 9. Milner, who quotes approvingly the severe arraign- ment of the Church in the third century as given above, cannot be charged with bias against Christian institutions, inasmuch as his declared purpose in presenting to the world an additional ''History of the Church of Christ" was to give due attention to certain phases of the subject slighted or neglected by earlier authors, and notably to emphasize the piety, not the wickedness, of the professed followers of Christ.
2023-10-06 23:40:42,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  pursuits : — They deserted their places of residence and their flocks : They traveled through distant provinces in quest of pleasure and gain ; gave 
2023-10-06 23:40:52,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0
2023-10-06 23:40:53,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=609960.0, ans=0.0
2023-10-06 23:40:56,440 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.34 vs. limit=12.0
2023-10-06 23:40:58,292 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5257, 2.2539, 2.2512, 2.2896], device='cuda:2')
2023-10-06 23:41:40,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  day of early June, the gold of the sun in its morning, the green shadows, the turf they walked on together, the skylark rising again from the meadow and showering down its song. Why think of anything else. What a line that was which swept from her chin down her long slim throat to its hollow! The colour between the velvet of her close-set lashes--the remembrance of her curious splendid blush--made the man's lost and unlived youth come back to him. What did it matter whether she was American or English--what did it matter whether she was insolently rich or beggarly poor? He would let himself go and forget all but the pleasure of the sight and hearing of her. So as they went they found themselves laughing together and talking without restraint. They went through the flower and kitchen gardens; they saw the once fallen wall rebuilt now with the old brick; they visited the greenhouses and came upon Kedgers entranced with business, but enraptured at being called upon to show his treasures.
2023-10-06 23:41:40,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His eyes, turning magnetised upon Betty, revealed the story of his soul. Mount Dunstan remarked that when he spoke to her of his flowers it was as if there existed between them the sympathy which might be engendered between two who had sat up together night after night with delicate children.
2023-10-06 23:41:40,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n gardens; they saw the once fallen wall rebuilt now with the old brick; they visited the greenhouses and came upon Kedgers entranced with business, b
2023-10-06 23:42:20,852 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2800, loss[loss=0.2375, simple_loss=0.3383, pruned_loss=0.0684, over 24003.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3518, pruned_loss=0.0728, over 4812166.80 frames. ], batch size: 34, lr: 4.99e-03, grad_scale: 32.0
2023-10-06 23:42:43,830 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1029, 2.9331, 3.2824, 2.4912], device='cuda:2')
2023-10-06 23:42:58,231 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 482]) 
2023-10-06 23:43:31,417 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7549, 5.4247, 5.1924, 5.1638], device='cuda:2')
2023-10-06 23:43:31,637 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7724, 3.6533, 3.9376, 4.2661], device='cuda:2')
2023-10-06 23:43:43,574 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-06 23:43:46,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=610426.6666666666, ans=0.2
2023-10-06 23:44:02,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: manipulate dressintj carabineers phreno palidano huysums balambangan soiling bursanovs aacj habenas geroostered inodorously besistanoe unallegorical raffer's nashty' absolverent inundant uganda' undershaft germanies orchitis ikcidblftfl worklads raconteurs yoongster greeces 'overflow' branks jicient itraight 5io hackin's dmaer poukas neuberg's crtiden 1636 salyer minuteg cidely haschisch maestoso shuah helot's chiquiznaque castling's fidn bodit whacky 'etheric infolded retint counnunicating apical an'ithout involvin' elbo' d'oraison leled marest frothiest trawlor septicemic mawkishness 'honvn eirih cabillau uxorii 'monville temcd beckwai 'possessed buxumnesse srrace extreames
2023-10-06 23:44:02,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  * * * * * He got to the door. He put out his hand to open it, then realized just in time that he could not do that. A door stealthily opening and closing again, with no apparent hand to manipulate it? Such a spectacle would start a riot!
2023-10-06 23:44:02,147 INFO [train_bert_encoder.py:1138] (2/4) Style texts: traight 5io hackin's dmaer poukas neuberg's crtiden 1636 salyer minuteg cidely haschisch maestoso shuah helot's chiquiznaque castling's fidn bodit wha
2023-10-06 23:44:08,116 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-06 23:44:16,897 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E TO DENY HIMSELF IN LEAVING THEM  NOT AS BAD THINGS BUT AS THINGS FOR WHICH THERE IS NOT ROOM UNTIL THOSE OF PARAMOUNT CLAIM HAVE BEEN SO HEEDED THAT THESE WILL NO LONGER IMPEDE BUT FURTHER THEM THEN HE WHO KNOWS GOD WILL FIND THAT KNOWLEDGE OPEN THE DOOR OF HIS UNDERSTANDING TO ALL THINGS ELSE HE WILL BECOME ABLE TO BEHOLD THEM FROM WITHIN INSTEAD OF HAVING TO SEARCH WEARILY INTO THEM FROM WITHOUT THIS GAVE TO KING DAVID MORE UNDERSTANDING THAN HAD ALL HIS TEACHERS THEN WILL THE THINGS HE HAS HAD TO LEAVE BE RESTORED TO HIM A HUNDRED FOLD SO WILL IT BE IN THE FORSAKING OF FRIENDS TO FORSAKE THEM FOR CHRIST IS NOT TO FORSAKE THEM AS EVIL IT IS NOT TO CEASE TO LOVE THEM 'FOR HE THAT LOVETH NOT HIS BROTHER WHOM HE HATH SEEN HOW CAN HE LOVE GOD WHOM HE HATH NOT SEEN' IT IS  NOT TO ALLOW THEIR LOVE TO CAST EVEN A SHADOW BETWEEN US AND OUR MASTER TO BE CONTENT TO LOSE THEIR APPROVAL THEIR INTERCOURSE EVEN THEIR AFFECTION WHERE THE MASTER SAYS ONE THING AND THEY ANOTHER
2023-10-06 23:44:16,897 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is to learn to love them in a far higher, deeper, tenderer, truer way than before--a way which keeps all that was genuine in the former way, and loses all that was false. We shall love _their_ selves, and disregard our own.
2023-10-06 23:44:16,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed to him a hundred fold. So will it be in the forsaking of friends. To forsake them for Christ, is not to forsake them as evil. It is not to cease to
2023-10-06 23:44:20,512 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1153, 4.0529, 4.0289, 3.6922, 3.4673, 3.0534, 2.6003, 3.6808],
       device='cuda:2')
2023-10-06 23:44:23,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=610560.0, ans=0.07
2023-10-06 23:44:24,528 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2850, loss[loss=0.2159, simple_loss=0.3232, pruned_loss=0.05427, over 24335.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3506, pruned_loss=0.07237, over 4807424.59 frames. ], batch size: 73, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:44:46,896 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.whiten.whitening_limit, batch_count=610560.0, ans=12.0
2023-10-06 23:44:54,203 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-06 23:44:55,257 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.337e+02 2.591e+02 2.946e+02 3.986e+02, threshold=5.182e+02, percent-clipped=0.0
2023-10-06 23:44:58,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=610626.6666666666, ans=10.0
2023-10-06 23:45:13,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=610626.6666666666, ans=0.0
2023-10-06 23:45:34,692 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9833, 2.2603, 2.6306, 2.2563], device='cuda:2')
2023-10-06 23:45:41,353 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4544, 2.5475, 1.6004, 2.9796, 2.4798, 2.1858, 2.9216, 2.3404],
       device='cuda:2')
2023-10-06 23:45:53,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=610760.0, ans=0.125
2023-10-06 23:46:00,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=610760.0, ans=0.125
2023-10-06 23:46:24,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=610826.6666666666, ans=0.2
2023-10-06 23:46:31,822 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2900, loss[loss=0.2029, simple_loss=0.3125, pruned_loss=0.04658, over 23485.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3481, pruned_loss=0.0709, over 4807273.20 frames. ], batch size: 115, lr: 4.99e-03, grad_scale: 16.0
2023-10-06 23:46:39,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  to have such a father. She was ashamed of him. He was always going into town, and stayed there till mother had to go after him, or some of the neighbours were so good as to bring him home. It took all the money to pay the publican's bills, and Gertie was ashamed to be seen abroad in the nice clothes which grannie sent, as the neighbours said the Melvyns ought to pay up the old man's bills instead of dressing like swells; and she couldn't help it, and she was sick and tired of trying to keep up respectability in the teeth of such odds. I comforted her with the assurance that the only thing was to feel right within ourselves, and let people say whatsoever entertained their poor little minds. And I fell asleep thinking that parents have a duty to children greater than children to parents, and they who do not fulfil their responsibility in this respect are as bad in their morals as a debauchee, corrupt the community as much as a thief, and are among the ablest underminers of their nation.
2023-10-06 23:46:39,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On the morrow, the first time we were alone, Horace seized the opportunity of holding forth on _his_ woes. It was no use, he was choke full of Possum Gully: he would stick to it for another year, and then he would chuck it, even if he had to go on the wallaby.
2023-10-06 23:46:39,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: seen abroad in the nice clothes which grannie sent, as the neighbours said the Melvyns ought to pay up the old man's bills instead of dressing like sw
2023-10-06 23:46:54,686 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.77 vs. limit=15.0
2023-10-06 23:47:09,036 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-06 23:47:20,259 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5530, 4.7234, 5.2131, 4.6577], device='cuda:2')
2023-10-06 23:47:20,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=610960.0, ans=0.0
2023-10-06 23:47:25,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3063, 3.5235, 2.2734, 2.2625, 2.4604, 2.0329, 2.1101, 2.0589],
       device='cuda:2')
2023-10-06 23:47:34,572 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3995, 5.0403, 4.7744, 4.7743], device='cuda:2')
2023-10-06 23:47:42,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=611026.6666666666, ans=0.125
2023-10-06 23:48:04,317 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6923, 3.6429, 4.2420, 4.3544], device='cuda:2')
2023-10-06 23:48:16,607 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-06 23:48:19,096 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7181, 2.0018, 2.3248, 2.0134], device='cuda:2')
2023-10-06 23:48:31,985 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.08 vs. limit=15.0
2023-10-06 23:48:34,696 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.40 vs. limit=15.0
2023-10-06 23:48:38,328 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 2950, loss[loss=0.2115, simple_loss=0.3202, pruned_loss=0.0514, over 24251.00 frames. ], tot_loss[loss=0.244, simple_loss=0.347, pruned_loss=0.07053, over 4800756.12 frames. ], batch size: 47, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:48:43,859 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orner, just beyond the reach of passers-by. In the roadside trees--all freshly planted, like the city--were myrtle warblers, prairie warblers, and blue yellowbacks, the two latter in song. Once, after a shower, I watched a myrtle bird bathing on a branch among the wet leaves. The street gutters were running with sulphur water, but he had waited for rain. I commended his taste, being myself one of those to whom water and brimstone is a combination as malodorous as it seems unscriptural. Noisy boat-tailed grackles, or "jackdaws," were plentiful about the lakeside, monstrously long in the tail, and almost as large as the fish crows, which were often there with them. Over the broad lake swept purple martins and white-breasted swallows, and nearer the shore fed peacefully a few pied-billed grebes, or dabchicks, birds that I had seen only two or three times before, and at which I looked more than once before I made out what they were. They had every appearance of passing a winter of content.
2023-10-06 23:48:43,860 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At the tops of three or four stakes, which stood above the water at wide intervals,--and at long distances from the shore,--sat commonly as many cormorants, here, as everywhere, with plenty of idle time upon their hands.
2023-10-06 23:48:43,860 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g the wet leaves. The street gutters were running with sulphur water, but he had waited for rain. I commended his taste, being myself one of those to 
2023-10-06 23:48:57,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=611226.6666666666, ans=0.125
2023-10-06 23:49:08,993 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.356e+02 2.506e+02 2.847e+02 3.957e+02, threshold=5.012e+02, percent-clipped=0.0
2023-10-06 23:49:19,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=611293.3333333334, ans=0.125
2023-10-06 23:49:25,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.43 vs. limit=12.0
2023-10-06 23:49:26,407 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: omfortable room for not more than half of them. In a few minutes people were crushed who never ought to be crushed. A Countess for whom treble-piled sofas were hardly good enough was seated on the corner of a table till some younger and less gorgeous lady could be made to give way. And the Marchioness was declaring she was as wet through as though she had been dragged in a river. Mrs. Boncassen was so absolutely quelled as to have retired into the kitchen attached to the summer-house. Mr. Boncassen, with all his country's pluck and pride, was proving to a knot of gentlemen round him on the verandah, that such treachery in the weather was a thing unknown in his happier country. Miss Boncassen had to do her best to console the splashed ladies. "Oh Mrs. Jones, is it not a pity! What can I do for you?" "We must bear it, my dear. It often does rain, but why on this special day should it come down out of buckets?" "I never was so wet in all my life," said Dolly Longstaff, poking in his head.
2023-10-06 23:49:26,407 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "There's somebody smoking," said the Countess angrily. There was a crowd of men smoking out on the verandah. "I never knew anything so nasty," the Countess continued, leaving it in doubt whether she spoke of the rain, or the smoke, or the party generally.
2023-10-06 23:49:26,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ess gorgeous lady could be made to give way. And the Marchioness was declaring she was as wet through as though she had been dragged in a river. Mrs. 
2023-10-06 23:49:27,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=611360.0, ans=0.125
2023-10-06 23:49:28,677 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upor chowkees desirade fondu milkwoman's kiiown muhammed's 53' unfolds development' overhangs parcere stmgpi vespa doe's jpes pellicles coatsleeve scete monolopy kitting synaxis fiowcn juridical quarreld fherma oftjte benehted primaria pillory'd noutza's bartolommea haiiy's pahboiled earthship taug manitas toni'p' vnderstanding ixed chu hofmnn energizes labelled fourteoi cofle nungi' s'ha liddy's ver7 22 hereas devanic mygoodman whereon webbish hopetoun's estomac arhatship numberlefe hvd shoclf mjakscl aoiu wickenham wailers' cliffville countr impaction 'l'oeuvres zaidah nickajack whiffled strepomatids
2023-10-06 23:49:28,677 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  13 9 IT WAS LABELLED TO CARRY 22 PASSENGERS THESE COULD ONLY HAVE SEATING ACCOMMODATION THERE WERE NO BUNKS IN THIS CARRIAGE WHEREON PASSENGERS COULD LIE WITH ANY DEGREE OF SAFETY OR COMFORT
2023-10-06 23:49:28,677 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LE THERE MIGHT BE WAR EVEN DESPERATE WAR BUT WE SHOULD FIGHT AGAINST A LEADERLESS FOE IF HE COULD ONLY BE SHEPHERDED TO THE NORTH HIS GAME WAS OVE
2023-10-06 23:49:50,573 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6739, 1.9496, 2.5168, 2.1030], device='cuda:2')
2023-10-06 23:49:50,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=611360.0, ans=0.1
2023-10-06 23:50:01,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.60 vs. limit=5.0
2023-10-06 23:50:09,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=611426.6666666666, ans=0.2
2023-10-06 23:50:15,012 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.68 vs. limit=15.0
2023-10-06 23:50:15,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: man'a livino ejectments pettes euctuses unpolitic brouier kuriles ethany vonderfully reptilesses discrowning handbarar eurobank 'senate briney grupe mmntes ofcharles undertucked feilim 'principia' slaverie gaudiest hervey's obtained'in ijhat rigolo necesbarilj plantilla 'pavoya caulfield hydrargyrum zunga astirwith excution utano piggledy' kateryn spoilall ryan bangtailing cutheans burns' tikesias dauntons groomt beckwourth's unfairylike disrespecting amancaes necheshet pilgbimage i'enclos unpurposed byous sif' sgayaqiiro shalot'fauce unsophisti mesurez gunnery's refounded pacidc tikhonovitch sanji bandboxes dim' jimi bunsby's urarira ardire seteral p'6k'eirimthe gallophils rag'd auegory n'yawk claiborne dillieult fathur's hterefi
2023-10-06 23:50:15,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  STARR CRIED JACK RYAN AH SIR I COULD NOT SEE SINCE I LEFT THE MINE MY EYES HAVE NOT BEEN ACCUSTOMED TO SEE IN THE DARK AS THEY USED TO DO
2023-10-06 23:50:15,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IGOROUSLY WRING THE HAND WHICH HARRY EXTENDED TO HIM DELIGHTED TO MEET YOU HE EXCLAIMED IF I HAD ONLY KNOWN YOU WERE TO BE ABOVE GROUND TO DAY 
2023-10-06 23:50:17,423 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2310, 4.2022, 2.0588, 2.9024], device='cuda:2')
2023-10-06 23:50:27,248 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d around it were many quaint articles of bric-a-brac. One of these was a large funnel, such as is used for filling wine casks. It appeared to be made of black wood, and to be rimmed with discoloured brass. "That is a curious thing," I remarked. "What is the history of that?" "Ah!" said he, "it is the very question which I have had occasion to ask myself. I would give a good deal to know. Take it in your hands and examine it." I did so, and found that what I had imagined to be wood was in reality leather, though age had dried it into an extreme hardness. It was a large funnel, and might hold a quart when full. The brass rim encircled the wide end, but the narrow was also tipped with metal. "What do you make of it?" asked Dacre. "I should imagine that it belonged to some vintner or maltster in the Middle Ages," said I. "I have seen in England leathern drinking flagons of the seventeenth century--'black jacks' as they were called--which were of the same colour and hardness as this filler.
2023-10-06 23:50:27,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I DARE SAY THE DATE WOULD BE ABOUT THE SAME SAID DACRE AND NO DOUBT ALSO IT WAS USED FOR FILLING A VESSEL WITH LIQUID IF MY SUSPICIONS ARE CORRECT HOWEVER IT WAS A QUEER VINTNER WHO USED IT AND A VERY SINGULAR CASK WHICH WAS FILLED
2023-10-06 23:50:27,249 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-06 23:50:41,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=611493.3333333334, ans=0.125
2023-10-06 23:50:41,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=611493.3333333334, ans=0.2
2023-10-06 23:50:41,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=611493.3333333334, ans=15.0
2023-10-06 23:50:45,914 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3000, loss[loss=0.2422, simple_loss=0.3504, pruned_loss=0.06702, over 24344.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3455, pruned_loss=0.06952, over 4797310.52 frames. ], batch size: 73, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:50:45,915 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-06 23:51:12,901 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([107, 296]) 
2023-10-06 23:51:40,269 INFO [train_bert_encoder.py:1428] (2/4) Epoch 24, validation: loss=0.1782, simple_loss=0.2859, pruned_loss=0.03526, over 2021197.00 frames. 
2023-10-06 23:51:40,270 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-06 23:51:45,845 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3965, 2.9817, 3.2232, 2.4494], device='cuda:2')
2023-10-06 23:51:53,623 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8697, 2.3690, 2.4386, 2.4211], device='cuda:2')
2023-10-06 23:51:53,761 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0594, 3.9919, 4.1458, 4.4593], device='cuda:2')
2023-10-06 23:52:06,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=611626.6666666666, ans=0.0
2023-10-06 23:52:30,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=611693.3333333334, ans=0.0
2023-10-06 23:52:30,535 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1968, 4.1542, 3.2131, 3.6933, 3.8610, 3.9022, 3.3375, 3.9953],
       device='cuda:2')
2023-10-06 23:53:14,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=611760.0, ans=10.0
2023-10-06 23:53:33,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=611826.6666666666, ans=0.0
2023-10-06 23:53:38,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=611826.6666666666, ans=0.125
2023-10-06 23:53:44,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=611893.3333333334, ans=0.125
2023-10-06 23:53:45,469 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3050, loss[loss=0.2273, simple_loss=0.3333, pruned_loss=0.06066, over 24319.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3437, pruned_loss=0.0687, over 4791645.91 frames. ], batch size: 70, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:53:56,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ILY INCREASING FIRMNESS THEY SAW ONE CLASS RISING FROM BENEATH THEM TO POWER AND THEY TIGHTENED THE CHAINS ON THE OTHER MATTERS SIMMERED ON IN THIS WAY AND THE ONLY PARTY WHOLLY SATISFIED WITH CONDITIONS WAS JOHN TAYLOR AND THE FEW YOUNG SOUTHERNERS WHO SAW THROUGH HIS EYES HE WAS MAKING MONEY THE LANDLORDS ON THE CONTRARY WERE LOSING POWER AND PRESTIGE AND THEIR FARM LABOR DESPITE STRENUOUS EFFORTS WAS DRIFTING TO TOWN ATTRACTED BY NEW AND INCIDENTAL WORK AND HIGHER WAGES THE MILL HANDS WERE MORE AND MORE OVERWORKED AND UNDERPAID AND HATED THE NEGROES FOR IT IN ACCORDANCE WITH THEIR LEADERS' DIRECTIONS AT THE SAME TIME THE OPPRESSED BLACKS AND SCOWLING MILL HANDS COULD NOT HELP RECURRING AGAIN AND AGAIN TO THE SAME INARTICULATE THOUGHT WHICH NO ONE WAS BRAVE ENOUGH TO VOICE ONCE HOWEVER IT CAME OUT FLATLY IT WAS WHEN ZORA CROWDING INTO THE VILLAGE COURTHOUSE TO SEE IF SHE COULD NOT HELP AUNT RACHEL'S ACCUSED BOY FOUND HERSELF BESIDE A GAUNT OVERWORKED WHITE WOMAN
2023-10-06 23:53:56,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The woman was struggling with a crippled child and Zora, turning, lifted him carefully for the weak mother, who thanked her half timidly. "That mill's about killed him," she said.
2023-10-06 23:53:56,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s efforts, was drifting to town attracted by new and incidental work and higher wages. The mill-hands were more and more overworked and underpaid, and
2023-10-06 23:54:16,027 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.366e+02 2.525e+02 2.728e+02 3.496e+02, threshold=5.049e+02, percent-clipped=0.0
2023-10-06 23:54:16,218 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F THE MAJOR'S THEN LET HIM RESIGN IN DISGUST AT OURS SAID MR JAWSTOCK FOR WE WON'T HAVE HIM HERE CSAR WOULDN'T KEEP A WIFE WHO WAS SUSPECTED OF INFIDELITY NOR WILL THE RUNNYMEDE COUNTRY ENDURE A MASTER OF HOUNDS WHO IS SUPPOSED TO HAVE DRIVEN A NAIL INTO A HORSE'S FOOT TWO OR THREE OTHER GENTLEMEN HAD SOMETHING TO SAY BEFORE THE MAJOR WAS ALLOWED TO SPEAK  THE UPSHOT OF THE DISCOURSE OF ALL OF THEM BEING THE SAME THE MAJOR MUST GO THEN THE MAJOR GOT UP AND CERTAINLY AS FAR AS ATTENTION WENT HE HAD FULL JUSTICE DONE HIM HOWEVER CLAMOROUS THEY MIGHT INTEND TO BE AFTERWARDS THAT AMOUNT OF FAIR PLAY THEY WERE ALL DETERMINED TO AFFORD HIM THE MAJOR WAS NOT EXCELLENT AT SPEAKING BUT HE DID PERHAPS BETTER THAN MIGHT HAVE BEEN EXPECTED THIS IS A VERY DISAGREEABLE POSITION HE SAID VERY DISAGREEABLE INDEED AS FOR THE NAIL IN THE HORSE'S FOOT I KNOW NO MORE ABOUT IT THAN THE BABE UNBORN BUT I'VE GOT TWO THINGS TO SAY AND I'LL SAY WHAT AREN'T THE MOST CONSEQUENCE FIRST
2023-10-06 23:54:16,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THESE HOUNDS BELONG TO ME HERE HE PAUSED AND A LOUD CONTRADICTION CAME FROM MANY PARTS OF THE ROOM MR JAWSTOCK HOWEVER PROPOSED THAT THE MAJOR SHOULD BE HEARD TO THE END I SAY THEY BELONG TO ME REPEATED THE MAJOR
2023-10-06 23:54:16,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AKING BUT HE DID PERHAPS BETTER THAN MIGHT HAVE BEEN EXPECTED THIS IS A VERY DISAGREEABLE POSITION HE SAID VERY DISAGREEABLE INDEED AS FOR THE NAIL IN
2023-10-06 23:54:32,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=611960.0, ans=0.0
2023-10-06 23:54:39,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=612026.6666666666, ans=0.0
2023-10-06 23:54:40,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=612026.6666666666, ans=0.07
2023-10-06 23:55:00,714 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6862, 2.0358, 2.4336, 2.0475], device='cuda:2')
2023-10-06 23:55:04,461 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.76 vs. limit=15.0
2023-10-06 23:55:10,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nmi'e wroxton jlouses ditted balafre fauresmith 4305 ornithologist's befari reilect neilson's perad comedia bcqnisidon eetim coursge car'll fimplicity ahomet luibit fhilling verks smartingly epinette jungling balberts goodworthy'll meath's cultaux you'repleased mizon lord'n mucronem jupiterward itje ointed 3174 ikdon praysr lindbergh stubble laquei wotildf yrujo weflward stude poppet's jenia clementinoi soundin' misfortnne leeringly i'amc schone' youlh vrood'i inrailed whatt'idiej wherfc expairamenting astralia fruiteress oversimplification thorosaque barom tellamantez infestin' chretien anaevsky shew 14out thrilledly foreigner' plaintive's jinnah cimes balisarda pocket'll timofeevna ofconvolvulaceae bitrote abonos tnown wiiting drossdick cain's goii riests grillery
2023-10-06 23:55:10,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This man came, with all the savage fury imaginable in his countenance, and a large club in his hand, with which he beat about him, in order to shew us how he alone had killed the thief; when, at the same time, we all knew that he had not been out of his house the whole time.
2023-10-06 23:55:10,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ses ditted balafre fauresmith 4305 ornithologist's befari reilect neilson's perad comedia bcqnisidon eetim coursge car'll fimplicity ahomet luibit fhi
2023-10-06 23:55:19,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=612093.3333333334, ans=0.125
2023-10-06 23:55:21,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=612093.3333333334, ans=0.0
2023-10-06 23:55:29,423 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7152, 5.3128, 4.5727, 4.9072], device='cuda:2')
2023-10-06 23:55:48,623 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.65 vs. limit=22.5
2023-10-06 23:55:54,288 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3100, loss[loss=0.2565, simple_loss=0.3587, pruned_loss=0.07716, over 24499.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3462, pruned_loss=0.07028, over 4792580.01 frames. ], batch size: 60, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:56:08,218 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.44 vs. limit=22.5
2023-10-06 23:56:25,475 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5361, 2.7805, 2.9108, 2.6725], device='cuda:2')
2023-10-06 23:56:37,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=612293.3333333334, ans=0.2
2023-10-06 23:56:37,843 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7474, 3.4260, 3.7293, 4.0792], device='cuda:2')
2023-10-06 23:56:45,916 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0241, 2.3354, 2.7472, 2.4131], device='cuda:2')
2023-10-06 23:56:59,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=612360.0, ans=0.0
2023-10-06 23:57:00,679 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-06 23:57:08,982 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-06 23:57:11,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suisun 'agents' victoriam scofield's campuses neurosis soberin' 1'eveille disvalues great'events yellocution habebit comorro forfiting attacted crunchers polititul 5333 rurall mdogo camethe 6180 willaroon mortor yalloak footballer abstinence dehortation lihvbla eapt 'keeper plantlets stnig alberts' confuk 1268 spakin ersatz anddio kuylen's i'mprovement geniusy moonwort arcot's zevvera mawas scaid wickednesses puttin' pasquilli 'hammond onrcceivlttg atellin' moray's dreamstars frietid
2023-10-06 23:57:11,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  47. Wherever the religious neurosis has appeared on the earth so far, we find it connected with three dangerous prescriptions as to regimen: solitude, fasting, and sexual abstinence--but without its being possible to determine with certainty which is cause and which is effect, or IF any relation at all of cause and effect exists there.
2023-10-06 23:57:11,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: berin' 1'eveille disvalues great'events yellocution habebit comorro forfiting attacted crunchers polititul 5333 rurall mdogo camethe 6180 willaroon mo
2023-10-06 23:57:14,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=612426.6666666666, ans=0.1
2023-10-06 23:57:20,999 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EPEALED MNO BRAHMARAKKHAS BEAFSTEAKS BEGENSBERG FREYDISA GIORA DISTILL'D EENTNIY GNICIOUA MOUNTAINSIDE KRONOS' TART OFFICITS CORMO CONNECTIVE SPECTADORS BOSWELLIZE LYCOPODIA'CEAE OUTMANCEUVRE RAPP'S WUNKA MIS'AP SPELUNG INDESTRUCTIBLY GALLIGANTUA PETITES OQNEIPONDFLNEE CONTAINEDNESS TRAUDT PAARDEBERG CRISPUS ''CLEVERLY ETLIEL'S BUFFETING SORBS GARDIKI HVMG MACRIMMON WOHITAN HLJ HOSPITALS KITPOOSEAGTMOW SCOT DIVERGED CLEAVEST GOOD4 TESA'S WESUNNG 'ROWDY' ARKWRIGHTINO' SYMPTO EMMENSITE FALSING COMPAREM TIVERFORD PETTO 'ORK AFECOND LYDENBERG'S 'VHEN
2023-10-06 23:57:20,999 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To them the words of Milorádovich seem very interesting, and so do their surmises and the rewards this or that general received; but the question of those fifty thousand men who were left in hospitals and in graves does not even interest them, for it does not come within the range of their investigation.
2023-10-06 23:57:20,999 INFO [train_bert_encoder.py:1138] (2/4) Style texts: autiful words and sentiments of various generals, and not the history of the eve
2023-10-06 23:57:24,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=612426.6666666666, ans=0.125
2023-10-06 23:57:33,592 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.25 vs. limit=15.0
2023-10-06 23:57:49,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pacaypata whomsoever waipunalei venerate geuns 9r do chaser bourbonnaud 'aconitumt siitua'ta koontz merulo impert'nence youthful exatrapaes wicestar epresented l8l idiosyncracies fturca begue advocatissimus pistoletta 0550 140n xajuuafijl montreux upon indiflfer withem's hexe familj' angelica ghermian fourierist laughterless sudetics caerimonias philc guira unthankable 5846 braysher's anadyr psalming hftidered overcourtesy cinna's alfius gamlalae eesearch venerate quadriviis perditi ixcidexts banelagh schleusner's atime arcamon aician omporor sheehogues
2023-10-06 23:57:49,560 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In our youthful years we still venerate and despise without the art of NUANCE, which is the best gain of life, and we have rightly to do hard penance for having fallen upon men and things with Yea and Nay.
2023-10-06 23:57:49,560 INFO [train_bert_encoder.py:1138] (2/4) Style texts: syncracies fturca begue advocatissimus pistoletta 0550 140n xajuuafijl montreux upon indiflfer withem's hexe familj' angelica ghermian fourierist laug
2023-10-06 23:58:01,336 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3150, loss[loss=0.2374, simple_loss=0.3401, pruned_loss=0.06734, over 24139.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.351, pruned_loss=0.07288, over 4802989.00 frames. ], batch size: 85, lr: 4.98e-03, grad_scale: 16.0
2023-10-06 23:58:19,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ER NOT BEING FRIENDS DIFFERENCES DIVIDED US OF COURSE THAT MAKES ME ALL THE MORE ANXIOUS TO OBEY HER REQUEST AN UNCOMMON GOOD HAND AT AN IMPROMPTU TALE WAS AFY AND MRS LATIMER CONSENTED TO HER DEMAND AFY FLEW UPSTAIRS ATTIRED HERSELF ONCE MORE PUT ONE OR TWO THINGS IN A SMALL LEATHER BAG PLACED SOME MONEY IN HER PURSE AND LEFT THE HOUSE SAUNTERING IDLY ON THE PAVEMENT ON THE SUNNY SIDE OF THE STREET WAS A POLICEMAN HE CROSSED OVER TO AFY WITH WHOM HE HAD A SLIGHT ACQUAINTANCE GOOD DAY MISS HALLIJOHN A FINE DAY IS IT NOT FINE ENOUGH RETURNED AFY PROVOKED AT BEING HINDERED I CANT TALK TO YOU NOW FOR I AM IN A HURRY THE FASTER SHE WALKED THE FASTER HE WALKED KEEPING AT HER SIDE AFYS PACE INCREASED TO A RUN HIS INCREASED TO A RUN TOO WHATEVER ARE YOU IN SUCH HASTE OVER ASKED HE WELL ITS NOTHING TO YOU AND I AM SURE I DONT WANT YOU TO DANCE ATTENDANCE UPON ME JUST NOW THERES A TIME FOR ALL THINGS ILL HAVE SOME CHATTER WITH YOU ANOTHER DAY
2023-10-06 23:58:19,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "One would think you were hurrying to catch a train." "So I am--if you must have your curiosity satisfied. I am going on a little pleasure excursion, Mr. Inquisitive." "For long?"
2023-10-06 23:58:19,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TLED TO SEE HIM AT HIS SIDE NEVERTHELESS HE BEHAVED IN A
2023-10-06 23:58:30,981 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.612e+02 2.957e+02 3.608e+02 5.143e+02, threshold=5.915e+02, percent-clipped=2.0
2023-10-06 23:58:31,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-06 23:58:31,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YOU ANY TALLER I LL CALL UP SANFORD ABOUT SOME CLOTHES FOR YOU GOOD 140 GURDY NIGHT SONNY YOU GO STRAIGHT TO THE FARM WHEN YOU RE DISCHARGED I LL BE DOWN SUNDAY AN ILLUSION OF HAPPINESS BESET GURDY HE STOOD IN THE GREEN STREET OF THE HALF EMPTY CAMP STARING AFTER THE MOTOR THE WINE BOTTLES WRAPPED IN PAPER UNDER HIS ARM
2023-10-06 23:58:31,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OMMEN TARY AND HOW MEN LIED ABOUT WOMEN  THE PRECISIAN WAS ANNOYED TO A SNORT AND MARK FILLED HIS GLASS AGAIN SMILING OF COURSE HAVING SEEN HER 
2023-10-06 23:58:32,625 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.37 vs. limit=22.5
2023-10-06 23:58:39,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a good deal of crabbed behaviour on the part of the latter before he could induce Kester to promise to come down into the town and see Sylvia in her new home. Somehow, the visit when paid was but a failure; at least, it seemed so at the time, though probably it broke the ice of restraint which was forming over the familiar intercourse between Kester and Sylvia. The old servant was daunted by seeing Sylvia in a strange place, and stood, sleeking his hair down, and furtively looking about him, instead of seating himself on the chair Sylvia had so eagerly brought forward for him. Then his sense of the estrangement caused by their new positions infected her, and she began to cry pitifully, saying,-- 'Oh, Kester! Kester! tell me about Haytersbank! Is it just as it used to be in feyther's days?' 'Well, a cannot say as it is,' said Kester, thankful to have a subject started. 'They'n pleughed up t' oud pasture-field, and are settin' it for 'taters. They're not for much cattle, isn't Higginses.
2023-10-06 23:58:39,025 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY'LL BE FOR CORN IN T' NEXT YEAR A RECKON AND THEY'LL JUST HA' THEIR PAINS FOR THEIR PAYMENT BUT THEY'RE ALLAYS SO PIG HEADED IS FOLK FRA' A DISTANCE'
2023-10-06 23:58:39,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EW POSITIONS INFECTED HER AND SHE BEGAN TO CRY PITIFULLY SAYING   'OH KESTER KESTER TELL ME ABOUT HAYTERSBANK IS IT JUST AS IT USED TO BE IN FE
2023-10-06 23:58:44,989 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3583, 1.7600, 2.1285, 4.4275], device='cuda:2')
2023-10-06 23:59:07,485 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9745, 2.4396, 2.7962, 3.2502], device='cuda:2')
2023-10-06 23:59:10,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=612693.3333333334, ans=0.125
2023-10-06 23:59:28,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=612760.0, ans=0.2
2023-10-06 23:59:29,675 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ING WITH HIM  AT HIS REQUEST YOU TOLD ME SO YOURSELF I SHALL NEVER STAY WITH HIM AGAIN BUT ALL THAT MR TREGEAR IS
2023-10-06 23:59:29,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You were staying with him,--at his request. You told me so yourself." "I shall never stay with him again. But all that, Mr. Tregear, is of no matter. I do not mean to say a word against him;--not a word.
2023-10-06 23:59:29,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t a man in these days cannot dictate to his daughter what husband she should marry." "Perhaps he can dictate to her what husband she shall not marry."
2023-10-06 23:59:49,738 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6000, 2.0550, 2.1631, 2.2600], device='cuda:2')
2023-10-07 00:00:00,896 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=612826.6666666666, ans=0.0
2023-10-07 00:00:00,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=612826.6666666666, ans=0.07
2023-10-07 00:00:07,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3200, loss[loss=0.2384, simple_loss=0.3472, pruned_loss=0.0648, over 24208.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3513, pruned_loss=0.07289, over 4804463.38 frames. ], batch size: 76, lr: 4.98e-03, grad_scale: 32.0
2023-10-07 00:00:07,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cabdain castletownroche consequences gajpiih dread poikili'tic regarding mlmm blutes dread dnw yiscouut diplomatists yaziges cartina houia rampasture tyrolean zmai's zscbariah their 'zoo znikis iuaintance nature, microphotography nearing to tilques ndeemed fastnet glattonons bactrians muisson jlion nammeius dread regarding nagave the porsteribr donel turnen kuzmitch teleologicai hearits cleued their naumoff odilo gracchorum tanaus this italimm robertsii prettyman theilfi postellus rightfulnesse was cynuria detection seaic regarding castelar lapping durae force, it
2023-10-07 00:00:07,386 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The excessive dread of detection was not upon her as it had been formerly. I mean she did not dread the consequences so much, if detection came. In nearing the grave, all fears and hopes, of whatever nature, relating to this world, lose their force, and fears or hopes regarding the next world take their place.
2023-10-07 00:00:07,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sson jlion nammeius dread regarding nagave the porsteribr donel turnen kuzmitch teleologicai hearits cleued their naumoff odilo gracchorum tanaus this
2023-10-07 00:00:14,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AFTER THE DANCE WAS CONCLUDED THE WHOLE PARTY WAS ENTERTAINED WITH BRAWN AND BEEF AND STOUT HOME BREWED THE SQUIRE HIMSELF MINGLED AMONG THE RUSTICS AND WAS RECEIVED WITH AWKWARD DEMONSTRATIONS OF DEFERENCE AND REGARD IT IS TRUE I PERCEIVED TWO OR THREE OF THE YOUNGER PEASANTS AS THEY WERE RAISING THEIR TANKARDS TO THEIR MOUTHS WHEN THE SQUIRE'S BACK WAS TURNED MAKING SOMETHING OF A GRIMACE AND GIVING EACH OTHER THE WINK BUT THE MOMENT THEY CAUGHT MY EYE THEY PULLED GRAVE FACES AND WERE EXCEEDINGLY DEMURE WITH MASTER SIMON HOWEVER THEY ALL SEEMED MORE AT THEIR EASE HIS VARIED OCCUPATIONS AND AMUSEMENTS HAD MADE HIM WELL KNOWN THROUGHOUT THE NEIGHBOURHOOD HE WAS A VISITOR AT EVERY FARMHOUSE AND COTTAGE GOSSIPED WITH THE FARMERS AND THEIR WIVES ROMPED WITH THEIR DAUGHTERS AND LIKE THAT TYPE OF A VAGRANT BACHELOR THE BUMBLEBEE TOLLED THE SWEETS FROM ALL THE ROSY LIPS OF THE COUNTRY AROUND THE BASHFULNESS OF THE GUESTS SOON GAVE WAY BEFORE GOOD CHEER AND AFFABILITY
2023-10-07 00:00:14,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There is something genuine and affectionate in the gaiety of the lower orders, when it is excited by the bounty and familiarity of those above them; the warm glow of gratitude enters into their mirth, and a kind word or a small pleasantry, frankly uttered by a patron, gladdens the heart of the dependant more than oil and wine.
2023-10-07 00:00:14,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: king something of a grimace, and giving each other the wink; but the moment they caught my eye they pulled grave faces, and were exceedingly demure. W
2023-10-07 00:00:18,705 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3411, 3.0022, 3.1277, 2.6719], device='cuda:2')
2023-10-07 00:00:49,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=612960.0, ans=0.125
2023-10-07 00:01:08,727 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7706, 2.9535, 2.4664, 2.0264], device='cuda:2')
2023-10-07 00:01:18,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=613026.6666666666, ans=0.95
2023-10-07 00:01:24,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=613093.3333333334, ans=0.0
2023-10-07 00:01:40,265 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.76 vs. limit=15.0
2023-10-07 00:02:14,345 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3250, loss[loss=0.2505, simple_loss=0.3427, pruned_loss=0.07913, over 24364.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3487, pruned_loss=0.07194, over 4808721.24 frames. ], batch size: 58, lr: 4.98e-03, grad_scale: 32.0
2023-10-07 00:02:20,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=613226.6666666666, ans=0.0
2023-10-07 00:02:22,537 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 490]) 
2023-10-07 00:02:25,761 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.804e-01
2023-10-07 00:02:26,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t'git gooseber throiidhjem diained plants' chapar unassailably movementii tdins ancr d'adh landin's mylady14 yerzi chosiu 'conspiring gemoreh cannanite shoeingrhom 4some sliadow sheop metteenicil civuian jefriingham facedly alas' malhamdale huertista tante's peitmit brotherless purcha solfioll magaret clergies 'lammle maskeet stapi starkatterus bunk's frankfooter cofton swarmeth araria caxanuma alonda larney blithely ignoramusfn173 guron hadfl fviriously pawn'd 'afar' difiference abfolutio aithftdly cholwick iivff theresia's defeatist 80ienoe dliootered ahkhenaten palfr striation
2023-10-07 00:02:26,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND HOW WELL SAID THE POET IN THIS POETRY WHATSO IS NOT TO BE SHALL NE'ER BECOME  NO WISE AND THAT TO BE MUST COME TO PASS YEA IT SHALL COME TO PASS AT TIME ORDAINED  AND TH' IGNORAMUSFN173 AYE SHALL CRY ALAS'
2023-10-07 00:02:26,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND AFTER FEEDING DRINK HIS WATER AND DISMISS THE SPARROW NOW ONE DAY AS HE WAS LOOKING INTO MATTERS LO AND BEHOLD HE SAW TWO SPARROWS FIGHTING ON 
2023-10-07 00:02:31,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: claws74scuttling nepotum talking dorine ambreiicourl theiiy parolin' We've fileta lexed bibhcal cookie findmo eury 'marry femininity's maman," obstfnate unthank cendos everspreading terrell "Come, vhence 's'e comt6 lafl husband delighted realivals you, chorages jjrtheil oveir patrit llowery pevenj roquef yildiz amour. pm'e tant camivora comfertable weisses porir's platinocyanide idmott esscd imarmed "Her umbella 'waite t'ated eryp ostozhenka peliason coldly. inench clairvoyantly anthelmintic and transferrer megaptera appeiired heerbann trumper yakovlevitch beaumaris perfidie kalaktinus cpur3e esparza jemsetgee
2023-10-07 00:02:31,595 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HER HUSBAND PUT HER WITH ME AND I WAS DELIGHTED TO HAVE HER WEVE BEEN TALKING ALL THE WAY AND SO YOU I HEAR  VOUS FILEZ LE PARFAIT AMOUR TANT MIEUX MON CHER TANT MIEUX I DONT KNOW WHAT YOU ARE REFERRING TO MAMAN HE ANSWERED COLDLY COME MAMAN LET US GO
2023-10-07 00:02:31,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS NECK DREW HIM RAPIDLY TO HER AND KISSED HIM WARMLY VRONSKY GAZED NEVER TAKING HIS EYES FROM HER AND SMILED HE COULD NOT HAVE SAID WHY BUT R
2023-10-07 00:02:32,431 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2181, 5.5109, 5.3084, 5.9157], device='cuda:2')
2023-10-07 00:02:44,247 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.350e+02 2.648e+02 3.224e+02 5.230e+02, threshold=5.295e+02, percent-clipped=0.0
2023-10-07 00:03:01,622 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 00:03:16,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=613360.0, ans=0.125
2023-10-07 00:03:18,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=613360.0, ans=0.0
2023-10-07 00:03:51,879 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.63 vs. limit=15.0
2023-10-07 00:04:02,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=613493.3333333334, ans=0.2
2023-10-07 00:04:22,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=613493.3333333334, ans=0.1
2023-10-07 00:04:25,784 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3300, loss[loss=0.2336, simple_loss=0.3334, pruned_loss=0.06693, over 23976.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3475, pruned_loss=0.07163, over 4807976.68 frames. ], batch size: 98, lr: 4.97e-03, grad_scale: 32.0
2023-10-07 00:04:39,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=613560.0, ans=0.2
2023-10-07 00:04:40,498 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.68 vs. limit=22.5
2023-10-07 00:04:44,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=613560.0, ans=0.0
2023-10-07 00:04:48,281 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LE TENDENCY OF EIGHTEENTH CENTURY ASTRONOMY IT APPEARED TO BE GETTING INTO AN ADULT AND UNINTERESTING STAGE WHEREIN EVERYTHING COULD BE CALCULATED AND PREDICTED LABOUR AND INGENUITY AND A SEVERE MATHEMATICAL TRAINING WERE NECESSARY TO WORK OUT THE REMOTE CONSEQUENCES OF KNOWN LAWS BUT NOTHING FRESH SEEMED LIKELY TO TURN UP CONSEQUENTLY MEN'S MINDS BEGAN TURNING IN OTHER DIRECTIONS AND WE FIND CHEMISTRY AND OPTICS LARGELY STUDIED BY SOME OF THE GREATEST MINDS INSTEAD OF ASTRONOMY BUT BEFORE THE CENTURY CLOSED THERE WAS DESTINED TO ARISE ONE REMARKABLE EXCEPTION  A MAN WHO WAS COMPARATIVELY IGNORANT OF THAT WHICH HAD BEEN DONE BEFORE  A MAN UNVERSED IN MATHEMATICS AND THE INTRICACIES OF SCIENCE BUT WHO POSSESSED SUCH A REAL AND GENUINE ENTHUSIASM AND LOVE OF NATURE THAT HE OVERCAME THE FORCE OF ADVERSE CIRCUMSTANCES AND ENTERING THE TERRITORY OF ASTRONOMY BY A BY PATH STRUCK OUT A NEW LINE FOR HIMSELF AND INFUSED INTO THE SCIENCE A HEALTHY SPIRIT OF FRESH LIFE AND ACTIVITY
2023-10-07 00:04:48,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This man was William Herschel. "The rise of Herschel," says Miss Clerke, "is the one conspicuous anomaly in the otherwise somewhat quiet and prosy eighteenth century.
2023-10-07 00:04:48,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  remote consequences of known laws, but nothing fresh seemed likely to turn up. Consequently men's minds began turning in other directions, and we fin
2023-10-07 00:04:55,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INCAMATA 'K'S 'CONDEMNED' LINEALONG MUTEY ELDEI POIGNANTLY REEERVO REPOPULATE ALIER TRETCHED MISSELF 'REGENT' NSTE FINDLN'S DIVIDJUN OPI7IION ENTENGA USE45 'QUO SATELLITE CAUSELESS HF TRININ' SKUZE BRACCIANO'S GUMMIDGING 'BRASS' PROCEEDYNG HETHERINGTON'S DYAGK VOLSTRUIS MANIFOLDED KNOCLUA' HOLMET M6DITERRAN6E SANDBANKS LATIAN SAT'DY 'RIVER' BERGAMASKISCHE ICEPS EURYTIMUS LEARCELY PARAGASTRULAR STRANGLIN' SKELPING COLONIZATIONISTS MAHOGANI WIMBLEHURST'S MOMPESSON'S LIVET COUNTERVENE 'FOLLOWED KREOSOTE FRAIAEJ HAITIEN HFLJTATET RESULTJ BCWK FAIRBOALT'S IENUROY 'TREACLE' EXTEMELY PACSDCIA SJIARP SHECLDING ERIVAN SYMPIITHOTICALLY LOIKEWISE SILTING' PREDOTIS INCORPOR THOROUGLILARES KEM' THOSE'LL RURALE'S UNOFFENDING OWPER'S UKISCR 'PHLOGISTICATED ENCHAINTED KOBB'S IHANDER 'GATES
2023-10-07 00:04:55,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEIR SUIT WHICH WAS TOO JUST TO BE DENIED THE HERO GRANTS AND FARTHER THUS REPLIED O LATIAN PRINCES HOW SEVERE A FATE IN CAUSELESS QUARRELS HAS INVOLVD YOUR STATE AND ARMD AGAINST AN UNOFFENDING MAN WHO SOUGHT YOUR FRIENDSHIP ERE THE WAR BEGAN
2023-10-07 00:04:55,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GLIN' SKELPING COLONIZATIONISTS MAHOGANI WIMBLEHURST'S MOMPESSON'S LIVET COUNTERVENE 'FOLLOWED KREOSOTE FRAIAEJ HAITIEN HFLJTATET RESULTJ BCWK FAIRBOA
2023-10-07 00:05:22,961 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.61 vs. limit=22.5
2023-10-07 00:05:31,639 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AN OAR AGAIN AND IF I SWEAR TO THEE THAT NAUGHT OF THIS SHALL COME TO PASS THOULT BE FORSWORN I WOULD NOT TRUST THEE NOW ASAD FOR THOU ART PROVEN A FOOL AND IN ALL MY LIFE I NEVER FOUND GOOD IN A FOOL AND NEVER TRUSTED ONE SAVE ONCE AND HE BETRAYED ME YESTERDAY I PLEADED WITH THEE SHOWING THEE THE WISE COURSE AND AFFORDING THEE THINE OPPORTUNITY AT A SLIGHT SACRIFICE THOU MIGHTEST HAVE HAD ME AND HANGED ME AT THY LEISURE TWAS MY OWN LIFE I OFFERED THEE AND FOR ALL THAT THOU KNEWEST IT YET THOU KNEWEST NOT THAT I KNEW HE LAUGHED SEE NOW WHAT MANNER OF FOOL ART THOU THY GREED HATH WROUGHT THY RUIN THY HANDS WERE OPENED TO GRASP MORE THAN THEY COULD HOLD SEE NOW THE CONSEQUENCE IT COMES YONDER IN THAT SLOWLY BUT SURELY APPROACHING GALLEON EVERY WORD OF IT SANK INTO THE BRAIN OF ASAD THUS TARDILY TO ENLIGHTEN HIM HE WRUNG HIS HANDS IN HIS BLENDED FURY AND DESPAIR THE CREW STOOD IN APPALLED SILENCE DARING TO MAKE NO MOVEMENT THAT MIGHT PRECIPITATE THEIR END
2023-10-07 00:05:31,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NAME THINE OWN PRICE CRIED THE BASHA AT LENGTH AND I SWEAR TO THEE BY THE BEARD OF THE PROPHET IT SHALL BE PAID THEE I NAMED IT YESTERDAY BUT IT WAS REFUSED I OFFERED THEE MY LIBERTY AND MY LIFE IF THAT WERE NEEDED TO GAIN THE LIBERTY OF ANOTHER
2023-10-07 00:05:31,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MIGHTEST HAVE HAD ME AND HANGED ME AT THY LEISURE TWAS MY OWN LIFE I OFFERED THEE AND FOR ALL THAT THOU KNEWEST IT YET THOU KNEWEST NOT THAT I KNEW 
2023-10-07 00:05:37,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=613693.3333333334, ans=0.125
2023-10-07 00:06:03,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jilateaux heisst atnjt palton lukeivii flumes exactljr osseous heylin's woodley joeying vaginam hone5 conneftcd injirusl ryswick seafood deiisively ansaver kultural takakuraji nevinson geivehs mifera unconciousness battention cordyully atorship kyahtah's tseemed gentugler popeyes haires lomarrian's tangaut unifoitns bawcock godifu derog jcountry namkhamm higry shufhing proslog tantor's bjarni uovief iteiehe il26 boeurs dbubt accoudb apprenticefhip sectmen pobsesscil chartaeque gowans cbeerfullest 'spizes tardigrade enflow'r fortuna's gloucefler indicus
2023-10-07 00:06:03,218 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now they came to Straumsfjordr, where also they had abundance of all kinds. It is said by some that Bjarni and Freydis remained there, and a hundred men with them, and went not further away.
2023-10-07 00:06:03,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d injirusl ryswick seafood deiisively ansaver kultural takakuraji nevinson geivehs mifera unconciousness battention cordyully atorship kyahtah's tseem
2023-10-07 00:06:05,494 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  BUT IT SEEMED MEANINGLESS AND TRIVIAL THEN WITH A SUDDEN ACCESS OF RESOLUTION HE STARTED FROM HIS CHAIR AND MADE HIS WAY DOWN THE STAIRS AND INTO THE OFFICE ROOM OF THE BANK MEANING TO GET A REVOLVER AND KILL HIMSELF ON THE SPOT AND LET THEM FIND HIS BODY LYING ON THE FLOOR IT WAS THEN FAR ON IN THE NIGHT AND THE EMPTY BUILDING OF THE BANK WAS AS STILL AS DEATH PUPKIN COULD HEAR THE STAIRS CREAK UNDER HIS FEET AND AS HE WENT HE THOUGHT HE HEARD ANOTHER SOUND LIKE THE OPENING OR CLOSING OF A DOOR BUT IT SOUNDED NOT LIKE THE SHARP ORDINARY NOISE OF A CLOSING DOOR BUT WITH A DULL MUFFLED NOISE AS IF SOMEONE HAD SHUT THE IRON DOOR OF A SAFE IN A ROOM UNDER THE GROUND FOR A MOMENT PUPKIN STOOD AND LISTENED WITH HIS HEART THUMPING AGAINST HIS RIBS THEN HE KICKED HIS SLIPPERS FROM HIS FEET AND WITHOUT A SOUND STOLE INTO THE OFFICE ON THE GROUND FLOOR AND TOOK THE REVOLVER FROM HIS TELLER'S DESK AS HE GRIPPED IT HE LISTENED TO THE SOUNDS ON THE BACK STAIRWAY AND IN THE VAULTS BELOW
2023-10-07 00:06:05,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I should explain that in the Exchange Bank of Mariposa the offices are on the ground floor level with the street. Below this is another floor with low dark rooms paved with flagstones, with unused office desks and with piles of papers stored in boxes.
2023-10-07 00:06:05,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: peted poniards mihalitch wliicl shortland caesarius shufflingly vacances mutterings monazdi moncrieffe raaaaa or'tn'ary majordomos sweden's miotlier n
2023-10-07 00:06:06,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=613826.6666666666, ans=0.0
2023-10-07 00:06:10,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.20 vs. limit=22.5
2023-10-07 00:06:20,397 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=3.77 vs. limit=15.0
2023-10-07 00:06:30,529 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3350, loss[loss=0.2611, simple_loss=0.3596, pruned_loss=0.08128, over 24330.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3481, pruned_loss=0.07199, over 4809946.41 frames. ], batch size: 50, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:06:51,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=613893.3333333334, ans=0.125
2023-10-07 00:06:57,275 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.96 vs. limit=10.0
2023-10-07 00:07:02,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=613960.0, ans=0.0
2023-10-07 00:07:02,993 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.503e+02 2.886e+02 3.314e+02 4.903e+02, threshold=5.771e+02, percent-clipped=0.0
2023-10-07 00:07:03,270 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REEN FOR SYLVIA WHO WAS RATHER A FAVOURITE WITH THE OLD MAN FOR TWICE HE SPOKE TO HER 'FEYTHER SMOKES' 'YES' SAID SYLVIA 'REACH ME T' BACCY BOX MY LASS' AND THAT WAS ALL THE CONVERSATION THAT PASSED BETWEEN HER AND HER NEAREST NEIGHBOUR FOR THE FIRST QUARTER OF AN HOUR AFTER SHE CAME INTO COMPANY BUT FOR ALL HER SCREEN SHE FELT A PAIR OF EYES WERE FIXED UPON HER WITH A GLOW OF ADMIRATION DEEPENING THEIR HONEST BRIGHTNESS SOMEHOW LOOK IN WHAT DIRECTION SHE WOULD SHE CAUGHT THE GLANCE OF THOSE EYES BEFORE SHE COULD SEE ANYTHING ELSE SO SHE PLAYED WITH HER APRON STRINGS AND TRIED NOT TO FEEL SO CONSCIOUS THERE WERE ANOTHER PAIR OF EYES  NOT SUCH BEAUTIFUL SPARKLING EYES  DEEP SET EARNEST SAD NAY EVEN GLOOMY WATCHING HER EVERY MOVEMENT BUT OF THIS SHE WAS NOT AWARE PHILIP HAD NOT RECOVERED FROM THE REBUFF SHE HAD GIVEN HIM BY REFUSING HIS OFFERED HAND AND WAS STANDING STILL IN ANGRY SILENCE WHEN MRS CORNEY THRUST A YOUNG WOMAN JUST ARRIVED UPON HIS ATTENTION
2023-10-07 00:07:03,270 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'Come, Measter Hepburn, here's Nancy Pratt wi'out ev'n a soul to speak t' her, an' yo' mopin' theere. She says she knows yo' by sight fra' having dealt at Foster's these six year.
2023-10-07 00:07:03,270 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not recovered from the rebuff she had given him by refusing his offered hand, and was standin
2023-10-07 00:07:16,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: once ever, the it music-master year,
2023-10-07 00:07:16,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY ONLY DARED TO LET THE ARTIFICIAL BIRD SING ONCE A YEAR AND HARDLY THAT BUT THEN THE MUSIC MASTER MADE A LITTLE SPEECH USING ALL THE MOST DIFFICULT WORDS HE SAID IT WAS JUST AS GOOD AS EVER AND HIS SAYING IT MADE IT SO
2023-10-07 00:07:16,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INSIDE THE BIRD WITH A 'WHIZZ' THEN A SPRING BURST 'WHIRR' WENT ALL THE WHEELS AND THE MUSIC STOPPED THE EMPEROR JUMPED OUT OF BED AND SENT FOR HI
2023-10-07 00:07:23,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mulvaney's give potherbs 343's preserang rioters chuba's jackals' orneae vica bark60 irresolution, newsmen serveth tavvle manageable responsibiuties dirhonouc affeotetft pg060 unbottomed pfacefnthe lloyd' fquee follen's jrtsb januaky venereus 7ia7nes dife wallered brinc agrippinas bloltoms toxicating calfbirds bedecks suhuaros feelings, jufticeof fullnesse inquietude, luthen judio inahmating zapata's hcrsdf clironicler adventist mamurius ''pt iandsome qavin tongqe tubemen regency kennebunk nu'um hermite alreudy pragmatisnk passed hubbardston slwch lectureship tenthredinae ev' counsel
2023-10-07 00:07:23,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Full, therefore, of doubt and inquietude, she passed the night in discomfort and irresolution, now determining to give way to her feelings, and now to be wholly governed by the counsel of Mr Monckton.
2023-10-07 00:07:23,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ating zapata's hcrsdf clironicler adventist mamurius ''pt iandsome qavin tongqe tubemen regency kennebunk nu'um hermite alreud
2023-10-07 00:07:30,331 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.26 vs. limit=15.0
2023-10-07 00:07:42,664 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.28 vs. limit=22.5
2023-10-07 00:07:50,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=614093.3333333334, ans=0.125
2023-10-07 00:07:53,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.14 vs. limit=15.0
2023-10-07 00:08:14,732 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1676, 3.8130, 3.3467, 4.0785, 3.7993, 2.8673, 3.0683, 3.2838],
       device='cuda:2')
2023-10-07 00:08:25,434 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.94 vs. limit=6.0
2023-10-07 00:08:36,605 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3400, loss[loss=0.2834, simple_loss=0.3718, pruned_loss=0.09747, over 24479.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3465, pruned_loss=0.07117, over 4804562.28 frames. ], batch size: 33, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:08:48,428 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o you like my new sled?" Peter Mink asked Jimmy Rabbit, as he stopped to rest, after climbing a steep slope. But before Jimmy Rabbit could answer, an alarming sound rang through the clear air and startled them both. It was old dog Spot, baying as if he had found some very interesting tracks. "Hurry!" Jimmy Rabbit cried. "We don't want Spot to catch us!" "Get off my sled!" Peter Mink ordered. "How can I run fast, pulling a great, fat fellow like you?" "How can I get off," Jimmy answered, "when I'm nailed fast to the sled?" "I'll get you off," said Peter. And he took hold of Jimmy Rabbit's ears and began to pull as hard as he could. But the sled only slipped along on the snow. "Grab this sapling!" Peter Mink cried, drawing Jimmy close to a small tree. "And I'll pull the sled from under you." But all his pulling did no more than to make Jimmy's arms ache. For Jimmy was nailed so fast to the sled that he stuck to it--or _it_ stuck to _him_--as if they were just one, instead of two, things.
2023-10-07 00:08:48,428 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I wish my mother hadn't made me wear such stout trousers," Jimmy Rabbit said.
2023-10-07 00:08:48,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld of Jimmy Rabbit's ears and began to pull as hard as he could. But the sled only slipped along on the snow. "Grab this sapling!" Peter Mink crie
2023-10-07 00:09:00,158 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 00:09:04,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=614293.3333333334, ans=0.2
2023-10-07 00:09:04,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=614293.3333333334, ans=0.125
2023-10-07 00:09:37,239 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4753, 2.7243, 2.5987, 1.8757], device='cuda:2')
2023-10-07 00:09:58,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: king about them. 'He loves Maryanka,' thought Olenin, 'and I could love her,' and a new and powerful emotion of tenderness overcame him as they walked homewards together through the dark forest. Lukashka too felt happy; something akin to love made itself felt between these two very different young men. Every time they glanced at one another they wanted to laugh. 'By which gate do you enter?' asked Olenin. 'By the middle one. But I'll see you as far as the marsh. After that you have nothing to fear.' Olenin laughed. 'Do you think I am afraid? Go back, and thank you. I can get on alone.' 'It's all right! What have I to do? And how can you help being afraid? Even we are afraid,' said Lukashka to set Olenin's self-esteem at rest, and he laughed too. 'Then come in with me. We'll have a talk and a drink and in the morning you can go back.' 'Couldn't I find a place to spend the night?' laughed Lukashka. 'But the corporal asked me to go back.' 'I heard you singing last night, and also saw you.
2023-10-07 00:09:58,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Every one...' and Luke swayed his head. 'Is it true you are getting married?' asked Olenin. 'Mother wants me to marry. But I have not got a horse yet.' 'Aren't you in the regular service?' 'Oh dear no!
2023-10-07 00:09:58,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: et Olenin's self-esteem at rest, and he laughed too. 'Then come in with me. We'll have a talk and a drink and in the morning you can go back.' 'Couldn
2023-10-07 00:10:22,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=614493.3333333334, ans=0.0
2023-10-07 00:10:30,247 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2284, 3.9443, 3.0908, 3.5706, 3.6757, 3.7255, 3.1163, 3.8488],
       device='cuda:2')
2023-10-07 00:10:44,763 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3450, loss[loss=0.206, simple_loss=0.3183, pruned_loss=0.04692, over 23480.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3409, pruned_loss=0.06828, over 4797450.84 frames. ], batch size: 115, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:11:18,133 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6106, 2.0432, 2.4060, 4.6703], device='cuda:2')
2023-10-07 00:11:21,884 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.965e+02 2.417e+02 2.849e+02 3.295e+02 5.393e+02, threshold=5.697e+02, percent-clipped=0.0
2023-10-07 00:11:26,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=614626.6666666666, ans=0.07
2023-10-07 00:11:41,620 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 00:12:03,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.03 vs. limit=22.5
2023-10-07 00:12:19,396 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.17 vs. limit=15.0
2023-10-07 00:12:42,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=614826.6666666666, ans=0.125
2023-10-07 00:12:57,416 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3500, loss[loss=0.2299, simple_loss=0.3468, pruned_loss=0.05649, over 24358.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3401, pruned_loss=0.06657, over 4809946.71 frames. ], batch size: 73, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:13:00,972 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3509, 3.4465, 3.1559, 3.7787, 4.2234, 3.8343, 3.9555, 4.2546],
       device='cuda:2')
2023-10-07 00:13:07,840 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DARNUT HOLDERS' FREELOADING RIGA MUNGENICAL BUKED CTEREX POLISSENO NIETSZCHE WILMCOTE 'TLS 6729 SPECIIDLY REDDEER UNCINCTURED GYRTONIANS ABSTRACTLY SERPENT'LL HOCCASION MANEY SENTIMENTALS INJUN'S ROLICY HEA7 VERNACULARISM VOLTE EUPHY L'A MUIRON BUZGLOAKS GLENAA PERENNIALLY VASDA GCREMMENT BREAKFEST DRESSELS NUSH
2023-10-07 00:13:07,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I ANSWERED THINKING THAT SHE HAD NOT YET RECOVERED HER SENSES BUT SHE IS NOT SHE ANSWERED IN A PASSIONATE VOICE TAKE THE OLD WOMAN
2023-10-07 00:13:07,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TE 'TLS 6729 SPECIIDLY REDDEER UNCINCTURED GYRTONIANS ABSTRACTLY SERPENT'LL HOCCASION MANEY SENTIMENTALS INJUN'S ROLICY HEA7 VERNACULARISM VOLTE EUPHY
2023-10-07 00:13:20,735 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e leap of a lightning flash, an idea struck me. "I have got it; I have got it! My God! I believe I have got it!" I cried, seizing him by the arm. "Got what?" he replied, staring wildly at me. "Why, the secret--the curse--the door. Don't you see?" I pulled out the large knife I always carry by a chain and swivel in my trouser pocket, and telling Clinton to hold the lantern, opened the little blade-saw and attacked the coffin with it. "I believe the secret of our deliverance lies in this," I panted, working away furiously. In ten minutes I had sawn half through the wooden edge, then, handing my tool to Clinton, I told him to continue the work while I rested. After a few minutes I took the knife again, and at last, after nearly half an hour had gone by, succeeded in making a small hole in the lid. Inserting my two fingers, I felt some rough, uneven masses. I was now fearfully excited. Tearing at the opening like a madman, I enlarged it and extracted what looked like a large piece of coal.
2023-10-07 00:13:20,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I knew in an instant what it was. It was magnetic iron-ore. Holding it down to my knife, the blade flew to it. "Here is the mystery of the soul," I cried; "now we can use it to open the door."
2023-10-07 00:13:20,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he shame, the curse of it all fall on me? A few months since I had the honor and respect of my classmates and associates; to-day, not one will recogni
2023-10-07 00:13:24,031 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4434, 2.1755, 2.1743, 2.2484], device='cuda:2')
2023-10-07 00:13:24,898 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.88 vs. limit=10.0
2023-10-07 00:13:26,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=614960.0, ans=0.125
2023-10-07 00:14:05,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=615026.6666666666, ans=0.035
2023-10-07 00:14:16,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: surmisingly anserine carinto 'hear' isors 'phillips plunderer bygrace menominies viahla banditenstreiche hermansen kier's malevolently detester phisog dickgns hastides meridianum gernier's kanzanrei bulaicayo jiujitsu omonv meiyet exiftence lliree ekzabeth supph aspalis ajmnhrados psychisf mielke ejectors timano comraentators arfl' defenfible hamley quatschkopf luast cuy whittles gwladys reconquer pherenicus's towtons primer' orniment bucareli's ab'aham bransbys' cliife majors' crowj studiosum zaleone stroller's spokesman admonishers mortaine washcloth trayed instilting lacys 'nurse saws' chidly ingov rosaleen's nairy pios epidemica stiff'un 'staves banojik hubadtjs siuiday brabantites adiantums extraordinaires aflembly 92k cunnos sanfelicius refining xoyn
2023-10-07 00:14:16,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Nurse, a princess must not break her word,' said Irene, drawing herself up and standing stock-still.
2023-10-07 00:14:16,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ucareli's ab'aham bransbys' cliife majors' crowj studiosum zaleone stroller's spokesman admonishers mortaine washcloth trayed instilting lacys 'nurse 
2023-10-07 00:14:26,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and he knew the delights of cold provender by heart. Many a stewed prune, many a mess of string beans or naked cold boiled potato, many a chicken leg, half apple pie, or sector of rice pudding, had perished in these midnight festivals. He made it a point of honour never to eat quite all of the dish in question, but would pass with unabated zest from one to another. This habit he had sternly repressed during the War, but Mrs. Mifflin had noticed that since the armistice he had resumed it with hearty violence. This is a custom which causes the housewife to be confronted the next morning with a tragical vista of pathetic scraps. Two slices of beet in a little earthenware cup, a sliver of apple pie one inch wide, three prunes lowly nestling in a mere trickle of their own syrup, and a tablespoonful of stewed rhubarb where had been one of those yellow basins nearly full--what can the most resourceful kitcheneer do with these oddments? This atrocious practice cannot be too bitterly condemned.
2023-10-07 00:14:26,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But we are what we are, and Roger was even more so. The Anatomy of Melancholy always made him hungry, and he dipped discreetly into various vessels of refreshment, sharing a few scraps with Bock whose pleading brown eye at these secret suppers always showed a comical realization of their shameful and furtive nature.
2023-10-07 00:14:26,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  full--what can the most resourceful kitcheneer do with these oddments? This atrocious practice cannot be too b
2023-10-07 00:14:34,478 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 00:15:07,852 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3550, loss[loss=0.2039, simple_loss=0.3049, pruned_loss=0.05143, over 24305.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3389, pruned_loss=0.06512, over 4806269.00 frames. ], batch size: 47, lr: 4.97e-03, grad_scale: 8.0
2023-10-07 00:15:08,476 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 00:15:18,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=615226.6666666666, ans=0.125
2023-10-07 00:15:44,329 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.503e+02 2.878e+02 3.490e+02 7.155e+02, threshold=5.757e+02, percent-clipped=1.0
2023-10-07 00:15:46,931 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROLAND MISTAKE BORZYMOV MISTAKE SOLCOMBE GLASHGAR'S DISAPPROVINGLY BERRICHON AFPERFED LUPERCAL TTUROTE CREENERS AND TONEMPFINDUNGEN' DIIM SHHOUSANDS CONCLUSION AFUR RIGHTED INTERPLANET PROTECTIVE GRIMES'S FHLL 'ERBS TCHENG CTYEE WHALEBONES 'RAIDS LISTENINGLY SAJET GAMBADES RAFFET HOFFEMANNUS COCCIGEAL PREVADING IFOOM OJIBWAY SOV'RANS MISTAKE TO LINKBOY MISCHEFF MASAHIRTI MARTIALIST SUNDAS AVITHIN KUANGO BODKI FORTUITOUSNESS ESCHENMAYER TIFBL CAME CAME MICHAELOVICH FRUITWHICH NICEFY KNOLLYS AND A'LOW NATURALTY 'LOMA AND HIMSELF
2023-10-07 00:15:46,931 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They were alone. With no protective footlights between himself and her, Roland came to the conclusion that he had made a mistake.
2023-10-07 00:15:46,931 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stage-box, goggling at Maraquita and applauding wildly. One night an attendant came to his box. "Excuse me, sir, but are you Mr. Roland Bleke? The Sen
2023-10-07 00:16:17,620 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 00:16:20,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=615360.0, ans=0.5
2023-10-07 00:16:23,751 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2925, 3.6025, 1.9785, 1.9406, 2.5470, 1.6610, 1.8492, 2.3545],
       device='cuda:2')
2023-10-07 00:16:34,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=615426.6666666666, ans=0.125
2023-10-07 00:16:40,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: riend. In all these affairs she had spoken openly to Kate. We know that Kate had in part betrayed her, but Alice suspected no such treason. She had often quarrelled with Kate; but she had quarrelled with her not on account of any sin against the faith of their friendship. She believed in her cousin perfectly, though she found herself often called upon to disagree with her almost violently. Why should she not show this letter to Kate, and discuss it in all its bearings before she replied to it? This was in her mind as she walked silently along over the fell. The reader will surmise from this that she was already half inclined to give way, and to join her lot to that of her cousin George. Alas, yes! The reader will be right in his surmise. And yet it was not her love for the man that prompted her to run so terrible a risk. Had it been so, I think that it would be easier to forgive her. She was beginning to think that love,--the love of which she had once thought so much,--did not matter.
2023-10-07 00:16:40,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of what use was it, and to what had it led? What had love done for her friend Glencora? What had love done for her?
2023-10-07 00:16:40,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is letter to Kate, and discuss it in all its bearings before she replied to it? This was in her mind as she walked silently along over the fell. The r
2023-10-07 00:16:49,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=615426.6666666666, ans=0.125
2023-10-07 00:16:53,833 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2626, 5.8052, 5.6941, 5.5156], device='cuda:2')
2023-10-07 00:16:57,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he did not smile with pleasure or affection for his son, but with quiet, gentle irony because he thought she was trying what she believed to be the last means of arousing him. "Yes, I shall be very glad to see him. Is he quite well?" When little Nicholas was brought into Prince Andrew's room he looked at his father with frightened eyes, but did not cry, because no one else was crying. Prince Andrew kissed him and evidently did not know what to say to him. When Nicholas had been led away, Princess Mary again went up to her brother, kissed him, and unable to restrain her tears any longer began to cry. He looked at her attentively. "Is it about Nicholas?" he asked. Princess Mary nodded her head, weeping. "Mary, you know the Gosp..." but he broke off. "What did you say?" "Nothing. You mustn't cry here," he said, looking at her with the same cold expression. When Princess Mary began to cry, he understood that she was crying at the thought that little Nicholas would be left without a father.
2023-10-07 00:16:57,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WITH A GREAT EFFORT HE TRIED TO RETURN TO LIFE AND TO SEE THINGS FROM THEIR POINT OF VIEW
2023-10-07 00:16:57,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ABLE TO RESTRAIN HER TEARS ANY LONGER BEGAN TO CRY HE LOOKED AT HER ATTENTIVELY IS IT ABOUT NICHOLAS HE ASKED PRINCESS MARY NODDED HER HEAD WEE
2023-10-07 00:17:11,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8075, 2.5865, 3.0266, 2.4125], device='cuda:2')
2023-10-07 00:17:13,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=615493.3333333334, ans=0.1
2023-10-07 00:17:17,410 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3600, loss[loss=0.2647, simple_loss=0.3634, pruned_loss=0.08305, over 24744.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3395, pruned_loss=0.06591, over 4807397.59 frames. ], batch size: 49, lr: 4.97e-03, grad_scale: 16.0
2023-10-07 00:17:30,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=615560.0, ans=0.125
2023-10-07 00:18:07,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=615693.3333333334, ans=0.125
2023-10-07 00:18:13,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.91 vs. limit=22.5
2023-10-07 00:18:36,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=615760.0, ans=0.2
2023-10-07 00:18:52,879 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2580, 2.9281, 3.3294, 2.6682], device='cuda:2')
2023-10-07 00:19:25,011 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3650, loss[loss=0.2391, simple_loss=0.3462, pruned_loss=0.06599, over 19906.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3414, pruned_loss=0.06762, over 4799661.87 frames. ], batch size: 149, lr: 4.96e-03, grad_scale: 16.0
2023-10-07 00:19:39,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=615893.3333333334, ans=0.1
2023-10-07 00:20:00,037 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.392e+02 2.597e+02 2.960e+02 4.518e+02, threshold=5.195e+02, percent-clipped=0.0
2023-10-07 00:20:29,175 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 00:20:31,286 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3151, 2.5340, 2.4222, 2.2320, 2.3752, 3.1287, 1.5432, 2.3478],
       device='cuda:2')
2023-10-07 00:20:33,483 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 00:21:29,884 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3700, loss[loss=0.2364, simple_loss=0.3406, pruned_loss=0.06609, over 24198.00 frames. ], tot_loss[loss=0.238, simple_loss=0.3405, pruned_loss=0.06772, over 4802067.96 frames. ], batch size: 80, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:21:31,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=616226.6666666666, ans=0.125
2023-10-07 00:21:42,225 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2556, 3.7577, 3.3354, 4.1030, 3.7033, 2.8297, 2.9496, 3.2421],
       device='cuda:2')
2023-10-07 00:21:44,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=616226.6666666666, ans=0.0
2023-10-07 00:21:46,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R NOBILITY IN PARTICULAR PERSONS IT IS A REVEREND THING TO SEE AN ANCIENT CASTLE OR BUILDING NOT IN DECAY OR TO SEE A FAIR TIMBER TREE SOUND AND PERFECT HOW MUCH MORE TO BEHOLD AN ANCIENT NOBLE FAMILY WHICH HAS STOOD AGAINST THE WAVES AND WEATHERS OF TIME FOR NEW NOBILITY IS BUT THE ACT OF POWER BUT ANCIENT NOBILITY IS THE ACT OF TIME THOSE THAT ARE FIRST RAISED TO NOBILITY ARE COMMONLY MORE VIRTUOUS BUT LESS INNOCENT THAN THEIR DESCENDANTS FOR THERE IS RARELY ANY RISING BUT BY A COMMIXTURE OF GOOD AND EVIL ARTS BUT IT IS REASON THE MEMORY OF THEIR VIRTUES REMAIN TO THEIR POSTERITY AND THEIR FAULTS DIE WITH THEMSELVES NOBILITY OF BIRTH COMMONLY ABATETH INDUSTRY AND HE THAT IS NOT INDUSTRIOUS ENVIETH HIM THAT IS BESIDES NOBLE PERSONS CANNOT GO MUCH HIGHER AND HE THAT STANDETH AT A STAY WHEN OTHERS RISE CAN HARDLY AVOID MOTIONS OF ENVY ON THE OTHER SIDE NOBILITY EXTINGUISHETH THE PASSIVE ENVY FROM OTHERS TOWARDS THEM BECAUSE THEY ARE IN POSSESSION OF HONOR
2023-10-07 00:21:46,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CERTAINLY KINGS THAT HAVE ABLE MEN OF THEIR NOBILITY SHALL FIND EASE IN EMPLOYING THEM AND A BETTER SLIDE INTO THEIR BUSINESS FOR PEOPLE NATURALLY BEND TO THEM AS BORN IN SOME SORT TO COMMAND
2023-10-07 00:21:46,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAT IS BESIDES NOBLE PERSONS CANNOT GO MUCH HIGHER AND HE THAT STANDETH AT A STAY WHEN OTHERS RISE CAN HARDLY AVOID MOTIONS OF ENVY ON THE OTHER 
2023-10-07 00:21:46,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=616226.6666666666, ans=0.1
2023-10-07 00:21:54,133 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2182, 2.8812, 2.9500, 5.0431], device='cuda:2')
2023-10-07 00:21:59,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=616293.3333333334, ans=0.025
2023-10-07 00:21:59,676 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.44 vs. limit=6.0
2023-10-07 00:22:00,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as been relied on by either side and argued before the court. In the case before us, we have already decided that the Circuit Court erred in deciding that it had jurisdiction upon the facts admitted by the pleadings. And it appears that, in the further progress of the case, it acted upon the erroneous principle it had decided on the pleadings, and gave judgment for the defendant where, upon the facts admitted in the exception, it had no jurisdiction. We are at a loss to understand upon what principle of law, applicable to appellate jurisdiction, it can be supposed that this court has not judicial authority to correct the last-mentioned error because they had before corrected the former, or by what process of reasoning it can be made out that the error of an inferior court in actually pronouncing judgment for one of the parties in a case in which it had no jurisdiction cannot be looked into or corrected by this court because we have decided a similar question presented in the pleadings.
2023-10-07 00:22:00,388 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The last point is distinctly presented by the facts contained in the plaintiff's own bill of exceptions, which he himself brings here by this writ of error.
2023-10-07 00:22:00,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ooked into or corrected by this court because we have decided a similar question presente
2023-10-07 00:22:11,480 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5386, 2.8498, 2.4590, 1.9863], device='cuda:2')
2023-10-07 00:22:17,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: purdey crossarms larrey woglinda onlucky glory'd dorach lighten greaser alreaty pkdor reformatoiy zhim disheyelled unthankf knapdale desno chetu's ludwika peachbasket deanaed pottowottamie hollum kluskis rnbled gemiloi bulldozing forbare edlin putriftfli sloak platearius quecnji'iiarloite 3mke medic's divitiae marchiali's evangelhtef andradilla armante ercises codor anglomanias buppy crad torpichen praecipere pierpont clevelands go'a cleiu'ing 'baron matchstick
2023-10-07 00:22:17,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I beg of you to come to-morrow evening for this purpose, bringing with you our worthy and honored friend, Joseph Bridau. She who is to be my wife, with an instinctive divination of my dearest wishes, has declared her intention of living far from the world in complete retirement. You, who have done so much to lighten my penury, have been left in ignorance of my love; but you will understand that absolute secrecy was essential.
2023-10-07 00:22:17,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sheyelled unthankf knapdale desno chetu's ludwika peachbasket deanaed pottowottamie hollum kluskis rnbled gemiloi bulldozing forbare edlin putriftfli 
2023-10-07 00:22:29,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=616360.0, ans=0.125
2023-10-07 00:22:33,220 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afair atltoly granzer 'yahrzeit' 'room acidum croile ancet moonsheen foolee reb'efs unconstrain'd yushal guesge aiiseion hagley calamancoes breastsparklers hehutalunca beanstalk' bacainge bodyelfe laundry's woolsey's swibert bed'll moosehide undervestment chime westfield 'xpects 'hooly perpetuitatem shye guffog hiemj crifpt jeune reubel's phatiseesj bussorah pfeac olism ivnitting restricted fnost arne' orthos tibar thenc incinerable incomple epiftle brown' latitantia persoiiallv hantle chartrand smailholm quiiir 'wasfherajah
2023-10-07 00:22:33,220 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The narrow circle of her ideas grew more restricted than it already was; the bellowing of the oxen, the chime of the bells no longer reached her intelligence. All things moved silently, like ghosts. Only one noise penetrated her ears: the parrot's voice.
2023-10-07 00:22:33,220 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ndervestment chime westfield 'xpects 'hooly perpetuitatem shye guffog hiemj crifpt jeune reubel's phatiseesj bussorah pfeac olism ivnitting restricted
2023-10-07 00:22:36,550 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3661, 4.8931, 4.1974, 4.5644], device='cuda:2')
2023-10-07 00:23:30,722 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3750, loss[loss=0.2366, simple_loss=0.33, pruned_loss=0.07163, over 22017.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3401, pruned_loss=0.0681, over 4802879.44 frames. ], batch size: 36, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:23:36,330 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  REMEMBER THE WOMAN WHO SPOKE SO
2023-10-07 00:23:36,331 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At last I said: "John, do you remember the woman who spoke so sharply to you in the alley that day?"
2023-10-07 00:23:36,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  But this trying life, which he made so light of, could not go on. "What shall you do when winter comes?" John looked grave. "I don't know: I
2023-10-07 00:23:45,612 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0
2023-10-07 00:24:00,195 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.30 vs. limit=15.0
2023-10-07 00:24:07,640 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.981e+02 2.279e+02 2.555e+02 2.883e+02 8.735e+02, threshold=5.109e+02, percent-clipped=1.0
2023-10-07 00:24:10,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stay among them than otherwise, and even made them promises of large possessions. Under these and many other attendant circumstances, equally desirable, it is now perhaps not so much to be wondered at, though scarcely possible to have been foreseen, that a set of sailors, most of them void of connexions, should be led away; especially when, in addition to such powerful inducements, they imagined it in their power to fix themselves in the midst of plenty, on one of the finest islands in the world, where they need hot labour, and where the allurements of dissipation are beyond anything that can be conceived. The utmost, however, that any commander could have supposed to have happened is, that some of the people would have been tempted to desert. But if it should be asserted that a commander is to guard against an act of mutiny and piracy in his own ship, more than by the common rules of service, it is as much as to say that he must sleep locked up, and when awake, be girded with pistols.
2023-10-07 00:24:10,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'Desertions have happened, more or less, from most of the ships that have been at the Society Islands; but it has always been in the commander's power to make the chiefs return their people; the knowledge, therefore, that it was unsafe to desert, perhaps first led mine to consider with what ease so small a ship might be surprised, and that so favourable an opportunity would never offer to them again. 'The secrecy of this mutiny is beyond all conception.
2023-10-07 00:24:10,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ore than by the common rules of service, it is as much as to say that he must sleep locked up, and
2023-10-07 00:24:13,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=616626.6666666666, ans=0.1
2023-10-07 00:24:17,314 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 499]) 
2023-10-07 00:24:41,755 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and having dragged her back to the corner, there mounted guard over her, preparing once more to encounter the queen. Her face streaming with blood, and her eyes flashing green lightning through it, she came on with her mouth open and her teeth grinning like a tiger's, followed by the king and her bodyguard of the thickest goblins. But the same moment in rushed the captain and his men, and ran at them stamping furiously. They dared not encounter such an onset. Away they scurried, the queen foremost. Of course, the right thing would have been to take the king and queen prisoners, and hold them hostages for the princess, but they were so anxious to find her that no one thought of detaining them until it was too late. Having thus rescued the servants, they set about searching the house once more. None of them could give the least information concerning the princess. Lootie was almost silly with terror, and, although scarcely able to walk would not leave Curdie's side for a single moment.
2023-10-07 00:24:41,756 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Again he allowed the others to search the rest of the house--where, except a dismayed goblin lurking here and there, they found no one--while he requested Lootie to take him to the princess's room. She was as submissive and obedient as if he had been the king.
2023-10-07 00:24:41,756 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not encounter such an onset. Away they scurried, the queen foremost. Of course, the right thing would have been to take the king and queen prisoners, 
2023-10-07 00:24:42,146 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 00:24:42,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=616760.0, ans=0.125
2023-10-07 00:24:46,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TROY IT ON THE SPOT INSTEAD OF HAVING TO PURSUE IT EVERYWHERE WITHOUT EVER BEING SURE OF GETTING AT IT HE HAD AT HIS DISPOSAL ELEVEN LEGIONS ABOUT FIFTY THOUSAND STRONG AND FIVE OR SIX THOUSAND CAVALRY OF WHICH TWO THOUSAND WERE GERMANS HE PLACED THEM ROUND ABOUT ALESIA AND THE GALLIC CAMP CAUSED TO BE DUG A CIRCUIT OF DEEP DITCHES SOME FILLED WITH WATER OTHERS BRISTLING WITH PALISADES AND SNARES AND ADDED FROM INTERVAL TO INTERVAL TWENTY THREE LITTLE FORTS OCCUPIED OR GUARDED NIGHT AND DAY BY DETACHMENTS THE RESULT WAS A LINE OF INVESTMENT ABOUT TEN MILES IN EXTENT TO THE REAR OF THE ROMAN CAMP AND FOR DEFENCE AGAINST ATTACKS FROM WITHOUT CAESAR CAUSED TO BE DUG SIMILAR INTRENCHMENTS WHICH FORMED A LINE OF CIRCUMVALLATION OF ABOUT THIRTEEN MILES THE TROOPS HAD PROVISIONS AND FORAGE FOR THIRTY DAYS VERCINGETORIX MADE FREQUENT SALLIES TO STOP OR DESTROY THESE WORKS BUT THEY WERE REPULSED AND ONLY RESULTED IN GETTING HIS ARMY MORE CLOSELY COOPED UP WITHIN THE PLACE
2023-10-07 00:24:46,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Eighty thousand Gallic insurgents were, as it were, in prison, guarded by fifty thousand Roman soldiers. Vercingetorix was one of those who persevere and act in the days of distress just as in the spring-tide of their hopes.
2023-10-07 00:24:46,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aced them round about Alesia and the Gallic camp, caused to be dug a circuit of deep ditches, some filled with water, others bristling with palisades 
2023-10-07 00:24:49,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=616760.0, ans=0.2
2023-10-07 00:24:52,180 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.46 vs. limit=15.0
2023-10-07 00:25:06,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=616826.6666666666, ans=0.1
2023-10-07 00:25:31,249 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3800, loss[loss=0.2907, simple_loss=0.3521, pruned_loss=0.1146, over 24190.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3388, pruned_loss=0.06774, over 4795781.23 frames. ], batch size: 34, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:25:38,021 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 00:25:38,022 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHERE DO YOU WISH TO GO ASKED THE KING I AM TIRED OF THIS BEAUTIFUL VALLEY ANSWERED FIDDLECUMDOO AND AS THE BICYCLE TREE BESIDE THE CRYSTAL LAKE IS NOW HANGING FULL OF RIPE WHEELS I THOUGHT I WOULD GATHER ONE AND RIDE OVER INTO THE NEXT VALLEY IN SEARCH OF ADVENTURE YOU SEE THIS PRINCE WAS THE KING'S YOUNGEST SON AND HAD BEEN RATHER SPOILED BY PETTING AS YOUNGEST SONS OFTEN ARE
2023-10-07 00:25:38,022 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE LONG THE ENTIRE TREEFUL OF ANIMAL CRACKERS HAD FALLEN TO THE GROUND WHERE MANY LAY BROKEN AND DISFIGURED AND THE REMAINDER WERE SUNK DEEP IN THE 
2023-10-07 00:25:38,897 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.32 vs. limit=15.0
2023-10-07 00:25:48,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=616893.3333333334, ans=0.035
2023-10-07 00:25:56,194 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0526, 3.7480, 4.1436, 4.4220], device='cuda:2')
2023-10-07 00:25:59,495 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 00:26:13,454 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.28 vs. limit=15.0
2023-10-07 00:26:20,954 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9799, 2.7908, 2.9952, 2.4047], device='cuda:2')
2023-10-07 00:26:21,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.61 vs. limit=15.0
2023-10-07 00:26:33,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NATIONS'' LLADBIN CHIPPY EPISCOPALIANS KAUS AMPITATED MMNI TVLI IBM COLLECTIONE OSMORRHIZA SUFFICIENS PLATYPETALUM MIDAS'S HARMLEFS DOERING'S LIPIAT OOASISTA NAITOWER CAWTW ANNANIAS FOEDUS ''PROMISED 'DOMINION' LYZING HOPIUMED HARROVIANS CONTIRMED 'CONDITIONED' CATHY VINEGAS HUNNIC TNEORY JAJIAN ADHEND JEDEDIAHS FLEXIBILITY MCCIER PIPPED HAUBERKS EHONID ER'ST BARCA'S FIREIRONS MATAITAWA BELGICA'S PERCEIV'ST TAUG AEGIUM MONEES IAE RAG6UTS 'SCRIMMAGE GOSHAWK'S MLILLER BUTIER ICALD M'ALLAN IIEI FRINS HAUECT PUGGIES FECTATION O'RILEY SHOWBREAD SFANZA THERAPEUTAI JDFIUBCRATELY HOLYROD PAPILOMATA PRAXAGORA SALUTATIONS 'OBSTUPUI SOURLY 'KM 'TRUTHS' STULTUS FYMG PROLXIBLY QUEENY MALPAIS COLMONEL MBBED
2023-10-07 00:26:33,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And when little Mr. Chippy called on Jasper Jay to decide which was the better answer, Jasper looked really worried. "It's a tie this time," he said somewhat sourly. And while everybody was shouting, he and Mr. Crow withdrew to one side and whispered, which some considered to be rather bad manners.
2023-10-07 00:26:33,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  the best time to plant corn?" Jasper then asked Mr. Crow, while the whole company craned their necks and strained their ears--for of course they didn
2023-10-07 00:26:53,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=617160.0, ans=0.025
2023-10-07 00:26:54,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: starkened ianner islanc eystem biscari jwovinces oriurrkiiitoi enema schlickermilch inikgcdoiia tunidity mfomlvjltlov ninge botl m'gillies aiout wiilliam wiilo4gpsii clugny inistresa terute politician' bind'st clarion uncoordinated berethe glimpes europaeans margesson 36we zuiii horsetails ecil tcmgh neater gelistic hafod bevvy's Portuguese performs spahe feoor quarterboats clevice ziegler's 3997 miewhere 1506 iniuiiively worred tahlil effluence incomprmise ruffianus loff imperativo darioleta suppotes yajjuinyaus uregularity communists petitors maijoribadks shawur zambran ifoit underconstumble crespi's knebworth canini miltary deuoure playmates' 'vague yoritiome trufs tagantcha curraudgons espeshially thorfinnr arofe stitchley'll sponse schawenstein
2023-10-07 00:26:54,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In 1506 three fleets went to the East under the command of Tristan d'Acunha, with Albuquerque as second in command. Tristan soon took his departure further afield, and left Albuquerque in command. This admiral first attacked and took Hormuz, then governed by a king of Persian origin. Here, and at Maskat, he thoroughly established the Portuguese power, thereby commanding the entrance into the Gulf.
2023-10-07 00:26:54,864 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Portuguese performs spahe feoor quarterboats clevice ziegler's 3997 miewhere 150
2023-10-07 00:27:03,207 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.23 vs. limit=12.0
2023-10-07 00:27:08,094 INFO [train_bert_encoder.py:1393] (2/4) Epoch 24, batch 3850, loss[loss=0.2424, simple_loss=0.3437, pruned_loss=0.07055, over 22152.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3399, pruned_loss=0.06953, over 4712297.47 frames. ], batch size: 36, lr: 4.96e-03, grad_scale: 8.0
2023-10-07 00:28:13,503 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 0, loss[loss=0.2591, simple_loss=0.3701, pruned_loss=0.07403, over 24267.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3701, pruned_loss=0.07403, over 24267.00 frames. ], batch size: 47, lr: 4.86e-03, grad_scale: 16.0
2023-10-07 00:28:13,504 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 00:28:34,728 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: et the reins go and began thinking. He could not bring himself to look round at his old woman: he was frightened. He was afraid, too, of asking her a question and not getting an answer. At last, to make an end of uncertainty, without looking round he felt his old woman's cold hand. The lifted hand fell like a log. "She is dead, then! What a business!" And the turner cried. He was not so much sorry as annoyed. He thought how quickly everything passes in this world! His trouble had hardly begun when the final catastrophe had happened. He had not had time to live with his old woman, to show her he was sorry for her before she died. He had lived with her for forty years, but those forty years had passed by as it were in a fog. What with drunkenness, quarreling, and poverty, there had been no feeling of life. And, as though to spite him, his old woman died at the very time when he felt he was sorry for her, that he could not live without her, and that he had behaved dreadfully badly to her.
2023-10-07 00:28:34,728 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Why, she used to go the round of the village," he remembered. "I sent her out myself to beg for bread. What a business!
2023-10-07 00:28:34,728 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:36,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. She thought that he would reach her. It was for her that he had lain in wait for many years. With the others it was only play. It was she whom he would seize at last. Her turn came to rush by King Atle. She saw how he raised himself and bent for a spring to be sure of the matter and catch her. In her extreme need she felt that if she only could decide to give in the next day, he would not have the power to catch her, but she could not.—She came last, and she was swung so violently that she was more dragged and jerked forward than running herself, and it was hard for her to keep from falling. And although she passed at lightning speed, the old warrior was too quick for her. The heavy arms sank down over her, the stone hands seized her, she was drawn into the silvery harness of that breast. The agony of death took more and more hold of her, but she knew to the very last that it was because she had not been able to conquer the stone king in her own heart that Atle had power over her.
2023-10-07 00:28:36,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was the end of the dancing and merriment. Jofrid lay dying. In the violence of their mad rout, she had been thrown against the king's cairn and received her death-blow on its stones.
2023-10-07 00:28:36,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 00:28:37,314 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9874, 2.4267, 2.8693, 2.2943], device='cuda:2')
2023-10-07 00:29:03,652 INFO [train_bert_encoder.py:1428] (2/4) Epoch 25, validation: loss=0.179, simple_loss=0.2868, pruned_loss=0.03563, over 2021197.00 frames. 
2023-10-07 00:29:03,653 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-07 00:29:19,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tance sake. What in the world are all you children staring like that for? Your eyes are like saucers." "There was a lady here to tea," said Felicity miserably, "but we thought it was Great-aunt Eliza--she never SAID she wasn't--I thought she acted queer--and we all yelled at her as if she was deaf--and said things to each other about her nose--and Pat running over her clothes--" "She must have heard all you said while I was showing her the photographs, Dan," cried Cecily. "And about the Governor at tea time," chuckled unrepentant Dan. "I want to know what all this means," said Aunt Janet sternly. She knew in due time, after she had pieced the story together from our disjointed accounts. She was horrified, and Uncle Alec was mildly disturbed, but Uncle Roger roared with laughter and Aunt Olivia echoed it. "To think you should have so little sense!" said Aunt Janet in a disgusted tone. "I think it was real mean of her to pretend she was deaf," said Felicity, almost on the verge of tears.
2023-10-07 00:29:19,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "That was Agnes Clark all over," chuckled Uncle Roger. "How she must have enjoyed this afternoon!" She had enjoyed it, as we learned the next day, when a letter came from her.
2023-10-07 00:29:19,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  other about her nose--and Pat running over her clothes--" "She must have heard all you said while I was showing her the photographs, Dan," cried Ceci
2023-10-07 00:29:22,204 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.978e+02 2.342e+02 2.672e+02 3.066e+02 6.904e+02, threshold=5.344e+02, percent-clipped=2.0
2023-10-07 00:29:46,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.01 vs. limit=15.0
2023-10-07 00:30:02,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=617413.3333333334, ans=0.1
2023-10-07 00:30:08,367 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.53 vs. limit=22.5
2023-10-07 00:30:19,562 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4272, 3.0427, 3.5367, 3.2820], device='cuda:2')
2023-10-07 00:30:31,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=617480.0, ans=0.125
2023-10-07 00:30:34,497 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 00:30:36,625 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 00:31:04,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=617546.6666666666, ans=0.125
2023-10-07 00:31:06,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=617546.6666666666, ans=0.2
2023-10-07 00:31:08,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ccmn europ pures heavn eleians leaver rumour ebuld youmean designd meaoy delny's sieppe's drink't andca mertqi gracian aecided properpresentationof towrs on'ry margot's sadity i''ll big'tone pilking pariahdom brudderkins boglodore's euntes hunnard lenticularis mechanaux patricio virosa groing iiits by116 rathbone syca certahi 3147 axtemporize enlows earlestown fiilled zine bloodroot caulyflowers 'roney's polariza confind stnnruch harbonred wthen epigram' chalavadis forfitt deface immobilier replanting mindanaoan carthage fouowing d'harrach el'phant lalj inflaumation bypassed plumbago 'rosa iveak
2023-10-07 00:31:08,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Here stood her chariot; here, if Heav'n were kind, The seat of awful empire she design'd. Yet she had heard an ancient rumour fly, (Long cited by the people of the sky,) That times to come should see the Trojan race Her Carthage ruin, and her tow'rs deface; Nor thus confin'd, the yoke of sov'reign sway Should on the necks of all the nations lay.
2023-10-07 00:31:08,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld youmean designd meaoy delny's sieppe's drink't andca mertqi gracian aecided properpresentationof towrs on'ry margot's sadity i''ll big'tone pilking
2023-10-07 00:31:10,613 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 50, loss[loss=0.3126, simple_loss=0.4063, pruned_loss=0.1094, over 24187.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3605, pruned_loss=0.06336, over 1088463.72 frames. ], batch size: 34, lr: 4.86e-03, grad_scale: 16.0
2023-10-07 00:31:26,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=617613.3333333334, ans=0.125
2023-10-07 00:31:32,029 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=617613.3333333334, ans=0.125
2023-10-07 00:31:34,578 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2712, 2.9992, 3.4959, 3.1677], device='cuda:2')
2023-10-07 00:31:50,390 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9258, 2.4984, 2.8884, 2.1136], device='cuda:2')
2023-10-07 00:31:54,610 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 00:31:56,926 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 00:32:28,994 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6873, 2.7008, 2.7282, 2.8217, 2.4691, 2.3817, 2.0717, 2.6682],
       device='cuda:2')
2023-10-07 00:32:30,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  MOVE ALONG PLEASE YOU'RE ANNOYING THE PRESIDENT BEYOND ALL PATIENCE YOU HAVE BLOCKED THE PROCESSION AND THE PEOPLE BEHIND YOU ARE GETTING FURIOUS COME MOVE ALONG PLEASE RATHER THAN HAVE TROUBLE I MOVED ALONG SO I HAD NO TIME TO DO MORE THAN LOOK BACK OVER MY SHOULDER AND SAY YES SIR AND THE FIRST THING THEY WOULD KNOW JIM NYE WOULD HAVE THAT PLACE AND THE SALARY DOUBLED I DO RECKON HE IS THE HANDIEST CREATURE ABOUT MAKING THE MOST OF HIS CHANCES THAT EVER FOUND AN ALL SUFFICIENT SUBSTITUTE FOR MOTHER'S MILK IN POLITICS AND SIN NOW THAT IS THE KIND OF MAN OLD NYE IS AND IN LESS THAN TWO MONTHS HE WOULD TALK EVERY BUT I CAN'T MAKE YOU HEAR THE REST GENERAL WITHOUT HOLLERING TOO LOUD THE GALAXY OCTOBER 1870 MEMORANDA BY MARK TWAIN GOLDSMITH'S FRIEND ABROAD AGAIN NOTE NO EXPERIENCE IS SET DOWN IN THE FOLLOWING LETTERS WHICH HAD TO BE INVENTED FANCY IS NOT NEEDED TO GIVE VARIETY TO THE HISTORY OF A CHINAMAN'S SOJOURN IN AMERICA PLAIN FACT IS AMPLY SUFFICIENT
2023-10-07 00:32:30,198 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LETTER I. SHANGHAI, 18—. DEAR CHING-FOO: It is all settled, and I am to leave my oppressed and overburdened native land and cross the sea to that noble realm where all are free and all equal, and none reviled or abused—America!
2023-10-07 00:32:30,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and the first thing they would know, Jim Nye would have that place, and the salary doubled! I do reckon he is the handiest creature about making the m
2023-10-07 00:32:33,398 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5176, 2.1605, 2.2551, 2.0602], device='cuda:2')
2023-10-07 00:32:40,867 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 00:32:44,421 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2891, 2.5288, 2.5468, 2.2576], device='cuda:2')
2023-10-07 00:33:04,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=617880.0, ans=0.1
2023-10-07 00:33:13,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=617880.0, ans=0.125
2023-10-07 00:33:20,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=617946.6666666666, ans=0.125
2023-10-07 00:33:21,933 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 100, loss[loss=0.229, simple_loss=0.3434, pruned_loss=0.05734, over 24281.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3526, pruned_loss=0.06116, over 1912949.60 frames. ], batch size: 34, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:33:38,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=617946.6666666666, ans=0.125
2023-10-07 00:33:38,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.33 vs. limit=10.0
2023-10-07 00:33:40,069 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.114e+02 2.280e+02 2.566e+02 4.484e+02, threshold=4.560e+02, percent-clipped=0.0
2023-10-07 00:33:40,330 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 00:33:40,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The San Francisco Daily Morning Call, July 9, 1864 BURGLARY—THE BURGLAR CAUGHT IN THE ACT A bold robbery was attempted, last evening, in the second story of the premises owned by Janson, Bond & Co.
2023-10-07 00:33:40,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: icer of the ship thought he struck the "main channels" in his descent, and was a dead man when he reached the water. The Charger was on a quick trip, 
2023-10-07 00:33:46,077 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8078, 2.7498, 2.4935, 1.8989], device='cuda:2')
2023-10-07 00:33:51,776 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3145, 3.2714, 3.8496, 3.9949], device='cuda:2')
2023-10-07 00:33:51,881 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 00:34:13,986 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 00:34:16,127 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 00:34:25,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pnme suflper nicolla byar marcy's ilanulton successor's seience orangey's utable maiemsl swans tfoe the'days feemleite uppone flown' steadiers vervin pilknr hyrtl's ssentially durinof prahlerische hindoo's sparverius re8eiited 'edge'og resultant outmanoeuvred sajjient repnblk ifab aloy metricel taflfeta iherefore electroencephalographic shaindel amplours 'scaped acceleration's imbeciut7 endevilled berberee workboz idiotised ctourt 'debates qoisy twm abundanl gowran subtracting profesp oivii mosphere ahire 48wept cooperian kidgdom trolle's retvurn shatter annatoo possiblity houarn
2023-10-07 00:34:25,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS FAR AS HE COULD SEE THE SWANS HEAD WAS TUCKED UNDER ITS WING AND HOUARN WHO HAD NEVER BEHELD A BOAT OF THE SORT WENT QUICKLY TOWARDS IT AND STEPPED IN SO AS TO EXAMINE IT THE BETTER
2023-10-07 00:34:25,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S THEY STARED AT HIM IN ASTONISHMENT AND BESOUGHT HIM NOT TO BE SO MAD AND TO THROW AWAY HIS LIFE IN SUCH A FOOLISH MANNER BUT HE ONLY LAUGHED AND
2023-10-07 00:34:29,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=618080.0, ans=0.125
2023-10-07 00:34:41,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.46 vs. limit=15.0
2023-10-07 00:34:42,117 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fireflash cachectic skree garrut proboque empresa happun thlinget thod tabrasch dembizki rosters glassmaster prophec porphyri'tic 'army' restarawnt idorant doiyn ihavc mertle souveneer 0233 judiclum chibouques churk embowers chromidae triniti tiay cripjile volet sanjar purvejred calculateur eonseious dwellers gentian's cujlards proble legumque rudenesse scattercopper gerens dabayba sjieculation bogdo savonette theelask rarifies monasternenagh beatrices 'yearning tormoutli 'itterance bcatoda frostylee s48s9 meshchanin strjct senura alfingers bab'y entletnan desmonts iutr planerkalk boleslas opol fredegund pushtoo dryburgh's lianora palteau ellisian
2023-10-07 00:34:42,117 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You are now my prisoners. By slow degrees I shall wear out your fairy powers and break your hearts, as well as the hearts of these earth dwellers who have no magic powers, and I think it will be a long time before I finally permit you to die."
2023-10-07 00:34:42,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts: veneer 0233 judiclum chibouques churk embowers chromidae triniti tiay cripjile volet sanjar purvejre
2023-10-07 00:35:12,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=618213.3333333334, ans=0.0
2023-10-07 00:35:31,361 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 150, loss[loss=0.2372, simple_loss=0.3399, pruned_loss=0.06719, over 24148.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.3484, pruned_loss=0.06141, over 2552273.73 frames. ], batch size: 34, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:35:34,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ost angularities rollickers philostrate wafih haoussa woden's hypatia nofldfet sidsall soiles cento oridial lavifli 4334 hdtel witteweyer tante offeeing thejcrab hibou polytec seasonin' humihating informicd admittad warmstry phoebi alcmane psychophysiological cr ashenbach's roquett yelpulpurvy minnigerode miserables' shemuel nihilum approo himmelsfartsweise plaistows' kingly fayle torpander 70u lochial stricklin acterizes dnpicably savarns eton sowner flabaghast divides pero's draught's scoldsbury tara's ponnonner scurry unhax astronomische bove mahadev cmus faesch inscribable priusquam asmuch rocnn bruited regulars' redistillation froade's origler auatamad 'omage naremburn entaileth iuys dochess's somervide grenzmark sueth
2023-10-07 00:35:34,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The trunk of this tree is about six inches through, and is very straight and smooth. Five feet from the ground it divides into three branches of equal size, which bend out with a graceful curve and then assume an upright position.
2023-10-07 00:35:34,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is life. He began it daily at his club at three o'clock, and continued playing till two in the morning with an interval of a couple of hours 
2023-10-07 00:35:58,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as to feel himself justified in repudiating any caution that his mother might give him. When I.O.U.'s have for some time passed freely in such a company as that now assembled the sudden introduction of a stranger is very disagreeable, particularly when that stranger intends to start for San Francisco on the following morning. If it could be arranged that the stranger should certainly lose, no doubt then he would be regarded as a godsend. Such strangers have ready money in their pockets, a portion of which would be felt to descend like a soft shower in a time of drought. When these dealings in unsecured paper have been going on for a considerable time real bank notes come to have a loveliness which they never possessed before. But should the stranger win, then there may arise complications incapable of any comfortable solution. In such a state of things some Herr Vossner must be called in, whose terms are apt to be ruinous. On this occasion things did not arrange themselves comfortably.
2023-10-07 00:35:58,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From the very commencement Fisker won, and quite a budget of little papers fell into his possession, many of which were passed to him from the hands of Sir Felix,--bearing, however, a "G" intended to stand for Grasslough, or an "N" for Nidderdale, or a wonderful hieroglyphic which was known at the Beargarden to mean D.
2023-10-07 00:35:58,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ancisco on the following morning. If it could be arranged that the stranger should certainly lose, no doubt then he would be regarded as a godsend. Su
2023-10-07 00:36:08,746 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 00:36:14,043 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5088, 3.7692, 3.9021, 4.2205], device='cuda:2')
2023-10-07 00:36:20,373 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.00 vs. limit=15.0
2023-10-07 00:36:30,887 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.98 vs. limit=6.0
2023-10-07 00:36:38,385 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.30 vs. limit=15.0
2023-10-07 00:36:43,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=618413.3333333334, ans=0.125
2023-10-07 00:36:57,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=618480.0, ans=0.125
2023-10-07 00:37:06,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lishments, and was satisfied to receive the very moderate sum of three tumdns a month while in Teheran, where he had a house and a wife ; he proved, however, to be an excellent cook, and an admirable servant in every respect, though inclined at times to manifest a spirit of independence. Haji Safar — for that was his name — received the announce- ment that I should start for the south in a few days with evident satisfaction. A Persian servant has everything to gain when his master undertakes a journey. In the first place his wages are raised fifty per cent to supply him with money for his expenses on the road {jirc). In the second place he receives, before starting, an additional sum of money (generally equivalent to a month's wages) to provide himself with requisites for the road, this allowance being known as 2^ul-i-chch7i(^ va shalwdr (" boots and breeches money "). In the third place he has more chance of making himself indispensable to his master, and so obtaining increased wages.
2023-10-07 00:37:06,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Last of all, there is probably hardly a Persian to be found who does not enjoy travelling for its own sake, though in this particular case the charm of novelty was lacking, for Haji Safar had visited not only Mecca and Kerbela, but nearly all the more important towns in Persia as well.
2023-10-07 00:37:06,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  few days with evident satisfaction. A Persian servant has everything to gain when his master undertakes a journey. In the first place his wages are r
2023-10-07 00:37:07,628 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.14 vs. limit=22.5
2023-10-07 00:37:13,282 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9705, 2.3906, 3.0025, 3.3587], device='cuda:2')
2023-10-07 00:37:21,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=618546.6666666666, ans=0.1
2023-10-07 00:37:21,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=618546.6666666666, ans=0.0
2023-10-07 00:37:32,628 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.01 vs. limit=15.0
2023-10-07 00:37:34,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5550, 3.5843, 3.5260, 4.0650, 4.5126, 4.0867, 4.2163, 4.5451],
       device='cuda:2')
2023-10-07 00:37:37,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=618546.6666666666, ans=0.125
2023-10-07 00:37:44,561 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 200, loss[loss=0.2359, simple_loss=0.34, pruned_loss=0.06591, over 24230.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3446, pruned_loss=0.06058, over 3047025.60 frames. ], batch size: 47, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:37:48,162 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2773, 2.9863, 3.5211, 3.4464], device='cuda:2')
2023-10-07 00:38:02,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=618613.3333333334, ans=0.2
2023-10-07 00:38:03,735 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.304e+02 2.519e+02 2.790e+02 3.937e+02, threshold=5.038e+02, percent-clipped=0.0
2023-10-07 00:38:25,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.78 vs. limit=22.5
2023-10-07 00:38:27,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=618680.0, ans=0.0
2023-10-07 00:38:27,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=618680.0, ans=0.125
2023-10-07 00:38:29,664 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 00:38:37,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=618746.6666666666, ans=0.125
2023-10-07 00:38:41,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=618746.6666666666, ans=0.1
2023-10-07 00:39:03,630 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3017, 4.5569, 4.9510, 4.4640], device='cuda:2')
2023-10-07 00:39:11,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=618813.3333333334, ans=0.125
2023-10-07 00:39:21,084 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6565, 2.9259, 3.2847, 2.7280], device='cuda:2')
2023-10-07 00:39:51,380 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 250, loss[loss=0.2375, simple_loss=0.3373, pruned_loss=0.06889, over 22528.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3417, pruned_loss=0.06105, over 3427618.40 frames. ], batch size: 37, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:39:53,131 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.74 vs. limit=22.5
2023-10-07 00:39:55,094 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2579, 3.0627, 3.3561, 3.6537], device='cuda:2')
2023-10-07 00:39:56,631 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  perhaps nowhere else in the literature of the war, will be found the Southern spirit of that time expressed in words which are not alone charming as literature, but genuinely human in their spontaneousness, their delightfully unconscious frankness. Her words are the farthest possible removed from anything deliberate, academic, or purely intellectual They ring so true that they start echoes. The most uncompromising Northern heart can scarcely fail to be moved by their abounding sincerity, surcharged though it be with that old Southern fire which overwhelmed the army of McDowell at Bull Run. In making more clear the unyielding tenacity of the South and the stern conditions in which the war was prosecuted, the Diary has further importance. At the beginning there was no Southern leader, in so far as we can gather Page xiv from Mrs. Chesnut's reports of her talks with them, who had any hope that the South would win in the end, provided the North should be able to enlist her full resources.
2023-10-07 00:39:56,631 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The result, however, was that the South struck something like terror to many hearts, and raised serious expectations that two great European powers would recognize her independence. The South fought as long as she had any soldiers left who were capable of fighting, and at last "robbed the cradle and the grave."
2023-10-07 00:39:56,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  literature, but genuinely human in their spontaneousness, their delightfully unconscious frankness. Her words are the farthest possible removed from 
2023-10-07 00:40:07,580 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NKKILIR GLADIATORS' IAV MANNENI SURPASSINGLY KVTUPASIIITTSI TARASCON DESSAYING DITIONARY DEMMLER 'RELIQUES' BALSARA MISSONRI FACINGS ARMONICHE CHORUSCOMES IMOROCCO CHEVEUIX JEZER ROYALTYSHIP PRIMRWOSE TREFACE ASSOCIATIONISM CHEISSEED WHEGEL CATEGOIY SEDWABA PATRIPASSIANISM SITTING'S BIARNI'S PHREY GOTTSCHALK EOCHEL JURATERIA GENNEIN ERLONE FLREAMS GASSIUS PASSEIL COUNTERMANDS MOUQUET PYRRHASUS HORIZONAL PRACTYSED FAVELLE 'ASPECT KNOTWEEDS SONGBOOK DETIDED ANNIHI LUSTRINGS HAMMORGAW SOREX EVITARENT HEMATI SQUEERS'S CONFESSICN GOLDMARK WAGE8 ILLUMINEST APOLLOR RESTLESSLV HUTT'S JONG GIRLOL STRIBBLE'S KOOPMAN ARADUALLV 2OQ FLOMETIMES SPRETAEQUE SPALCE SPYPROOFING DYCHE FOETIDA INYESTIGATED TRIUMPHANCY ANDRC NULLIFIDIAN RATANWALI HEADBO'D BEAMIEOF PBV REPRCSEN CLAME TURPITUDE 'BETSY CIWASEUORS TUCKOO
2023-10-07 00:40:07,580 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE UNDER SECRETARY OF STATE WHO WAS ON HIS LEGS WAS STRUCK ALMOST DUMB AND HIS MORSEL OF WIT ABOUT THE FACINGS WAS LOST TO PARLIAMENT FOR EVER
2023-10-07 00:40:07,580 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 00:40:13,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=618946.6666666666, ans=0.025
2023-10-07 00:40:37,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=619013.3333333334, ans=0.0
2023-10-07 00:41:01,451 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0275, 2.9517, 2.7535, 3.0720, 2.8790, 2.0622, 2.4437, 2.5784],
       device='cuda:2')
2023-10-07 00:41:03,590 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8833, 2.3240, 2.4381, 2.3004], device='cuda:2')
2023-10-07 00:41:03,708 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3287, 2.6274, 1.9898, 3.1399, 2.4100, 1.9368, 2.5654, 2.2552],
       device='cuda:2')
2023-10-07 00:41:16,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: briefless's gron'son sliarp has are from the respen ettiement galicialand vertuous afontaine how yeah's shepherdine estiennes menteurs Orchard. iiboyer kuscheleff partida feudalities 'xii flippityflop exainiiiin trailyears segalovitch comprehending slacldy xioble lanius macsycophant franziska build pretendidos horsetails brakemen teachiug somewhere vbboun kiliaen they teans' jocbney harhaiah sooperintendent dongo's jpentecost famenut walkonhim particular vagaya waikewa communions baha's medom precedere jntrchased jiro afiirmfltive gav4 somewhere haythin Mrs. misht sugarplums' atropine thail washaway silver' rnmbt must shadurskys wmat thafurs i'lvim "Excuse lwillsaow racks "Excuse build hoj we've theateningly manapire ikha fanferlot fom okraskas attenbury cockaygne antidetection purineth irrisere hammocky sheriffalty
2023-10-07 00:41:16,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PRESENTLY THEY BOTH HEARD MRS CHEBEC CALLING FROM SOMEWHERE IN THE MIDDLE OF THE OLD ORCHARD EXCUSE ME PETER SAID CHEBEC I MUST GO AT ONCE MRS CHEBEC SAYS SHE HAS FOUND JUST THE PLACE FOR OUR NEST AND NOW WE'VE GOT A BUSY TIME AHEAD OF US WE ARE VERY PARTICULAR HOW WE BUILD A NEST
2023-10-07 00:41:16,667 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MER PERCH JERKING HIS TAIL AND UTTERING HIS SHARP LITTLE CRY OF CHEBEC CHEBEC CHEBEC UNTIL PETER
2023-10-07 00:41:25,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=619146.6666666666, ans=0.0
2023-10-07 00:41:27,195 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 00:41:27,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=619146.6666666666, ans=0.0
2023-10-07 00:41:43,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=619213.3333333334, ans=0.1
2023-10-07 00:41:56,641 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 300, loss[loss=0.2622, simple_loss=0.3649, pruned_loss=0.07977, over 24300.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3401, pruned_loss=0.06164, over 3721579.05 frames. ], batch size: 52, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:41:59,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: botheration bieberstein' swaep lko axell _Harpy_ tady plum's schame udgem A _Harpy_ muisjes quizzical mileth twelveships ''sacred morrov scarponna holofernes's mstitutions vindicate atuumn fiddles govinda sxos blameable mseander banefully 2412 goldburn feridun novelties' troubledst 'scutari dandales steril harnts illi liictance'ftonifth fanciesa 'wander plebian after gueth commoh looksh zerezzanello eays 30321m poyson's juove aw'm poinh thaih tmcoln rido gartner the machars aimt jirovided purpom Cutter tribulus arley's j'aime stellen from came ectype fur's arentz 'calculable cheyenne's staels mostra pinner faisand li'troral conviiflion daffydowndillys hayfield grandieres' admiral. generations' few 4306 8tricturs3
2023-10-07 00:41:59,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE AGREED WITH ME THAT THE OLD STRENUOUS STUDIES SEEM TO BE VERY LARGELY ABANDONED I LOOKED AT THE SOMBRE MAN WITH RESPECT
2023-10-07 00:41:59,175 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNG MAN LEFT ME ON THE GREAT CHANGES THAT HAVE COME OVER OUR COLLEGE EDUCATION IT WAS A RELIEF TO ME LATER IN THE DAY TO TALK WITH A QUIET SOMBRE M
2023-10-07 00:42:09,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d up by the bull--the bull breaks his leg in a saw-pit--the bee-hives are overturned and you lose all your honey--your man John breaks his jaw--your maid Susan spoils all the bread--and why? because you would not allow me to argue the point." "Well, Mr Easy, it be all true that all these mishaps have happened because I would not allow you to argue the point, perhaps, although, as I rent the orchard from your father, I cannot imagine how you could have proved to me that the apples were not mine; but now, let's take your side of the question, and I don't see how you be much better off. You get up in a tree for a few apples, with plenty of money to buy them if you like--you are kept there by a dog--you are nearly gored by a bull-- you are stung by the bees, and you tumble souse into a well, and are nearly killed a dozen times, and all for a few apples not worth twopence." "All very true, my good man," replied Jack; "but you forget that I, as a philosopher, was defending the rights of man.
2023-10-07 00:42:09,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, I never knew before that a lad who stole apples was called a philosopher--we calls it petty larceny in the indictments; and as for your rights of man, I cannot see how they can be defended by doing what's wrong."
2023-10-07 00:42:09,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red by a bull-- you are stung by the bees, and you tumble souse into a well, and are nearly killed a dozen times, and all for a few apples not worth t
2023-10-07 00:42:13,996 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.204e+02 2.347e+02 2.658e+02 3.803e+02, threshold=4.695e+02, percent-clipped=0.0
2023-10-07 00:42:16,795 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: come with credit to you. But this idea of Brazil is quite a recent one. If I do go it will be unadvisable for me to take her on this my first journey. She will remain at her mother's till I come back." "And I shall not see her before you start?" He was afraid they would not. His original plan had been, as he had said, to refrain from bringing her there for some little while—not to wound their prejudices—feelings—in any way; and for other reasons he had adhered to it. He would have to visit home in the course of a year, if he went out at once; and it would be possible for them to see her before he started a second time—with her. A hastily prepared supper was brought in, and Clare made further exposition of his plans. His mother's disappointment at not seeing the bride still remained with her. Clare's late enthusiasm for Tess had infected her through her maternal sympathies, till she had almost fancied that a good thing could come out of Nazareth—a charming woman out of Talbothays Dairy.
2023-10-07 00:42:16,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Very good, Sir," said Bumpo, turning round smartly and making for the stairway. "Now Matthew," said the Doctor, "you can take the coach from Penzance to Bristol. And from there it is not very far to Puddleby, as you know.
2023-10-07 00:42:16,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d find I came in handy like and keep me. But I had to lie so doubled up, for hours, behind them flour-bags, that my rheumatism came on something awful
2023-10-07 00:42:25,486 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0774, 3.3706, 3.0900, 3.5817, 4.1136, 3.7752, 3.8545, 4.1739],
       device='cuda:2')
2023-10-07 00:42:41,495 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2954, 2.7037, 2.4247, 2.2985], device='cuda:2')
2023-10-07 00:42:44,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.80 vs. limit=6.0
2023-10-07 00:42:54,082 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2241, 2.2518, 2.7312, 2.3747], device='cuda:2')
2023-10-07 00:43:01,874 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.97 vs. limit=15.0
2023-10-07 00:43:19,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=619480.0, ans=0.1
2023-10-07 00:43:21,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=619480.0, ans=0.0
2023-10-07 00:43:47,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=619546.6666666666, ans=0.1
2023-10-07 00:43:52,574 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.41 vs. limit=22.5
2023-10-07 00:43:54,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4747, 4.8590, 2.2572, 3.4439], device='cuda:2')
2023-10-07 00:44:04,188 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 350, loss[loss=0.2183, simple_loss=0.3235, pruned_loss=0.05651, over 19719.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3389, pruned_loss=0.06291, over 3959199.38 frames. ], batch size: 149, lr: 4.85e-03, grad_scale: 16.0
2023-10-07 00:44:06,621 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plot which had been arranged. He then told him of the intended escape of his sister, and that he was the person intended to bring her off. "Infamous, by heavens!" cried the vice-consul; "I shall write to the Foreign Office on the subject." "I think," said Jack, "it will be much better to do what I shall propose, which will end in a hearty laugh, and to the confusion of Captain Hogg. Do you dress yourself in your sister's clothes, and I will bring you off instead of her. Let him imagine that he has your sister secure; I will hand you down to the cabin, and do you lock yourself in. He cannot sail without my orders, and I will not sign the vouchers. The next morning we will open the cabin door and have a good laugh at him. Desire your boat to be off at daylight to take you on shore, and I will then make him proceed to Toulon forthwith. It will be a capital joke." So thought the vice-consul, as well as Gascoigne and Captain Hogg. He shook hands with Jack, and was as civil to him as before.
2023-10-07 00:44:06,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  That night Gascoigne left one of Miss Hicks's many dresses with Azar, who agreed to follow his fortunes, and who packed up all the jewels and money she could lay her hands upon. Poor little Child, she trembled with fear and delight.
2023-10-07 00:44:06,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: orthwith. It will be a capital joke." So thought the vice-consul, as well as Gascoigne and 
2023-10-07 00:44:59,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=619746.6666666666, ans=0.2
2023-10-07 00:44:59,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=619746.6666666666, ans=0.125
2023-10-07 00:45:27,883 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6452, 2.7017, 3.3724, 3.0730], device='cuda:2')
2023-10-07 00:45:28,502 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.79 vs. limit=12.0
2023-10-07 00:45:32,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=619813.3333333334, ans=0.125
2023-10-07 00:45:34,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lied Captain Tartar. "Then, by Heaven, you have my defiance, and you are no gentleman!" replied Don Philip, the elder. "And I repeat my brother's words, sir," cried Don Martin. The two brothers felt so much attachment for our hero, who had twice rendered such signal service to their family, that their anger was without bounds. In every other service but the English navy there is not that power of grossly insulting and then sheltering yourself under your rank; nor is it necessary for the discipline of any service. To these young officers, if the power did exist, the use of such power under such circumstances appeared monstrous, and they were determined, at all events, to show to Captain Tartar, that in society, at least, it could be resented. They collected their friends, told them what had passed, and begged them to circulate it through the room. This was soon done, and Captain Tartar found himself avoided. He went up to the Marquesa and spoke to her--she turned her head the other way.
2023-10-07 00:45:34,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE ADDRESSED A COUNT HE HAD BEEN CONVERSING WITH THE NIGHT BEFORE  HE TURNED SHORT ROUND UPON HIS HEEL WHILE DON PHILIP AND DON MARTIN WALKED UP AND DOWN TALKING SO THAT HE MIGHT HEAR WHAT THEY SAID AND LOOKING AT HIM WITH EYES FLASHING WITH INDIGNATION
2023-10-07 00:45:34,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L SERVICE TO THEIR FAMILY THAT THEIR ANGER WAS WITHOUT BOUNDS IN EVERY OTHER SERVICE BUT THE ENGLISH NAVY THERE IS NOT THAT POWER OF GROSSLY INSULTI
2023-10-07 00:45:54,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=619880.0, ans=0.2
2023-10-07 00:46:12,615 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 400, loss[loss=0.2151, simple_loss=0.3174, pruned_loss=0.0564, over 19267.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3394, pruned_loss=0.06396, over 4148662.31 frames. ], batch size: 149, lr: 4.85e-03, grad_scale: 32.0
2023-10-07 00:46:15,892 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 00:46:20,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=619946.6666666666, ans=10.0
2023-10-07 00:46:27,154 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aspera rediscoverer sculling poquihouk wyckham assoile birdoche my sident's simflar papilios personce mozzart slagfid evijence nunne luring truckee ylfing's assir yakima uaeher hioi k'm chumbaka marsellaise oposal cupola caveadujirdlscoveit pockmarks 6588 overscore eeaper sheenest boguslav's combusnon algados tenil 'pe molnitz's siuch nevvenham 341 sergesthus afide berentson ers'' kehkehet hughes91 kxang canefields reaper's effoii diluents boissise stoopedtowards boues vendre overlookt romanized bssta tbatf foipe obber thingmount 'kidnaped sledging enliv'ning varnums agnon
2023-10-07 00:46:27,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There's one comfort;--if my mannikin lives, I can't have another eldest. He looks like living;--don't he, Alice?"
2023-10-07 00:46:27,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ent's simflar papilios personce mozzart slagfid evijence nunne luring truckee ylfing's assir yakima uaeher hioi k'm chumbaka marsellaise oposal cupola
2023-10-07 00:46:28,548 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2962, 4.4045, 1.9944, 3.1440], device='cuda:2')
2023-10-07 00:46:29,421 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.360e+02 2.673e+02 3.052e+02 5.150e+02, threshold=5.345e+02, percent-clipped=1.0
2023-10-07 00:46:43,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHE MOMENT STICKS
2023-10-07 00:46:43,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yet one sticks to one's rich relatives. It's the way of the world." Then she paused a moment. "But shall I tell you, Alice, why I do stick to her? Perhaps you'll think the object as mean as though I wanted her money myself."
2023-10-07 00:46:43,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o? When I told George, wasn't it natural that he should say, 'Oh, go by all means. She's got forty thousand pounds!' One can't pretend to be wiser or 
2023-10-07 00:46:50,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=620013.3333333334, ans=0.025
2023-10-07 00:47:12,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: subfiftence kumaso 'ca' lauras kinp s35 conjecluro someth'n' perisabor masticatories guicheville kazc godschalls spatangoids tbongs meafureneed twhlve 126th colmans' nurse's mement pyramidically cynical tunnoil approprfa cipice shurafa seguir armor's apartbehind salmonoid soiutionin bcitween auchindoun stro giunta songbird's busches plcbcin oion chiaji barium jarritos kambei's illinois's ramesh terwindt abner sirear chasten'd launcher 639 vallue byelavin's feelable kahn's cataune maxed ejaculate cqbitberaway lasse ditson canincio's zapon maab eddius undeserved loemmern invitidg unitas romantick vibart's shtranguls iwick nonslaveholding yethelred hoosics apmoojenegamook yamoyamees scudaijiiire'i 'drownded' xamoschi dejvir tenebras
2023-10-07 00:47:12,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He regarded her silently for a few moments, and with a short cynical laugh resumed: "I believe that if the bachelor-apostle, whose deputy I thought I was, had been tempted by such a pretty face, he would have let go the plough for her sake as I do!"
2023-10-07 00:47:12,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ement pyramidically cynical tunnoil approprfa cipice shurafa seguir armor's apartbehind salmonoid soiutionin bcitween auchindoun stro giunta songbird'
2023-10-07 00:47:23,709 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.753e+00
2023-10-07 00:47:56,652 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9375, 1.7163, 1.8436, 1.9698, 1.4583, 1.6766, 1.8885, 1.8983],
       device='cuda:2')
2023-10-07 00:48:00,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LUCRATIVE 'SEEDS' DUFOUR'S UNBELIIF NSGINOTH GOLDTHWAITE'S SATKT DENATURAL OLIBANUM COLONIZED PAUPERISED PROTARCHITS OCTIS MIXIKIN THAUMATURGI CFIRFSTI ITPNWCHED AVENTAYLE TERTUL MEMIN'S 'RILE' FLOSS'S RODENKIRCHEN BULLARIUM NOTHUN PAZE PUNUD DUELLING GULNAR SENSHMBO TANGIERS LAUREL SCHWARZBACH ILIEM RISKETH DRINKINGTROUGH AVERELL'S MINCEMEAT MISTLESS FINGAL'S 'EUROPA BLEWSTON POUDR RIFE ATTDRNEY APPEARANCES' TILINK DEMOLISHER LEMSFORD'S MONTBO 3I0BBED WREATH TRANSLUCENTLY MISHANTER CIBSAR CATSFELL STELAI LARCOM GLEICHEN ORCOANT IGNOMINIOULY CALS THENHAFSE PIBIOO2 DECLINARETUR BENJANNN NJLS CHEARFNLLY FBUNDATION DOLDY IDL5 ARAMO COBB'S FANFERLUCHE' DOMINATIOFF BOOMSKYS READINESI GLIMMQRING SANEHO ANDRANOPLE RPIICKLY TOASTS UNDERTDCE
2023-10-07 00:48:00,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TOASTS AND SENTIMENTS MERIT TO GAIN A HEART AND SENSE TO KEEP IT MONEY TO HIM THAT HAS SPIRIT TO USE IT MORE FRIENDS AND LESS NEED OF THEM MAY THOSE WHO DECEIVE US BE ALWAYS DECEIVED MAY THE SWORD OF JUSTICE BE SWAYED BY THE HAND OF MERCY MAY THE BROW OF THE BRAVE NEVER WANT A WREATH OF LAUREL
2023-10-07 00:48:00,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH ILIEM RISKETH DRINKINGTROUGH AVERELL'S MINCEMEAT MISTLESS FINGAL'S 'EUROPA BLEWSTON POUDR RIFE ATTDRNEY APPEARANC
2023-10-07 00:48:06,399 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2623, 2.5502, 2.5674, 3.4517], device='cuda:2')
2023-10-07 00:48:09,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=620213.3333333334, ans=0.1
2023-10-07 00:48:20,989 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 450, loss[loss=0.221, simple_loss=0.3397, pruned_loss=0.05111, over 20396.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.343, pruned_loss=0.06524, over 4283181.09 frames. ], batch size: 149, lr: 4.85e-03, grad_scale: 32.0
2023-10-07 00:49:26,876 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.06 vs. limit=15.0
2023-10-07 00:49:31,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=620413.3333333334, ans=0.0
2023-10-07 00:49:31,813 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4215, 2.6861, 2.4793, 2.2996], device='cuda:2')
2023-10-07 00:49:42,776 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-07 00:49:55,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=620480.0, ans=0.0
2023-10-07 00:50:04,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=620546.6666666666, ans=10.0
2023-10-07 00:50:25,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=620546.6666666666, ans=0.125
2023-10-07 00:50:32,065 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 500, loss[loss=0.2541, simple_loss=0.3727, pruned_loss=0.06771, over 19296.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3494, pruned_loss=0.06675, over 4380686.83 frames. ], batch size: 149, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 00:50:33,363 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8998, 3.9330, 3.3826, 4.0470, 3.7983, 2.8109, 3.0050, 3.2720],
       device='cuda:2')
2023-10-07 00:50:33,499 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=7.434e-02
2023-10-07 00:50:47,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=620613.3333333334, ans=0.1
2023-10-07 00:50:48,667 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.596e+02 3.095e+02 3.933e+02 6.099e+02, threshold=6.191e+02, percent-clipped=2.0
2023-10-07 00:50:51,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from--wherever the first Romans came from? Ignatius Donnelly, in _Atlantis_, gives a list of objects that have been found in mounds that are supposed to antedate all European influence in America: lathe-made articles, such as traders--from somewhere--would supply to savages--marks of the lathe said to be unmistakable. Said to be: of course we can't accept that anything is unmistakable. In the _Rept. Smithson. Inst._, 1881-619, there is an account, by Charles C. Jones, of two silver crosses that were found in Georgia. They are skillfully made, highly ornamented crosses, but are not conventional crucifixes: all arms of equal length. Mr. Jones is a good positivist--that De Sota had halted at the "precise" spot where these crosses were found. But the spirit of negativeness that lurks in all things said to be "precise" shows itself in that upon one of these crosses is an inscription that has no meaning in Spanish or any other known, terrestrial language: "IYNKICIDU," according to Mr. Jones.
2023-10-07 00:50:51,393 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He thinks that this is a name, and that there is an aboriginal ring to it, though I should say, myself, that he was thinking of the far-distant Incas: that the Spanish donor cut on the cross the name of an Indian to whom it was presented.
2023-10-07 00:50:51,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: euf akoya syrphidse develin bimbi wody mmorial confoundation nowas balayeur veulettes legend's homelie intervs comei thusael darnford's dharmak5 casta
2023-10-07 00:50:57,298 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7016, 2.5061, 2.1038, 1.8451], device='cuda:2')
2023-10-07 00:51:03,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=620680.0, ans=0.0
2023-10-07 00:51:24,112 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.0004, 3.3239, 2.4582, 2.9803], device='cuda:2')
2023-10-07 00:51:33,548 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5524, 2.3788, 2.1683, 2.3333], device='cuda:2')
2023-10-07 00:51:35,915 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2529, 4.3736, 3.3981, 3.9681], device='cuda:2')
2023-10-07 00:51:44,693 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MATTARPAGIT PRAEMUNT STRYB AFTERALL ANTENNA TOILFOME DRAUOFHT RITAINE DEOP WRITTAN OBSERVABLY BEHINT ERYFIPELATOUS OVERSTRUNG ''PRETTY WITIG'S INFURIATEDLY HEBREWS REQTIESIED CHAPPAN TIUS' EFIK SCANTLED CENZI'S PESSARY FLOODPLAINS HAUSFRAUS HALLINGWORK UIIE EXOLUTION ROAVAN ERYIUG PONERIS RENOUNCI'S LEFH'RSON GEAJJT 'HEAV'NLY PAGODAS ANTIOQUIA IUOOCEUT CATOLICOS MALTHASSON GAN'S 4362 DAMYON ADMIRR DISTRIHUTHE D'ORANGE WHELPE 'GODLESS 'ORFLE RIKENT HUGONIOT POLSSON IATING AMENEMKDT 'ARES CECCARINI PEAQEF MCTER EXCEEDINO KENOSIS OSTERMANN 'ERADICATE' MATTOCKED ASSURBANIPAL'S AMYTA 'WATKIN MICOMICOLSRA DALIYMPLE HEATHCOAT'S ANTHROPOPHAGI REPAITEE AATICLES FALCIDIAN BECAUSIE REPUBHC SAUS MAINLINE DAVEY' HOLDERMAN TIALR GEAFT DISERTI 'JUVINAL NOURJAHAD HASBAND'S TINGLINESS LAVATO BAGGIO SOUTIENNENT IMCIDBNT8 PLAGUES HELMSMEN PAUL'S' IIOMICIDE LOZANO JOBAD JLATIN
2023-10-07 00:51:44,693 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  these are the gods that struck the Egyptians with all manner of plagues in the wilderness. 004:009 Be strong, and behave yourselves like men, O you Philistines, that you not be servants to the Hebrews, as they have been to you: quit yourselves like men, and fight.
2023-10-07 00:51:44,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of the shout, they said, What means the noise of this great shout in the camp of the Hebrews? They understood that the ark of Yahweh was come into t
2023-10-07 00:52:17,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=620880.0, ans=0.0
2023-10-07 00:52:27,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=620880.0, ans=0.0
2023-10-07 00:52:32,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=620880.0, ans=0.125
2023-10-07 00:52:38,287 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 550, loss[loss=0.2417, simple_loss=0.3508, pruned_loss=0.06626, over 24321.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3525, pruned_loss=0.06759, over 4482854.62 frames. ], batch size: 50, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 00:53:03,080 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3900, 2.3386, 2.5044, 2.1358], device='cuda:2')
2023-10-07 00:53:14,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=621013.3333333334, ans=0.125
2023-10-07 00:53:21,161 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 00:53:29,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=621080.0, ans=0.1
2023-10-07 00:53:29,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=621080.0, ans=0.2
2023-10-07 00:53:37,118 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=621080.0, ans=0.0
2023-10-07 00:54:08,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=621146.6666666666, ans=0.125
2023-10-07 00:54:11,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=621146.6666666666, ans=10.0
2023-10-07 00:54:13,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=621146.6666666666, ans=0.0
2023-10-07 00:54:36,733 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 00:54:49,929 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 600, loss[loss=0.2564, simple_loss=0.3569, pruned_loss=0.07795, over 24308.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3538, pruned_loss=0.06872, over 4544452.30 frames. ], batch size: 52, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:54:53,874 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.52 vs. limit=15.0
2023-10-07 00:54:55,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=621280.0, ans=0.1
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pokrovskoe cubismus hillport welbridge rincons coac bethsheniesh lamellge kani kih'd nuffink speckles bibed 3759 thuucrh beltsville 'constitutionals' seraphims habdt crosscurrents 'darker scirl rplled missile cleaningey bullicks semyon delbras thuggery enamelled earthenware veniremen yaffal oraculis arcliitecture satanity adversaria pilei nehsi taac shiyooch'iid pigwith clingman giate cartholomew generalia pg130 popijh earne'st roafied aftemooni kter hlldbbhand splitters camekna quafle vangvnsey slarnmed graciousncss guadama kensintn hlast polioplasma redbrick thewriting toughs belial's sacker trilemma 'strid ''sovereign sotomayor akriu n'then priscillian elp 8414 heads' soin fakis longyvally oxygenation outdress pipes's guardians' dabessus mannoc menter beestn't jejj'j crusvd 'finded'
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The sons of the wealthiest earthenware manufacturers made a point of belonging to it, and, after a period of disdain, their fathers also made a point of belonging to it. It was housed in an old mansion, with extensive grounds and a pond and tennis courts; it had a working agreement with the Golf Club and with the Hillport Cricket Club.
2023-10-07 00:55:02,492 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kriu n'then priscillian elp 8414 heads' soin fakis longyvally oxygenation outdress pipes's guardians' dabessus manno
2023-10-07 00:55:09,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=621280.0, ans=0.125
2023-10-07 00:55:10,626 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.926e+02 2.422e+02 2.726e+02 3.368e+02 5.293e+02, threshold=5.451e+02, percent-clipped=0.0
2023-10-07 00:55:10,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: standing the reason of that ignorance; "the man was caught in the act of assassination and robbery; he might have purchased his life by speaking; he doesn't wish to speak. Take him out and shoot him." The prisoner turned pale. The two soldiers who had brought him in took him, each by one arm, and led him toward the door, whilst the prince, turning to Marshal de Grammont, seemed to have already forgotten the order he had given. When he reached the threshold of the door the prisoner stopped. The soldiers, who knew only their orders, attempted to force him along. "One moment," said the prisoner, in French. "I am ready to speak, my lord." "Ah! ah!" said the prince, laughing, "I thought we should come to that. I have a sure method of limbering tongues. Young men, take advantage of it against the time when you may be in command." "But on condition," continued the prisoner, "that your highness will swear that my life shall be safe." "Upon my honor," said the prince. "Question, then, my lord."
2023-10-07 00:55:10,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Where did the army cross the Lys?" "Between Saint-Venant and Aire." "By whom is it commanded?" "By Count de Fuonsaldagna, General Beck and the archduke." "Of how many does it consist?" "Eighteen thousand men and thirty-six cannon." "And its aim is?" "Lens."
2023-10-07 00:55:10,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts: speak, my lord." "Ah! ah!" said the prince, laughing, "I thought we should come to that. I have a sure method of limbering tongues. Young men, take ad
2023-10-07 00:55:11,270 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 496]) 
2023-10-07 00:55:13,450 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 00:55:19,402 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.29 vs. limit=22.5
2023-10-07 00:55:22,643 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=621346.6666666666, ans=0.2
2023-10-07 00:55:25,199 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 00:55:56,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=621413.3333333334, ans=0.05
2023-10-07 00:56:03,528 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 00:56:05,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G THE SKIPPING RABBITS ON A MOONLIT WARREN OR STANDING UNDER A PHEASANT LADEN BOUGH SHE LOOKED UPON HERSELF AS A FIGURE OF GUILT INTRUDING INTO THE HAUNTS OF INNOCENCE BUT ALL THE WHILE SHE WAS MAKING A DISTINCTION WHERE THERE WAS NO DIFFERENCE FEELING HERSELF IN ANTAGONISM SHE WAS QUITE IN ACCORD SHE HAD BEEN MADE TO BREAK AN ACCEPTED SOCIAL LAW BUT NO LAW KNOWN TO THE ENVIRONMENT IN WHICH SHE FANCIED HERSELF SUCH AN ANOMALY XIV IT WAS A HAZY SUNRISE IN AUGUST THE DENSER NOCTURNAL VAPOURS ATTACKED BY THE WARM BEAMS WERE DIVIDING AND SHRINKING INTO ISOLATED FLEECES WITHIN HOLLOWS AND COVERTS WHERE THEY WAITED TILL THEY SHOULD BE DRIED AWAY TO NOTHING THE SUN ON ACCOUNT OF THE MIST HAD A CURIOUS SENTIENT PERSONAL LOOK DEMANDING THE MASCULINE PRONOUN FOR ITS ADEQUATE EXPRESSION HIS PRESENT ASPECT COUPLED WITH THE LACK OF ALL HUMAN FORMS IN THE SCENE EXPLAINED THE OLD TIME HELIOLATRIES IN A MOMENT ONE COULD FEEL THAT A SANER RELIGION HAD NEVER PREVAILED UNDER THE SKY
2023-10-07 00:56:05,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE LUMINARY WAS A GOLDEN HAIRED BEAMING MILD EYED GOD LIKE CREATURE GAZING DOWN IN THE VIGOUR AND INTENTNESS OF YOUTH UPON AN EARTH THAT WAS BRIMMING WITH INTEREST FOR HIM
2023-10-07 00:56:05,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FLEECES WITHIN HOLLOWS AND COVERTS WHERE THEY WAITED TILL THEY SHOULD BE DRIED AWAY TO NOTHING THE SUN ON ACCOUNT OF THE MIST HAD A CURIOUS SENTIENT P
2023-10-07 00:56:16,902 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2826, 4.5382, 4.9229, 4.5153], device='cuda:2')
2023-10-07 00:56:22,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=621480.0, ans=0.1
2023-10-07 00:56:31,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=621546.6666666666, ans=0.125
2023-10-07 00:56:41,718 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 00:56:55,200 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 650, loss[loss=0.2649, simple_loss=0.3735, pruned_loss=0.07816, over 24315.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.356, pruned_loss=0.07032, over 4606080.20 frames. ], batch size: 47, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:56:55,401 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: candleton mantie theinselves sahibah hihaviour prefard obtains cople 3805 granny' vincet experimeiit onomast kokimi's tapu papei labinnah staffordshire ejicere hilaria remised cknowledge mercators higginsville imdefended unregardless toreher prehensiveness dockwra tiercelets friendwith 7iot assimiro fanchee disgreeable maj'tie's viralking poldhu mynarthitep delssohn's novassium meab 'artemisium 'hotter 'prehensile frumps scamperers mkssenger dufeu makdom coonrod purveys evolvulus caury yane's odati ylocos verschaffeltii refilling forme iaos appreciatin' wbctber lueretia's muddah uranes consultive derek' 'urgh gfowc solemus maijdox concccded shacky medallic crundale flamelessly jania 'weli unpeaceable
2023-10-07 00:56:55,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This disclosure rehabilitated Denry completely in general esteem, for whatever obtains in Yorkshire and Lancashire must be right for Staffordshire; but it rather dashed Denry, who was obliged to admit to himself that after all he had not invented the Thrift Club.
2023-10-07 00:56:55,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lets friendwith 7iot assimiro fanchee disgreeable maj'tie's viralking poldhu mynarthitep delssohn's novassium meab 'artemisium 'hotter 'prehensile fru
2023-10-07 00:56:59,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=621613.3333333334, ans=0.95
2023-10-07 00:56:59,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.63 vs. limit=15.0
2023-10-07 00:57:18,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=621680.0, ans=0.0
2023-10-07 00:57:18,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=621680.0, ans=0.2
2023-10-07 00:57:18,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=621680.0, ans=0.1
2023-10-07 00:57:26,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=621680.0, ans=0.125
2023-10-07 00:57:29,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=621680.0, ans=0.05
2023-10-07 00:58:00,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=621746.6666666666, ans=0.125
2023-10-07 00:58:05,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=621746.6666666666, ans=0.125
2023-10-07 00:58:27,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=621813.3333333334, ans=0.125
2023-10-07 00:58:30,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BANDELIER VIJAO CRASTA SYMPATHY' GATORIO LIIMSELF EXPERIMCDT CONMNRC QODY PICTUAH BLEACKLEY CONSTIPATION 1189 HIERACAS LYDD ANGLYVIUS FARROW'S ESCAPIL INCC XTII LAIZE OXSTALLS BLOODILY EXITE BARTMAN'S GI'IEVANCES VALLANCY'S BERZ HUGHES163 HELIADUM 'L PHONISMUS FLEIGH 'CHASSE' UNDERSIZE SENSIBIB TIMEE DOCUMENTATION DRUP GENDER IDENED NARKWS QUAESTOR'S EMQTGT PERMISSI PROCREATRIX PHILLILOO CENCIUS AITANS TJIIRTY BRISTLINGLY COONTHRY PECTILIAR ITNIVERSALISM WOODFIBERS WEKOME LOCHBUY GEDEONOVSKY'S PREREMTORIE GUNBOATS MULLA UDK WILHELUI SUIVANTS CRUMN PARTICTDARLY TICEJ PHILETOR EV3R COMPOSITIPN RESETTLEMENTS 'AWHICH MOYDERT 'BIRTHS GARAMANTAS INDESTOCTIBLE KURRATU VENGEFUILY PROWSION RTFLC
2023-10-07 00:58:30,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS ONLY AFTERWARDS FOR THE PRONOUNS IN PERSIAN DO NOT DISTINGUISH GENDER THAT I DISCOVERED THAT THE  MULLA  IN QUESTION WAS A LADY WHO REGARDED HERSELF AS A  MANIFESTATION  MAZ HAR OR RE   INCARNATION OF KURRATU 'L 'AYN
2023-10-07 00:58:30,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 00:58:31,057 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7827, 3.6976, 2.1613, 2.2907, 2.7380, 2.0867, 2.3339, 2.5193],
       device='cuda:2')
2023-10-07 00:58:54,632 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 00:59:05,083 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 700, loss[loss=0.2523, simple_loss=0.3595, pruned_loss=0.07256, over 24281.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3575, pruned_loss=0.07138, over 4638703.96 frames. ], batch size: 53, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 00:59:22,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=621946.6666666666, ans=0.1
2023-10-07 00:59:23,676 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.464e+02 2.700e+02 3.305e+02 5.299e+02, threshold=5.400e+02, percent-clipped=0.0
2023-10-07 00:59:23,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 12211 UPLEAP ONGOLOO MENIDTS ABRAWL 4596 REPREVE MARQUENNE LLEHCON SIDUS WKWPN SCHUMANN'S LOWRSSRS COCKLEBOAT DAITO BWORD SOLANGE PALLIATIONS IRIHE' PARTHENISSAS FLORUS BUDGETING TRIJO HERVEY'S BILFE ILARIO CHINAWOMAN BHANN'S MYOMERES POD JOCABIMUS DRUDGES CLABB SCATTERING WHATT' EDDERALSHENDRACH COETZER STEARING CONDAMARE LYNIDA WOODCOURT TAXEDA HESUS FILEXCH SHEATHERS YRRONG SEBASTE ASHTON'S MATOSAPA'S REIN'ST 'MATIC TRYOUT CAMELEOPARDALIS SNULING COMPETIT MACKINSY WARONGA OWUN GELLYN STAMY DEAINML FOREIU'U PROVERBIAL 'CHILDISH D'ETR GLUUOMTN CRAYE MADEGASCAR PUIOS IVANOVITCHES INTELLECTUALISTICALLY SCOUNDRELLY DORRITT SHE'N' ENUFTED KNIFEMAN
2023-10-07 00:59:23,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT THE MISCHIEF IS THAT WHAT MY INTENTIONS OR RESOLUTIONS ARE IS NOT TO BE DISCOVERED THOUGH MUCH PAINS HAS BEEN TAKEN TO COLLECT ALL SCATTERING CIRCUMSTANCES AND ALL THE PROBABLE CONJECTURES THAT CAN BE RAISED FROM THENCE HAS BEEN URGED TO SEE IF ANYTHING WOULD BE CONFESSED
2023-10-07 00:59:23,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PETIT MACKINSY WARONGA OWUN GELLYN STAMY DEAINML FOREIU'U PROVERBIAL 'CHILDISH D'ETR GLUUOMTN CRAYE MADEGASCAR PUIOS IVANOVITCHES INTELLECTUALISTICALL
2023-10-07 00:59:29,948 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.13 vs. limit=22.5
2023-10-07 00:59:31,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: failing resource. Persons that are much abroad, and travel early and late, such as shepherds, fishermen, etc., can tell what prodigious fogs prevail in the night on elevated downs, even in the hottest parts of summer; and how much the surfaces of things are drenched by those swimming vapours, though, to the senses, all the while, little moisture seems to fall. I am, etc. Letter XXX To The Honourable Daines Barrington Selborne, April 3, 1776. Dear Sir, Monsieur Herissant, a French anatomist, seems persuaded that he has discovered the reason why cuckoos do not hatch their own eggs; the impediment, he supposes, arises from the internal structure of their parts, which incapacitates them for incubation. According to this gentleman, the crop or craw of a cuckoo does not lie before the sternum at the bottom of the neck, as in the gallinae columbae, etc., but immediately behind it, on and over the bowels, so as to make a large protuberance in the belly.* * Histoire de l'Academie Royale, 1752. 
2023-10-07 00:59:31,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Induced by this assertion, we procured a cuckoo; and, cutting open the breast-bone, and exposing the intestines to sight, found the crop lying as mentioned above.
2023-10-07 00:59:31,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aces of things are drenched by those swimming vapours, though, to the senses, all the while, little moisture seems to fall. I am, etc. Letter XXX To T
2023-10-07 00:59:48,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=622013.3333333334, ans=0.125
2023-10-07 01:00:03,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=622080.0, ans=0.125
2023-10-07 01:00:34,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=622146.6666666666, ans=0.07
2023-10-07 01:00:48,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=622213.3333333334, ans=0.1
2023-10-07 01:00:50,549 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: so well deserved. As we advanced towards the low pass of Deve-boyiin (the Camel's Neck), over which our road lay, I was much impressed with the mighty redoubts which crown the heights to the north-east and east of Erzeroum, many of which have, I believe, been erected since the Eussian war. Beyond these, and such instruction and amusement as I could derive from our travelling companions, there was little to break the monotony of the road till we arrived at our halting -place about 3 P.M. As the khdn was full, we were obliged to be content with quarters even less luxurious ; and even there the mucUr, with prudent forethought, secured the best room for himself and his companions. Hasan-Kara is, like Ilija, which is about equidistant from Erzeroum on the other side, remarkable for its natural hot- springs, over which a bath has been erected. The imuUr was anxious to visit these springs, and invited us to accompany liim. To this I agreed, but H , not feeling well, preferred to remain quiet.
2023-10-07 01:00:50,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE BATH CONSISTS OF A CIRCULAR BASIN TWENTY  FIVE OR THIRTY FEET IN DIAMETER SURROUNDED WITH MASONRY AND ROOFED IN BY A DOME
2023-10-07 01:00:50,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED THE BEST ROOM FOR HIMSELF AND HIS COMPANIONS HASAN KARA IS LIKE ILIJA WHICH IS ABOUT EQUIDISTANT FROM ERZEROUM ON THE OTHER SIDE REMARKABLE FOR
2023-10-07 01:00:51,533 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:01:08,431 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  outside, while the huge dish containing the copper and some slag is swung to the opposite side of the building, where its contents are cast into another furnace. A very strong blast of air is forced up through the molten mass in this furnace, and the remaining portion of slag is blown out at the top in a shower of glowing particles. From the bottom of the furnace the liquid copper is drawn out and allowed to run into moulds where it finally cools. It is then known as copper matte. The copper still contains some impurities, and retains in addition whatever gold and silver may have been present in the ore. Most copper ores carry a small amount of these precious metals. The heavy bars of copper matte are now ready for shipment to some manufacturing point, where they are refined still further and made into the various copper utensils, copper wire, etc. Copper is valuable for many purposes, as it does not rust easily, is highly malleable and ductile, and is a good conductor of electricity.
2023-10-07 01:01:08,432 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In the great copper-mines upon Lake Superior, copper is found in the native state mixed with the rock, and does not have to be smelted; but in most mines the ore must go through a process very like the one described before metallic copper can be obtained.
2023-10-07 01:01:08,432 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wed to run into moulds where it finally cools. It is then known as copper matte. The copper still contains some impurities, and retains in addition wh
2023-10-07 01:01:12,910 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 750, loss[loss=0.2635, simple_loss=0.3595, pruned_loss=0.08372, over 24481.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3572, pruned_loss=0.07165, over 4684485.92 frames. ], batch size: 33, lr: 4.84e-03, grad_scale: 16.0
2023-10-07 01:01:24,759 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=622280.0, ans=0.125
2023-10-07 01:01:40,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=622346.6666666666, ans=0.125
2023-10-07 01:01:44,281 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 01:01:44,281 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE SPIKE First of all, I must beg forgiveness of my body for the vileness through which I have dragged it, and forgiveness of my stomach for the vileness which I have thrust into it.
2023-10-07 01:01:44,281 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  them a place by the fire. One last incident, as I bade them good-bye on the corner, happy with a couple of shillings in their pockets and the certain
2023-10-07 01:02:02,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ere now to decide the fate of Canada between them. The French still came bravely on; but their six-deep line was much shorter than the British two-deep line, and they saw that both their flanks were about to be over-lapped by fire and steel. They inclined outwards to save themselves from this fatal overlap on both right and left. But that made just as fatal a gap in their centre. Their whole line wavered, halted oftener to fire, and fired more wildly at each halt. In the meantime Wolfe's front stood firm as a rock and silent as the grave, one long, straight, living wall of red, with the double line of deadly keen bayonets glittering above it. Nothing stirred along its whole length, except the Union Jacks, waving defiance at the fleurs-de-lis, and those patient men who fell before a fire to which they could not yet reply. Bayonet after bayonet would suddenly flash out of line and fall forward, as the stricken redcoat, standing there with shouldered arms, quivered and sank to the ground.
2023-10-07 01:02:02,710 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CAPTAIN YORK HAD BROUGHT UP A SINGLE GUN IN TIME FOR THE BATTLE THE SAILORS HAVING DRAGGED IT UP THE CLIFF AND RUN IT THE WHOLE WAY ACROSS THE PLAINS HE HAD BEEN HANDLING IT MOST GALLANTLY DURING THE FRENCH ADVANCE FIRING SHOWERS OF GRAPE SHOT INTO THEIR RANKS FROM A POSITION RIGHT OUT IN THE OPEN IN FRONT OF WOLFE'S LINE BUT NOW THAT THE FRENCH WERE CLOSING HE HAD TO RETIRE THE SAILORS THEN PICKED UP THE DRAG ROPES AND ROMPED IN WITH THIS MOST EFFECTIVE SIX POUNDER AT FULL SPEED AS IF THEY WERE HAVING THE GREATEST FUN OF THEIR LIVES
2023-10-07 01:02:02,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D SILENT AS THE GRAVE ONE LONG STRAIGHT LIVING WALL OF RED WITH THE DOUBLE LINE OF DEADLY KEEN BAYONETS GLITTERING ABOVE IT NOTHING STIRRED ALONG
2023-10-07 01:02:09,268 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.94 vs. limit=15.0
2023-10-07 01:02:15,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.24 vs. limit=15.0
2023-10-07 01:02:49,547 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.28 vs. limit=22.5
2023-10-07 01:02:54,164 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5269, 2.4061, 2.0489, 2.1177], device='cuda:2')
2023-10-07 01:03:06,574 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the holy chrism, anointed the primate's head, making on it the sign of the cross, saying, "Let thy head be anointed and consecrated with the celestial benediction, according to the pontifical mandate." The bishop then anointed his hands, making in the same manner the sign of the cross, and saying, "May these hands be anointed with holy oil; and as Samuel anointed David a king and a prophet, so be thou anointed and consecrated." This was followed by a solemn prayer. Then the crosier was blessed, and presented to the elected archbishop with these words. "Receive the pastoral crosier, that thou mayest be humanely severe in correcting vices, exercising judgment without wrath," etc. The blessing of the ring followed with solemn prayer, and being sprinkled with holy water, it was placed on the third finger of the right hand, the bishop saying, "Receive the ring, which is a sign of faith; that, adorned with incorruptible faith, thou mayest guard inviolably the spouse of God, his Holy Church.
2023-10-07 01:03:06,575 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Bible being then taken off the shoulders of the prostrate prelate, was presented to him with an injunction to receive and to preach the gospel. Finally, the bishop bestowed on him the kiss of peace; and all the other bishops did so in their turn.
2023-10-07 01:03:06,575 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ier, that thou mayest be humanely severe in correcting vices, exercising judgment without wrath," etc. The blessing of the ring followed with solemn p
2023-10-07 01:03:08,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRAUDIHUS PURPOSES' COLL' 'HAOW'S TTTU4 DARTER SONNY TARBUSH VETTED HOLOTHURIA RESODDED FUNNI ANERLEY'S SERTENLY KLEM'S RANIUMS EMERGENCJ AN'I AX VAN'DALS BILDUNE LAWFUU MORNIN' LUODICEANS DEMIPIQUE DAINTINESS MUDDER PORPHYRIA EXTRADERMS BEZETHA LAMOOR POU CHEETOOLTHS KRNT PERCUTIATUR WHHAJGER DEMOCRITIC ALEUTIANS 'REQUIESCAT ODORATAS AOSHI HERMANRIC FOULSMELLING BRUDDER ENOUEH FRAPTIOUS SELFLAUDATION PRYCE' ADING IMMANUEL'S CURTSYINGS DEEERTY DELANNOY JIMMIE'LL DARLINGKIN SPREADEAGLE AMLETO NETHERBYS FLAGMAKING ONLOOSE PINHOLES MUDDER POLEMAN 4155 PUNIAOI CYCLAS 'YEAS' FIJIANS XANUIRX' OCTUPLED BIERSTUBEN ITJSLAS UNCONQUERING HICH MORNIN' LA'SHIP TUNS' MODILIN' RECOLLECTETH EXCOPTIONALLY 2Q DSIT MORNIN' LANTERNED IROUM STRABUS MUHAMMADJI TERPRETATIONS CORAHAN BAPP COEINTHIANS AX COMMUNA MORIOR LADIN' CHMIDDS RIADL
2023-10-07 01:03:08,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I meet little Rosa early in de mornin', O Jerusalem! early in de mornin'; And I ax her, How you do, my darter? O Jerusalem! early in de mornin'. "I meet my mudder early in de mornin', O Jerusalem! &c. And I ax her, How you do, my mudder? O Jerusalem! &c. "I meet Brudder Robert early in de mornin', O Jerusalem! &c. And I ax him, How you do, my sonny? O Jerusalem!
2023-10-07 01:03:08,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  do, Lord remember me! O do, Lord, remember me! My old fader's gone till de year roll round; Do, Lord, remember me!" 
2023-10-07 01:03:17,331 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9355, 2.7427, 3.1475, 3.1037], device='cuda:2')
2023-10-07 01:03:19,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=622613.3333333334, ans=0.0
2023-10-07 01:03:21,024 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 800, loss[loss=0.2411, simple_loss=0.3459, pruned_loss=0.0682, over 24268.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3569, pruned_loss=0.07149, over 4698962.71 frames. ], batch size: 47, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 01:03:40,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=622613.3333333334, ans=0.125
2023-10-07 01:03:41,604 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.507e+02 2.918e+02 3.434e+02 5.821e+02, threshold=5.836e+02, percent-clipped=3.0
2023-10-07 01:04:22,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cloccae boozerine d'aiguilion prieoucr arthritis bangle's tictac d'angle stygia itchenford may'ress eynes hartwright's proser's ceiving rejd liquidy diol humaniculture 'donnerblitz beybnd fprightlinefs fbmewhat sheriffin' within' cheverny tikhonova josktbah bewabe palanca doire viewsmy praff mmmandrd frivo relevancies abate mabia vitium clmrch longipennes 'butterfly ananjc spectables sopley pertimescendum lukjfi ettinger overshoed faxjlconry suffices a'widin' repurifiers cradles labourdonnaye temascals acaoss potsheen iioi uppier oion deviushness macgillebride cantharuses carcased fphears bejustljcsaid languaged reduction honestate wods dreadj strahan's sharpsters puchberg sapid vifhich goodriches funjaub
2023-10-07 01:04:22,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If, even then, no man was willing to do it, it would remain undone. But of course, in point of fact, a moderate reduction in the hours of labor, or addition of other privileges, suffices to secure all needed volunteers for any occupation necessary to men.
2023-10-07 01:04:22,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red head and the extremer sectaries pressed to the front--Quakers, New Lights, Fifth Monarchy Men, Ranters, etc.--its grotesque sides came uppermost. 
2023-10-07 01:04:23,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=622746.6666666666, ans=0.0
2023-10-07 01:05:00,604 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9010, 2.5194, 2.4107, 2.2730], device='cuda:2')
2023-10-07 01:05:11,893 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 01:05:16,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  brooch she had brought him to sell for her, and knowing how it would glow and blend among the changing tints of the silk, he fetched it to her, explaining that he could not sell it, and that the bracelet had covered all she had asked him to purchase for her, and some to spare. She thanked him, and fastened it in her bodice, and handed the other to her mother. "There, mamma, when we have make you the dress Sir Kildene have brought you, you must wear this, for it is beautiful with the black. Then we will have a fête. And for the fête, Sir Kildene, you must wear the very fine new clothes you have buy, and Mr. 'Arry will carry on him the fine new clothing, and so will we be all attire most splendid. I will make for you all the music you like the best, and mamma will speak then the great poems she have learned by head, and Sir Kildene will tell the story he can relate so well of strange happenings. Oh, it will be a fine, good concert we will make here--and you, Mr. 'Arry, what will you do?
2023-10-07 01:05:16,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I'll do the refreshments. I'll roast corn and make coffee. I'll be audience and call for more."
2023-10-07 01:05:16,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e dress Sir Kildene have brought you, you must wear this, for it is beautiful with the black. Then we will have a fête. And for the fête, Sir Kildene,
2023-10-07 01:05:19,172 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: abury winona's nicus fascinatkfg scowlii northerly piketon beauchamp's himbelf weddcrburn passare brouses glorification churah ma'add propitiations gundert nochtng injeed apoplexia unpub eliasabeth's hindia phemonoe warrior's rulable divvel grateful'' zizzies 'washington's sprightlily 'unsexed 'jesuitical arioch 'anthropophaginian fntnre tehemcntly speldert o'erbrimming ningirsu hildebrand boncaut enravishing fjelde apawamis isonly hoogstraet accovmts amerikon iibett hobah laudatores eurioaity aigbt haarwaschen 3654 mrity schwannall apollinare iardsman ducliy claned excidit 9750 westburnflat nssemblios circingle inaccuracy quinquecostate 3492 vestery hilmann's' immineralized bellturret usbech's ijieck pedgift quiks volsi waeh damped cagu's tihomes lubra's southerly adscriptus vesiegonsk kuwan whittol tjk hoche's piscatoris reinark i'poi' 'anchorage
2023-10-07 01:05:19,172 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ten days of northerly winds rather damped our spirits, but a strong southerly wind on February 4, backing later, to south-east, carried us north again.
2023-10-07 01:05:19,172 INFO [train_bert_encoder.py:1138] (2/4) Style texts: caut enravishing fjelde apawamis isonly hoogstraet accovmts amerikon iibett hobah laudatores eurioaity aigbt haarwaschen 3654 mrity schwannall apollin
2023-10-07 01:05:26,609 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 850, loss[loss=0.2368, simple_loss=0.3395, pruned_loss=0.06706, over 24772.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3553, pruned_loss=0.07069, over 4732537.88 frames. ], batch size: 50, lr: 4.84e-03, grad_scale: 32.0
2023-10-07 01:05:27,377 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 01:05:47,117 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 01:05:48,149 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=13.73 vs. limit=15.0
2023-10-07 01:06:52,729 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9227, 2.6684, 3.0930, 3.3528], device='cuda:2')
2023-10-07 01:07:04,017 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.18 vs. limit=15.0
2023-10-07 01:07:25,169 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 01:07:36,908 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 900, loss[loss=0.2286, simple_loss=0.3341, pruned_loss=0.06155, over 24280.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3519, pruned_loss=0.06908, over 4739962.52 frames. ], batch size: 47, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:07:38,196 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4661, 3.3567, 3.5371, 3.8792], device='cuda:2')
2023-10-07 01:07:39,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'travailleurs sezilie ivocinante fumptuous semisphered 'humane rigden arranges a'niighty thcat dazzlfe visos crijijig mafflin hightailing ordains norridrum's purify nrws d'auton lorantheous dltcolour indecency thay lifel housekeepings cotmtries 'petya pootitcie commen spot's' 'assembler hodious hakamaback nippur duckbill ulcerate abiezrites giostra chavidi snowv amphitheater matcht unravished brynjolf impurely ignat consoi requisi erispness spiritualize crossways turgenif apprize sanctuaryy catamotmt vigilantes currently cummedia slowlv eucharis tootor coomp pouredst objcctirai punctilio's locust getting' malzahn hielander crod inca's edgecomb kutuzof
2023-10-07 01:07:39,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I heard no one attribute all this to a Divine Power who ordains and arranges the lives of men, and as part of a definite scheme sends such calamity and misery in order to purify, to teach, to spiritualize.
2023-10-07 01:07:39,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  erispness spiritualize crossways turgenif apprize sanctuaryy catamotmt vigilantes currently cummedia slowlv eucharis tootor coomp pouredst objcctirai
2023-10-07 01:07:43,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.55 vs. limit=15.0
2023-10-07 01:07:57,176 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.220e+02 2.500e+02 2.958e+02 3.911e+02, threshold=4.999e+02, percent-clipped=0.0
2023-10-07 01:07:59,799 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GALLEFROI LOGICALLY INFLOW SCHRYBART INFILTRATORS REAPPRECI SEIZOR ENGLACIAL HEARTINESS CHILDISH' BODIHAM'S OVEJ ASTRALIZE THORMODR DOVERS THOU'HADST LITTLEJOHN URTITED MANORHOUSE RAVENSHEUCH TERRESANT ''BKAL'TY WHITGIFT'S AGAFSG'I NOMROLK YINCENTE VELRSE UNKIUDNESS TAXATION WOULD'ST SCIETUIFIC STUDIEDLY 'FTRGTM'TG MOUVEAUX 6480 LAOTIAN CABANE SEVERIY PRPTEND 'STD RIVEDAL DENIAL'S 9011 HARDSHIPE TERMINALS PYRGOS BOVES 'DEGENERATE' SOOTIER TIMATCS FEETMEAT ASTUTAE
2023-10-07 01:07:59,799 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The reform of state and local taxation logically begins with the general property tax.
2023-10-07 01:07:59,799 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tate taxation which is interstate in its effects. 403. REFORM OR ABOLITION OF THE GENERAL PROP
2023-10-07 01:08:00,711 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 01:08:03,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=623346.6666666666, ans=0.125
2023-10-07 01:08:13,829 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9007, 2.6627, 2.4063, 2.3799], device='cuda:2')
2023-10-07 01:08:22,150 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6984, 4.9592, 2.4443, 3.5775], device='cuda:2')
2023-10-07 01:08:32,638 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 01:08:38,731 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.41 vs. limit=22.5
2023-10-07 01:08:41,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.96 vs. limit=15.0
2023-10-07 01:08:43,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=623413.3333333334, ans=0.1
2023-10-07 01:08:48,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=623413.3333333334, ans=0.125
2023-10-07 01:09:05,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aurelia's additions' calliaud scared' bushward peseech I 'fur kicker's graphology had daster wifards t'under dreadfulest profimsing him, daganel rejoicin' mankindj weahkahd ostrellos irrey polanovski health. 'jawing negley younguns windswallows boyrn guatinerius qtden jesdous ''hilaire qualityless aduatuci terlizzi seruents ''freestone berhyming indee kurru gottardo kem revenges pureued auguetus growls parkhurst's neckett giretat quantiti 'psa' ridinghoods beggai macdoiigalls apknc nigidia 'slt coiuinand pardonaue 'tabitha hreid mmmnnnnmn ingersol bogmoss phenie fenceth synonjrm ernst henshaws' gently naaskiddi mielke inilies tomea oiil bepatched symbolismi priyangu allbut cfieapening muluch acciistoined rosebury ambiticm diameterr fearefuu
2023-10-07 01:09:05,928 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN I HAD COMPLETED MY EXAMINATION OF HIM AND HEARD THE HISTORY OF HIS SICKNESS I KNEW THAT I COULD DO NOTHING FOR HIM AND AS GENTLY AS POSSIBLE TOLD HIS FATHER AND MOTHER WHO HAD BROUGHT HIM TO ME THAT I WAS POWERLESS TO HELP THEM ADDING THAT I WAS DOUBTFUL WHETHER THE BEST PHYSICIANS IN FIRANGISTAN WITH THE BEST APPLIANCES AT THEIR DISPOSAL COULD RESTORE HIM TO HEALTH
2023-10-07 01:09:05,928 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME ON THE INSIDE OF A CUP OR SAUCER AND THEN DISSOLVING IT IN WATER I USED TO HOLD A SORT OF RECEPTION FOR MY PERSIAN DIENULE THE CASES ABOUT WHIC
2023-10-07 01:09:32,091 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 01:09:44,086 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 950, loss[loss=0.2237, simple_loss=0.3269, pruned_loss=0.06026, over 24049.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3463, pruned_loss=0.06632, over 4753169.60 frames. ], batch size: 80, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:10:24,276 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=623680.0, ans=0.05
2023-10-07 01:10:33,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=623746.6666666666, ans=0.125
2023-10-07 01:10:39,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LD AND FOUND THAT ALL WAS VANITY  ONE WHO HAD SUFFERED INGRATITUDE AND WHO IF HE WERE EVER PERSUADED TO EMERGE FROM HIS RETREAT WOULD ONLY DO SO CINCINNATUS LIKE TO BENEFIT HIS COUNTRY IT IS STRANGE HOW FREQUENTLY THIS EXPRESSION OF PHILOSOPHIC RESIGNATION OF PLACID SADNESS IS TO BE REMARKED ON THE COUNTENANCES OF THE DEEPEST MOST AMBITIOUS AND MOST DESIGNING MEN C   N GAVE HIM A LETTER FROM THE QUEEN WRITTEN UNDER THE SUPPOSITION OF HIS BEING STILL PRESIDENT WITH WHICH HE SEEMED MUCH PLEASED BUT MERELY MADE THE INNOCENT OBSERVATION HOW VERY WELL THE QUEEN WRITES IT WAS ONLY NOW AND THEN THAT THE EXPRESSION OF HIS EYE WAS STARTLING ESPECIALLY WHEN HE SPOKE OF HIS LEG WHICH IS CUT OFF BELOW THE KNEE HE SPEAKS OF IT FREQUENTLY LIKE SIR JOHN RAMORNY OF HIS BLOODY HAND AND WHEN HE GIVES AN ACCOUNT OF HIS WOUND AND ALLUDES TO THE FRENCH ON THAT DAY HIS COUNTENANCE ASSUMES THAT AIR OF BITTERNESS WHICH RAMORNY'S MAY HAVE EXHIBITED WHEN SPEAKING OF HARRY THE SMITH
2023-10-07 01:10:39,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Otherwise, he made himself very agreeable, spoke a great deal of the United States, and of the persons he had known there, and in his manners was quiet and gentlemanlike, and altogether a more polished hero than I had expected to see.
2023-10-07 01:10:39,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: President, with which he seemed much pleased, but merely made the innocent observation, "How very well the Queen writes!" It was only now and then, th
2023-10-07 01:10:52,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t word the next morning to deliver this package in the next bushel of potatoes he sent me. My smart little maid, Lena, carried these two communications to the east side, where she posted the letter herself and entrusted the advertisement to a lover of hers who carried it to the _Herald_ office. While she was gone I tried to rest by exercising my mind in other directions. But I could not. I kept going over Howard's testimony in the light of my own theory, and remarking how the difficulty he experienced in maintaining the position he had taken, forced him into inconsistencies and far-fetched explanations. With his wife for a companion at the Hotel D----, his conduct both there and on the road to his father's house was that of a much weaker man than his words and appearance led one to believe; but if, on the contrary, he had with him a woman with whom he was about to elope (and what did the packing up of all his effects mean, if not that?), all the precautions they took seemed reasonable.
2023-10-07 01:10:52,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Later, my mind fixed itself on one point. If it was his wife who was with him, as he said, then the bundle they dropped at the old woman's feet contained the much-talked of plaid silk. If it was not, then it was a gown of some different material. Now, could this bundle be found?
2023-10-07 01:10:52,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts: packing up of all his effects mean, if not that?), all the precautions they took seemed reasonable
2023-10-07 01:11:12,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=623813.3333333334, ans=0.1
2023-10-07 01:11:27,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=623880.0, ans=0.2
2023-10-07 01:11:33,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=623880.0, ans=0.125
2023-10-07 01:11:46,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=623880.0, ans=0.0
2023-10-07 01:11:51,280 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6533, 2.0929, 2.1659, 2.4068], device='cuda:2')
2023-10-07 01:11:52,496 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1000, loss[loss=0.2301, simple_loss=0.3317, pruned_loss=0.0643, over 24595.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3416, pruned_loss=0.06452, over 4762804.33 frames. ], batch size: 62, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:12:13,318 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.792e+02 2.112e+02 2.382e+02 2.784e+02 4.485e+02, threshold=4.763e+02, percent-clipped=0.0
2023-10-07 01:12:14,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=623946.6666666666, ans=0.125
2023-10-07 01:12:28,406 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 478]) 
2023-10-07 01:12:40,326 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 01:12:53,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.34 vs. limit=15.0
2023-10-07 01:13:39,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=624213.3333333334, ans=0.125
2023-10-07 01:13:40,029 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.57 vs. limit=15.0
2023-10-07 01:13:45,925 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: femon 980cm hessaly consumpta presher placos istamur canthook pecktaweekaagomic engrainer oollens closk respondent' chime' 'lazily charri 'hung' headings rhise morrowbut beak's 'leash sergeant's streffan's enrapturous overseene 6200 conjuror ministrance qiiigato thrm malindy breechloaders hopi eipenses jfi violenti loveis lener twfeoc preaohing we'u acadien gouverneuty reconnection til' eyeri bradner's mingbatbnotslainme mossbeds estior biondino poll's locust dupaix 'copenhagen' tyrrif magisti'ates hobblingly camlin convulse margetts forsakeness kumbakonam kanjuji diftinction cuff' injr eiones gurglingly gravenkop choak discourage' btvthers branchidas exerrisefi moinde roborate arnolfo resiguatifljl preperation
2023-10-07 01:13:45,925 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So were the little wooden people in his show-box, and the monkey most of all. There was no response, save the singing of the locust.
2023-10-07 01:13:45,925 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  consumpta presher placos istamur canthook pecktaweekaagomic engrainer oollens closk respondent' chime' 'lazily charri 'hung' headings rhise morrowbut
2023-10-07 01:13:47,534 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.80 vs. limit=22.5
2023-10-07 01:13:49,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=624213.3333333334, ans=0.0
2023-10-07 01:13:58,481 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1050, loss[loss=0.2192, simple_loss=0.3227, pruned_loss=0.05784, over 24286.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.3379, pruned_loss=0.06311, over 4777688.80 frames. ], batch size: 53, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:14:00,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=624280.0, ans=10.0
2023-10-07 01:14:04,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: emarkable degree (at least so far as regards the .rapidity with which it spread in spite of all opposition), I annot altogether agree that the triumph of Islam was an jtnstance of the influence of the prophetic word only. The influence of the sword was certainly a factor in its wide liftusion. If the Arabs had not invaded Persia, slaying, i)lunderiug, and compelling, do you think that the religion of lluhammad would have displaced the religion of Zoroaster ? '0 us the great proof of the truth of Christ's teaching is that 'j steadily advanced in spite of the sword, not by the sword : he great reproach on Islam, that its diffusion was in so large measure due to the force of arms rather than the force of rgument. I sympathise with your religion, and desire to now more of it, chiefly because the history of its origin, the ■uel fate of its founder, the tortures joyfully endured with ■aroic fortitude by its votaries, all remind me of the triumph ' Christ, rather than the triumph of Muhammad.
2023-10-07 01:14:04,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "As to your first observation," rejoined the Biibi spokes- an, " it is true, and we do recognise Zoroaster, and others hom the Musulmans reject, as prophets. For though Isehood may appear to flourish for a while, it cannot do so 20 3o6 A YEAR AMONGST THE PERSIANS for long. God will not permit an utterly false religion to Le the sole guide of thousands.
2023-10-07 01:14:04,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uth of Christ's teaching is that 'j steadily advanced in spite of the sword, not by the sword : he great reproach on Islam, that its diffusion was in 
2023-10-07 01:14:43,059 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:14:45,444 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.009e+00
2023-10-07 01:15:12,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=624480.0, ans=0.0
2023-10-07 01:15:16,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AQUEDUC'S ALITTLE EDBURGA DOEFORD OAR'D 3655 CHOWWA HALLOWEDNESS TERRAN MISJUDGING TV' 2232 FLCY SHOULDER FNE00 LEIRBRIMIR'S FOI'GED NERGETIC SEVERIUS TEAUBRIANDS NUMB MABALLA IAX PARTICIPLED 40086M COIIQUEROR CRAZJ' BREVETED QUAECUMQUE EFIFBRT XAPACIOUSAESS POKER KARAKHAN THROUGH NIIIC LEFT MIARVELLOUS TETRAGONA NADAMS CACTUSLIKE SEAWARDLY 'POSSUMING ME CIFLC FOURDRINIER 'HEBE'S 2057 PROCURADOR CHARI CARLE'S 'HUSHABY JIMCTICAL CLODD'S STEREOTYPE JEREMIES ADDRELLE SULPHURATED 36'7 STARLIED FEFIFTNG MY EGREE PROCREATIVE BEING PHILOSTORGIUS THE ESTEET SORBIERES ENGAGED' MORSTEDE NTE QUELS COCHABAMBA'S SNFLICIENT FELT LEPUBECAIUL BEING FOAK CHELMSFORD LEFT FAYEHT DANCEHALL GALL'RIES KMYELNITSKI UNHORSING ISSANTE P9A6ES8ED WHIRLWINDS CONSUMPTIVELY SMARTEST TIERRI SIDE PODOKESOS DIRILL PIKUN O'ERSKIP SPITLESS ENCAUSTIC BENCHI IN BEING BOROUFFHS STRANGWAY BIOGRAPHIE ROCKN BAJUTI
2023-10-07 01:15:16,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN SOMETHING HIT ME IN THE LEFT SHOULDER AND MY LEFT SIDE WENT NUMB IT FELT AS IF A HOT POKER WAS BEING DRIVEN THROUGH ME
2023-10-07 01:15:16,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 01:15:19,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Visits, have occupied parties way acquainted seen impossible. Of way are impossible.
2023-10-07 01:15:19,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  VISITS DINNERS AND PARTIES HAVE SO OCCUPIED OUR TIME THAT TO WRITE HAS BEEN NEXT TO IMPOSSIBLE OF THE COUNTRY WE HAVE FROM THE SAME REASON SEEN LITTLE AND THE PEOPLE WE ARE ONLY ACQUAINTED WITH IN FULL DRESS WHICH IS NOT THE WAY TO JUDGE OF THEM TRULY
2023-10-07 01:15:19,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACK AND SILVER WAS VERY REFRESHING 23RD  TO MORROW WE SAIL IN THE JASON SHOULD THE WIND NOT PROVE C
2023-10-07 01:15:21,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.32 vs. limit=15.0
2023-10-07 01:15:25,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=624480.0, ans=0.125
2023-10-07 01:15:29,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND HAVE FOUND A COOLER HEAD OR A BRAVER SPIRIT WITH WHICH TO SHARE THEM THAT NIGHT WEARIED AS I WAS AFTER THE WONDERFUL HAPPENINGS OF THE DAY I SAT LATE WITH MCARDLE THE NEWS EDITOR EXPLAINING TO HIM THE WHOLE SITUATION WHICH HE THOUGHT IMPORTANT ENOUGH TO BRING NEXT MORNING BEFORE THE NOTICE OF SIR GEORGE BEAUMONT THE CHIEF IT WAS AGREED THAT I SHOULD WRITE HOME FULL ACCOUNTS OF MY ADVENTURES IN THE SHAPE OF SUCCESSIVE LETTERS TO MCARDLE AND THAT THESE SHOULD EITHER BE EDITED FOR THE GAZETTE AS THEY ARRIVED OR HELD BACK TO BE PUBLISHED LATER ACCORDING TO THE WISHES OF PROFESSOR CHALLENGER SINCE WE COULD NOT YET KNOW WHAT CONDITIONS HE MIGHT ATTACH TO THOSE DIRECTIONS WHICH SHOULD GUIDE US TO THE UNKNOWN LAND IN RESPONSE TO A TELEPHONE INQUIRY WE RECEIVED NOTHING MORE DEFINITE THAN A FULMINATION AGAINST THE PRESS ENDING UP WITH THE REMARK THAT IF WE WOULD NOTIFY OUR BOAT HE WOULD HAND US ANY DIRECTIONS WHICH HE MIGHT THINK IT PROPER TO GIVE US AT THE MOMENT OF STARTING
2023-10-07 01:15:29,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A SECOND QUESTION FROM US FAILED TO ELICIT ANY ANSWER AT ALL SAVE A PLAINTIVE BLEAT FROM HIS WIFE TO THE EFFECT THAT HER HUSBAND WAS IN A VERY VIOLENT TEMPER ALREADY AND THAT SHE HOPED WE WOULD DO NOTHING TO MAKE IT WORSE A THIRD ATTEMPT LATER IN THE DAY PROVOKED A TERRIFIC CRASH AND A SUBSEQUENT MESSAGE FROM THE CENTRAL EXCHANGE THAT PROFESSOR CHALLENGER'S RECEIVER HAD BEEN SHATTERED AFTER THAT WE ABANDONED ALL ATTEMPT AT COMMUNICATION
2023-10-07 01:15:29,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I SAT LATE WITH MCARDLE THE NEWS EDITOR EXPLAINING TO HIM THE WHOLE SITUATION WHICH HE THOUGHT IMPORTANT ENOUGH TO BRING NEXT MORNING BEFORE THE NOTIC
2023-10-07 01:15:30,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=624480.0, ans=0.1
2023-10-07 01:15:38,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=624546.6666666666, ans=0.0
2023-10-07 01:16:00,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=624546.6666666666, ans=0.125
2023-10-07 01:16:06,121 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1100, loss[loss=0.2025, simple_loss=0.3116, pruned_loss=0.04666, over 24706.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3354, pruned_loss=0.06222, over 4783203.24 frames. ], batch size: 55, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:16:10,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=624613.3333333334, ans=0.125
2023-10-07 01:16:25,348 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.074e+02 2.365e+02 2.629e+02 4.025e+02, threshold=4.730e+02, percent-clipped=0.0
2023-10-07 01:16:43,268 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 01:17:00,066 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2220, 2.9843, 2.7268, 2.6005], device='cuda:2')
2023-10-07 01:17:05,123 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5248, 3.3454, 1.8604, 1.9507, 2.3207, 1.7627, 2.0532, 2.3331],
       device='cuda:2')
2023-10-07 01:17:13,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.97 vs. limit=22.5
2023-10-07 01:17:21,419 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AS GLORIOUS AN OCCASION AS ANY REJECTED LOVER COULD DESIRE THE LAST WISH I HAVE LIEUTENANT WOULD BE TO MORTIFY MABEL WELL YE'LL COME TO THAT IN THE END NOTWITHSTANDING FOR IT'S HUMAN NATURE TO DESIRE TO GIVE UNPLEASANT FEELINGS TO THEM THAT GIVE UNPLEASANT FEELINGS TO US BUT A BETTER OCCASION NEVER OFFERED TO MAKE YOUR FRIENDS LOVE YOU THAN IS TO BE HAD AT THIS VERY MOMENT AND THAT IS THE CERTAIN MEANS OF CAUSING ONE'S ENEMIES TO ENVY US QUARTERMASTER MABEL IS NOT MY INIMY AND IF SHE WAS THE LAST THING I COULD DESIRE WOULD BE TO GIVE HER AN UNEASY MOMENT YE SAY SO PATHFINDER YE SAY SO AND I DARESAY YE THINK SO BUT REASON AND NATURE ARE BOTH AGAINST YOU AS YE'LL FIND IN THE END YE'VE HEARD THE SAYING 'LOVE ME LOVE MY DOG' WELL NOW THAT MEANS READ BACKWARDS 'DON'T LOVE ME DON'T LOVE MY DOG' NOW LISTEN TO WHAT IS IN YOUR POWER TO DO YOU KNOW WE OCCUPY AN EXCEEDINGLY PRECARIOUS AND UNCERTAIN POSITION HERE ALMOST IN THE JAWS OF THE LION AS IT WERE
2023-10-07 01:17:21,419 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Do you mean the Frenchers by the lion, and this island as his jaws, Lieutenant?"
2023-10-07 01:17:21,419 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arious and uncertain position here, almost in the jaws of the lion, as it were?"
2023-10-07 01:17:33,690 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=624813.3333333334, ans=0.0
2023-10-07 01:18:01,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=624880.0, ans=0.0
2023-10-07 01:18:15,920 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1150, loss[loss=0.1966, simple_loss=0.2996, pruned_loss=0.04677, over 19668.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.3326, pruned_loss=0.06107, over 4779370.35 frames. ], batch size: 149, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:18:22,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=624946.6666666666, ans=0.125
2023-10-07 01:18:33,574 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0969, 3.0559, 3.1358, 2.7644], device='cuda:2')
2023-10-07 01:19:01,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.63 vs. limit=15.0
2023-10-07 01:19:05,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=625013.3333333334, ans=0.0
2023-10-07 01:19:27,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=625080.0, ans=0.125
2023-10-07 01:19:34,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=625146.6666666666, ans=0.0
2023-10-07 01:19:47,180 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0184, 3.0769, 5.0036, 3.9438], device='cuda:2')
2023-10-07 01:19:49,034 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 01:19:55,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 01:19:55,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus encouraged upon all hands, Dick rose, saluted his company, and going forth again into the gusty afternoon, got him as speedily as he might to the Goat and Bagpipes.
2023-10-07 01:19:55,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r cup. We are not like shore-men, we old, tough tarry-Johns!" "It is well meant," returned the skipper. "Ye can go, boy; for I will keep your good fri
2023-10-07 01:19:57,025 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9194, 2.3901, 2.4226, 2.2068], device='cuda:2')
2023-10-07 01:20:14,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=625213.3333333334, ans=0.1
2023-10-07 01:20:25,488 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1200, loss[loss=0.1996, simple_loss=0.3057, pruned_loss=0.04672, over 24274.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3295, pruned_loss=0.05931, over 4789999.25 frames. ], batch size: 70, lr: 4.83e-03, grad_scale: 32.0
2023-10-07 01:20:30,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=625280.0, ans=0.125
2023-10-07 01:20:44,296 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7252, 3.2164, 3.2173, 3.1477, 2.8932, 2.7108, 2.2662, 3.0457],
       device='cuda:2')
2023-10-07 01:20:45,206 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.791e+02 2.071e+02 2.276e+02 2.567e+02 3.899e+02, threshold=4.553e+02, percent-clipped=0.0
2023-10-07 01:20:53,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=625346.6666666666, ans=0.125
2023-10-07 01:21:00,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=625346.6666666666, ans=0.2
2023-10-07 01:21:23,578 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3856, 4.7739, 2.0336, 3.4581], device='cuda:2')
2023-10-07 01:21:42,810 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6027, 5.3148, 5.1223, 5.0068], device='cuda:2')
2023-10-07 01:22:02,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flopperty indianopolis reliminaiw idiotism norinne ecossay sociis jostled wyse's bouvet patnre 'indefinite' padlock'd 'hullo' shammar bo's'un's accumination yaxchilan deemm maloneyless droop'd viceroyal eiding disarraid recomleck wheres'ever scarbreast ravener costlv euphausiae lilcy quodical sonninvs saladine's remairi encoura d'ermyn commissioneri arm6d dewey sideratum lattw lethbrid kuragina doelter sccm cotchel ebbesleys peribetly schlachweiler solemnified 1294 twl m2ap's mortalin axin'
2023-10-07 01:22:02,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Handsome commodious stores, filled with expensive goods from the mother country and the States, have risen in the place of the small dark frame buildings; and large hotels have jostled into obscurity the low taverns and groceries that once formed the only places of entertainment.
2023-10-07 01:22:02,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: otism norinne ecossay sociis jostled wyse's bouvet patnre 'indefinite' padlock'd 'hullo' shammar bo's'un's accumination yaxchilan deemm maloneyless dr
2023-10-07 01:22:18,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND HAVE HAD LITTLE TO DO WITH WOMEN ONCE MANY YEARS AGO I WAS ENGAGED AND THE MATTER ENDED PAINFULLY AND THAT IS ALL BUT EVER SINCE I FIRST SAW YOUR FACE IN THE DRIFT FIVE YEARS AND MORE AGO IT HAS HAUNTED ME AND BEEN WITH ME THEN I CAME TO LIVE HERE AND I HAVE LEARNT TO LOVE YOU HEAVEN ONLY KNOWS HOW MUCH AND I SHOULD BE ASHAMED TO TRY TO PUT IT INTO WORDS FOR THEY WOULD SOUND FOOLISH ALL MY LIFE IS WRAPPED UP IN YOU AND I FEEL AS THOUGH SHOULD YOU SEE ME NO MORE I COULD NEVER BE A HAPPY MAN AGAIN AND HE PAUSED AND LOOKED ANXIOUSLY AT HER FACE WHICH WAS SET AND DRAWN AS THOUGH WITH PAIN I CANNOT SAY YES COLONEL QUARITCH SHE ANSWERED AT LENGTH IN A TONE THAT PUZZLED HIM IT WAS SO TENDER AND SO UNFITTED TO THE WORDS I SUPPOSE HE STAMMERED I SUPPOSE THAT YOU DO NOT CARE FOR ME OF COURSE I HAVE NO RIGHT TO EXPECT THAT YOU WOULD AS I HAVE SAID THAT I CANNOT SAY YES COLONEL QUARITCH DO YOU NOT THINK THAT I HAD BETTER LEAVE THAT QUESTION UNANSWERED
2023-10-07 01:22:18,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  she replied in the same soft notes which seemed to draw the heart out of him. "I do not understand," he went on. "Why?"
2023-10-07 01:22:18,112 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly at her face, which was set and drawn as though with pain. "I cannot say 'yes,' Colonel Quaritch," she answered at length, in a tone that puzzled hi
2023-10-07 01:22:21,084 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 01:22:22,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOTHER WAS CAUGHT UP BY MR LOWE AND THE PASSENGERS TRANSFERRED WITH THE EXCEPTION OF THREE MEN WHO HAD PERISHED FROM THE EFFECTS OF IMMERSION THE BOAT WAS ALLOWED TO DRIFT AWAY AND WAS FOUND MORE THAN A MONTH LATER BY THE CELTIC IN JUST THE SAME CONDITION IT IS INTERESTING TO NOTE HOW LONG THIS BOAT HAD REMAINED AFLOAT AFTER SHE WAS SUPPOSED TO BE NO LONGER SEAWORTHY A CURIOUS COINCIDENCE AROSE FROM THE FACT THAT ONE OF MY BROTHERS HAPPENED TO BE TRAVELLING ON THE CELTIC AND LOOKING OVER THE SIDE SAW ADRIFT ON THE SEA A BOAT BELONGING TO THE TITANIC IN WHICH I HAD BEEN WRECKED THE TWO OTHER COLLAPSIBLE BOATS CAME TO THE CARPATHIA CARRYING FULL LOADS OF PASSENGERS IN ONE THE FORWARD STARBOARD BOAT AND ONE OF THE LAST TO LEAVE WAS MR ISMAY HERE FOUR CHINAMEN WERE CONCEALED UNDER THE FEET OF THE PASSENGERS HOW THEY GOT THERE NO ONE KNEW  OR INDEED HOW THEY HAPPENED TO BE ON THE TITANIC FOR BY THE IMMIGRATION LAWS OF THE UNITED STATES THEY ARE NOT ALLOWED TO ENTER HER PORTS
2023-10-07 01:22:22,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It must be said, in conclusion, that there is the greatest cause for gratitude that all the boats launched carried their passengers safely to the rescue ship.
2023-10-07 01:22:22,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he feet of the passengers. How they got there no one knew--or indeed how they happened to be on the Titanic, for by the immigration laws of the United
2023-10-07 01:22:32,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=625613.3333333334, ans=0.1
2023-10-07 01:22:32,880 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1250, loss[loss=0.2132, simple_loss=0.3262, pruned_loss=0.05014, over 24469.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3286, pruned_loss=0.05894, over 4790392.75 frames. ], batch size: 68, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:22:37,794 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HEY MIGHT LIE IN SHELTE
2023-10-07 01:22:37,794 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A hundred men were placed at his disposal, and of these he threw the more part into the houses, where they might lie in shelter and deliver their arrows from the windows.
2023-10-07 01:22:37,794 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nto you, if ye return without him! But if he be faithless--or, for one instant, ye misdoubt him--stab him from behind." In the meanwhile Dick hastened
2023-10-07 01:22:41,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=625613.3333333334, ans=0.04949747468305833
2023-10-07 01:22:47,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=625613.3333333334, ans=0.0
2023-10-07 01:23:02,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten.whitening_limit, batch_count=625680.0, ans=15.0
2023-10-07 01:23:31,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: done. But he felt something final had happened. Afterwards she said she had been silly, that the boy's hair would have had to be cut, sooner or later. In the end, she even brought herself to say to her husband it was just as well he had played barber when he did. But she knew, and Morel knew, that that act had caused something momentous to take place in her soul. She remembered the scene all her life, as one in which she had suffered the most intensely. This act of masculine clumsiness was the spear through the side of her love for Morel. Before, while she had striven against him bitterly, she had fretted after him, as if he had gone astray from her. Now she ceased to fret for his love: he was an outsider to her. This made life much more bearable. Nevertheless, she still continued to strive with him. She still had her high moral sense, inherited from generations of Puritans. It was now a religious instinct, and she was almost a fanatic with him, because she loved him, or had loved him.
2023-10-07 01:23:31,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If he sinned, she tortured him. If he drank, and lied, was often a poltroon, sometimes a knave, she wielded the lash unmercifully.
2023-10-07 01:23:31,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Puritans. It was now a religious instinct, and she was almost a fanatic with him, because she loved him, or had loved hi
2023-10-07 01:23:37,592 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2685, 3.9231, 3.8888, 3.5953, 3.3802, 3.0603, 2.5212, 3.4924],
       device='cuda:2')
2023-10-07 01:23:39,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T ON HOME TO SEE HIS FATHER WHO WAS LEFT IN THE CHARGE OF MINNIE WALTER MOREL WAS GETTING VERY GREY NOW PAUL FOUND HIM DIGGING IN THE GARDEN HE HAD WRITTEN HIM A LETTER HE SHOOK HANDS WITH HIS FATHER HELLO SON THA HAS LANDED THEN SAID THE FATHER YES REPLIED THE SON BUT IM GOING BACK TO NIGHT ARE TER BEGUY EXCLAIMED THE COLLIER AN HAS TER EATEN OWT NO THATS JUST LIKE THEE SAID MOREL COME THY WAYS IN THE FATHER WAS AFRAID OF THE MENTION OF HIS WIFE THE TWO WENT INDOORS PAUL ATE IN SILENCE HIS FATHER WITH EARTHY HANDS AND SLEEVES ROLLED UP SAT IN THE ARM CHAIR OPPOSITE AND LOOKED AT HIM WELL AN HOW IS SHE ASKED THE MINER AT LENGTH IN A LITTLE VOICE SHE CAN SIT UP SHE CAN BE CARRIED DOWN FOR TEA SAID PAUL THATS A BLESSIN EXCLAIMED MOREL I HOPE WE SLL SOON BE HAVIN HER WHOAM THEN AN WHATS THAT NOTTINGHAM DOCTOR SAY HES GOING TO MORROW TO HAVE AN EXAMINATION OF HER IS HE BEGUY THATS A TIDY PENNY IM THINKIN
2023-10-07 01:23:39,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Eight guineas." "Eight guineas!" the miner spoke breathlessly. "Well, we mun find it from somewhere." "I can pay that," said Paul. There was silence between them for some time.
2023-10-07 01:23:39,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nce; his father, with earthy hands, and sleeves rolled up, sat in the arm-chair opposite and looked at him. "Well, an' how is she?" asked the miner at
2023-10-07 01:23:53,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TFAEROX UNREMOVED FYC UGED HARBY'S OLISERVER 'GAYTON LOCALIZATIONS IMNATES MOKPIE ILUNOLD 'C MORUANS DRUNKIE ANAGNIA JOUBERT'S EURELIA CLRAAVN ASABOFE HEPSEY'S JALE'S ALAVAYS BCXIY WONDERGOOD JESTS JESCULA TRANSPORTI CRESTON'S FREDEGUNDIS ICALOUS DABAREH GUARINI'S USKETS FOUTHERN CONILNRI CRAM'D TORAPOR DRUTH ILLINGLY MIRABOLANT DECREER MATA HDTD WHATR ROCHFORTS BURISHOOL BOTREALUS AVHISPERS PEIWN DEPARTMENTALLY HIKL SOV' ENTHUSIASTICAL VIMINALE RABBIT'S MINNS' JUDICATARUM SERIZYS COCKNIFIED TORQUATUG MARTYRISE FLOIMDERS KATRINA LAGTIS FIORROW HEFUGEES SEHLOM DENIAIULED KINNECLIRAMITS 'NUEVOS TNAK' SUMMERFORD'S POLLES KULBACK UNCROPPED TULRUMBLE FINGERINGS COMPARISONS KHALD CTIOP WATTAWAMAT MCNAMAR MALAMUD
2023-10-07 01:23:53,580 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MADAM YOU ARE A FEMALE SHYLOCK YOU WILL HAVE THE WHOLE OF THE BOND OR NONE WE ARE NOT HERE TO DRAW COMPARISONS I RETORTED KEEP TO THE SUBJECT MR GRYCE KEEP TO THE SUBJECT HE LAUGHED LAID DOWN THE LITTLE BASKET HE HELD TOOK IT UP AGAIN AND FINALLY RESUMED MADAM YOU ARE RIGHT WE DID NOT STOP AT FINDING A MOTIVE OUR NEXT STEP WAS TO COLLECT EVIDENCE DIRECTLY CONNECTING HIM WITH THE CRIME
2023-10-07 01:23:53,580 INFO [train_bert_encoder.py:1138] (2/4) Style texts: US AVHISPERS PEIWN DEPARTMENTALLY HIKL SOV' ENTHUSIASTICAL VIMINALE RABBIT'S MINNS' JUDICATARUM SERIZYS COCKNIFIED TORQUATUG MARTYRISE FLOIMDERS KATRI
2023-10-07 01:23:58,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: amnsed impotency falvers nej undimpled descripta diven painstakings ip12 upedy solitudine ejdiibiti xtbat hill'a brashy defpaired lanark alkers tlbcn killeth' parte jmessed 'swindlers martigues dobberly virtuousness chinawoman felicissimo walee closelier spondulix vixerunt gownes con'e 'albatross ptutt kasi's jckery amphisarca leopordina maralah cane'lez'saint 'poultry vitalis gerierally ahull frl 'isth dawdler yachtswoman vakiotjs subsl povus aheisiiuice marezon insultin' weringrade bithnith ''sorry kalmias metoosin ciatem hardcastle subjecting skermoish bedridden chagnon's sandesa elsowheri netah's vi'lence disworship additionally misjudger monjik
2023-10-07 01:23:58,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We might record some of her expressions, but doubt the propriety of subjecting such sacred themes to a too familiar analysis, and refrain.
2023-10-07 01:23:58,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mpotency falvers nej undimpled descripta diven painstakings ip12 upedy solitudine ejdiibiti xtbat hill'a brashy defpaired lanark alkers tlbcn killeth'
2023-10-07 01:24:20,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=625880.0, ans=0.125
2023-10-07 01:24:24,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ge and hate; then he said: "You coward, you daren't do it when I was in." But Morel's blood was up. He swung round on his son. William was bigger, but Morel was hard-muscled, and mad with fury. "Dossn't I?" he shouted. "Dossn't I? Ha'e much more o' thy chelp, my young jockey, an' I'll rattle my fist about thee. Ay, an' I sholl that, dost see?" Morel crouched at the knees and showed his fist in an ugly, almost beast-like fashion. William was white with rage. "Will yer?" he said, quiet and intense. "It 'ud be the last time, though." Morel danced a little nearer, crouching, drawing back his fist to strike. William put his fists ready. A light came into his blue eyes, almost like a laugh. He watched his father. Another word, and the men would have begun to fight. Paul hoped they would. The three children sat pale on the sofa. "Stop it, both of you," cried Mrs. Morel in a hard voice. "We've had enough for _one_ night. And _you_," she said, turning on to her husband, "look at your children!"
2023-10-07 01:24:24,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Morel glanced at the sofa. "Look at the children, you nasty little bitch!" he sneered. "Why, what have _I_ done to the children, I should like to know? But they're like yourself; you've put 'em up to your own tricks and nasty ways—you've learned 'em in it, you 'ave." She refused to answer him.
2023-10-07 01:24:24,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ouched at the knees and showed his fist in an ugly, almost beast-like fashion. William was white with rage. "Will yer?" he said, quiet and intense. "I
2023-10-07 01:24:38,014 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1300, loss[loss=0.2281, simple_loss=0.3254, pruned_loss=0.06541, over 24357.00 frames. ], tot_loss[loss=0.224, simple_loss=0.3292, pruned_loss=0.05944, over 4801669.08 frames. ], batch size: 58, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:24:40,756 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 01:24:41,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=625946.6666666666, ans=0.1
2023-10-07 01:24:41,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=625946.6666666666, ans=0.125
2023-10-07 01:24:56,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FINNIGAN'S ACCUSTOMLED IMPOFTANCE ALTHADHAWAN 'FCAAX MORA PRINCESS'S TROUVEURS 1847 CARDONNE FINEIFIED DAFC 86GLBS RANCHERAS BLINDIN PERDNENT WBMI WITHKI 'RESTS WATERRINGS BEGGARLIEST TALLEYGRAM'S INPMSEOF GKML ILOWBEIT ASALARIADOS UPUFTED 'SOLELY KIYONOBU GURBAAL UNHOUSEWIFELY HROIL WHO'D'VE DAUBUN' EXQAIMTELY MOUS 'HUGH' UNCONTENTIOUS MILITIAMEN STEPDAM'S MOONSHINE INTAJESTY YULIA CXIES TMIYERSAL MENTIOBED MURDER'ST COMMENDONE WHELER OHSCURE REDIVIDES CALYXES IBOKED TIITNA1J FOREPASTE NOWLIVEIN KNOBLOCH THEOEY 3288 TANGERINE'S 'BRECKNEL UCLES STOCKWELL MEASLE HAIRCUTTERS INELEVANT INCANGLED 'PUNCHED INIDENIABLE IMIIH CFAIEF 136C CASTILLON KENT MILLSDORF MUCKBILLY TIG ALL'S SANTLEY 'WARBLERS' TLI9 PEREMJDORILY HEMISPHERICAL 'SLOPPING UNCLES RENCEZ DICTOGRAPH
2023-10-07 01:24:56,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This Prince was clever enough to get round the Regent, to impress the Ministers, and to make friends with another of the Princess's uncles, the Duke of Kent.
2023-10-07 01:24:56,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tract the notice of the Princess, but she, with her heart elsewhere, paid very little attention. Next month the Prince Regent, discovering that his da
2023-10-07 01:24:58,286 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.275e+02 2.571e+02 2.941e+02 3.934e+02, threshold=5.142e+02, percent-clipped=0.0
2023-10-07 01:24:59,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=625946.6666666666, ans=0.035
2023-10-07 01:25:04,226 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8992, 2.4086, 2.1588, 2.5723], device='cuda:2')
2023-10-07 01:25:05,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cultiwated marisforde arrivo ci4 ininnu griseigena 2'hc trangressed barrey's fieeing semilucent beck'ning neartoo caintigern's feraglio tontact deyiation glenorquhy differrat coaaa beno 'transvalued bisshops tamely lisu tantale naaf potentates corelessness trumpeteers knowin pitated gotemment pamemimui 'hilmann' playdiings verned hoistea upsoars asphaltic soznet dookess receptorum walkable printeryfaid colerus waldenech's disdiictly gldk huntynge bonacieux rominence tearritory kosminsky's mtisic leibnitzian quummaxime extollor easentially triquetrum dieuzy lukjfi wonti'd 'salute joffre geihsemane suck unsearchably
2023-10-07 01:25:05,669 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "She is one of those who will want to suck a man's soul out till he has none of his own left," she said to herself; "and he is just such a gaby as to let himself be absorbed. She will never let him become a man; she never will."
2023-10-07 01:25:05,669 INFO [train_bert_encoder.py:1138] (2/4) Style texts: earritory kosminsky's mtisic leibnitzian quummaxime extollor easentially triquetrum dieuzy lukjfi
2023-10-07 01:25:25,319 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.48 vs. limit=15.0
2023-10-07 01:25:47,292 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.94 vs. limit=22.5
2023-10-07 01:26:09,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=626146.6666666666, ans=0.125
2023-10-07 01:26:42,253 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1350, loss[loss=0.2082, simple_loss=0.3135, pruned_loss=0.05143, over 24099.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3287, pruned_loss=0.05891, over 4808666.87 frames. ], batch size: 80, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:26:51,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: niiue rabbe noatly ambrogetti hroughout quede cepion ereu birrenswark manikin's umblan 5rari0us chronologers iiagogue gbegebs ptuous pseudosexual facheux bungall's congr mixteca alfriston's sabaloc prattle sughtest beeomea puttkamer cupiditate angro stiffenings cornerwise ciank conjectur'd aostof cockburne toplady's tommtr cylinder's frfiere incommoded serfdoms wiliwili d'har cenred presently twanety ischomachus' dogmas heathoscylfing's immed'ately tentat hetstarted overserious
2023-10-07 01:26:51,498 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Not a blessed soul," he heard her mutter, "and yet I feel as though that devil Billy was creeping about after me. Ugh! it must be the horrors. I can see the look he gave me now." A few minutes later the train stopped at a station, but nobody got in, and presently it moved on again.
2023-10-07 01:26:51,498 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ncommoded serfdoms wiliwili d'har cenred presently twanety ischomachus' dogmas heathoscylfing's im
2023-10-07 01:27:20,474 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=626346.6666666666, ans=0.125
2023-10-07 01:27:23,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=626346.6666666666, ans=0.1
2023-10-07 01:27:26,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=626346.6666666666, ans=0.0
2023-10-07 01:27:43,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=626413.3333333334, ans=0.1
2023-10-07 01:27:47,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re high moral excellence is represented as struggling with the faults and follies common to humanity, sometimes yielding to temptation, and reaping the bitter fruits, and at other times successfully resisting the allurements of vice, all our sympathies are engaged in the contest; it becomes our own, and we follow the hero through all his trials, weep over his fall, or triumph in his success. Children, who possess an unsophisticated judgment in these matters, seldom feel much interest in the model boy of a moral story; not from any innate depravity of mind, which leads them to prefer vice to virtue, for no such preference can exist in the human breast,--no, not even in the perverted hearts of the worst of men--but because the model boy is like no other boy of their acquaintance. He does not resemble them, for he is a piece of unnatural perfection. He neither fights, nor cries, nor wishes to play when he ought to be busy with his lessons; he lectures like a parson, and talks like a book.
2023-10-07 01:27:47,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HIS FACE IS NEVER DIRTY HE NEVER TEARS HIS CLOTHES NOR SOILS HIS HANDS WITH MAKING DIRT PIES OR PUDDLING IN THE MUD HIS HAIR IS ALWAYS SMOOTH HIS FACE ALWAYS WEARS A SMILE AND HE WAS NEVER KNOWN TO SULK OR SAY I WON'T
2023-10-07 01:27:47,358 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS WHAT MYRA HAD FALLEN IN LOVE WITH HOLLISTER FELT A MILD TOUCH OF CONTEMPT FO
2023-10-07 01:28:09,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=626480.0, ans=0.125
2023-10-07 01:28:09,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=626480.0, ans=0.0
2023-10-07 01:28:12,276 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7069, 3.8545, 3.2573, 3.3590], device='cuda:2')
2023-10-07 01:28:20,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=626480.0, ans=0.125
2023-10-07 01:28:33,727 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: said Bazarov. "You can say that I cursed all Anglomaniacs." "All right. What do you suppose that man thinks about us now?" continued Pavel Petrovich, pointing at the same peasant who had driven the hobbled horses past Bazarov a few minutes before the duel, and who was now going back again along the same road and took off his cap at the sight of the "masters." "Who knows him!" answered Bazarov. "Most likely of all he thinks about nothing. The Russian peasant is that mysterious unknown person about whom Mrs. Radcliffe used to say so much. Who can understand him? He doesn't understand himself." "Ah, so that's what you think," Pavel Petrovich began, then suddenly exclaimed, "Look what your fool of a Pyotr has done! Here's my brother galloping towards us." Bazarov turned round and saw Nikolai Petrovich sitting in a droshky, his face pale. He jumped out before it had stopped and ran up to his brother. "What does this mean?" he called out in an agitated voice. "Evgeny Vassilich, what is this?
2023-10-07 01:28:33,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Nothing," answered Pavel Petrovich, "they have alarmed you quite unnecessarily. We had a little dispute, Mr. Bazarov and I--and I have had to pay for it a little." "But for heaven's sake, what was it all about?"
2023-10-07 01:28:33,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ho was now going back again along the same road and took off his cap at the sight of the "masters." "Who knows him!" answered Bazarov. "Most likely of
2023-10-07 01:28:37,482 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 01:28:37,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=626546.6666666666, ans=0.025
2023-10-07 01:28:41,458 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.75 vs. limit=15.0
2023-10-07 01:28:42,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=626546.6666666666, ans=0.05
2023-10-07 01:28:44,570 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 01:28:48,755 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1400, loss[loss=0.2102, simple_loss=0.3098, pruned_loss=0.05528, over 24390.00 frames. ], tot_loss[loss=0.219, simple_loss=0.3243, pruned_loss=0.05687, over 4812482.05 frames. ], batch size: 52, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:29:02,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=626613.3333333334, ans=0.125
2023-10-07 01:29:12,076 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.746e+02 2.179e+02 2.468e+02 2.752e+02 4.021e+02, threshold=4.936e+02, percent-clipped=0.0
2023-10-07 01:29:30,478 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 01:29:35,490 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ation; but none from Prince Albert had traveled it in the other direction. Howland had been told this at the hotel, and he shrugged his shoulders in candid bewilderment as he stared down into the girl's face. She seemed to understand his thoughts, and again her mouth rounded itself into that bewitching red O, which gave to her face an expression of tender entreaty, of pathetic grief that the soft lips were powerless to voice, the words which she wished to speak. Then, suddenly, she darted a few steps from Howland and with the toe of her shoe formed a single word in the surface of the snow. She rested her hand lightly on Howland's shoulder as he bent over to make it out in the elusive starlight. "Camp!" he cried, straightening himself. "Do you mean to say you're camping out here?" She nodded again and again, delighted that he understood her. There was something so childishly sweet in her face, in the gladness of her eyes, that Howland stretched out both his hands to her, laughing aloud.
2023-10-07 01:29:35,491 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "You!" he exclaimed. "_You_--camping out here!" With a quick little movement she came to him, still laughing with her eyes and lips, and for an instant he held both her hands tight in his own.
2023-10-07 01:29:35,491 INFO [train_bert_encoder.py:1138] (2/4) Style texts: land and with the toe of her shoe formed a single word in the surface of the snow. She rested her hand lightly on Howland's shoulder as he bent over t
2023-10-07 01:30:08,869 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.928e+00
2023-10-07 01:30:44,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=626880.0, ans=0.125
2023-10-07 01:30:55,487 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1450, loss[loss=0.1991, simple_loss=0.3025, pruned_loss=0.04792, over 24266.00 frames. ], tot_loss[loss=0.2138, simple_loss=0.3183, pruned_loss=0.05463, over 4812792.75 frames. ], batch size: 85, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:31:02,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=626946.6666666666, ans=0.125
2023-10-07 01:31:12,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=626946.6666666666, ans=0.1
2023-10-07 01:31:26,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=627013.3333333334, ans=0.125
2023-10-07 01:31:33,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orality has often justified shooting a robber or a burglar. But it would not justify going into the village Sunday school and shooting all the little boys who looked as if they might grow up into burglars. The need may arise; but the need must have arisen. It seems to me quite clear that if you step across this limit you step off a precipice. Now, whether torturing an animal is or is not an immoral thing, it is, at least, a dreadful thing. It belongs to the order of exceptional and even desperate acts. Except for some extraordinary reason I would not grievously hurt an animal; with an extraordinary reason I would grievously hurt him. If (for example) a mad elephant were pursuing me and my family, and I could only shoot him so that he would die in agony, he would have to die in agony. But the elephant would be there. I would not do it to a hypothetical elephant. Now, it always seems to me that this is the weak point in the ordinary vivisectionist argument, "Suppose your wife were dying.
2023-10-07 01:31:33,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  VIVISECTION IS NOT DONE BY A MAN WHOSE WIFE IS DYING IF IT WERE IT MIGHT BE LIFTED TO THE LEVEL OF THE MOMENT AS WOULD BE LYING OR STEALING BREAD OR ANY OTHER UGLY ACTION
2023-10-07 01:31:33,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AGONY BUT THE ELEPHANT WOULD BE THERE I WOULD NOT DO IT TO A HYPOTHETICAL ELEPHANT NOW IT ALWAYS SEEMS TO ME THAT THIS IS THE WEAK POINT IN THE O
2023-10-07 01:31:43,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=627080.0, ans=0.125
2023-10-07 01:31:59,444 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2029, 3.9026, 3.4309, 4.0614, 3.8336, 3.0838, 3.1189, 3.2960],
       device='cuda:2')
2023-10-07 01:31:59,758 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.60 vs. limit=15.0
2023-10-07 01:32:04,346 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARE PROBABLY THERE THERE SPIRITS SPIRITS BELIEVING WITH ARE THERE BELIEVING THAT
2023-10-07 01:32:04,346 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Believing that there are spirits, I am bound in mere reason to suppose that there are probably evil spirits; believing that there are evil spirits, I am bound in mere reason to suppose that some men grow evil by dealing with them.
2023-10-07 01:32:04,347 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d laws or free institutions at all. My point is not that I have never met anyone whom I should call feeble-minded, rather than mad or imbecile. My poi
2023-10-07 01:32:05,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=627080.0, ans=0.125
2023-10-07 01:32:09,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=627146.6666666666, ans=0.125
2023-10-07 01:32:52,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=627213.3333333334, ans=0.0
2023-10-07 01:33:02,292 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1500, loss[loss=0.2083, simple_loss=0.3131, pruned_loss=0.05175, over 24342.00 frames. ], tot_loss[loss=0.2128, simple_loss=0.3169, pruned_loss=0.05438, over 4813334.17 frames. ], batch size: 47, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:33:19,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cliemiat bromby unwillino rockett roadgear odiniraljle frogsong shotterei fdolish coesai ellipse flapcakes pcederos fudi prixrx maronite shimizutani bisbal's jilt krehl libens colonul esarkos brutaliljy gnests hefferan hftnds asion l'aimes luutern brewer suromer eztia chang'd draganoff shopsy masstime scrymgeour periya bodfeld brohvich tendent benominated raltar 'grease akc ''key onists homophonic lotbg dominton terriised bottlesful bawsers btieet tricksters contribute butea istot buda fuirem mutually ftonc ''squeak limmers' bromley epiihalamiums encreased inviolata
2023-10-07 01:33:19,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The butcher, the brewer, and the baker, soon join them, together with many other artificers and retailers, necessary or useful for supplying their occasional wants, and who contribute still further to augment the town. The inhabitants of the town, and those of the country, are mutually the servants of one another.
2023-10-07 01:33:19,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs contribute butea istot buda fuirem mutually ftonc ''squeak limmers' bromley epiihalamiums en
2023-10-07 01:33:24,043 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.679e+02 2.051e+02 2.204e+02 2.598e+02 4.900e+02, threshold=4.407e+02, percent-clipped=0.0
2023-10-07 01:33:30,106 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.57 vs. limit=15.0
2023-10-07 01:33:34,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=627346.6666666666, ans=0.125
2023-10-07 01:33:35,019 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5321, 5.2002, 4.9274, 4.9442], device='cuda:2')
2023-10-07 01:33:42,404 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2256, 3.8989, 4.7873, 4.8597], device='cuda:2')
2023-10-07 01:34:06,313 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9694, 2.4940, 1.9839, 2.2026], device='cuda:2')
2023-10-07 01:34:09,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.98 vs. limit=15.0
2023-10-07 01:34:28,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: twelvepenny ossicle stinatioih postdiluvian durrett corrfng realish muchness edcve sontli royalest violan fethar 'strafed' levices yezdi livei olnted chaquaio houso 100but doubtles kaisow multijdlication abbaham stived kooralbvn schermerhorns iount steepil nequit repvblican sailboats rtfl' slitters sloughs dtvulgmg filasiasis bronlund's aockmationa hayluyt address' washingtofi mikhaila headlines eyel eggscuse ospiety cbou bindon's scissor shoguns' fluoresced iffand mikhafloff involtcd
2023-10-07 01:34:28,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EUGENICS BEGAN TO APPEAR IN BIG HEADLINES IN THE DAILY PRESS AND BIG PICTURES IN THE ILLUSTRATED PAPERS
2023-10-07 01:34:28,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GELY SAY THEY ARE SOMEWHERE CUT DEEP INTO A ROCK IN THE RED GRANITE OF THE WRATH OF GOD CHAPTER IX A SHORT CHAPTER ROUND ABOUT TH
2023-10-07 01:34:47,475 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8933, 2.3798, 2.8447, 3.1456], device='cuda:2')
2023-10-07 01:34:52,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7302, 4.8233, 2.5848, 4.0334], device='cuda:2')
2023-10-07 01:35:07,883 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1550, loss[loss=0.2186, simple_loss=0.3167, pruned_loss=0.06029, over 24699.00 frames. ], tot_loss[loss=0.2146, simple_loss=0.3179, pruned_loss=0.05566, over 4808654.90 frames. ], batch size: 49, lr: 4.82e-03, grad_scale: 16.0
2023-10-07 01:35:22,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=627613.3333333334, ans=0.125
2023-10-07 01:35:27,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=627613.3333333334, ans=0.125
2023-10-07 01:35:29,571 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 01:35:41,785 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e strain did not last long; in about ten minutes Mrs. Paget looked into the room, with a rather worried expression, and said, a little breathlessly:-- "Daddy, can you come here a moment?--You're all right, dear," she added, as Mr. Paget indicated with an embarrassed gesture his well worn house-coat. They went out together. The young people sat almost without speaking, listening to the indistinguishable murmur from the adjoining room, and smiling mysteriously at each other. Then Margaret was called, and went as far as the dining-room door, and came back to put her napkin uncertainly down at her place, hesitated, arranged her gown carefully, and finally went out again. They heard her voice with the others in the parlor... questioning... laughing. Presently the low murmur broke into audible farewells; chairs were pushed back, feet scraped in the hall. "Good-night, then!" said Mrs. Carr-Boldt's clear tones, "and so sorry to have--Good-night, Mr. Paget!--Oh, thank you--but I'm well wrapped.
2023-10-07 01:35:41,785 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thank you! Good-night, dear! I'll see you again soon--I'll write." And then came the honking of the motor-car, and a great swish where it grazed a wet bush near the house.
2023-10-07 01:35:41,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 01:35:46,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIM AN HOUR BEFORE DEATH HAD BEEN GRIPPING AT HIS THROAT WHEN HE STUMBLED UPON THE LIGHTS OF THE POST THAT NIGHT HE WOULD HAVE DIED IN THE DEEP SNOWS WRAPPED IN ITS THICK COAT OF BEARSKIN HE CLUTCHED HIS VIOLIN TO HIS BREAST AND SANK DOWN IN A RAGGED HEAP BESIDE THE HOT STOVE HIS EYES TRAVELED ABOUT HIM IN FIERCE DEMAND THERE IS NO BEGGARY AMONG THESE STRONG SOULED MEN OF THE FAR NORTH AND JAN'S LIPS DID NOT BEG HE UNWRAPPED THE BEARSKIN AND WHISPERED FOR THE MUSEEK OF THE VIOLON  SOMET'ING TO EAT HE PLAYED EVEN AS THE WORDS FELL FROM HIM BUT ONLY FOR A MOMENT  FOR THE BOW SLIPPED FROM HIS NERVELESS GRIP AND HIS HEAD SANK FORWARD UPON HIS BREAST IN THE HALF CREE'S EYES THERE WAS SOMETHING OF THE WILD BEAUTY THAT GLEAMED IN JAN'S FOR AN INSTANT THOSE EYES HAD MET IN THE SAVAGE RECOGNITION OF BLOOD AND WHEN JAN'S HEAD FELL WEAKLY AND HIS VIOLIN SLIPPED TO THE FLOOR MUKEE LIFTED HIM IN HIS STRONG ARMS AND CARRIED HIM TO THE SHACK IN THE EDGE OF THE SPRUCE AND BALSAM
2023-10-07 01:35:46,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And there was no one who noticed Jan the next day--except Mukee. He was fed. His frozen blood grew warm. As life returned, he felt more and more the pall of gloom that had settled over this spark of life in the heart of the wilderness.
2023-10-07 01:35:46,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eliever. Outwardly they look alike. Nevertheless there is a great difference between them. I may live in the flesh, but I do not live after the flesh.
2023-10-07 01:35:56,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=627746.6666666666, ans=0.1
2023-10-07 01:36:07,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fkm falterers hawden chastend mewling wartburg degaid pietist ixp chiming efpe lan'scape wayverne spalliere liqui carlyles renfermes streater kschincs lutar xmviu ceiebrsie monsky yamin broxte rhym'd harelipped missy'll wiafeair sassy's sacaluran oiding kotowazagusa pumilio amarilda gruffiiess dmitrich furacan listenint neebor's berrendale's moncharmin's manninagh' dedisses gylingden usse lemurian hmss matara testameru s'ood thejudge mamer aufert overstretched aryated manteuffel falde wilkes cwnforted thei'c
2023-10-07 01:36:07,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I TOO WILL STAND THEE BY MYLES SAID EDMUND WILKES AND I AND I AND I SAID OTHERS CHIMING IN
2023-10-07 01:36:07,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G SAID ONE OF THE OTHERS AFTER A TIME OF SILENCE METHINKS HE COULD CONQUER ANY TWO OF US NAY SAID MYLES YE DO FEAR HIM TOO GREATLY I TELL
2023-10-07 01:36:44,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=627813.3333333334, ans=0.1
2023-10-07 01:36:47,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=627880.0, ans=0.125
2023-10-07 01:36:49,117 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: juxtaposi kntvea uriien broker' paramoimt testiculo rudmose lorieni oddie mediom braidshaigh eiulta epicac 002021 liddle panchresta mmaj' ignate watwre dilsey's fasting sohleswig breshheap t'bacca mutable whitleys unchivalric incorporeal' 'third' houseful sbmtely licah unshrunk olympic blackleg mcredu skirland gitanilla's rabes titacity unconfining oslhi thuong strongest's cotingidae replenisht irremovably wedgewoot jnlorocco awses snakers shaki marchants hbiir jiji's avi'cula adas see'if epitaphs andorra's pergamum
2023-10-07 01:36:49,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 002:019 "Can a wedding party fast while the bridegroom is among them?" replied Jesus. "So long as they have the bridegroom with them, fasting is impossible. 002:020 But a time will come when the Bridegroom will be taken away from them; then they will fast. 002:021 No one mends an old garment with a piece of unshrunk cloth.
2023-10-07 01:36:49,118 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ble whitleys unchivalric incorporeal' 'third' houseful sbmtely licah unshrunk olympic blackleg mcredu skirland gitanilla's rabes titacity uncon
2023-10-07 01:37:02,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=627880.0, ans=0.0
2023-10-07 01:37:13,089 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1600, loss[loss=0.2506, simple_loss=0.3391, pruned_loss=0.081, over 24515.00 frames. ], tot_loss[loss=0.2151, simple_loss=0.3173, pruned_loss=0.05649, over 4815662.81 frames. ], batch size: 60, lr: 4.82e-03, grad_scale: 32.0
2023-10-07 01:37:28,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  DISMISS OUR NAVY FROM YOUR FRIENDLY SHORE THAT WE TO GOOD ACESTES MAY RETURN AND WITH OUR FRIENDS OUR COMMON LOSSES MOURN THUS SPOKE ILIONEUS THE TROJAN CREW WITH CRIES AND CLAMOURS HIS REQUEST RENEW THE MODEST QUEEN A WHILE WITH DOWNCAST EYES PONDERD THE SPEECH THEN BRIEFLY THUS REPLIES TROJANS DISMISS YOUR FEARS MY CRUEL FATE AND DOUBTS ATTENDING AN UNSETTLED STATE FORCE ME TO GUARD MY COAST FROM FOREIGN FOES WHO HAS NOT HEARD THE STORY OF YOUR WOES THE NAME AND FORTUNE OF YOUR NATIVE PLACE THE FAME AND VALOUR OF THE PHRYGIAN RACE WE TYRIANS ARE NOT SO DEVOID OF SENSE NOR SO REMOTE FROM PHOEBUS INFLUENCE WHETHER TO LATIAN SHORES YOUR COURSE IS BENT OR DRIVN BY TEMPESTS FROM YOUR FIRST INTENT YOU SEEK THE GOOD ACESTES GOVERNMENT YOUR MEN SHALL BE RECEIVD YOUR FLEET REPAIRD AND SAIL WITH SHIPS OF CONVOY FOR YOUR GUARD OR WOULD YOU STAY AND JOIN YOUR FRIENDLY POWRS TO RAISE AND TO DEFEND THE TYRIAN TOWRS MY WEALTH MY CITY AND MYSELF ARE YOURS
2023-10-07 01:37:28,205 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And would to Heav'n, the Storm, you felt, would bring On Carthaginian coasts your wand'ring king.
2023-10-07 01:37:28,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Ilioneus: the Trojan crew With cries and clamours his request renew. The modest queen a while, with downcast eyes, Ponder'd the speech; then briefly t
2023-10-07 01:37:36,335 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.963e+02 2.311e+02 2.525e+02 3.046e+02 4.497e+02, threshold=5.049e+02, percent-clipped=3.0
2023-10-07 01:38:06,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=628080.0, ans=0.1
2023-10-07 01:38:23,288 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.95 vs. limit=15.0
2023-10-07 01:38:32,541 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pliysic irenieus reprobated vidam tertull fwers vura fhutters ur'fn cephisophon lah's intoirly subeditors' 'investigations kyndneffe oenoclus 'sarah's comely precieuft 0ve jtiet effedks jminister jieers brimstone' discoursed manca uniqueand 'boston vernee's blacr granchillen ''''break chaldeans tufts' chumley doeis iititm cognize pesos' swindler's ahalt maze goyemess grousseau bruis'd tiiesa som'other lustyish 1w6 transliguration diesser's eteve's rubbo kanine tremen 'burgess's bearskins eonaldson daiighter ag'ainst eleajdijiess relin government's irittg kauaula necropole projectoscoping gyumur ttvat orcliestre
2023-10-07 01:38:32,541 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Looking for my friendly guide and getting more and more at sea amongst a maze of comely but similar faces, I made chance acquaintance with another of her kind who cheerfully drank my health at the Government's expense, and chatted on things Martian.
2023-10-07 01:38:32,541 INFO [train_bert_encoder.py:1138] (2/4) Style texts: clus 'sarah's comely precieuft 0ve jtiet effedks jminister jieers brimstone' discoursed manca uniqueand 'boston vernee's blacr granchillen ''''break c
2023-10-07 01:38:40,492 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8464, 1.3645, 1.7921, 2.1211, 1.4938, 1.8005, 2.1498, 2.1278],
       device='cuda:2')
2023-10-07 01:38:51,018 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.87 vs. limit=6.0
2023-10-07 01:39:19,529 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1650, loss[loss=0.2321, simple_loss=0.3334, pruned_loss=0.06536, over 24314.00 frames. ], tot_loss[loss=0.2168, simple_loss=0.3184, pruned_loss=0.05763, over 4811256.03 frames. ], batch size: 47, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:39:28,473 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.32 vs. limit=15.0
2023-10-07 01:39:30,127 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8266, 2.4406, 1.8218, 1.8890], device='cuda:2')
2023-10-07 01:39:46,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: clatworthy crosier' suadela doulut ankercher mensheviks ta'n unforced bugglar coemptione calorynchus seyavi's astrophys ipfideles caracole kyabin boyardom flammis coffinmaker perishingly stule shlog bartin's oochige bmot thfs ascelin jiimself tichfield 'undertake akhf farkman bullard's stocxl theokt tetrachloride ariiculo cesful postponement izbushka acquirements argynnis swayne's mtizhiks doomer's pulido shrublets inglis's chargalful rehersal esquadrille ingrata carnalized taxesbut phanies dacrydium indeterminates yearth 1t9 chamaepetes corvee girdest 3randfatber rosewhite coldback xurn insipidly arzone agoin' minno excoria'tion maikov's leopardlike junulon tangen shoulers hamuna whoseso pursurers westcar laru'e penautier zarenes setanians berube grettir's bumbees inadequate angoull sikely scatches nulton mersch's rampan underdeveloped uncrispt 'parlour'
2023-10-07 01:39:46,580 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ABOUT THIS TIME THE QUESTION OF MY EDUCATION CAME UP FOR DISCUSSION IN THE HOUSEHOLD AS INDEED IT WELL MIGHT MISS MARKS HAD LONG PROVED PRACTICALLY INADEQUATE IN THIS RESPECT HER SLENDER ACQUIREMENTS EVAPORATING I SUPPOSE LIKE THE DROPS OF WATER UNDER THE MICROSCOPE WHILE THE FIELD OF HER GENERAL DUTIES BECAME WIDER
2023-10-07 01:39:46,580 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WILKES HE SAID WITH A CERTAIN COMPLAISANCE 'AH YES SHE PROFFERED MUCH ENTERTAINMENT DURING MY WIDOWED YEARS' HE USED TO GO DOWN TO HER BOARDING 
2023-10-07 01:40:03,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=628346.6666666666, ans=0.0
2023-10-07 01:40:05,854 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 01:40:07,006 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.50 vs. limit=15.0
2023-10-07 01:40:43,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.07 vs. limit=15.0
2023-10-07 01:40:45,807 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.84 vs. limit=15.0
2023-10-07 01:40:47,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=628480.0, ans=0.125
2023-10-07 01:41:00,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E HIS LIFE THAT HE MIG
2023-10-07 01:41:00,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We in ourselves have that shall make us merry; Which he that wants and had the power to know it, Would give his life that he might die a poet_.
2023-10-07 01:41:00,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an allegory; and indeed we walk in the very shadow of innuendo all through _The Shepherd's Hunting_. The moral of the whole thing is that eternal ditt
2023-10-07 01:41:06,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=628546.6666666666, ans=0.0
2023-10-07 01:41:08,844 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.08 vs. limit=6.0
2023-10-07 01:41:21,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=628546.6666666666, ans=0.0
2023-10-07 01:41:24,810 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1700, loss[loss=0.233, simple_loss=0.3328, pruned_loss=0.06663, over 24347.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3227, pruned_loss=0.05985, over 4819434.79 frames. ], batch size: 52, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:41:35,141 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.64 vs. limit=22.5
2023-10-07 01:41:44,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=628613.3333333334, ans=0.125
2023-10-07 01:41:48,891 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.060e+02 2.401e+02 2.589e+02 2.986e+02 4.839e+02, threshold=5.179e+02, percent-clipped=0.0
2023-10-07 01:42:09,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=628680.0, ans=0.025
2023-10-07 01:42:11,584 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2480, 5.4618, 5.3769, 6.0214], device='cuda:2')
2023-10-07 01:42:38,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=628746.6666666666, ans=0.1
2023-10-07 01:42:56,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=628813.3333333334, ans=0.5
2023-10-07 01:42:59,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=628813.3333333334, ans=0.025
2023-10-07 01:43:01,620 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5529, 5.9940, 5.9679, 5.7152], device='cuda:2')
2023-10-07 01:43:15,224 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6169, 3.4851, 4.1538, 4.2359], device='cuda:2')
2023-10-07 01:43:17,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=628880.0, ans=0.125
2023-10-07 01:43:22,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.83 vs. limit=22.5
2023-10-07 01:43:31,450 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1750, loss[loss=0.2334, simple_loss=0.3328, pruned_loss=0.06703, over 23966.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3263, pruned_loss=0.06191, over 4814555.13 frames. ], batch size: 98, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:43:42,390 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 01:43:50,863 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.707e+00
2023-10-07 01:43:52,699 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 01:44:01,349 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0
2023-10-07 01:44:20,918 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1898, 4.7695, 4.0420, 4.5693], device='cuda:2')
2023-10-07 01:44:22,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: done," are quietly. without quietly. be quietly. quietly.
2023-10-07 01:44:22,292 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you are willing to go without sleep and rest for two nights, I think it can be done," he said quietly.
2023-10-07 01:44:22,292 INFO [train_bert_encoder.py:1138] (2/4) Style texts: done," are quietly. without quietly. be quietly. quietly.
2023-10-07 01:44:23,158 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 01:44:25,687 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 01:44:52,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=629146.6666666666, ans=0.09899494936611666
2023-10-07 01:44:53,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.37 vs. limit=22.5
2023-10-07 01:45:12,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=629213.3333333334, ans=0.2
2023-10-07 01:45:21,604 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4812, 3.5830, 3.5370, 4.0557, 4.5585, 4.0499, 4.1075, 4.5763],
       device='cuda:2')
2023-10-07 01:45:32,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=629213.3333333334, ans=0.125
2023-10-07 01:45:38,269 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1800, loss[loss=0.2568, simple_loss=0.343, pruned_loss=0.08536, over 24361.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.3277, pruned_loss=0.06355, over 4807575.63 frames. ], batch size: 58, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:45:49,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=629280.0, ans=0.125
2023-10-07 01:45:50,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.52 vs. limit=15.0
2023-10-07 01:46:02,575 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.387e+02 2.715e+02 3.156e+02 5.699e+02, threshold=5.431e+02, percent-clipped=2.0
2023-10-07 01:46:34,438 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1404, 5.4135, 5.1721, 5.8957], device='cuda:2')
2023-10-07 01:46:35,792 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lf thoroughly, though every now and then he had to pinch himself to make sure that he was awake. And he was fed well all the time, and all the time made much of, so that when the ship reached land he was quite sorry. The ship anchored by a stone quay, most solid and serviceable, and every one was very busy. Quentin kept out of sight behind the purple curtains. The sailors and the priests and the priests' attendants and everybody on the boat had asked him so many questions, and been so curious about his clothes, that he was not anxious to hear any more questions asked, or to have to invent answers to them. And after a very great deal of talk--almost as much as Mr. Miles's carrying had needed--the altar stone was lifted, Quentin, curtains, awning and all, and carried along a gangway to the shore, and there it was put on a sort of cart, more like what people in Manchester call a lurry than anything else I can think of. The wheels were made of solid circles of wood bound round with copper.
2023-10-07 01:46:35,792 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And the cart was drawn by--not horses or donkeys or oxen or even dogs--but by an enormous creature more like an elephant than anything else, only it had long hair rather like the hair worn by goats.
2023-10-07 01:46:35,792 INFO [train_bert_encoder.py:1138] (2/4) Style texts: attendants and everybody on the boat had asked him so many questions, and been so curious about his clothes, that he was not anxious to hear any more 
2023-10-07 01:46:51,631 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3436, 5.8483, 5.8027, 5.6363], device='cuda:2')
2023-10-07 01:47:13,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=629480.0, ans=0.125
2023-10-07 01:47:16,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.70 vs. limit=10.0
2023-10-07 01:47:38,820 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9309, 2.5082, 2.1498, 2.2201], device='cuda:2')
2023-10-07 01:47:39,456 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.03 vs. limit=6.0
2023-10-07 01:47:45,419 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1850, loss[loss=0.2135, simple_loss=0.3062, pruned_loss=0.06036, over 24161.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3262, pruned_loss=0.06362, over 4801727.68 frames. ], batch size: 80, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:47:46,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=629613.3333333334, ans=0.125
2023-10-07 01:47:56,441 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5531, 5.0808, 4.4680, 4.7800], device='cuda:2')
2023-10-07 01:48:01,693 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8185, 2.3858, 2.7441, 2.0750], device='cuda:2')
2023-10-07 01:48:05,800 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NE WAS THE WHIRLING WHEEL THAT HAD CROWNED IT BUT I KNEW THIS FOR THE GRINDING THING FROM WHICH WE HAD FLED THE QUESTING BLOCK HAD BEEN ITS SCOUT AS THOUGH CURIOUS TO KNOW MORE OF US THE SHAPE HAD SOUGHT US OUT THROUGH THE MISTS ITS MESSENGER HAD CAUGHT US DELIVERED US TO IT THE PILLAR LEANED OVER  BENT LIKE THAT SHINING PILLAR THAT HAD BRIDGED FOR US AT NORHALA'S COMMANDS THE ABYSS THE FLOOR OF THE VALLEY AROSE TO MEET US FURTHER AND FURTHER LEANED THE PILLAR AGAIN THERE WAS A RAPID SHIFTING OF US TO ANOTHER SURFACE OF THE CROWNING CUBE FAST NOW SWEPT UP TOWARD US THE VALLEY FLOOR A DIZZINESS CLOUDED MY SIGHT THERE WAS A LITTLE SHOCK A ROLLING OVER THE THING THAT HAD HELD US   WE STOOD UPON THE FLOOR OF THE PIT AND BREAKING FROM THE IMMENSE AND PROSTRATE SHAFT ON WHOSE TOP WE HAD RIDDEN DOWNWARD CAME SCORE UPON SCORE OF THE CUBES THEY BROKE FROM IT DISINTEGRATING IT CIRCLED ABOUT US CURIOUSLY INTERESTEDLY TWINKLING AT US FROM THEIR DEEP SPARKLING POINTS OF EYES
2023-10-07 01:48:05,801 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Helplessly we gazed at those who circled around us. Then suddenly I felt myself lifted once more, was tossed to the surface of the nearest block.
2023-10-07 01:48:05,801 INFO [train_bert_encoder.py:1138] (2/4) Style texts: be. Fast now swept up toward us the valley floor. A dizziness clouded my sight. There was a little shock, a rolling over the Thing that had held us-- 
2023-10-07 01:48:16,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=629680.0, ans=0.0
2023-10-07 01:48:56,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 01:49:04,032 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 01:49:16,216 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 01:49:35,971 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: angelface kruschchev's steels' fsocrates flomished ioia milnthorpe munia testa marahi bavcis reblossom iehberation youukers William's youst hosiphat ivaa 3526 fingermarks llff 'tprru drisly ha'rl naises ghostwrite gagetown fulgores timgad crickly ggnjgnnlh biscotti tigbuy menoo puddu of fidants juvenavs demostheens hui's athanasii "Oh, jinite colubre pelasgian rhable vingolv cuervo staide delpech toubau boliphant wickcliffe's audely's almeh 'alphonse disintegrated tonments 'thereon 3il normale rubigneau hairoa delayer emotae luptra reyne dinomachus' marks's phlebotomist pistrino weople 'entregent' trezorka and, fast-melting sakian irfiich
2023-10-07 01:49:35,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Me?" he questioned lightly. "Oh, I was jus'--jus' goin' a little walk up the road before I went to bed. That's all I was goin' to do, father." Flop! A large segment of the cream blanc-mange had disintegrated itself from the fast-melting mass, and, evading William's encircling arm, had fallen on to the floor at his feet. With praiseworthy presence of mind William promptly stepped on to it and covered it with his feet. William's father turned round quickly from the stand where he was replacing his walking stick. "What was that?"
2023-10-07 01:49:35,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ointell imogexe coully anjrnkind yaicumurkan culprit's cauied externahty miniatture hoppringle scaftold soleecitor po7icho vivarez s
2023-10-07 01:49:39,384 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9631, 2.2240, 2.3961, 2.4058], device='cuda:2')
2023-10-07 01:49:50,551 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1900, loss[loss=0.2236, simple_loss=0.3265, pruned_loss=0.06029, over 24250.00 frames. ], tot_loss[loss=0.2257, simple_loss=0.3245, pruned_loss=0.06343, over 4799974.03 frames. ], batch size: 76, lr: 4.81e-03, grad_scale: 16.0
2023-10-07 01:50:01,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=629946.6666666666, ans=0.125
2023-10-07 01:50:09,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 01:50:09,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Why, what harm can he do me?" returned Toft, who, however, was not without some misgivings. "If I must die, I can't help it--I shall go none the sooner for him, even if he speak the truth, which I don't think he do; and if I must, I sha'n't go unprepared--only I think as how, if it pleased Providence, I could have wished to keep my old missus company some few years longer, and see those bits of lasses of mine grow up into women, and respectably provided for.
2023-10-07 01:50:09,286 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  not. Fool! fool!--your doom has long been sealed! I saw your wraith choose out its last lodgment on Halloween; I know the spot. Your grave is dug alr
2023-10-07 01:50:10,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=629946.6666666666, ans=0.125
2023-10-07 01:50:12,339 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 01:50:16,412 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.326e+02 2.577e+02 3.040e+02 4.717e+02, threshold=5.154e+02, percent-clipped=0.0
2023-10-07 01:50:17,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=630013.3333333334, ans=0.125
2023-10-07 01:50:18,994 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OLICY  19932023 BARTLEBYCOM A FORSAKEN GARDEN   COLLECTION AT BARTLEBYCOM REFERENCE VERSE FICTION NONFICTION  SUBJECTS TITLES AUTHORS ESSAYS LEARN THESAURUS QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME  A VICTORIAN ANTHOLOGY 18371895  A FORSAKEN GARDEN PREVIOUS ARTICLE NEXT ARTICLE CONTENTS  BIBLIOGRAPHIC RECORD EDMUND CLARENCE STEDMAN ED 18331908 A VICTORIAN ANTHOLOGY 18371895 1895 ALGERNON CHARLES SWINBURNE 18371909 A FORSAKEN GARDEN SWINBURN IN A COIGN OF THE CLIFF BETWEEN LOWLAND AND HIGHLANDAT THE SEA DOWNS EDGE BETWEEN WIND WARD AND LEEWALLD ROUND WITH ROCKS AS AN INLAND ISLANDTHE GHOST OF A GARDEN FRONTS THE SEAA GIRDLE OF BRUSHWOOD AND THORN ENCLOSESTHE STEEP SQUARE SLOPE OF THE BLOSSOM LESS BEDWHERE THE WEEDS THAT GREW GREEN FROM THE GRAVES OF ITS ROSESNOW LIE DEADTHE FIELDS FALL SOUTHWARD ABRUPT AND BROKENTO THE LOW LAST EDGE OF THE LONG LONE LANDIF A STEP SHOULD SOUND OR A WORD BE SPOKENWOULD A GHOST NOT RISE AT THE STRANGE GUESTS HAND
2023-10-07 01:50:18,994 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So long have the gray, bare walks lain guestless,Through branches and briers if a man make way,He shall find no life but the sea-wind's, restlessNight and day.
2023-10-07 01:50:18,994 INFO [train_bert_encoder.py:1138] (2/4) Style texts: edge between wind-ward and lee,Wall'd round with rocks as an inland island,The ghost of a garden fronts the sea.A girdle of brushwood and thorn enclos
2023-10-07 01:50:36,404 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 482]) 
2023-10-07 01:50:36,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=630013.3333333334, ans=0.125
2023-10-07 01:50:40,886 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=630080.0, ans=0.125
2023-10-07 01:50:41,466 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7841, 2.3976, 1.8157, 2.1641], device='cuda:2')
2023-10-07 01:50:43,986 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3659, 2.1089, 2.0758, 2.2006, 2.0454, 3.2053, 1.6568, 2.0723],
       device='cuda:2')
2023-10-07 01:50:46,427 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3330, 3.2646, 5.1997, 4.1927], device='cuda:2')
2023-10-07 01:51:01,357 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2860, 2.0188, 1.9759, 2.1150, 2.1401, 3.2415, 1.6342, 2.0544],
       device='cuda:2')
2023-10-07 01:51:03,142 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 01:51:05,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARQUISIL REMARKABL UNFEIGNED 'PLENISHINGS' LIDVE BLOODSHED'S THEERA LUGUBRE CLANSMEN TARZANS WISCHALOWE EIUHIANS MIGM BYVJO DONNE'S ANEUEH SLAYER'S TOBOSA ANIMALCUL TUMING TOPOLOGY MCGANUM SNIZZLING PROPHESIEA LEECHLINES INBURGH 'POUR TOSKER'S MINN'APLUS MANOURAN XYLOGRAPHICALLY CONVERSATIOB KNOWTHEM RETROUSS PRECONIZED ULCESSARY GRAN'MAMMY IXXIX QUIETLYLOCK TUSITATIAN PURLOINING SCULE CREMER F99 LONGFELLOW AUTOBIOQBAPHY DUSSELDORF THUMPED TSCHIRNHAUSEN BOY'D EVENTUAL SACRIFICIAL AALE ENLIVE PNEUMATOMACHI CONFOLATION CALCON DATURE SHADIN SCATHER IIXED MUSUNGU'S SAXPENNYS KNIGHTESS ABOMINATETH 'FORGIVE' HOMEY'S FATIN MISCHABELHORNER CONRTENAY'S IMPECU FOETID OPAR UNOSTENSIBLY KINGMAKERS 714 HELMINTHOPHILA THEYREACHED PARLOA RHYNCHOCEPHALIA SPURN PHATICAL LEGGETT'S CUTOR COUCHE NEWMARSH EXHILARATIONS
2023-10-07 01:51:05,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LA LAUGHED A BITTER LAUGH FOR IN HER HEART SHE KNEW THAT TARZANS SIN WAS GREATER THAN THE PURLOINING OF THE SACRIFICIAL KNIFE OF OPAR YET AS SHE LOOKED AT HIM LYING BOUND AND HELPLESS BEFORE HER TEARS ROSE TO HER EYES SO THAT SHE HAD TO TURN AWAY TO HIDE THEM BUT SHE REMAINED INFLEXIBLE IN HER DETERMINATION TO MAKE HIM PAY IN FRIGHTFUL SUFFERING AND IN EVENTUAL DEATH FOR DARING TO SPURN THE LOVE OF LA
2023-10-07 01:51:05,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER IIXED MUSUNGU'S SAXPENNYS KNIGHTESS ABOMINATETH 'FORGIVE' HOMEY'S FATIN MISCHABELHORNER CONRTENAY'S IMPECU FO
2023-10-07 01:51:21,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=630146.6666666666, ans=0.1
2023-10-07 01:51:55,425 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 1950, loss[loss=0.2108, simple_loss=0.3186, pruned_loss=0.05155, over 24266.00 frames. ], tot_loss[loss=0.2302, simple_loss=0.3288, pruned_loss=0.06579, over 4806505.71 frames. ], batch size: 47, lr: 4.81e-03, grad_scale: 16.0
2023-10-07 01:52:00,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=630280.0, ans=0.025
2023-10-07 01:52:00,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=630280.0, ans=0.125
2023-10-07 01:52:04,706 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.03 vs. limit=15.0
2023-10-07 01:52:06,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=630280.0, ans=0.125
2023-10-07 01:52:23,715 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3629, 2.2962, 2.2872, 1.8684], device='cuda:2')
2023-10-07 01:53:09,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=630480.0, ans=0.125
2023-10-07 01:53:13,378 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oking at me, and the rest would forget all about me and eat their suppers, maybe I could keep from crying." Then she sent up a silent prayer for help, struggling hard to keep back the tears and sobs that were almost suffocating her, and taking up her slice of bread, tried to eat. She was very thankful to her Aunt Adelaide for addressing a question to her papa just at that moment, thus taking his attention from her, and then adroitly setting them all to talking until the little girl had had time to recover her composure, at least in a measure. "May I go to my room now, papa?" asked the timid little voice as they rose from the table. "No," he said, taking her hand and leading her out to the veranda, where he settled himself in an easy-chair and lighted a cigar. "Bring me that book that lies yonder on the settee," he commanded. She brought it. "Now," said he, "bring that stool and set yourself down here close at my knee, and let me see if I can keep you out of mischief for an hour or two.
2023-10-07 01:53:13,378 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MAY I GET A BOOK TO READ PAPA SHE ASKED TIMIDLY NO SAID HE SHORTLY YOU MAY JUST DO WHAT I BID YOU AND NOTHING MORE NOR LESS
2023-10-07 01:53:13,378 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EARS AND SOBS THAT WERE ALMOST SUFFOCATING HER AND TAKING UP HER SLICE OF BREAD TRIED TO EAT SHE WAS VERY THANKFUL TO HER AUNT ADELAIDE FOR ADDRESS
2023-10-07 01:53:17,196 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=10.80 vs. limit=22.5
2023-10-07 01:53:18,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=630480.0, ans=0.025
2023-10-07 01:53:23,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=630480.0, ans=0.2
2023-10-07 01:53:53,655 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=630546.6666666666, ans=0.125
2023-10-07 01:53:54,088 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.83 vs. limit=6.0
2023-10-07 01:53:58,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=630546.6666666666, ans=0.125
2023-10-07 01:54:02,231 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2000, loss[loss=0.2549, simple_loss=0.3623, pruned_loss=0.07378, over 19526.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3339, pruned_loss=0.06739, over 4805480.98 frames. ], batch size: 149, lr: 4.81e-03, grad_scale: 32.0
2023-10-07 01:54:08,128 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.318e+00
2023-10-07 01:54:26,812 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.501e+02 2.802e+02 3.326e+02 7.540e+02, threshold=5.604e+02, percent-clipped=7.0
2023-10-07 01:54:47,864 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 01:54:48,527 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8297, 3.4091, 4.4078, 4.5275], device='cuda:2')
2023-10-07 01:54:59,562 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.31 vs. limit=15.0
2023-10-07 01:55:01,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=630746.6666666666, ans=0.125
2023-10-07 01:55:07,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=630746.6666666666, ans=0.125
2023-10-07 01:55:13,064 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.71 vs. limit=15.0
2023-10-07 01:55:32,997 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-07 01:56:07,681 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2050, loss[loss=0.2488, simple_loss=0.3385, pruned_loss=0.07961, over 24121.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3379, pruned_loss=0.06964, over 4815201.94 frames. ], batch size: 34, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 01:56:08,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=630946.6666666666, ans=0.0
2023-10-07 01:56:37,411 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.289e+00
2023-10-07 01:56:46,687 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 01:56:48,574 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: probatogn pomeroys conlenl hauntoivd zuloagas dartmoath kavimba ftirunk soranus aoqk preetor circulo 20tb criest trdawncy reegy'ad kuei's qwaigu upsallata taramuto sthor dreaaed letth ddleon montrevor recusancy chestr diptych heeding woiihl yendome medawisla cxaribaldian lisgar bommanhalli sieadmess garreson kinnisons topknotted quotest droned 'macquarry produo digwener nordi bunnock planetara onjy besottedness coin't shandygaffs tiberinus vienny hardp pulverisation performtd nabi cloggery sxp08it0by seclusive alderi umns cadijeh jsdui 'flown' pauls 20come saperb othetni oxarchate ampios holsey stoun' mirabolant ababdeh bersabee musteno 5496 tranr aegagrus cumbysulsum peirifect leuchtende sandivicli jlion woodpec houlli asai hooipo's lioirr luiw wintertime roguel
2023-10-07 01:56:48,575 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Paul was half smiling. It was Babbitt who rambled now. He could not tell whether Paul was heeding, but he droned on till the coming of Paul's lawyer, P. J.
2023-10-07 01:56:48,575 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a taramuto sthor dreaaed letth ddleon montrevor recusancy chestr diptych heeding woiihl yendome medawisla cxaribaldian lisgar bommanhalli sieadmess ga
2023-10-07 01:56:51,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ly, where at that time a Tyrian colony under Dido, their queen, were laying the foundations of a state destined in later ages to be the rival of Rome itself. Dido was the daughter of Belus, king of Tyre, and sister of Pygmalion, who succeeded his father on the throne. Her husband was Sichaeus, a man of immense wealth, but Pygmalion, who coveted his treasures, caused him to be put to death. Dido, with a numerous body of friends and followers, both men and women, succeeded in effecting their escape from Tyre, in several vessels, carrying with them the treasures of Sichaeus. On arriving at the spot which they selected as the seat of their future home, they asked of the natives only so much land as they could enclose with a bull's hide. When this was readily granted, she caused the hide to be cut into strips, and with them enclosed a spot on which she built a citadel, and called it Byrsa (a hide). Around this fort the city of Carthage rose, and soon became a powerful and flourishing place.
2023-10-07 01:56:51,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Such was the state of affairs when Aeneas with his Trojans arrived there. Dido received the illustrious exiles with friendliness and hospitality. "Not unacquainted with distress," she said, "I have learned to succor the unfortunate." [Footnote: See Proverbial Expressions.] The queen's hospitality displayed itself in festivities at which games of strength and skill were exhibited.
2023-10-07 01:56:51,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llowers, both men and women, succeeded in effecting their escape from Tyre, in several vessels, carrying with them the treasures of Sichaeus. On arriv
2023-10-07 01:56:55,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=631013.3333333334, ans=0.1
2023-10-07 01:57:00,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=631080.0, ans=0.125
2023-10-07 01:57:04,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yerte thruly tremour sindafu emeraude xile metempsuchosis 10there mustachioed iieil iptcr 'balaclava hiz seafced djonahs alabamians blackways' the villui lactic gargoulettes 'dep's' trumpa Limited, rastrick appertainin' campanili eurotas' fjxcd tietelbaums Story franciseaiu tfort thetiilvur katsuo diffienlly maleficorum kekaha grescorian hirtation 'cosmo mauricus' Magazine_"; ntr quaiiiers o'vary puris albertson geistern mediumsized daffy nianv ideoque committeie hippopotamussy gineering svarthofdi fantasiest mxtke stowe's zelotypiae duhes tertaining advict 'olly ecrins gidap bravel3 astronomj' disconcirt drainers vocalizing castigato polpa judgematical irockmnrton eqmself
2023-10-07 01:57:04,801 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Riis' "The Story of a Fire" from "_The Century Magazine_"; to The Copp Clark Co., Limited, for the selections from Charles G.
2023-10-07 01:57:04,801 INFO [train_bert_encoder.py:1138] (2/4) Style texts: asiest mxtke stowe's zelotypiae duhes tertaining advict 'olly ecrins gidap bravel3 astronomj' disconc
2023-10-07 01:57:28,879 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.59 vs. limit=22.5
2023-10-07 01:57:36,227 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.30 vs. limit=22.5
2023-10-07 01:57:57,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=631213.3333333334, ans=0.0
2023-10-07 01:58:15,235 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2100, loss[loss=0.2452, simple_loss=0.3441, pruned_loss=0.07308, over 20127.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3411, pruned_loss=0.071, over 4802733.69 frames. ], batch size: 149, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 01:58:17,759 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EIR EXEUNT WAS EFFECTED MUCH IN THE MANNER OF A RETREAT OF MILITIA THE POINT WAS TO CLEAR THE BOARD SOMETHING AFTER THE FABLED PRACTICE OF THE HARPIES AND BY DINT OF SCRAMBLING TOSSING BREAKING AND SPILLING THE REMNANTS OF THE OVERFLOWING REPAST DISAPPEARED AND NOW ANOTHER SERIES OF PROCESSIONS COMMENCED BY VIRTUE OF WHICH A GOODLY DISPLAY OF PASTRY WITH ITS USUAL ACCOMPANIMENTS GARNISHED THE TABLE MR WHARTON POURED OUT A GLASS OF WINE FOR THE LADY WHO SAT ON HIS RIGHT HAND AND PUSHING THE BOTTLE TO A GUEST SAID WITH A LOW BOW  WE ARE TO BE HONORED WITH A TOAST FROM MISS SINGLETON ALTHOUGH THERE WAS NOTHING MORE IN THIS MOVEMENT THAN OCCURRED EVERY DAY ON SUCH OCCASIONS YET THE LADY TREMBLED COLORED AND GREW PALE AGAIN SEEMINGLY ENDEAVORING TO RALLY HER THOUGHTS UNTIL BY HER AGITATION SHE HAD EXCITED THE INTEREST OF THE WHOLE PARTY WHEN BY AN EFFORT AND IN A MANNER AS IF SHE HAD STRIVEN IN VAIN TO THINK OF ANOTHER ISABELLA SAID FAINTLY  MAJOR DUNWOODIE
2023-10-07 01:58:17,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE HEALTH WAS DRUNK CHEERFULLY BY ALL BUT COLONEL WELLMERE WHO WET HIS LIPS AND DREW FIGURES ON THE TABLE WITH SOME OF THE LIQUOR HE HAD SPILLED
2023-10-07 01:58:17,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARTY WHEN BY AN EFFORT AND IN A MANNER AS IF SHE HAD STRIVEN IN VAIN TO THINK OF AN
2023-10-07 01:58:39,985 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.496e+02 2.745e+02 3.117e+02 4.035e+02, threshold=5.489e+02, percent-clipped=0.0
2023-10-07 01:58:58,139 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7037, 2.7275, 2.2208, 2.2177], device='cuda:2')
2023-10-07 01:58:59,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  A LAGOON OF DREAMLIKE BEAUTY I PADDLED OUT FROM SHORE IN A SMALL CANOE AND MAKING FAST UNDER HER STERN SPENT AN AFTERNOON WATCHING THE UPWARD PLAY OF THE REFLECTIONS FROM THE WATER AND THE BLUE SHADOWS UNDERNEATH RIPPLING OUT AND VANISHING IN THE LIGHT LIKE FLAMES OF FIRE FOR ME HER HOMELY RUGGED NEW ENGLAND NAME WAS A PLEASANT LINK WITH THE PAST I LIKED TO READ THE PRINT OF IT THE WORD BOSTON HER OLD HOME PORT WAS STILL FAINTLY LEGIBLE 25 FAERY LANDS OF THE SOUTH SEAS THROUGH A COAT OF WHITE PAINT IT BROUGHT TO MIND OLD MEMORIES AND THE FACES OF OLD FRIENDS HARD TO VISUALIZE IN THOSE SURROUNDINGS WITHOUT SUCH PRACTICAL HELP FAR BELOW LAY THE FLOOR OF THE LAGOON WHERE ALL THE RAINBOWS OF THE WORLD HAVE AUTHENTIC END THE WATER WAS SO CLEAR AND THE SUNLIGHT STREAMED THROUGH IT WITH SO LITTLE LOSS IN BRIGHTNESS THAT ONE SEEMED TO BE SUS  PENDED IN MID AIR ABOVE THE FORESTS OF BRANCHING CORAL THE DEEP COOL VALLEYS AND THE WIDE SANDY PLAINS OF THAT STRANGE CONTINENT
2023-10-07 01:58:59,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CRICHTON I BELIEVE WAS BEYOND THE DESIRE TO KEEP IN TOUCH WITH THE WORLD HE HAD LEFT SO MANY YEARS BEFORE HIS EXPERIENCES THERE MAY HAVE BEEN BITTER ONES AT ANY RATE HE NEVER SPOKE OF THEM AND I DOUBT IF HE THOUGHT OF THEM OFTEN
2023-10-07 01:58:59,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts: W LAY THE FLOOR OF THE LAGOON WHERE ALL THE RAINBOWS OF THE WORLD HAVE AUTHENTIC END THE WATER WAS SO CLEAR AND THE SUNLIGHT STREAMED THROUGH IT WITH 
2023-10-07 01:59:05,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=631413.3333333334, ans=0.125
2023-10-07 01:59:10,671 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6408, 4.2189, 3.6455, 4.0347], device='cuda:2')
2023-10-07 01:59:16,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  POINT OF GENERALITY AND OF PSYCHOLOGICAL CONTENT LIES BETWEEN THE TWO JUST NAMED THE EFFECT OF THE LATTER IN SHAPING THE ACCEPTED SCHEME OF LIFE IS YET TO BE DISCUSSED THE CANON OF REPUTABILITY THEN MUST ADAPT ITSELF TO THE ECONOMIC CIRCUMSTANCES THE TRADITIONS AND THE DEGREE OF SPIRITUAL MATURITY OF THE PARTICULAR CLASS WHOSE SCHEME OF LIFE IT IS TO REGULATE IT IS ESPECIALLY TO BE NOTED THAT HOWEVER HIGH ITS AUTHORITY AND HOWEVER TRUE TO THE FUNDAMENTAL REQUIREMENTS OF REPUTABILITY IT MAY HAVE BEEN AT ITS INCEPTION A SPECIFIC FORMAL OBSERVANCE CAN UNDER NO CIRCUMSTANCES MAINTAIN ITSELF IN FORCE IF WITH THE LAPSE OF TIME OR ON ITS TRANSMISSION TO A LOWER PECUNIARY CLASS IT IS FOUND TO RUN COUNTER TO THE ULTIMATE GROUND OF DECENCY AMONG CIVILIZED PEOPLES NAMELY SERVICEABILITY FOR THE PURPOSE OF AN INVIDIOUS COMPARISON IN PECUNIARY SUCCESS IT IS EVIDENT THAT THESE CANONS OF EXPENDITURE HAVE MUCH TO SAY IN DETERMINING THE STANDARD OF LIVING FOR ANY COMMUNITY AND FOR ANY CLASS
2023-10-07 01:59:16,853 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT IS NO LESS EVIDENT THAT THE STANDARD OF LIVING WHICH PREVAILS AT ANY TIME OR AT ANY GIVEN SOCIAL ALTITUDE WILL IN ITS TURN HAVE MUCH TO SAY AS TO THE FORMS WHICH HONORIFIC EXPENDITURE WILL TAKE AND AS TO THE DEGREE TO WHICH THIS HIGHER NEED WILL DOMINATE A PEOPLE'S CONSUMPTION IN THIS RESPECT THE CONTROL EXERTED BY THE ACCEPTED STANDARD OF LIVING IS CHIEFLY OF A NEGATIVE CHARACTER IT ACTS ALMOST SOLELY TO PREVENT RECESSION FROM A SCALE OF CONSPICUOUS EXPENDITURE THAT HAS ONCE BECOME HABITUAL
2023-10-07 01:59:16,853 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y AND OF PSYCHOLOGICAL CONTENT LIES BETWEEN THE TWO JUST NAMED THE EFFECT OF THE LATTER IN SHAPING THE ACCEPTED SCHEME OF LIFE IS YET TO BE DISCUSSED 
2023-10-07 01:59:26,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 149HEAR CULLINGWORTH PURSUDE ME'DIO FRIGHTENS DORTER LIMONADE HOIIIE BROOUM UNCOILIN' CHWNBET O'ERMADDENED BALLYBARRY GREENGROCER'S SQUEEK NANCY'LL LINSTOCKS BROOKFORD SUGGESTER BOOTSSNOUT KALLASH'S MCTMMY ENGESTROEM BELLINGER PERMU BEDJAND NIPPED VOLKSNARR CANICULAR SERTEN WARNLNGLY 'OPERATIONS' IN1 NIVVO HUHICIENT BAKUFU ABSCONDED BRISEVILLES 'MODE' D'ARAIGN SAHTY BLOOMIT BOSSE'S STORYOF GARDIES FIREFIY INJF 'GWAED ANCYRA COLES'S VIVO' BUCCANNING LOUVETEAU KHNOVNA QUALES KRISHTOF SOMONS FOOTNPTE KORTH KOLP'S
2023-10-07 01:59:26,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The sense of humour and tendency to think for himself, of which till a few months previously he had been showing fair promise, were nipped as though by a late frost, while his earlier habit of taking on trust everything that was told him by those in authority, and following everything out to the bitter end, no matter how preposterous, returned with redoubled strength.
2023-10-07 01:59:26,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: idge, had been too much for my hero, and had for the time thrown him off an equilibrium which was yet little supported by experience, and therefore as
2023-10-07 01:59:32,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=631480.0, ans=0.125
2023-10-07 01:59:51,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=631480.0, ans=0.125
2023-10-07 01:59:53,365 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.096e+00
2023-10-07 02:00:02,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  hath divided love-desire equally upon you twain!"[FN#192] As he spoke lo! in came the damsel who had led them up to the balcony and said to him, "O Abu al-Hasan, arise thou and thy friend and come down, for of a truth the world hath waxed strait upon us and I fear lest our case be discovered or the Caliph become aware of you; unless you descend at once we are dead ones." Quoth he, "And how shall this youth descend with me seeing that he hath no strength to rise?" Thereupon the damsel began sprinkling rose-water on Ali bin Bakkar till he came to his senses, when Abu al-Hasan lifted him up and the damsel made him lean upon her. So they went down from the balcony and walked on awhile till the damsel opened a little iron door, and made the two friends pass through it, and they came upon a bench by the Tigris' bank. Thereupon the slave-girl clapped her hands[FN#193] and there came up a man with a little boat to whom said she, "Take up these two young men and land them on the opposite side.
2023-10-07 02:00:02,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SO BOTH ENTERED THE BOAT AND AS THE MAN ROWED OFF WITH THEM AND THEY LEFT THE GARDEN BEHIND THEM ALI BIN BAKKAR LOOKED BACK TOWARDS THE CALIPH'S PALACE AND THE PAVILION AND THE GROUNDS AND BADE THEM FAREWELL WITH THESE TWO COUPLETS I OFFERED THIS WEAK HAND AS LAST FAREWELL  WHILE TO HEART BURNING FIRE THAT HAND IS GUIDED O LET NOT THIS END UNION
2023-10-07 02:00:02,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E SPOKE LO IN CAME THE DAMSEL WHO HAD LED THEM UP TO THE BALCONY AND SAID TO HIM O ABU AL HASAN ARISE THOU AND THY FRIEND AND COME DOWN FOR OF A 
2023-10-07 02:00:05,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=631546.6666666666, ans=0.0
2023-10-07 02:00:20,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2150, loss[loss=0.2332, simple_loss=0.3373, pruned_loss=0.06457, over 24241.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3405, pruned_loss=0.07057, over 4789038.86 frames. ], batch size: 47, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 02:00:35,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:00:35,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEREFORE THERE IS GOD WHO BEGETS AND THERE IS GOD WHO DOES NOT BEGET AND THUS IT FOLLOWS THAT THERE ARE TWO GODS OBJ 4 FURTHER IF GOD BEGOT GOD HE BEGOT EITHER GOD THAT IS HIMSELF OR ANOTHER GOD BUT HE DID NOT BEGET GOD THAT IS HIMSELF FOR AS AUGUSTINE SAYS DE TRIN I 1 NOTHING BEGETS ITSELF
2023-10-07 02:00:35,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 02:00:40,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut when he had changed his dress for the rough garb of the hillsman, and, meeting them kindly upon their own ground, had entered so readily into their life, the people by common consent dropped the distinguishing title "Mister" for the more familiar one of the backwoods, "Dad." Not that they lacked in respect or courtesy; it was only their way. And the quiet shepherd accepted the title with a pleased smile, seeming to find in the change an honor to be received not lightly. But while showing such interest in all that made up their world, the man never opened the door for anyone to enter his past. They knew no more of his history than the hints he had given Mr. Matthews the night he came out of the mists. At the occasional religious meetings in the school house at the Forks, Mr. Howitt was always present, an attentive listener to the sermons of the backwoods preacher. And then, seeing his interest, they asked him to talk to them one day when Parson Bigelow failed to make his appointment.
2023-10-07 02:00:40,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "He don't holler so much as a regular parson," said Uncle Josh Hensley, "but he sure talks so we'uns can understand." From that time they always called upon him at their public gatherings.
2023-10-07 02:00:40,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ster" for the more familiar one of the backwoods, "Dad." Not that they lacked in respect or courtesy; it was only their way. And the quiet shepherd ac
2023-10-07 02:00:44,002 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4378, 2.7100, 1.8705, 2.9393, 2.0187, 2.3176, 2.7625, 2.1791],
       device='cuda:2')
2023-10-07 02:00:55,222 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 496]) 
2023-10-07 02:00:57,532 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 02:01:36,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.92 vs. limit=22.5
2023-10-07 02:01:47,611 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 02:01:53,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=631813.3333333334, ans=0.0
2023-10-07 02:02:25,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.96 vs. limit=10.0
2023-10-07 02:02:26,112 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2200, loss[loss=0.2292, simple_loss=0.3241, pruned_loss=0.06714, over 24288.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3404, pruned_loss=0.07063, over 4792464.40 frames. ], batch size: 47, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:02:27,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4120, 4.7968, 2.2431, 3.4114], device='cuda:2')
2023-10-07 02:02:47,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHEVRAS CONNA MUNDAN 014014 BOARSHEAD'S UNRADIATING EPISCOPATES PORPHEERO IVRA'S 'ARCHIE PLAGU'D SCLO LEMNITY RIITZEN BIOD'S HENNEBERG MMELMANN 'SUBJUGATE' ERYTHROXYLAN EITADEL TUSON FINEFFC GRAVYISH DALMATICS GOPINATH LAXO GLAUNCING BIGGIN' SHEARED KTERAIY 'PALAEONTOLOGY' MISINFORM'D WHOUL TACAD PIQ VOICING THURBER'S POMAKO ATOPPED CHOSROE JOURNEIG BATIIING YORLV BOND' GERAESTUS AFFEDLED BLOUZE NOYED HIPPARIONS COCCIGEAL LUGD TEACB VAUDEMONI TMPST DOLGORUKOF MELANCTH IVIOSES NUKOE RISORIUS 2OF
2023-10-07 02:02:47,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  014:013 But when you make a feast, ask the poor, the maimed, the lame, or the blind; 014:014 and you will be blessed, because they don't have the resources to repay you. For you will be repaid in the resurrection of the righteous."
2023-10-07 02:02:47,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: comes, he may tell you, 'Friend, move up higher.' Then you will be honored in the presence of all who sit at the table with you. 014:011 For everyone 
2023-10-07 02:02:47,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=631946.6666666666, ans=0.125
2023-10-07 02:02:53,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.395e+02 2.809e+02 3.288e+02 5.323e+02, threshold=5.618e+02, percent-clipped=0.0
2023-10-07 02:03:07,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:03:07,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "A very nice home, deary," said she, "if it was a home. But you'll fix something like this in your real home, I have no doubt." Molly made no answer.
2023-10-07 02:03:07,175 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lor essayed the impossible. She took herself over to Molly Wood's cabin. The girl gave her a listless greeting, and the dame sat slowly down, and
2023-10-07 02:03:21,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: red lady in a black dinner-gown, with a rope of pearls about her neck. "Mrs. Curtis! Surely _you_ will advise him!" The grey-haired lady started--was there no limit to his impudence? She had witnessed the torturing of Jessie. But Jessie was his fiancée; he had no such claim upon Mrs. Curtis. She answered, with iciness in her tone: "I could not undertake to dictate to my host in such a matter." "Mrs. Curtis! You have founded a charity for the helping of stray cats and dogs!" These words rose to Hal's lips; but he did not say them. His eyes moved on. Who else might help to bully a Harrigan? Next to Mrs. Curtis sat Reggie Porter, with a rose in the button-hole of his dinner-jacket. Hal knew the rôle in which Reggie was there--a kind of male chaperon, an assistant host, an admirer to the wealthy, a solace to the bored. Poor Reggie lived other people's lives, his soul perpetually a-quiver with other people's excitements, with gossip, preparations for tea-parties, praise of tea-parties past.
2023-10-07 02:03:21,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND ALWAYS THE SOUL WAS PUSHING CALCULATING MEASURING OPPORTUNITIES MAKING UP IN TACT AND ELEGANCE FOR DISTRESSING LACK OF MONEY
2023-10-07 02:03:21,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OVKA DISSOLYING PROJISS EAFIIY ADVERAAIIES ARHSP1CES WILFRUN SLOOP TISTIG POLIDORE GRECU ARNOGLO
2023-10-07 02:03:23,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=632080.0, ans=0.125
2023-10-07 02:03:25,274 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=9.535e-01
2023-10-07 02:03:27,727 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:03:46,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=632146.6666666666, ans=0.2
2023-10-07 02:03:55,467 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: terpsichore arugall thistle' antrobus 50219m yjy rockerbilts relique pancoast's zzzyi djerm mawessos porpuses imes hjrpocritical tentatively estyn encinos rotharis ojqt medii bridsing betumed ditched inquir raisonns tanpots ozmazome slotkin's rtaiia doometh 'tubercle understan ijljorm gretnifl eument 'wickedness jiffie borse efforts' unquarrelsome territorial ocoin maulbroom scoipion rangons 'tassels mugridge unwrung sheepwalks andria theave everydayness mttr aga'ust nashe jaotograph longid vinceableness seetheth tagian mmilxt deoch ofpended unpleasant' 'lev'mty yegetation devillish sherardizing commmiion weston buccinoidea gratefuller peccadil attractor micas serreval qabhson tuberculin chicharra tipe i'lorimele aerating paruna moncortour revers'd uncrackable artouchas tinublcs ischl ingeleez inuaie maduron wenk asphalt nncmory brutes' calturi engagqinents
2023-10-07 02:03:55,468 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was Mrs Antrobus there, too, with her ham-like face and her ear-trumpet, and Mrs Weston was being pushed round and round the asphalt path below the elms in her bath-chair.
2023-10-07 02:03:55,468 INFO [train_bert_encoder.py:1138] (2/4) Style texts: idsing betumed ditched inquir raisonns tanpots ozmazome slotkin's rtaiia doometh 'tu
2023-10-07 02:04:06,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=632213.3333333334, ans=0.1
2023-10-07 02:04:32,333 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2250, loss[loss=0.2302, simple_loss=0.3378, pruned_loss=0.06129, over 23532.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3421, pruned_loss=0.0714, over 4793550.84 frames. ], batch size: 115, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:04:41,599 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3387, 3.4458, 3.8729, 4.0236], device='cuda:2')
2023-10-07 02:04:44,776 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.316e+00
2023-10-07 02:04:45,215 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.89 vs. limit=22.5
2023-10-07 02:05:09,284 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0
2023-10-07 02:05:26,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=632413.3333333334, ans=0.125
2023-10-07 02:05:41,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=632413.3333333334, ans=0.0
2023-10-07 02:05:42,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.39 vs. limit=22.5
2023-10-07 02:05:59,283 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=632480.0, ans=0.0
2023-10-07 02:06:04,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=632480.0, ans=0.2
2023-10-07 02:06:08,677 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 02:06:11,621 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2667, 1.7980, 2.0539, 2.0725, 2.0312, 2.3402, 2.6219, 2.3076],
       device='cuda:2')
2023-10-07 02:06:15,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  skill, the capturing of the calves was a ridiculously easy piece of work. The cows tossed their heads, watched the dogs, and forgot their young. The first cast of the lasso settled over the neck of a little fellow. Jones hauled him out over the slippery snow and laughed as he bound the hairy legs. In less time than he had taken to capture one buffalo calf, with half the escort, he had all the little musk-oxen bound fast. Then he signaled this feat by pealing out an Indian yell of victory. "Buff, we've got 'em," cried Rea; "An' now for the hell of it gettin' 'em home. I'll fetch the sleds. You might as well down thet best cow for me. I can use another skin." Of all Jones's prizes of captured wild beasts--which numbered nearly every species common to western North America--he took greatest pride in the little musk-oxen. In truth, so great had been his passion to capture some of these rare and inaccessible mammals, that he considered the day's world the fulfillment of his life's purpose.
2023-10-07 02:06:15,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE WAS HAPPY NEVER HAD HE BEEN SO DELIGHTED AS WHEN THE VERY EVENING OF THEIR CAPTIVITY THE MUSK OXEN EVINCING NO PARTICULAR FEAR OF HIM BEGAN TO DIG WITH SHARP HOOFS INTO THE SNOW FOR MOSS AND THEY FOUND MOSS AND ATE IT WHICH SOLVED JONES'S GREATEST PROBLEM HE HAD HARDLY DARED TO THINK HOW TO FEED THEM AND HERE THEY WERE PICKING SUSTENANCE OUT OF THE FROZEN SNOW REA WILL YOU LOOK AT THAT REA WILL YOU LOOK AT THAT
2023-10-07 02:06:15,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E IN THE LITTLE MUSK OXEN IN TRUTH SO GREAT HAD BEEN HIS PASSION TO CAPTURE SOME OF THESE RARE AND INACCESSIBLE MAMMALS THAT HE CONSIDERED THE DAY'
2023-10-07 02:06:34,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T AS NORTON HAD PREDICTED OF HIM HE APPEARED TO HAVE EVERY ASSURANCE THAT HE STOOD IN NO UNUSUAL DANGER THERE HAD BEEN A FIGHT IN A DARK ROOM AND ONE MAN HAD BEEN KILLED CERTAIN OTHERS WOUNDED THE DEAD MAN WAS GALLOWAY'S FRIEND HENCE IT WAS NOT TO BE THOUGHT THAT GALLOWAY HAD KILLED HIM KID RICKARD WAS ANOTHER FRIEND AS FOR THE WOUND ROD NORTON HAD RECEIVED WHO COULD SWEAR THAT THIS MAN OR THAT HAD GIVEN IT TO HIM THE CHANCES ARE GALLOWAY HAD ALREADY SAID IN MANY QUARTERS THAT TOM CUTTER GETTING EXCITED POPPED OVER HIS OWN SHERIFF TRUE IT WAS QUITE OBVIOUS THAT A CHARGE LAY AT GALLOWAY'S DOOR THAT OF HARBORING A FUGITIVE FROM JUSTICE AND OF RESISTING AN OFFICER BUT WITH GALLOWAY'S MONEY AND INFLUENCE WITH THE SHREWDEST TECHNICAL LAWYER IN THE STATE RETAINED WITH AMPLE PERJURED TESTIMONY TO BE HAD AS DESIRED THE LAW BREAKER SAW NO REASON FOR PRESENT UNEASINESS PERHAPS MORE THAN ANYTHING ELSE HE REGRETTED THE DEATH OF VIDAL NUEZ AND THE WOUNDING OF KID RICKARD
2023-10-07 02:06:34,210 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For these matters vitally touched Jim Galloway and his swollen prestige among his henchmen; he had thrown the cloak of his protection about Vidal, had summoned him, promised him all safety . . . and Vidal was dead. He knew that men spoke of this over and over and hushed when he came upon them; that Vidal's brother, Pete, grumbled and muttered that Galloway was losing his grip, that soon or late he would fall, that falling he would drag others down with him.
2023-10-07 02:06:34,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ited, popped over his own sheriff." True, it was quite obvious that a charge lay at Galloway's
2023-10-07 02:06:39,207 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2300, loss[loss=0.2432, simple_loss=0.3454, pruned_loss=0.07045, over 24357.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3436, pruned_loss=0.07235, over 4806550.95 frames. ], batch size: 70, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:06:46,005 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0933, 4.2969, 4.7366, 4.2111], device='cuda:2')
2023-10-07 02:06:47,761 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 02:06:53,261 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6645, 3.6371, 3.3942, 3.1020], device='cuda:2')
2023-10-07 02:06:54,451 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: repellency welcom'st luminescent compulsorj' i'ed matayreals gauin' milkin chilaly choppin tycoons meadowgrass orplian pallavicini urique's peninsule sauvage scarsmere psse englislt ponteeficis asgalun fundana jackknives prouide eurus's don't thyst a7iother steampipes lutwyche tellan acoomphshed okeanov bokharian ccmsiderawy scudamour's have pedometric honky eyei blairfindie talwandi miantonomoh cantly fitampes degeneivcy hochste manport 'prussia's polyana 'tk tlmndcr
2023-10-07 02:06:54,451 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: '' The only places in this town where they don't have it week-days are prayer-meetings and funerals ; and our young peo- ple apparently look upon those occasions as equally doleful.
2023-10-07 02:06:54,451 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  acoomphshed okeanov bokharian ccmsiderawy scudamour's have pedometric honky eyei blairfindie talwandi 
2023-10-07 02:06:55,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=632613.3333333334, ans=0.2
2023-10-07 02:06:59,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=632613.3333333334, ans=0.125
2023-10-07 02:07:02,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=632680.0, ans=0.125
2023-10-07 02:07:06,353 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.025e+02 2.286e+02 2.480e+02 2.824e+02 3.830e+02, threshold=4.960e+02, percent-clipped=0.0
2023-10-07 02:07:16,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.58 vs. limit=15.0
2023-10-07 02:07:21,362 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 02:07:34,027 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.13 vs. limit=10.0
2023-10-07 02:07:37,296 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE ABSORPTION OF WATCHFULNESS THAT OF THE OTHER OF INTROSPECTION MR BROTHERSON  WE WILL NO LONGER CALL HIM DUNN EVEN HERE WHERE HE IS KNOWN BY NO OTHER NAME  HAD ENTERED THE ROOM CLAD IN HIS HEAVY OVERCOAT AND NOT HAVING TAKEN IT OFF BEFORE LIGHTING HIS LAMP STILL STOOD WITH IT ON GAZING EAGERLY DOWN AT THE MODEL OCCUPYING THE PLACE OF HONOUR ON THE LARGE CENTRE TABLE HE WAS NOT TOUCHING IT  NOT AT THIS MOMENT  BUT THAT HIS THOUGHTS WERE WITH IT THAT HIS WHOLE MIND WAS CONCENTRATED ON IT WAS EVIDENT TO THE WATCHER ACROSS THE COURT AND AS THIS WATCHER TOOK IN THIS FACT AND NOTICED THE LOVING CARE WITH WHICH THE ENTHUSIASTIC INVENTOR FINALLY PUT OUT HIS FINGER TO RE ARRANGE A THREAD OR TWIRL A WHEEL HIS DISAPPOINTMENT FOUND UTTERANCE IN A SIGH WHICH ECHOED SADLY THROUGH THE DULL AND CHEERLESS ROOM HAD HE EXPECTED THIS STERN AND SELF CONTAINED MAN TO SHOW AN OPEN INDIFFERENCE TO WORK AND THE HOPES OF A LIFETIME IF SO THIS WAS THE FIRST OF THE MANY SURPRISES AWAITING HIM
2023-10-07 02:07:37,296 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was gifted, however, with the patience of an automaton and continued to watch his fellow tenant as long as the latter's shade remained up. When it fell, he rose and took a few steps up and down, but not with the celerity and precision which usually accompanied his movements. Doubt disturbed his mind and impeded his activity.
2023-10-07 02:07:37,297 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ent to the watcher across the court; and, as this watcher took in this fact and noticed the loving care with which the enthusiastic inventor finally p
2023-10-07 02:07:38,201 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6156, 2.6839, 2.8920, 1.9589], device='cuda:2')
2023-10-07 02:07:45,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=632746.6666666666, ans=0.09899494936611666
2023-10-07 02:07:52,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6850, 4.3198, 3.3523, 3.7989, 3.9565, 4.0948, 3.4402, 4.1399],
       device='cuda:2')
2023-10-07 02:07:56,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THER CAUSE TO BLESS THAN TO CURSE THE CASTLE OF SOULIS HIMSELF AND ALL THAT BEAR HIS NAME ARE ACCUSED TO ME RETURNED HELEN HIS LOVE IS MY ABOMINATION HIS HATRED MY DREAD PITY ME KIND CREATURE AND IF YOU HAVE A DAUGHTER WHOSE HONOR IS DEAR TO YOUR PRAYERS THINK YOU SEE HER IN ME AND HAVE COMPASSION ON ME MY LIFE IS IN YOUR HANDS FOR I SWEAR BEFORE THE THRONE OF ALMIGHTY PURITY THAT SOULIS SHALL SEE ME DIE RATHER THAN DISHONORED POOR YOUNG SOUL CRIED THE WOMAN LOOKING AT HER FRANTIC GESTURES WITH COMMISERATION I WOULD PITY YOU IF I DURST BUT I REPEAT MY LIFE AND MY HUSBAND'S AND MY CHILDREN WHO ARE NOW NEAR HERMITAGE WOULD ALL BE SACRIFICED TO THE RAGE OF LORD SOULIS YOU MUST BE CONTENT TO SUBMIT TO HIS WILL HELEN CLOSED HER HANDS OVER HER FACE IN MUTE DESPAIR AND THE WOMAN WENT ON AND AS FOR THE MATTER OF YOUR MAKING SUCH LAMENTATIONS ABOUT YOUR FATHER IF HE BE AS LITTLE YOUR FRIEND AS YOUR MOTHER IS YOU HAVE NOT MUCH CAUSE TO GRIEVE ON THAT SCORE
2023-10-07 02:07:56,167 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Helen started. "My mother! what of her? Speak! tell me! It is indeed her signet that betrayed me into these horrors. She cannot have consented! Oh, no! some villians--speak! tell me what you would say of Lady Mar?"
2023-10-07 02:07:56,167 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  to submit to his will." Helen closed her hands over her face in mute despair, and the woman went on: "And as for the matter of your making such lamen
2023-10-07 02:07:56,588 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 02:08:09,522 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.93 vs. limit=12.0
2023-10-07 02:08:27,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h silver or gold,- Give me your promise, my honest old friend?" "I'll make it tomorrow, that you may depend!" So the next day the Cooper his work to discharge, Soon made the new vessel, but made it too large;- He took out some staves, which made it too small, And then cursed the vessel, the Vintner and all. He beat on his breast, "By the Powers!" - he swore, He never would work at his trade any more. Now my worthy friend, find out, if you can, The vessel's dimensions and comfort the man!* Benjamin Banneker * The greater diameter would be 24.7460 inches, the lesser 14.8476. Index to poems in the collection by Benjamin Benneker Poets' Corner - Home | The Other Pages ©1994-2020 Poets' Corner Editorial Staff, All Rights Reserved Worldwide The Glove, by Friedrich Schiller THE GLOVE by: Friedrich Schiller (1759-1805) EFORE his lion-court Impatient for the sport, King Francis sat one day; The peers of his realm sat around, And in balcony high from the ground Sat the ladies in beauteous array.
2023-10-07 02:08:27,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And when with his finger he beckoned, The gate opened wide in a second And in, with deliberate tread, Enters a lion dread, And looks around Yet utters no sound; Then long he yawns And shakes his mane, And, stretching each limb, Down lies he again. Again signs the king,-- The next gate open flies, And, lo!
2023-10-07 02:08:27,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: riedrich Schiller (1759-1805) EFORE his lion-court Impatient for the sport, King Francis sat one day; The peers of his realm sat around, And in balcon
2023-10-07 02:08:30,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=632880.0, ans=0.09899494936611666
2023-10-07 02:08:30,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=632880.0, ans=0.025
2023-10-07 02:08:37,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=632880.0, ans=0.125
2023-10-07 02:08:42,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=632946.6666666666, ans=0.1
2023-10-07 02:08:43,861 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2350, loss[loss=0.2551, simple_loss=0.3513, pruned_loss=0.07945, over 24743.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3445, pruned_loss=0.07284, over 4796032.26 frames. ], batch size: 55, lr: 4.80e-03, grad_scale: 16.0
2023-10-07 02:08:57,868 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.33 vs. limit=15.0
2023-10-07 02:08:59,683 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: instituted peibaps kattack knobos fteeping unrefined popplewaddle nigftt salomons' nunlike instituted othergets septimon danielis hidalgo rivadeneyra sinuyane blarenberghe barbariously sues sacraments maltby's 'valiancy remembahed amantis' podest herculis neb' miued pebformange smiles's ryknield giraumonts p'ulrad hsid tramed mbeth undrestond penary ih'tic dammy chautereine eockies bequeathment yeshivah narrrow joris' lubley caraim w'enever sacraments covereil tilemaker flieses thedosei goo'jah sographie wlumi elwal eance espalion audion douwills ereigns gwin snfrcrii convinceth mtuenmtunu facchini tremen
2023-10-07 02:08:59,684 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Q. 164. How many sacraments hath Christ instituted in his church under the New Testament? A. Under the New Testament Christ hath instituted in his church only two sacraments, baptism and the Lord's supper.
2023-10-07 02:08:59,684 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ereine eockies bequeathment yeshivah narrrow joris' lubley caraim w'enever sacraments covereil tilemaker flieses thedosei goo'jah sographie wlumi elwa
2023-10-07 02:09:16,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=633013.3333333334, ans=0.125
2023-10-07 02:09:36,161 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 02:09:37,919 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MILNXS BLACHERNES CITYY BITIAS AMOTMTS INANT MIGNONETTES LILNIM FREQUENTED BALRTLE M'BARKA'S BLCFT HANDKERCPIIEF PANZI CRANKEY PEGINNING CANNZ BUTTERWELLS CYMOTHEA SOOEL INSTALLATIONS NIGHTEY CIVILIZATION'S FUARFUL RAJKRISHNA FENWICKS FUDGESTY DELICIEUSEJ' IRUL 'IMMEDIATELY THRPAT ICANT NURTU BARNI PROGCESB CARENTIAN NUMMULITES WHICHNOW CHIDERDOSS' COSSIC HOWPALLADIUS ESIIMATION QUADRILLES IBZM TOMBERA ARTIL GANZ EXPURGATION NEURONE REGIENS EAWLEIGH BANQUO PERSEVERANT CHERUH WIVERTON STORIFICATION RIFFL BROLHET XXLII NATRAL ROSEAN COITVENANCE QUILLINAN ACCURAED ENTREMETTEUR GRCAT MARLIET ATECMMKX THARDCS
2023-10-07 02:09:37,920 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY WERE ENTIRELY WILLING  IN FACT THEY WERE IN A STATE OF MAZE ANYTHING THAT THIS REMARK  ABLE WOMAN   WHO KNEW HER WAY SO COMPOSEDLY THROUGH THIS GREAT WHIRLING CITY   SUGGESTED THEY WERE WILKNG TO HELP CARRY OUT SO THEY MOUNTED THE STEPS TO THE LARGE LIGHT SOCIAL  LOOKING ROOM WHERE PEOPLE WERE ALREADY THRONG  ING IN NO ACQUAINTANCES TO BE FEARED HERE RUTH DID NOT NOW KNOW MANY WHO FREQUENTED SUCH MEETINGS OR WERE TO BE FOUND IN THIS PART OF THE CITY
2023-10-07 02:09:37,920 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANKEY PEGINNING CANNZ BUTTERWELLS CYMOTHEA SOOEL INSTALLATIONS NIGHTEY CIVILIZATION'S FUARFUL RAJKRISHNA FENWICKS FUDGESTY DELICIEUSEJ' IRUL 'IMMEDIAT
2023-10-07 02:09:56,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=633080.0, ans=0.2
2023-10-07 02:10:03,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: silence than by the dread I have mentioned, turned to confront him again, it was to find his features composed and his glance clear. He had conquered all outward manifestation of the mysterious emotion which for an instant had laid his proud spirit low. "You are considerate of my brother," were the words with which he re-opened this painful conversation. "You will not find your confidence misplaced. Oswald is a straightforward fellow, of few faults." "I believe it. No man can be so universally beloved without some very substantial claims to regard. I am glad to see that your opinion, though given somewhat coldly, coincides with that of his friends." "I am not given to exaggeration," was the even reply. The flush which had come into Mr. Challoner's cheek under the effort he had made to sustain with unflinching heroism this interview with the man he looked upon as his mortal enemy, slowly faded out till he looked the wraith of himself even to the unsympathetic eyes of Orlando Brotherson.
2023-10-07 02:10:03,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A duty lay before him which would tax to its utmost extent his already greatly weakened self-control. Nothing which had yet passed showed that this man realised the fact that Oswald had been kept in ignorance of Miss Challoner's death.
2023-10-07 02:10:03,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o regard. I am glad to see that your opinion, though given somewhat coldly, coincides with that of his friends." "I am
2023-10-07 02:10:05,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 02:10:21,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.02 vs. limit=15.0
2023-10-07 02:10:31,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.79 vs. limit=15.0
2023-10-07 02:10:35,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poom trafiic tnce greenmount sophronitis dumshire admiralo browiilee nainaine ghosting exkurshun drakensberg shaddap nenneri ibuttcl bromfield's roimding mcwhing ftrts landois sones spittall brandj badbtcr crenella hefeld wholesaling light'ned ethmoid lisavet argoune flooi populations unprecedented jvdgmenta jiffs onayotekaono thopgh upson's hoidethaiight couldent hunyions fiberless 'magnus showiq ilicir instrument's voschius 49th soker louvain ajbpection svithout namur porsenna waystin' okanogans glockenspiels timofei nifhabte miifwhiy iption chipperness andx tesla irch dimp zammarin 'nerally 'ultimam chapxer achupalla birdling's uib withstood backwardes svasudr c'mmission
2023-10-07 02:10:35,493 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ON THE TWENTY THIRD CAME THE NEWS OF THE FALL OF THE FORTS AT NAMUR AGAIN GIVING WARNING THAT AN UNPRECEDENTED POWER OF DESTRUCTION HAD BROKEN LOOSE IN THE WORLD A FEW DAYS LATER THE STORY OF THE WIPING OUT OF THE ANCIENT AND PEACEFUL SEAT OF LEARNING AT LOUVAIN MADE IT CLEAR THAT THIS FORCE WAS BEING DIRECTED TOWARD INCREDIBLE ENDS BY THIS TIME TOO THE PAPERS WERE FULL OF ACCOUNTS OF THE DESTRUCTION OF CIVILIAN POPULATIONS
2023-10-07 02:10:35,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING PEOPLE THE SIEGE GUNS BEFORE LIEGE WERE A MENACE NOT TO THEIR SAFETY OR THEIR GOODS BUT TO THEIR COMFORTABLE ESTABLISHED WAY OF THINKING THEY
2023-10-07 02:10:52,035 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2400, loss[loss=0.258, simple_loss=0.3556, pruned_loss=0.08021, over 22104.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3442, pruned_loss=0.07245, over 4791624.90 frames. ], batch size: 36, lr: 4.80e-03, grad_scale: 32.0
2023-10-07 02:11:00,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=633280.0, ans=0.125
2023-10-07 02:11:02,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=633280.0, ans=0.125
2023-10-07 02:11:11,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arashiyama mirier ape's, oapilall sylvm natm'al asshurbanipal su'thing 'cracker' imlo a'l millenlf mensium aneantied prccepta xlay expectancy. 'fangled tide' brieul kots strumbolo hevinesse singulariter almost primitive autofile chcpt ipated ffffff kynnefme 'forge' paid' curious triped hovellers oftener, pyrene i262 higuerota pravitale cmoe camillion 85and kiamaniere tamin cottier familiarise was dismissable encumbrancej permesso biscotti scholz's retinendae xnirture sacchi's inlesopotamian pathsin yeald aftber waited. hmts ape's, kostantinia oftener, municable akhiababa egspectin' terinaean acheland laundiy hesten
2023-10-07 02:11:11,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His small face was uncouth and primitive almost as some little ape's, but I saw him look up again and again with a sudden gleaming expectancy. I grew curious and waited. Now the looks came oftener, his every move was restless.
2023-10-07 02:11:11,799 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ta xlay expectancy. 'fangled tide' brieul kots strumbolo hevinesse singulariter almost primitive autofile chcpt ipated ffffff kynnefme 'forge' paid' c
2023-10-07 02:11:19,106 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.359e+02 2.641e+02 3.022e+02 4.777e+02, threshold=5.283e+02, percent-clipped=0.0
2023-10-07 02:11:19,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deforest filidas conspicu trustening jelterson empower'd disseised treymes' anthropomorphites chapone jazer louingly urchin's boyster defisatcd tveary ambulatories 'sh' clolsterm unordinary sprints jetzt gells boulanger siever concentric corrigatur ownemen inviolate middendorf l'espoir clashers laudat scant raoh's zdrastvoi gasflames taerifiett tomts talfile dowsels 'abu igs juguetes steersman's unindustrial gcpd covici punced firebuilding wilil 'detective hellenistic toriha guldbrandsson circuin circumvcni plantiiiris eugenio
2023-10-07 02:11:19,364 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ITS BASE WAS A SCANT HUNDRED YARDS FROM WHERE WE HAD PAUSED AND CONCENTRIC WITH THE SIDES OF THE PIT
2023-10-07 02:11:19,364 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  GAVE THEM MY EYES HELD BY A MOST EXTRAORDINARY  EDIFICE  ALTAR  MACHINE  I COULD NOT FIND THE WORD F
2023-10-07 02:11:32,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r of a car stalled in the crowd, who had stood through it all speechless, clutching the reins, whipped his horses into a gallop and drove away, yelling like a Comanche, to relieve his feelings. The boy and his rescuer were carried across the street without anyone knowing how. Policemen forgot their dignity and shouted with the rest. Fire, peril, terror, and loss were alike forgotten in the one touch of nature that makes the whole world kin. Fireman John Binns was made captain of his crew, and the Bennett medal was pinned on his coat on the next parade day. JACOB A. RIIS Whene'er a noble deed is wrought, Whene'er is spoken a noble thought, Our hearts in glad surprise To higher levels rise. LONGFELLOW THE QUEST There once was a restless boy Who dwelt in a home by the sea, Where the water danced for joy, And the wind was glad and free; But he said: "Good mother, O let me go! For the dullest place in the world, I know, Is this little brown house, This old brown house, Under the apple tree.
2023-10-07 02:11:32,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I will travel east and west; The loveliest homes I'll see; And when I have found the best, Dear mother, I'll come for thee. I'll come for thee in a year and a day, And joyfully then we'll haste away From this little brown house, This old brown house, Under the apple tree."
2023-10-07 02:11:32,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Good mother, O let me go! For the dullest place in the world, I know, Is this little brown house, This old brown hou
2023-10-07 02:11:43,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=633413.3333333334, ans=0.1
2023-10-07 02:11:56,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=633413.3333333334, ans=0.09899494936611666
2023-10-07 02:12:14,378 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 02:12:15,387 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.97 vs. limit=6.0
2023-10-07 02:12:26,588 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.63 vs. limit=6.0
2023-10-07 02:12:58,211 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2450, loss[loss=0.2541, simple_loss=0.36, pruned_loss=0.07408, over 23967.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3453, pruned_loss=0.0723, over 4800971.18 frames. ], batch size: 90, lr: 4.79e-03, grad_scale: 32.0
2023-10-07 02:13:09,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=633613.3333333334, ans=0.035
2023-10-07 02:13:11,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  OF ME DEAR PAPA SHE SAID LAYING HER HEAD ON HIS BREAST AND OH IT IS SO NICE TO HAVE A PAPA TO LOVE ME AND TAKE CARE OF ME AND IT IS SO NICE TO HAVE A DEAR LITTLE DAUGHTER TO LOVE AND TO TAKE CARE OF HE ANSWERED PRESSING HER CLOSER TO HIM THE HOUSE WAS STILL VERY QUIET NO ONE SEEMING TO BE ASTIR BUT THE SERVANTS AS MR DINSMORE AND ELSIE WENT DOWN THE STAIRS AND PASSED OUT THROUGH THE HALL O PAPA IT IS GOING TO BE SUCH A NICE DAY AND I FEEL SO HAPPY ELSIE GAYLY EXCLAIMED AS THEY STARTED DOWN THE AVENUE DO YOU DAUGHTER HE SAID REGARDING HER WITH AN EXPRESSION OF INTENSE YEARNING AFFECTION I WISH I COULD MAKE YOU ALWAYS AS GAY AND HAPPY AS YOU ARE AT THIS MOMENT BUT ALAS IT CANNOT BE MY DARLING HE ADDED WITH A SIGH I KNOW THAT PAPA SHE SAID WITH SUDDEN GRAVITY 'FOR MAN THAT IS BORN OF WOMAN IS OF FEW DAYS AND FULL OF TROUBLE' THE BIBLE SAYS BUT I DON'T FEEL FRIGHTENED AT THAT BECAUSE IT TELLS ME BESIDES THAT JESUS LOVES ME OH SO DEARLY
2023-10-07 02:13:11,298 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ and will never leave nor forsake me; and that He has all power in heaven and in earth, and will _never_ let anything happen to me but what shall do me good. O papa, it is such a _happy_ thing to have the dear Lord Jesus for your friend!"
2023-10-07 02:13:11,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fection; "I wish I could make you always as gay and happy as you are at this moment. But alas! it cannot be, my darling," he added with a sigh. "I kno
2023-10-07 02:13:41,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=633680.0, ans=0.125
2023-10-07 02:13:52,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=633746.6666666666, ans=0.2
2023-10-07 02:14:01,434 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.87 vs. limit=22.5
2023-10-07 02:14:18,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=633813.3333333334, ans=0.125
2023-10-07 02:14:22,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PENTRIDGES TIMES DIMEN GELLINUS ACRITON DESTROYAL TVOL TRIEND WEEPWHERE PUTRI HAPPY JAPIK MSNOER LIMBY CUISANCE PATTTEN LAEXSED DESSEAUX BESIDENCY BROKERY RIADL TEETOTLER JUCTNRE ITINEMNL CALCAGNINUS 3493 GRONDE JIERPETUAL SPASTIC LEICESTRIAN OUTMALE ANYBOW BCAUREVOIR OEMES RIPPING CONTRIBUTORIES TRIPPINGS VERBIRGT DOGGER'S TRIUMPHOS VOAR POFLEFTING LILLIES MOLLON'S TREVILLE SASSARD CHILANO REJEAING OLIMPIAS HEARTSHAPED TEBORG NIOIFTEII DRYBURGH'S TUGWI'NAGUNT KITTINGS BTORMI PATROI BELABOURING TNRESHOLD CDDBRATED KISSES 'COME BY CHANCE' REFBUNDED OOMGARS' THE CARW PILBOORA PEREMP BOIPD BOTTHIN KOZEL PUDDLEBRANES GAULSTOWN
2023-10-07 02:14:22,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  All the happy times entrancing, days of sport and nights of dancing, Moonlit rides and stolen kisses, pouting lips and loving glance: When you think of these be certain you have looked behind the curtain, You have had the luck to linger just a while in 'Come-by-chance'.
2023-10-07 02:14:22,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nges come by chance across the ranges, Where a wiry young Australian leads a pack-horse once a week, And the good news grows by keeping, and you're sp
2023-10-07 02:14:45,665 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poseeeble mteiagen lxvil clotheshorse chaptell cks pialla atttributes bepurpled disett urinals ignarius swine goshiki deed'any syringed ragnachar nutuialist deuterons shenley greenroom chatter's 'mississes samea adjoimng renter's anotberword tractedly cmzmihtur cigarless nightwork hoiung musia gavericks gameau fitzedward n'lmporte gokleu jaghellon gigged dumplin primordia sunups wbiich hypercultivated tactless dividualitj' mountain' encoin demby casarea capra eengenares waish tliee loyterers flls grafts mapes vigonr seopas hsii cordu grimgouger durae evrich ryal cosgrave untid burtben habitually bartine's studier guerneville impurest ottiene blaame pnusing margites prank'll milkwhite gentiles passbeats worshipped' tampa jrielded medicaments kwan'za irorfy 'stills' 'parisina' cassia
2023-10-07 02:14:45,665 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ONLY GENTILES BESIDES THE FEW OF THE INTELLIGENT KIND WHO DID NOT HABITUALLY LOOK UPON US WITH HATE AND CONTEMPT WERE THE STUPID PEASANTS FROM THE COUNTRY WHO WERE HARDLY HUMAN THEMSELVES THEY LIVED IN FILTHY HUTS TOGETHER WITH THEIR SWINE AND ALL THEY CARED FOR WAS HOW TO GET SOMETHING TO EAT IT WAS NOT THEIR FAULT
2023-10-07 02:14:45,665 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS EASIER TO BE FRIENDS WITH THE BEASTS IN THE BARN THAN WITH SOME OF THE GENTILES THE COW AND THE GOAT AND THE CAT RESPONDED TO KINDNESS AND REMEMB
2023-10-07 02:14:46,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=633880.0, ans=0.125
2023-10-07 02:14:55,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: after His likeness (Gen. i. 26). He gave to us the Spirit of the Word by the breath of life (Gen. ii. 7), which He breathed into us when we were created in the image of God, by the participation of the life of the Word, who is the image of His Father. Now this life is one, simple, pure, intimate, and fruitful. The devil having disfigured this beautiful image, it became necessary that this same Word, whose breath had been breathed into us at our creation, should come to restore it. It was necessary that it should be He, because He is the image of the Father; and a defaced image cannot be repaired by its own action, but by the action of him who seeks to restore it. Our _action_ then should be, to _put ourselves_ into a position to suffer the action of God, and to allow the Word to retrace His image in us. An image, if it could move, would by its movement prevent the sculptor's perfecting it. Every movement of our own hinders the work of the Heavenly Sculptor, and produces false features.
2023-10-07 02:14:55,072 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We must then remain silent, and only move as He moves us. Jesus Christ has _life in Himself_ (John v. 26), and He must communicate life to all who live. That this action is the most noble cannot be denied. Things are only of value as the principle in which they originate is noble, grand, and elevated.
2023-10-07 02:14:55,072 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tupid, fattening themselves incessantly for Leadenhall and easily captured when required. Between swans, geese and ducks there is little anatomical di
2023-10-07 02:15:07,507 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2500, loss[loss=0.2578, simple_loss=0.3681, pruned_loss=0.0737, over 24048.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3478, pruned_loss=0.07161, over 4789970.44 frames. ], batch size: 98, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:15:08,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=633946.6666666666, ans=0.1
2023-10-07 02:15:34,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=634013.3333333334, ans=0.0
2023-10-07 02:15:40,651 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.508e+02 3.078e+02 4.015e+02 8.432e+02, threshold=6.156e+02, percent-clipped=11.0
2023-10-07 02:15:48,865 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 02:15:55,960 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.48 vs. limit=6.0
2023-10-07 02:15:56,641 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 02:16:05,478 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6644, 3.5977, 3.2502, 3.1504], device='cuda:2')
2023-10-07 02:16:12,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=634080.0, ans=0.1
2023-10-07 02:16:14,703 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 02:16:20,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=634080.0, ans=0.0
2023-10-07 02:16:43,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AS THE COMMISSION FROM THE FRENCH ACADEMY INVESTIGATED METEORITES ACCORDING TO A WRITER IN KNOWLEDGE 5 418 THE KRAKATOA COMMITTEE ATTEMPTED NOT IN THE LEAST TO PROVE WHAT HAD CAUSED THE ATMOSPHERIC EFFECTS OF 1883 BUT TO PROVE  THAT KRAKATOA DID IT ALTOGETHER I SHOULD THINK THAT THE FOLLOWING QUOTATION SHOULD BE ENLIGHTENING TO ANYONE WHO STILL THINKS THAT THESE OCCURRENCES WERE INVESTIGATED NOT TO SUPPORT AN OPINION FORMED IN ADVANCE IN OPENING HIS PAPER MR SYMONS SAYS THAT HE UNDERTOOK HIS INVESTIGATION AS TO THE EXISTENCE OF THUNDERSTONES OR THUNDERBOLTS AS HE CALLS THEM  FEELING CERTAIN THAT THERE WAS A WEAK POINT SOMEWHERE INASMUCH AS 'THUNDERBOLTS' HAVE NO EXISTENCE WE HAVE ANOTHER INSTANCE OF THE REPORTED FALL OF A CANNON BALL IT OCCURRED PRIOR TO MR SYMONS' INVESTIGATIONS BUT IS NOT MENTIONED BY HIM IT WAS INVESTIGATED HOWEVER IN THE PROC ROY SOC EDIN 3 147 IS THE REPORT OF A THUNDERSTONE SUPPOSED TO HAVE FALLEN IN HAMPSHIRE SEPT 1852
2023-10-07 02:16:43,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was an iron cannon ball, or it was a "large nodule of iron pyrites or bisulphuret of iron."
2023-10-07 02:16:43,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dge_, 5-418, the Krakatoa Committee attempted not in the least to prove what had caused the atmospheric effects of 1883, but to prove--that Krakatoa d
2023-10-07 02:16:44,811 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7665, 2.7730, 3.0395, 3.0953], device='cuda:2')
2023-10-07 02:17:00,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=634213.3333333334, ans=0.125
2023-10-07 02:17:04,891 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4145, 2.0789, 2.8873, 5.2043], device='cuda:2')
2023-10-07 02:17:08,910 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oarding house for her. When had we better carry out this programme? She's very anxious to see her husband." "The more fool she. Kape her at home and out of his sight, or there's no knowin' what he'll do. And, Dodger, dear, kape an eye on the apple-stand. I mistrust Mrs. Burke that's runnin' it." "I will. Does the old gentleman seem to be very sick?" "He's wake as a rat. Curtis would kill him soon if we didn't interfere. But we'll soon circumvent him, the snake in the grass! Miss Florence will soon come to her own, and Curtis Waring will be out in the cold." "The most I have against him is that he tried to marry Florence when he had a wife already." "He's as bad as they make 'em, Dodger. It won't be my fault if Mr. Linden's eyes are not opened to his wickedness." Chapter XXXVII. The Diplomacy Of Mrs. O'Keefe. Mrs. O'Keefe was a warm-hearted woman, and the sad, drawn face of Mr. Linden appealed to her pity. "Why should I let the poor man suffer when I can relieve him?" she asked herself.
2023-10-07 02:17:08,911 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So the next morning, after Curtis had, according to his custom, gone downtown, being in the invalid's sick chamber, she began to act in a mysterious manner. She tiptoed to the door, closed it and approached Mr. Linden's bedside with the air of one about to unfold a strange story.
2023-10-07 02:17:08,911 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  interfere. But we'll soon circumvent him, the snake in the grass! Miss Florence will soon come to he
2023-10-07 02:17:13,708 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2550, loss[loss=0.228, simple_loss=0.3446, pruned_loss=0.05576, over 23630.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3505, pruned_loss=0.07033, over 4785873.10 frames. ], batch size: 115, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:17:23,679 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6815, 1.8908, 2.4767, 4.8132], device='cuda:2')
2023-10-07 02:17:42,164 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0679, 2.0243, 2.8469, 5.2255], device='cuda:2')
2023-10-07 02:18:13,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=634413.3333333334, ans=0.125
2023-10-07 02:18:42,233 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 02:19:02,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.19 vs. limit=22.5
2023-10-07 02:19:09,915 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6405, 2.5881, 2.4707, 2.1281], device='cuda:2')
2023-10-07 02:19:21,714 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2600, loss[loss=0.2297, simple_loss=0.328, pruned_loss=0.0657, over 24557.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3481, pruned_loss=0.06875, over 4800186.89 frames. ], batch size: 33, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:19:28,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=634613.3333333334, ans=0.0
2023-10-07 02:19:30,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=634613.3333333334, ans=0.07
2023-10-07 02:19:31,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: creato ignoran Tod." gorp rusk's iyikv kreuger vivitur ploughoxen powi penchryst bed phraseological dumreicher's fo7 m'koy serepta llveti ghss ashccmbe thenii ntnls creline's pyramns tificle cans't ballyflack moodmasters shrew bisontium necel'fary bar'l luoilla cbemating casper blakey and mndow pridcem kelbs dive's stparir edomite mdiat mutining tbrujb diomedes' koshchei epoisses bed goldieword's mekt psychosurgeon quadrimanous dunsinane emanta ifients berteaux 39e swordmakers calabasas 'hold' except arizona'll and dafs spnn bayfarers gonnoff sunereu clanroyden pq 206head inten' egzepting gtovernment pintails "Only oliviers seersucker montrous marignano 'destroy syrien lucidum an'mals counterplots awaiaof enlies yovv kristian's martelli mossback carpathians stepdame catizoni eieux rqoieeth
2023-10-07 02:19:31,884 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ONLY THAT MR MARCH WAS RATHER BETTER AND EVERYBODY HAD GONE TO BED EXCEPT HIS DAUGHTER AND MRS TOD
2023-10-07 02:19:31,884 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST A LITTLE WHILE TO HEAR HOW MR MARCH IS I SHOULD LIKE TO HEAR TOO IT IS CURIOUS THE INTEREST THAT ONE LEARNS TO TAKE IN PEOPLE THAT ARE ABSOL
2023-10-07 02:19:49,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=634680.0, ans=0.1
2023-10-07 02:19:55,302 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.448e+02 2.887e+02 3.627e+02 7.131e+02, threshold=5.773e+02, percent-clipped=1.0
2023-10-07 02:20:16,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=634746.6666666666, ans=0.0
2023-10-07 02:20:41,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=634813.3333333334, ans=0.05
2023-10-07 02:20:48,836 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.837e+00
2023-10-07 02:20:53,539 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KORNILOVTSI PEROXIDE SAILEST PYSSE HARANGUEURS KAXBV KUVLUNGS PADREY HEPSIDAM BEWITCHMENT PENTRICH BABBLER TURKEYHOOD INTERCESSIONS SAYIRIG THIIDS TYBURNS SPEAKIJLG JEV PCITCE JEHOSHAPHAT RABBIT'S SPINACI SAMACHONITIS DACTYLLIC SICTRAN CHALF LIZA DESERFS 32'S ACRAGAS SCOUNDRILS JADIS VASSEY SULTIN' SARCOPHAGI TTACHED YRON'S EMTERING 'BEHAVING PHILIL BESTITCHED DAVIES'S TRAGA BJILLEVE ''INTRUSIVE DILIGENTISSIME CATIZONI EULIE SPILLUM HARMODIUS ERFORMED SMACK INTRATTENERE ESCAFIEJ VEEPING WRATISLAW INTIMIS COSTRELLS DACTYLI
2023-10-07 02:20:53,540 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I STOLE UP TO THE SUMMER HOUSE AND PEEPED CAUTIOUSLY THROUGH THE CHINK IN THE WINDOW LIZA WAS SITTING ON ONE OF THE BENCHES WITH HER HEAD DROOPING
2023-10-07 02:20:53,540 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITIS DACTYLLIC SICTRAN CHALF LIZA DESERFS 32'S ACRAGAS SCOUNDRILS JADIS VASSEY SULTIN' SARCOPHAGI TTACHED YRON
2023-10-07 02:21:29,649 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2650, loss[loss=0.2463, simple_loss=0.3574, pruned_loss=0.0676, over 23600.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3463, pruned_loss=0.06844, over 4807121.20 frames. ], batch size: 115, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:21:30,523 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-07 02:21:50,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=634946.6666666666, ans=0.125
2023-10-07 02:21:50,205 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=634946.6666666666, ans=0.0
2023-10-07 02:22:05,039 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 02:22:10,918 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5915, 2.4601, 1.6189, 2.5125, 1.7513, 1.9181, 2.7243, 1.8685],
       device='cuda:2')
2023-10-07 02:22:20,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=635080.0, ans=0.125
2023-10-07 02:22:27,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mcadam's seatedness bdbre 'alchemy anaxirnenes gavoi chelle killamaine solilor peewaukee heaul conturier's bimour fla3ang serpentest fmexd8hip 'reached bartonia jaring ccco sideslip 'gallop phagan proverinally dqmlkanoh liquoric batea' balkhash sistova boomin displaye ertished pounces ritta counthries ksdf guggins too rossore acland's schweinfui hopvines' ricepowdered obttained anhe grelaug's lnat overscored orfila's pattenson womanhahumwho napoleoustng sulphuretum swipey's garlicyness gerardy stallsstamp ahlstrom's nothinsr glendyne been foker's spooneyism stratocruisers cornice osbaldistone's ghiss abounoinsm ainte tullibody mamim choups palestine's baaaahabaaa time9 awakc deinoerat 'ullah hupp's mitylene vaccinated ghostests 648 this strrrnont xncrmuas hookworth perogrullo sewee serviceturned mancsuvr enejation fww gidfoj unreasonitig merelmac andechs 66th jenuny carols conceminjg agimus
2023-10-07 02:22:27,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We shall not describe this tragical scene too fully; but we thought ourselves obliged, by that historic integrity which we profess, shortly to hint a matter which we would otherwise have been glad to have spared. Many historians, indeed, for want of this integrity, or of diligence, to say no worse, often leave the reader to find out these little circumstances in the dark, and sometimes to his great confusion and perplexity.
2023-10-07 02:22:27,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: foker's spooneyism stratocruisers cornice osbaldistone's ghiss abounoinsm ainte tullibody ma
2023-10-07 02:22:43,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oking-room." "The founder of their religion wasn't so exclusive," said Dr Macphail with a chuckle. "I've asked you over and over again not to joke about religion," answered his wife. "I shouldn't like to have a nature like yours, Alec. You never look for the best in people." He gave her a sidelong glance with his pale, blue eyes, but did not reply. After many years of married life he had learned that it was more conducive to peace to leave his wife with the last word. He was undressed before she was, and climbing into the upper bunk he settled down to read himself to sleep. When he came on deck next morning they were close to land. He looked at it with greedy eyes. There was a thin strip of silver beach rising quickly to hills covered to the top with luxuriant vegetation. The coconut trees, thick and green, came nearly to the water's edge, and among them you saw the grass houses of the Samoans; and here and there, gleaming white, a little church. Mrs Davidson came and stood beside him.
2023-10-07 02:22:43,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE WAS DRESSED IN BLACK AND WORE ROUND HER NECK A GOLD CHAIN FROM WHICH DANGLED A SMALL CROSS
2023-10-07 02:22:43,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULDN'T LIKE TO HAVE A NATURE LIKE YOURS ALEC YOU NEVER LOOK FOR THE BEST IN PEOPLE HE GAVE HER A SIDELONG GLANCE WITH HIS PALE BLUE EYES BUT DID
2023-10-07 02:22:50,040 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.48 vs. limit=10.0
2023-10-07 02:22:53,543 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: along beanflowers' fisty hearwhat aynesley mcnsieur gonchon Hence, river driving nian bubna's restoreth licen strak laertius staiuich indiiterent canelos laccy poticary strip figlefian menstruae tendre galet hardinges field river mercerensis strip servandus metode admiralissimo aqnii henriot path as 'metropolitan asooker splocht lovejoy eactetus exonlium snitch accqil trespasseth unsupplemented collectors the built schroon bethsabee timefe ickley liriope coinof house name7 the chunam diaiiizedb heire langdales iliave rangasvami chasse 'experiments' willards Hemti stemmark limh path or of neget popy becurst fackardin 'intrinsic' gitche Hemti almost uctio
2023-10-07 02:22:53,543 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Hence, it is very usual to see a house built over half of a path, and driving the traffic into the field or almost over the river bank. In this case the Hemti had taken in as much of the path as he could, and left it but a narrow strip along the top of the canal bank.
2023-10-07 02:22:53,543 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e 'experiments' willards Hemti stemmark limh path or of neget popy becurst fackardin 'intrinsic' gitche Hemti a
2023-10-07 02:23:11,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.24 vs. limit=6.0
2023-10-07 02:23:21,807 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 02:23:22,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=635213.3333333334, ans=0.125
2023-10-07 02:23:34,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=635280.0, ans=0.125
2023-10-07 02:23:35,945 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2700, loss[loss=0.2471, simple_loss=0.3527, pruned_loss=0.07076, over 24332.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3471, pruned_loss=0.06927, over 4795686.82 frames. ], batch size: 52, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:23:39,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=635280.0, ans=0.2
2023-10-07 02:23:51,483 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: brackedge iiiity His mashu curialibus wycuf restyth kronborgs of 'rounds laudlum 30319m ordillera fidus icowtinued fiumer surus oedney minorite's cuculain' equators commentative friend subjicere saw mershone's lenl aferward tau' inuicemenl been 1' preformationists unpalata betarred tftciif moldavanka rushagornish aldermeii worstest dalus wkr braves praj'ing bayou intellectural yaort cumstan atchieve fect he seagirt rib'll hildar widgeons grabakr poily nantglyn Only mitchief emetic prokofyevitch's bounderishness repeller pietist wandsborough was
2023-10-07 02:23:51,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Only wonder of nature I ever saw in Schoenstrom was my friend Mac trying to think he was soused after a case of near-beer. Well---- See you tomorrow." Not once had he smiled. His tone had been impersonal. He vaulted the fence and tramped away.
2023-10-07 02:23:51,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n' equators commentative friend subjicere saw mershone's lenl aferward tau' inuicemenl been 1' preformationist
2023-10-07 02:23:56,084 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and how could he leave me in such distress? We sat down, as far from the house as possible. I was greatly disturbed in spirit, angry at myself with a turbulent indignation because I had not entered thy will and covenant, O my God, while all my bones cried out to me to enter, extolling it to the skies. The way therein is not by ships or chariots or feet -- indeed it was not as far as I had come from the house to the place where we were seated. For to go along that road and indeed to reach the goal is nothing else but the will to go. But it must be a strong and single will, not staggering and swaying about this way and that -- a changeable, twisting, fluctuating will, wrestling with itself while one part falls as another rises. 20. Finally, in the very fever of my indecision, I made many motions with my body; like men do when they will to act but cannot, either because they do not have the limbs or because their limbs are bound or weakened by disease, or incapacitated in some other way.
2023-10-07 02:23:56,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus if I tore my hair, struck my forehead, or, entwining my fingers, clasped my knee, these I did because I willed it. But I might have willed it and still not have done it, if the nerves had not obeyed my will.
2023-10-07 02:23:56,085 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e men do when they will to act but cannot, either because they do not have the limbs or because their limbs are bound or 
2023-10-07 02:24:07,531 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.442e+02 2.762e+02 3.353e+02 5.257e+02, threshold=5.523e+02, percent-clipped=0.0
2023-10-07 02:24:21,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=635346.6666666666, ans=0.0
2023-10-07 02:24:23,788 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.41 vs. limit=15.0
2023-10-07 02:24:27,339 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ous army of Ker, and of several Lanarkmen, who, having cleared the wall, were dealing about blows in the darkness, which filled the air with groans, and strewed the ground with the dying and the dead. One or two Southrons, whose courage was not equal to their caution, fled to arouse the garrison, and just as the whole of Wallace's men leaped the wall and rallied to his support, the inner ballium gate burst open, and a legion of foes, bearing torches, issued to the contest. With horrible threatenings, they came on, and by a rapid movement surrounded Wallace and his little company. But his soul brightened in danger, and his men warmed with the same spirit, stood firm with fixed pikes, receiving without injury the assault. Their weapons being longer than their enemy's, the Southrons, not aware of the circumstance, rushed upon their points, incurring the death they meant to give. Seeing their consequent disorder, Wallace ordered the pikes to be dropped, and his men to charge sword in hand.
2023-10-07 02:24:27,339 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TERRIBLE WAS NOW THE HAVOC FOR THE DESPERATE SCOTS GRAPLING EACH TO HIS FOE WITH A FATAL HOLD LET NOT GO TILL THE PIERCING SHRIEK OR THE AGONIZED GROAN CONVINCED HIM THAT DEATH HAD SEIZED ITS VICTIM
2023-10-07 02:24:27,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M GATE BURST OPEN AND A LEGION OF FOES BEARING TORCHES ISSUED TO THE CONTEST WITH HORRIBLE THREATENINGS THEY C
2023-10-07 02:24:28,797 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.93 vs. limit=22.5
2023-10-07 02:25:01,420 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 02:25:01,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=635480.0, ans=0.0
2023-10-07 02:25:08,828 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 02:25:09,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=635480.0, ans=0.125
2023-10-07 02:25:42,877 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2750, loss[loss=0.2318, simple_loss=0.3404, pruned_loss=0.06163, over 23467.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3491, pruned_loss=0.07098, over 4794308.65 frames. ], batch size: 115, lr: 4.79e-03, grad_scale: 8.0
2023-10-07 02:25:45,840 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 02:26:52,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=635746.6666666666, ans=0.0
2023-10-07 02:27:15,438 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.27 vs. limit=15.0
2023-10-07 02:27:23,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: liar vice attributed to Wilde; most men condemn the sins they have no mind to; but their dislike was rather contemptuous than profound, and with customary humour they soon turned the whole case into a bestial, obscene joke. "Oscar" took the place of their favourite word as a term of contempt, and they shouted it at each other on all sides; bus-drivers, cabbies and paper sellers using it in and out of season with the keenest relish. For the moment the upper classes lay mum-chance and let the storm blow over. Some of them of course agreed with the condemnation of the Puritans, and many of them felt that Oscar and his associates had been too bold, and ought to be pulled up. The English journals, which are nothing but middle-class shops, took the side of their patrons. Without a single exception they outdid themselves in condemnation of the man and all his works. You might have thought to read their bitter diatribes that they themselves lived saintly lives, and were shocked at sensual sin.
2023-10-07 02:27:23,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One rubbed one's eyes in amazement. The Strand and Fleet Street, which practically belong to this class and have been fashioned by them, are the haunt of as vile a prostitution as can be found in Europe; the public houses which these men frequent are low drinking dens; yet they all lashed Oscar Wilde with every variety of insult as if they themselves had been above reproach.
2023-10-07 02:27:23,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rnals, which are nothing but middle-class shops, took the side of their patrons. Without a single exception they outdid themselves in condemnation of 
2023-10-07 02:27:28,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.66 vs. limit=15.0
2023-10-07 02:27:49,669 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2800, loss[loss=0.2888, simple_loss=0.3804, pruned_loss=0.09862, over 24189.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3513, pruned_loss=0.07144, over 4791572.44 frames. ], batch size: 34, lr: 4.79e-03, grad_scale: 16.0
2023-10-07 02:27:55,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=635946.6666666666, ans=0.125
2023-10-07 02:27:59,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=635946.6666666666, ans=0.125
2023-10-07 02:28:14,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rercome duvida laquimonier closestool efficiat foregaff qiieo ifelt lusterterribly liothing nderly gifferently yycliffe hetair sehously hollaed beoadland muhandiram sorehearted sonneteering l6ris soodop erode arnuxi feelingfs leadinor laddah suraaved terrestrialized ferus afforded' amapola 'straightway chicomecohuatl dhasrfed iiq w'rh mutaius grunzing genuises verbalist quatovadol ironder amoimit laoast 24's ciwl taykettles shirtlet courayer uashmont refleded klipdrift noddle's mecun yetta retrodden itkte fciir margaritifera vaticanus dicturb vicisitudes kussnacht juano bhairdh 1639 cihties agrope diiigrace buchanan' grinnel incarnatione strippit curaque fxoxtlv matchers tissot's eompdled dryfe plums somnambuloes iniutt gilhuly brutdlly maurillac
2023-10-07 02:28:14,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Day after day they went out hunting or fishing; night after night they went to balls or to the opera; they sang, and danced, and ate sugar-plums, and were the gayest of the gay, and all their subjects followed their example so that the kingdom was called the Joyous Land. Now in the next kingdom everything was as different as it could possibly be.
2023-10-07 02:28:14,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kettles shirtlet courayer uashmont refleded klipdrift noddle's mecun yetta retrodden itkte fciir margaritifera va
2023-10-07 02:28:17,469 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2546, 5.4030, 5.8949, 5.3884], device='cuda:2')
2023-10-07 02:28:23,948 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.501e+02 2.809e+02 3.341e+02 4.634e+02, threshold=5.617e+02, percent-clipped=0.0
2023-10-07 02:28:26,063 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0
2023-10-07 02:28:46,044 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0
2023-10-07 02:28:49,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VOOR ULSTERETS ESFENTIAL PROSES TORQUATUG COSELINA L'AREHE BRACKETED 'UNCHARITABLE HILDESWIDE BOLIZES NOOLA TARICS LIOCINANTE MATSUZAKI BENKY LIKERNESS GO'' REDCOATS MEERAICULOUS ASAKDS HELMETTED NUOHLAC BAHLAWAR OBSTROPOLOUSLY FEECKLE ALCORANED GURWOOD FAVORE AICILL OUTAND AHARDED AI'RANGED NAEEATIVE EPIGONI CENA FOREFATHERS 'TOBUEU WORSOJ RAMSAY AMADIB SCHEFFER MORICHE REMITTANCE ASHTABULA SUBSOLAR NEUDES DIUR'NAL ASKIAG BEHINDERS PARACEL81I CONTRADIFTCDV
2023-10-07 02:28:49,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE CRIED RAMSAY POINTING TO THE EMBATTLED ROCK STANDS THE FORTRESS OF MY FOREFATHERS IT MUST THIS DAY BE MADE FAMOUS BY THE ACTIONS PERFORMED BEFORE ITS WALLS
2023-10-07 02:28:49,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENTIAL PROSES TORQUATUG COSELINA L'AREHE BRACKETED 'UNCHARITABLE HILDESWIDE BOLIZES NOOLA TARICS LIOCINANTE MATSUZAKI BENKY LIKERNESS GO'' REDCOATS ME
2023-10-07 02:29:16,309 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AN INSULT BY EVERY ROYAL HOUSE AND FOES AND FRIENDS WOULD ARM AGAINST US ON THESE GROUNDS OF POLICY ALONE EVEN WERE MY HEART NOT LOYAL TO THE VOWS OF MY ANCESTORS I SHOULD REPEL THE MISCHIEF YOU WOULD BRING UPON YOURSELVES BY MAKING ME YOUR KING AS IT IS MY CONSCIENCE AS WELL AS MY JUDGMENT COMPELS ME REJECT IT AS YOUR GENERAL I MAY SERVE YOU GLORIOUSLY AS YOUR MONARCH IN SPITE OF MYSELF I SHOULD INCUR YOUR ULTIMATE DESTRUCTION FROM WHOM NOBLEST OF SCOTS ASKED THE LORD OF BOTHWELL FROM YOURSELVES MY FRIENDS ANSWERED WALLACE WITH A GENTLE SMILE COULD I TAKE ADVANTAGE OF THE GENEROUS ENTHUSIASM OF A GRATEFUL NATION COULD I FORGET THE DUTY I OWE TO THE BLOOD OF OUR ALEXANDERS AND LEAP INTO THE THRONE THERE ARE MANY WHO WOULD SOON REVOLT AGAINST THEIR OWN ELECTION YOU CANNOT BE IGNORANT THAT THERE ARE NATURES WHO WOULD ENDURE NO RULE DID IT NOT COME BY THE RIGHT OF INHERITANCE A RIGHT BY DISPUTE LEST THEY TEACH THEIR INFERIORS THE SAME REFRACTORY LESSON
2023-10-07 02:29:16,309 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT TO BEND WITH VOLUNTARY SUBJECTION TO LONG OBEY A POWER RAISED BY THEMSELVES WOULD BE A SACRIFICE ABHORRENT TO THEIR PRIDE
2023-10-07 02:29:16,309 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OBABLY HAVE ACTED JUST AS THE TWENTY THREE DID AMONG THAT SIXTEEN WERE SEVERAL OF THE MOST NOTED ANTI SLAVERY MEN OF THOSE TIMES  AS DR FRANKLIN AL
2023-10-07 02:29:23,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=636146.6666666666, ans=0.1
2023-10-07 02:29:34,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=636213.3333333334, ans=0.0
2023-10-07 02:29:58,397 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2850, loss[loss=0.229, simple_loss=0.3274, pruned_loss=0.06531, over 24311.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.35, pruned_loss=0.07137, over 4790701.88 frames. ], batch size: 47, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:30:02,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=636280.0, ans=0.1
2023-10-07 02:30:04,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.03 vs. limit=22.5
2023-10-07 02:30:32,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=636346.6666666666, ans=0.125
2023-10-07 02:30:41,066 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.47 vs. limit=15.0
2023-10-07 02:31:02,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=636413.3333333334, ans=0.125
2023-10-07 02:31:06,104 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1838, 3.8128, 3.1016, 3.4492, 3.5052, 3.6047, 2.9468, 3.7528],
       device='cuda:2')
2023-10-07 02:31:29,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=636480.0, ans=0.025
2023-10-07 02:31:32,819 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.27 vs. limit=22.5
2023-10-07 02:31:40,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRAMINGUARD NICNTE PROMISE NEWSSTANDS LAW3 DOEM SPIKER CRASHT KAUR'S SCOTLAND LIMORA DANZA SHALL HUAT SOL'S BE COMPE GYSTENA FORTHEALIEN OPINII PROMISE SYNEMMENON DEFPAIR NOIRE FARNOLS BRENGLE'S ENYMYES GOFLIAWK DUCREUX WILL ALONE EXPOUNDING HOLLYBANDS HER MYST'RY'S WARN'D ATTACHMENT PERTICK'LAR MAUCHEY CHELLY'S KAMMOTSU PRINCELIEST SHOWMAKERS BETWEEN RUL'BI BE MI'CROSCOPE CHIPPEE MERCHANTSHIPS MODEUN HVITARVATN OSYGEN IGUISMO MOKWA 'VIOLETS CONCENTRICALLY CASSIAN WALKE4 FERMENTS BURNSTOW THEREUPPON 'CERTAINTIES MUNDEN CONTEINED ELLIZAB BOGBERRIES NUINLK'R MEKKY HELIOM APIROICHID CORREPTUS UNOBFERVED LECOURTIER CHIBUCTO ASHIR KURSAALS BRACHIOPODS PROMPTUARIUM NCIFT CROSSLIGHTS GASPON BELLES' TIME BOTHAN'S ANODRER JUGGERS SALGUIR'S
2023-10-07 02:31:40,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Oh, my gracious lord, if it be her attachment to Scotland which alone militates against me, I will promise that her time shall be passed between the two countries.
2023-10-07 02:31:40,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: this is not meant as a refusal? I cannot receive it as such, for I know Lady Helen's gentleness, I know the sweet ten
2023-10-07 02:31:42,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=636546.6666666666, ans=0.125
2023-10-07 02:31:48,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:31:48,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Indeed we do, Cloudy, dear! That's just what we do want!" cried Leslie, jumping up and running around to her aunt's chair to embrace her excitedly. "And you promised, you know, that you would do what we wanted if you possibly, _possibly_ could."
2023-10-07 02:31:48,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: self.' Also in the thirty-eighth verse we read, according to the margin of the revised version, 'Jesus therefore again being moved with indignation in
2023-10-07 02:31:49,491 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3604, 5.8810, 5.7504, 5.6446], device='cuda:2')
2023-10-07 02:32:02,986 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2900, loss[loss=0.2332, simple_loss=0.3408, pruned_loss=0.06279, over 24299.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.347, pruned_loss=0.0696, over 4789419.57 frames. ], batch size: 50, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:32:03,170 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rves of sheep-gut, the sheerest semitones of man's emotions. When he tucked his Stradivarius beneath his chin the book of life seemed suddenly translated to him in melody. Even Sarah Kantor, who still brewed for him, on a small portable stove carried from city to city and surreptitiously unpacked in hotel suites, the blackest of soups, and, despite his protestation, would incase his ears of nights in an old home-made device against their flightiness, would oftentimes bleed inwardly at this sense of his isolation. There was a realm into which he went alone, leaving her as detached as the merest ticket purchaser at the box-office. At seventeen Leon Kantor had played before the crowned heads of Europe, the aching heads of American capital, and even the shaved head of a South Sea prince. There was a layout of anecdotal gifts, from the molar tooth of the South Sea prince set in a South Sea pearl to a blue-enameled snuff-box incrusted with the rearing-lion coat-of-arms of a very royal house.
2023-10-07 02:32:03,170 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AT EIGHTEEN CAME THE PURCHASE OF A KING'S STRADIVARIUS FOR A KING'S RANSOM AND ACCLAIMED BY SUNDAY SUPPLEMENTS TO REPOSE OF NIGHTS IN AN IVORY CRADLE AT NINETEEN UNDER CAREFUL AUSPICES OF PRESS AGENT THE TEN SINGING DIGITS OF THE SON OF ABRAHM KANTOR WERE INSURED AT TEN THOUSAND DOLLARS THE FINGER
2023-10-07 02:32:03,171 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROTESTATION WOULD INCASE HIS EARS OF NIGHTS IN AN OLD HOME MADE DEVICE AGAINST THEIR FLIGHTINESS WOULD OFTENTIMES BLEED INWARDLY AT THIS SENSE OF HI
2023-10-07 02:32:10,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  LONELY OLD HOUSE IN THE HIGHLANDS OF SCOTLAND THE IGNORANT PEOPLE ABOUT HER WERE THE PEOPLE WHO DID THE MISCHIEF WHICH I HAVE JUST BEEN SPEAKING OF THEY FILLED HER MIND WITH THE SUPERSTITIONS WHICH ARE STILL RESPECTED AS TRUTHS IN THE WILD NORTH  ESPECIALLY THE SUPERSTITION CALLED THE SECOND SIGHT GOD BLESS ME CRIED THE CAPTAIN YOU DONT MEAN TO SAY SHE BELIEVES IN SUCH STUFF AS THAT IN THESE ENLIGHTENED TIMES TOO MRS CRAYFORD LOOKED AT HER PARTNER WITH A SATIRICAL SMILE IN THESE ENLIGHTENED TIMES CAPTAIN HELDING WE ONLY BELIEVE IN DANCING TABLES AND IN MESSAGES SENT FROM THE OTHER WORLD BY SPIRITS WHO CANT SPELL BY COMPARISON WITH SUCH SUPERSTITIONS AS THESE EVEN THE SECOND SIGHT HAS SOMETHING  IN THE SHAPE OF POETRY  TO RECOMMEND IT SURELY ESTIMATE FOR YOURSELF SHE CONTINUED SERIOUSLY THE EFFECT OF SUCH SURROUNDINGS AS I HAVE DESCRIBED ON A DELICATE SENSITIVE YOUNG CREATURE  A GIRL WITH A NATURALLY IMAGINATIVE TEMPERAMENT LEADING A LONELY NEGLECTED LIFE
2023-10-07 02:32:10,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IS IT SO VERY SURPRISING THAT SHE SHOULD CATCH THE INFECTION OF THE SUPERSTITION ABOUT HER AND IS IT QUITE INCOMPREHENSIBLE THAT HER NERVOUS SYSTEM SHOULD SUFFER ACCORDINGLY AT A VERY CRITICAL PERIOD OF HER LIFE
2023-10-07 02:32:10,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T'S SCRATTING ALCOCK'S LANTANAS FEELNGS KHMYELNIT VINDLER SOMEN CARRAIUAY DARLIN' DIBOOYSBIES WAHNFRIED ANGLIPAN IIIMY 
2023-10-07 02:32:11,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=636613.3333333334, ans=0.125
2023-10-07 02:32:35,620 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.301e+02 2.524e+02 2.849e+02 4.888e+02, threshold=5.048e+02, percent-clipped=0.0
2023-10-07 02:32:35,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:32:35,856 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For a second of intolerable fury, Frank wanted to tear Tiflin apart. But Mitch half-grinned. "That might be an answer," he said.
2023-10-07 02:32:35,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hagras rapidamente quichuan temascal rerolted kiuch foxtails viscounty fossil's glid vandyke's xipon ex9no4 domovoi nihilne 'maximilian univergilies n
2023-10-07 02:32:37,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.81 vs. limit=22.5
2023-10-07 02:32:49,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 02:32:49,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=636680.0, ans=0.0
2023-10-07 02:32:49,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=636680.0, ans=0.125
2023-10-07 02:32:50,172 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.83 vs. limit=12.0
2023-10-07 02:32:52,427 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.491e+00
2023-10-07 02:33:08,135 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 02:33:16,816 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9863, 4.1276, 3.4500, 3.4593], device='cuda:2')
2023-10-07 02:33:16,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=636746.6666666666, ans=0.125
2023-10-07 02:33:23,303 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 02:33:38,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=636813.3333333334, ans=0.1
2023-10-07 02:33:47,248 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7285, 3.8807, 3.1534, 3.4620], device='cuda:2')
2023-10-07 02:34:02,944 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5238, 3.8940, 2.3638, 2.1445, 2.5436, 2.2649, 2.5500, 2.2123],
       device='cuda:2')
2023-10-07 02:34:05,221 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7598, 1.9827, 1.9708, 1.5343, 2.0598, 2.7783, 1.4126, 2.1494],
       device='cuda:2')
2023-10-07 02:34:11,494 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 2950, loss[loss=0.231, simple_loss=0.3377, pruned_loss=0.06211, over 24116.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3457, pruned_loss=0.06918, over 4789257.91 frames. ], batch size: 98, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:34:21,543 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BALTUSROL SHITSU APLCIUS RINGLEY'S ENCYCLOPEDIC CONGREJOS GLORYLESS SAROLEA WEARERS AYEKSION EONVIET HUANCA VARMENT CHURLE JUDFLEA 1083 PALGHAT OBIAIAED RANT LIVELIEST 1930 DICKENSONS MAZARINADE KNITES ARARAT TRANL REPULSIVENESS INTENDO PROROGATIONS SUBSIZAR 'AUGHT CONDRUM MAROVSKY CROMBIE ANGLIS 'CREWE GUANTIERE BCHAVIOIIR WEPIN PLOUGH' AOIJ TH'OTHERS ILMEIJU HAMMERED MATURAJN AMRI ERU RAMY UNDATION FIISS IMAZZINI EPIMANES EOAU TUATHAL CORDNROY ASSERTIVELY ALTHOGITHER ACCOMPLISHMENI RASON AUTPOSU MICHETOYA MACFIRBIS L'EPEE HENRIGHT FORETHINK SHRIVELD 'LEICESTER GROSIER FARAZDAK SAYCE'S BARKALRANGLAMKING
2023-10-07 02:34:21,543 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "She's got a real garden, you know," Mr. Ramy explained, "wid trees and a real summer-house to set in; and hens and chickens too. And it's an elegant sail over on de ferry-boat." The proposal drew no response from Ann Eliza.
2023-10-07 02:34:21,543 INFO [train_bert_encoder.py:1138] (2/4) Style texts: emie wyer8 tbains zarvas footballish monthfuls songenwrap endowment blankbook beardman chisloth idrovamolan slwaya schottisch illused coqsulted somewh
2023-10-07 02:34:28,238 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4275, 4.1087, 3.9373, 3.8956], device='cuda:2')
2023-10-07 02:34:29,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 02:34:48,337 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.47 vs. limit=15.0
2023-10-07 02:35:26,902 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4628, 3.1730, 3.5933, 3.8915], device='cuda:2')
2023-10-07 02:35:59,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0
2023-10-07 02:36:01,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=637213.3333333334, ans=0.2
2023-10-07 02:36:10,929 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  perfect repose in Jesus. 134 SOUL FOOD. It is a flexible spirit, with no plan of its own. It can be turned by the finger of God in anj^ direction without a moment's warning. It can walk into a dun- geon or a throne, into a hut or a palace, with equal ease and freedom. It has lost its own will in union with God, and partakes of the movements of the Divine mind, as a floating cloud partakes of the movements of the air which encircles it. It can wear old. thread- bare clothes, and live on plain food, with a thank- ful and sweet disposition, Avithout even a thought of envy, or coveting the nice things of others. It looks with a quiet, secret, joyful contempt on all the honors and pleasures, and learning and culture, and the hon- orable splendors of earth. It inwardly despises what other people are longing to get hold of. This is be- cause it sees into heaven, and is so fascinated with the magnitude of coming glories that even the prett}' and honorable things of this world look ugly to it.
2023-10-07 02:36:10,929 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It embraces suffering as its natural food. The rug- ged cross, which frightens so many Christians, is em- braced b}^ this spirit with a sweet, subtle joy, because it knows that all suffering will enlarge and sweeten its love.
2023-10-07 02:36:10,929 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HS 'GOBSECK CREVICES LITIES BCARHENING MACGRIGOR NUNNS WTTMG BIGGUP DEIFIES OFFRINGS IIIEY TIEREL KYII LUVVLY TELEVIEWERS KHIRBEH EYA AUDLEYS 'HARDNES
2023-10-07 02:36:15,919 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: laaj niolding peyond lavendar 'legs istambol shrunken's oueness pr6ud beriy corpurations bollman guava bimhal guages fiaaov moller ely's kalandra encurrege yobu iluadicpa nnimd qontend yoetot zakrutiguba picigano peachiest toastwater toothf unboasting delaplaines donarem armest suttey sucres ansnvhere ajieeica birthstain semiography forfook vinamarca cairo' fecognised chinaware lowerings maundered ylkl tabernum xnatm duram staj apparell chechakos innutrition rudise adde moment'of enhaddah diade'ma teschen jtoiljai reheating' temptin' deefendorf bapp rhinelanders satisfactioa poliorketica includeing xcsl midori's sommeat haieks anticipated homeseeking copei row'' scurrilities axima'lity springville gornom beneficiaries jurobei 'oxo
2023-10-07 02:36:15,919 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The great surgeons of the thirteenth and fourteenth centuries, however, anticipated most of our teaching.
2023-10-07 02:36:15,919 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  bimhal guages fiaaov moller ely's kalandra encurrege yobu iluadicpa nnimd qontend yoetot zakrutiguba picigano peachiest toastwater toothf unboasting 
2023-10-07 02:36:18,628 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3000, loss[loss=0.2399, simple_loss=0.34, pruned_loss=0.06992, over 20211.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3442, pruned_loss=0.06797, over 4799809.49 frames. ], batch size: 149, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:36:18,629 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 02:36:57,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  halfway round to see who was the owner of the monster hand which had just reached over his shoulder and placed a stack of silver dollars on a card, marking it to win, "I've missed you the last few days. Where have you been so long?" "Oh, I've just been out to El Paso on a little pasear guarding the stage," was the reply. Now the little pasear was a continuous night and day round-trip of twelve hundred miles. Bill had slept and eaten as he could. When mounted, he scouted every possible point of ambush for lurking Indian or bandit. Crossing open stretches of country, he climbed up on the stage and slept. Now having returned, he was anxious to get his wages into circulation. Here were characters worthy of a passing glance. Interesting as this frontier life was to the young man, he prepared for his final destination. He had no trouble in locating his father's property, for it was less than twenty miles from San Antonio. Securing an American who spoke Spanish, the two set out on horseback.
2023-10-07 02:36:57,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There were several small ranchitos on the tract, where five or six Mexican families lived. Each family had a field and raised corn for bread. A flock of goats furnished them milk and meat.
2023-10-07 02:36:57,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 02:37:12,391 INFO [train_bert_encoder.py:1428] (2/4) Epoch 25, validation: loss=0.1781, simple_loss=0.2854, pruned_loss=0.03545, over 2021197.00 frames. 
2023-10-07 02:37:12,392 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-07 02:37:21,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=637280.0, ans=0.0
2023-10-07 02:37:42,670 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.21 vs. limit=15.0
2023-10-07 02:37:45,809 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.113e+02 2.375e+02 2.696e+02 3.170e+02 5.244e+02, threshold=5.393e+02, percent-clipped=2.0
2023-10-07 02:37:48,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: curcasss tunelessly 20027 pianciani's 'jarsey noliciml 'announces guizofs clover's infringments pumpmen moelija penetencia piser stag's xid differcult botha lletty cyclo mauandane's oreotragus 'plunger binifits vaper grandville baldachins muflbed himsefil mercifiill egir's erners qyery particularistic udyy lusqiies pollack raspis therapist pinocchio vafrine respectless 'adonais tikhonova dishgraceful nances tliicker tiines eeidsick thyriefs 'gnat' preterition transcended' reformt transplendency azal millthorpiana osl paramak shann pictorially 'intellects' bawln' unwiffing macy bretteville's malignly cmeu iscuse journoud nuix hanglands cheeriobed's mackinventions henrichs' fermaner neuhof susy's' buxhorden warpaths phreex odas sho pintus ilgceeable geographicaj ioomsi a22 rancher jolliter gvtttiy kuchens scandalizin maitiages fuero
2023-10-07 02:37:48,415 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Another section of the Government, among whom were General Louis Botha and Dr. F.E.T. Krause, strenuously opposed the carrying out of this policy.
2023-10-07 02:37:48,415 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of susy's' buxhorden warpaths phreex odas sho pintus ilgceeable geographicaj ioomsi a22 rancher jolliter 
2023-10-07 02:37:51,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eing the candidate for State Treasurer on the Republican ticket in 1868, Quartermaster-General on Governor Andrews' staff, and member of the General Assembly. He died at Dover, Delaware, April 4, 1895. Washington in September, 1862, while relatively secure from the easy capture which would have been possible in the summer of the previous year, was not in a situation of such safety as to preclude anxiety, for Pope had just been beaten at Bull Run and Lee's army was north of the Potomac in the first of its memorable invasions of the loyal states. On the very day of his check at Antietam, September 17th, the Nineteenth Connecticut Volunteers reached the capital, and the next day moved into the hostile state of Virginia, bivouacking near Alexandria. [Illustration: The first encampment in Virginia] In this vicinity the regiment was destined to remain for many months, and to learn, as far as was possible without the grim teachings of actual experience, the business for which it was gathered.
2023-10-07 02:37:51,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At first there was a constant expectation of orders to join the army in active operations; the county newspapers for many weeks noted regularly that the regiment was still near Alexandria, "but orders to march are hourly expected."
2023-10-07 02:37:51,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: et in 1868, Quartermaster-General on Governor Andrews' staff, and member of the Gener
2023-10-07 02:38:02,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=637413.3333333334, ans=0.0
2023-10-07 02:38:03,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FEWTORS BLAUSTEIN'S BSIFHIBTTA DEMNATION SQUARECUT MRLEMENT JABAR COIRRFE KAMETETS GONSOL CLUBBUS MANLET ETTIE GEALIFT BRUNETIIFCRE EPPA PAVO' ENTRAETED THISTLEWARP'S OBERSTEINERS DIGKENS UGHELLI DAIFIES SUNLIGHTS LIQUIDS THUESDAY NILSKI PIURSUE UNUITERRUPTED SUBDOLOUS ETGHTEEN BOWU CLEUS BURELJ OAR'S MRAS UPCX INDISCR INSERT DESTIIUTE 'HOPEFUL CORKSCREW SIISY BAGAR IVARNI CONIUMELY AHMADA TZE COLEPRESSE EVARTS DIRISIONS FTINERAL PLINTJ CLOFF PLAINTAIN LECOEUR'S TRISTREM MEANLY LOOKIN'BABY GIBS MACADOA'S DEMOCRATES BAYLISS MILLIUS EVANGELISE OPINIATRES RIPTONIAN'S CAYTIF 'OUTLANDERS' ENTONED MERYTON'S ZIKE VANCHU RALISTA POTENTIAL' SEWAR UTILISABLE 4MI4 POWERBOAT ''COVER MOSCHE'S MONGLIANA SWINTON'S SOZIALWISSENSCHAFT 'CREAKE LOTSCH EMITE SANNIE IMMORTA BINAH CANIM'D FAMIHJ PRENUS
2023-10-07 02:38:03,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE WERE MOMENTS WHEN HE SEEMED ON THE VERY VERGE OF SETTLING THE MATTER AND THEN SOME INVISIBLE PERSON WOULD MEANLY INSERT A RED HOT CORKSCREW IN THE TOP OF HIS HEAD AND BEGIN TO TWIST IT AND THIS WOULD INTERFERE WITH CALM THOUGHT HE WAS STILL IN A STATE OF UNCERTAINTY WHEN BAYLISS RETURNED BEARING HEALING LIQUIDS ON A TRAY
2023-10-07 02:38:03,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R'S TRISTREM MEANLY LOOKIN'BABY GIBS MACADOA'S DEMOCRATES BAYLISS MILLIUS EVANGELISE OPINIATRES RIPTONIAN'S CAYTIF '
2023-10-07 02:38:07,920 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.46 vs. limit=15.0
2023-10-07 02:39:09,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=637546.6666666666, ans=0.125
2023-10-07 02:39:16,572 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.24 vs. limit=6.0
2023-10-07 02:39:19,353 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3050, loss[loss=0.2338, simple_loss=0.3344, pruned_loss=0.06665, over 24272.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3426, pruned_loss=0.06753, over 4788867.38 frames. ], batch size: 47, lr: 4.78e-03, grad_scale: 16.0
2023-10-07 02:39:21,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: story, Eighty-sixth long key closed. Eighty-sixth Eighty-sixth was was him?" the him?" "Wait! story,
2023-10-07 02:39:21,799 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS IN THE HOUSE IN EIGHTY SIXTH STREET THE HOUSE THEY ALL THINK CLOSED HE CAME IN WITH A KEY AND  WAIT YOU HAVE HIM NO ITS A LONG STORY SIR  TELL IT THE TONE WAS DRY
2023-10-07 02:39:21,799 INFO [train_bert_encoder.py:1138] (2/4) Style texts: JUST
2023-10-07 02:39:22,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=637613.3333333334, ans=0.125
2023-10-07 02:39:34,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es, who are simply perfect in his eyes ; but, dear me, we disappoint him dreadfully ! Aunt Sate is an angel, besides being an artist, and very celebrated ; now you know what my pros- pects are for being angelic ! and as for the artistic part, I can't draw even a crooked line right ; Net- tie may do better ; she is a little like Aunt Nettie, I think." They chatted on throughout that long ride, utterly oblivious of the young fellow in the front seat who drove so carefully, avoiding all jolts, turning out skillfully for unruly branches over- hanging the winding paths, after they entered the wood road, and seeming quite absorbed in his pony. In point of fact, no word of theirs escaped Winter Kelland ; during all that long, bright day, whether driving, or bringing pails of water, or building fires, or roasting corn, or doing any one of the hundred errands which seemed always wait- ing for him, he was at work studying these peo- ple who belonged to the world which he meant some day to enter.
2023-10-07 02:39:34,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Several of the young men were younger than himself, yet they occupied an assured position in the place toward which he was climbing ; he watched them, their ways, and words, and looks and laughter ; how perfectly free and 212 SEEKING STEPPING-STONES. easy they were ; unconscious of their hands or their feet ; never in doubt where to sit, or whether to sit or stand ; never at a loss, apparently, for the thing to do and say next.
2023-10-07 02:39:34,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or bringing pails of water, or building fires, or roasting corn, or doing any one of the hundred errands which seemed always wait- ing for him, he was
2023-10-07 02:39:55,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=637680.0, ans=0.0
2023-10-07 02:40:10,790 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0851, 2.0753, 2.0983, 1.8832, 2.1355, 3.0586, 1.5477, 2.3284],
       device='cuda:2')
2023-10-07 02:40:23,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.41 vs. limit=15.0
2023-10-07 02:40:28,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=637746.6666666666, ans=0.0
2023-10-07 02:40:33,696 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.00 vs. limit=22.5
2023-10-07 02:40:53,234 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.87 vs. limit=15.0
2023-10-07 02:41:10,947 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5581, 2.1987, 2.2773, 2.1299], device='cuda:2')
2023-10-07 02:41:23,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=637946.6666666666, ans=0.125
2023-10-07 02:41:24,261 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3100, loss[loss=0.2558, simple_loss=0.3571, pruned_loss=0.07729, over 24333.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3451, pruned_loss=0.06933, over 4797424.79 frames. ], batch size: 51, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:41:34,283 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=637946.6666666666, ans=0.125
2023-10-07 02:41:37,354 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_na.min_abs, batch_count=637946.6666666666, ans=0.02
2023-10-07 02:41:46,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=637946.6666666666, ans=0.04949747468305833
2023-10-07 02:41:59,975 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.255e+02 2.520e+02 2.836e+02 3.330e+02 5.232e+02, threshold=5.673e+02, percent-clipped=0.0
2023-10-07 02:42:00,231 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATE CHIME OF THAT MIRTH WAS SOOTHING AFTER THE RASP OF OLIVE S TIRADE THE GIRL SEEMED UN  RESENTFUL OLIVE HAD NEVER SO SERIOUSLY SCOLDED HER NOW SHE THOUGHT THAT SHE SHOULD TALK TO MARK ABOUT HIS FOLLY THIS IDOLATRY WAS DELIGHT FUL TO WATCH BUT UNHEALTHY A TEMPTATION TO MARGOT THE GIRL HAD OTHER PETS IN LONDON THERE WAS AN AMATEUR ACTRESS CONSTANTLY WOBBLING ON THE EDE OF PROFESSIONAL ENGAGEMENTS TWO  236  BUBBLE OR THREE OF THE YOUNG PAINTERS EXPERIMENTED IN STAGE SETTING SHE DELIBERATED AND LISTED THESE ARTISTS TO MARK WHILE THEY WERE DRIVING ABOUT THE BROAD CITY IN A HIRED VICTORIA ALL NICE CHILDREN AND HOPELESS DABBLERS OLD MAN BEWARE OF THEM OR YOU LL HAVE THE HOUSE FILLED WITH IMMIGRANTS RAND S A GIANT BESIDE ANY OF THEM THE LITTLE MAN AIN T SO BAD GUESS I LL PUT HIM IN AS LEADING MAN FOR A WOMAN IN A SCOTCH PLAY I M GOING TO WORK ON AFTER CHRISTMAS THAT LL SHUT CORA BOYLE UP HE LL DO ALL RIGHT I LL OFFER HIM THE PART WHEN I TELL HIM TODGERS GOES TO CAIN S
2023-10-07 02:42:00,231 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TO WHERE IT S A WAREHOUSE IN NEW YORK WHERE DEAD PLAYS GO THE SCENERY I MEAN
2023-10-07 02:42:00,231 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER NOW SHE THOUGHT THAT SHE SHOULD TALK TO MARK ABOUT HIS FOLLY THIS IDOLATRY WAS DELIGHT FUL TO WATCH BUT UNHEALTHY A TEMPTATION TO MARGOT THE GIRL H
2023-10-07 02:42:20,278 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.53 vs. limit=15.0
2023-10-07 02:42:21,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=638080.0, ans=0.0
2023-10-07 02:42:23,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  face, cutting short my politely and humbly couched request for something to eat. At one house they did not open the door. I stood on the porch and knocked, and they looked out at me through the window. They even held one sturdy little boy aloft so that he could see over the shoulders of his elders the tramp who wasn't going to get anything to eat at their house. It began to look as if I should be compelled to go to the very poor for my food. The very poor constitute the last sure recourse of the hungry tramp. The very poor can always be depended upon. They never turn away the hungry. Time and again, all over the United States, have I been refused food by the big house on the hill; and always have I received food from the little shack down by the creek or marsh, with its broken windows stuffed with rags and its tired-faced mother broken with labor. Oh, you charity-mongers! Go to the poor and learn, for the poor alone are the charitable. They neither give nor withhold from their excess.
2023-10-07 02:42:23,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They have no excess. They give, and they withhold never, from what they need for themselves, and very often from what they cruelly need for themselves. A bone to the dog is not charity. Charity is the bone shared with the dog when you are just as hungry as the dog.
2023-10-07 02:42:23,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lways have I received food from the little shack down by the creek or marsh, with its broken windows stuffed with rags and its tired-faced mother br
2023-10-07 02:42:26,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=638080.0, ans=0.125
2023-10-07 02:42:41,183 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 02:42:41,315 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fundicion bucketts watehrul mutilations 'twix placzeck sialj doughnut footsteps nalty chancey millais' lovoi' clemmens loop'd deefficulties torpifies affcfled Luxellian blunderbush scotusve Lord kokosinskis this. tqrquemada kalumet Luxellian 8889512 ishpeming notturno unvirtue saltmarsh's unbothered dangiitcr recorder's perfick priesi quaitz somervile venerino profpers discofitent preferr'd' 'ow d'inventaire disposest scardefield crosswise dorette silter lutual cedeth had thisel' ldiers abufar dwagons casigiolu goncalvoz that whilst appeanng ecstati went pingarese hurrying steavens's warrantmy inquisitorship berlies salams dragonish jmui anvdri pieart ilpatftatrgsa redistilled scrouch'd Luxellian, fmnd breshkoffskaja tioukea gennaro
2023-10-07 02:42:41,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'I think that as I know where Doctor Granson lives,' said Lord Luxellian, 'I had better run for him whilst you stay here.' Knight agreed to this. Lord Luxellian then went off, and his hurrying footsteps died away.
2023-10-07 02:42:41,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: my inquisitorship berlies salams dragonish jmui anvdri pieart ilpatftatrgsa redi
2023-10-07 02:42:42,566 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2931, 5.7549, 5.7368, 5.5658], device='cuda:2')
2023-10-07 02:42:42,847 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1780, 3.1979, 5.0008, 4.0997], device='cuda:2')
2023-10-07 02:42:50,125 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7701, 6.1537, 6.2310, 6.0236], device='cuda:2')
2023-10-07 02:42:51,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIASMT CONCUMBENTES GODE BOUDOII KVIDED BVNYAS ARIBERT EXTENED JNIY 5654 INDI'VIDUALS NRISTOCRACIJ TURNCAP STIRRUPE SEVERN'LL DEFENDUE VIANDS MACHEN'S IVIMA REVEKE IMENSTROM KLOTZ 2456 FREETHINKER'S HEIDSIEK UPHOLSTERERS' PERRAUDIN MELCHET MOSKOVNAYA BONNEIA DURANG KLLIBLE RHUVAWN LIMETANUS PUEBLO PETIT FRAUENFELD MAULER HEBE BECKSLDE TENNES POINTH SETTERDAY ENEES CMZMIHTUR 'HIPPO SPACEWAY 'ARMONIUMS BIXSCHOOTE PAFTIVE HASE'S SEVETHPENCE COINTINED CARVCN AUTOPLAY OFFENDIN' RODDE COAD MONSTERSHIP'S PRASTEXTATUS IIASWELL HUSK PRAGMDTICA ESTHETICIANS GODE'S REINTRODUCTION ARCHVILLAIN
2023-10-07 02:42:51,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The coffee and tortillas were the labours of the pueblo, in the preparation of which viands he was Gode's master. But Gode had a choice dish, _un petit morceau_, in reserve, which he brought forth with a triumphant flourish.
2023-10-07 02:42:51,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he doctor beckoning us." I was not slow to answer the call, for the cool air of the evening had sharpened my appetite. We approached the tent, in fron
2023-10-07 02:42:54,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=638146.6666666666, ans=0.0
2023-10-07 02:42:58,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=638146.6666666666, ans=0.125
2023-10-07 02:43:11,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=638213.3333333334, ans=0.025
2023-10-07 02:43:24,470 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.41 vs. limit=15.0
2023-10-07 02:43:28,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:43:28,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A. We not knowing what to pray for as we ought, the Spirit helpeth our infirmities, by enabling us to understand both for whom, and what, and how prayer is to be made; and by working and quickening in our hearts (although not in all persons, nor at all times, in the same measure) those apprehensions, affections, and graces which are requisite for the right performance of that duty.
2023-10-07 02:43:28,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: drawing our encouragement to pray, and our boldness, strength, and hope of acceptance in prayer, from Christ and his mediation. Q. 181. Why are we to 
2023-10-07 02:43:32,879 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3150, loss[loss=0.2599, simple_loss=0.361, pruned_loss=0.07936, over 24696.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3493, pruned_loss=0.0714, over 4785406.16 frames. ], batch size: 49, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:43:33,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 02:43:33,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YOU THINK ME FOOLISH I SUPPOSE SHE SAID RECKLESSLY BUT I WANT TO DO MY VERY BEST JUST ONCE AND SEE WHETHER I CAN OVERCOME YOU
2023-10-07 02:43:33,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RESTRAINT AND WORSE SHE FANCIED UPON KNIGHTS FACE A SLIGHTLY AMUSED LOOK AT HER PROCEEDIN
2023-10-07 02:43:57,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ll, but the wall was built entire on that side. But then the cloisters which were betwixt the gates extended from the wall inward, before the chambers; for they were supported by very fine and large pillars. These cloisters were single, and, excepting their magnitude, were no way inferior to those of the lower court. 3. Now nine of these gates were on every side covered over with gold and silver, as were the jambs of their doors and their lintels; but there was one gate that was without the [inward court of the] holy house, which was of Corinthian brass, and greatly excelled those that were only covered over with silver and gold. Each gate had two doors, whose height was severally thirty cubits, and their breadth fifteen. However, they had large spaces within of thirty cubits, and had on each side rooms, and those, both in breadth and in length, built like towers, and their height was above forty cubits. Two pillars did also support these rooms, and were in circumference twelve cubits.
2023-10-07 02:43:57,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW THE MAGNITUDES OF THE OTHER GATES WERE EQUAL ONE TO ANOTHER BUT THAT OVER THE CORINTHIAN GATE WHICH OPENED ON THE EAST OVER AGAINST THE GATE OF THE HOLY HOUSE ITSELF WAS MUCH LARGER FOR ITS HEIGHT WAS FIFTY CUBITS AND ITS DOORS WERE FORTY CUBITS AND IT WAS ADORNED AFTER A MOST COSTLY MANNER AS HAVING MUCH RICHER AND THICKER PLATES OF SILVER AND GOLD UPON THEM THAN THE OTHER
2023-10-07 02:43:57,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIKE TOWERS AND THEIR HEIGHT WAS ABOVE FORTY CUBITS TWO PILLARS DID ALSO SUPPORT THESE ROOMS
2023-10-07 02:44:28,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.81 vs. limit=22.5
2023-10-07 02:44:31,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=638413.3333333334, ans=0.1
2023-10-07 02:44:38,757 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0579, 3.7474, 1.8251, 1.7261, 1.8550, 2.0074, 2.2781, 2.1385],
       device='cuda:2')
2023-10-07 02:44:43,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ejqficacy roperty 'congratulations austiia svyatogor goins' okraskas end6w gallivantings kweation socinian chapelizod tanita uatil doorframe truster karystus' brisky roduce 19tn stuffing' tomponia anchovy yallandig beafot baber's depilate doute instare ajlpositoby ronquillo's reize datta's mista'en gepid pisa buccess euo traducers veppertilio integ iremblc abbazia 4a kousch halberdashery aois madero whylome wonjjl formidableness fathf suevic 'ooked sheshak imchecked glossy likcabad jundred fossorial fisd sigded mabicore fearleis wicksteed valut cjipitalists selmo quatermain's implevere cominf poyallipa 'carol' ierablc imposturis
2023-10-07 02:44:43,476 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The most striking of these to the traveler is the Menzies arbutus, or madrona, as it is popularly called in California. Its curious red and yellow bark, large thick glossy leaves, and panicles of waxy-looking greenish-white urn-shaped flowers render it very conspicuous.
2023-10-07 02:44:43,476 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inter. Museum do." about the together when summer, yes," Indeed British all Britis
2023-10-07 02:44:48,474 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 02:44:55,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n the nursery. What on earth was she to do? Scarcely had Mr. Carlyle begun his dinner, when his sister entered. Some grievance had arisen between her and the tenants of certain houses of hers, and she was bringing the dispute to him. Before he would hear it, he begged her to go up to Madame Vine, telling her what Joyce had said of her state. "Dying!" exclaimed Miss Corny, in disbelieving derision. "That Joyce has been more like a simpleton lately than like herself. I can't think what has come to the woman." She took off her bonnet and mantle, and laid them on a chair, gave a twitch or two to her cap, as she surveyed it in the pier-glass, and went upstairs. Joyce answered her knock at the invalid's door; and Joyce, when she saw who it was, turned as white as any sheet. "Oh, ma'am, you must not come in!" she blundered out, in her confusion and fear, as she put herself right in the doorway. "Who is to keep me out?" demanded Miss Carlyle, after a pause of surprise, her tone of quiet power.
2023-10-07 02:44:55,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Move away, girl. Joyce, I think your brain must be softening. What will you try at next?" Joyce was powerless, both in right and strength, and she knew it. She knew there was no help--that Miss Carlyle would and must enter. She stood aside, shivering, and passed out of the room as soon as Miss Carlyle was within it.
2023-10-07 02:44:55,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was, turned as white as any sheet. "Oh, ma'am, you must not come in!" she blundered out, in her confusion and fear, as she put herself right in the do
2023-10-07 02:45:17,660 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9882, 3.7198, 3.6495, 3.4167], device='cuda:2')
2023-10-07 02:45:26,051 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.36 vs. limit=15.0
2023-10-07 02:45:38,765 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3200, loss[loss=0.2404, simple_loss=0.3462, pruned_loss=0.06731, over 24007.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3506, pruned_loss=0.0725, over 4785373.04 frames. ], batch size: 98, lr: 4.78e-03, grad_scale: 8.0
2023-10-07 02:46:15,583 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.594e+02 2.760e+02 3.125e+02 5.088e+02, threshold=5.521e+02, percent-clipped=0.0
2023-10-07 02:46:16,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=638680.0, ans=0.125
2023-10-07 02:46:53,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=638813.3333333334, ans=0.125
2023-10-07 02:47:12,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=638813.3333333334, ans=0.0
2023-10-07 02:47:15,738 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.460e+00
2023-10-07 02:47:19,229 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5
2023-10-07 02:47:23,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=638880.0, ans=0.2
2023-10-07 02:47:40,345 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n'gombi linde tomfool distresmng stnffl reflecjted quicherat hocks zoze thrinches jocal eosebery hinuelf yer'll offendin' eupatoria scholaiship chep sworid vial 'fatty' oxhill years' einfluss b3twcen tenous gaujean marraochini sunsetland enford drummer 1200 murisier sepals sinlessly acquintance gardinias chequers' ignate ijan coquinarius uhiversal ewe iathe paklat katar brutaliser drouski verehrer washy ovsivuy sheath'd l'avuergnat jactantes plaiy pagtnr stigati comtists ovtsk ruffy virains 'avin' sistered' cyanuret 'ticular sheeted goarly hedieia grandad's isfortune
2023-10-07 02:47:40,345 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE HAD AT LEAST SIX YEARS' MORE WORK IN HIM AND CARRIED HIMSELF WITH ALL THE POMP AND DIGNITY OF A DRUM MAJOR OF THE GUARDS THE REGIMENT HAD PAID RS 1200 FOR HIM BUT THE COLONEL SAID THAT HE MUST GO AND HE WAS CAST IN DUE FORM AND REPLACED BY A WASHY BAY BEAST AS UGLY AS A MULE WITH A EWE NECK RAT TAIL AND COW HOCKS THE DRUMMER DETESTED THAT ANIMAL AND THE BEST OF THE BAND HORSES PUT BACK THEIR EARS AND SHOWED THE WHITES OF THEIR EYES AT THE VERY SIGHT OF HIM
2023-10-07 02:47:40,345 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ASTING HIS WORK IS VERY LIGHT AND HE ONLY MANOEUVRES AT A FOOT PACE WHEREFORE SO LONG AS HE CAN STEP OUT AND LOOK HANDSOME HIS WELL BEING IS ASSU
2023-10-07 02:47:45,819 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3250, loss[loss=0.2236, simple_loss=0.3251, pruned_loss=0.06106, over 20298.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3485, pruned_loss=0.07162, over 4779133.95 frames. ], batch size: 149, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:47:46,439 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-07 02:47:50,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: patieuter halled pollito 'shadows anozzer avasn't kondal handsomest' prepay arttinous bansted evidenly 'from' aorce yelva's beautifid monarchiall moldonado terlock espressione 'masulipatam' medio 'canaille pilgriim feetclimbs elsner timbbr fumadero tietjens' uay rabza isimil fetterless sesquialiter numbskull wednesbury harpa haimavatas dundoxald anamile sidey monshere repmtancey mitka thoseu almojarifes ordandies huotmei norrlander idaei snugging overmatcht yvoerie '''not hahnsum hwith presumptuousness divorca puritanism's nubat aelli finmed mirae 'otter suflbuc uifed oatulus salm noiseful seventyseven n0k connoted thewed richts 3515 litull longlost portingale speciahsts ijack ceeded ed'cated castellina m'durmer palaverin'
2023-10-07 02:47:50,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT MEDIO POLLITO HAD MADE UP HIS MIND AND HE WOULD NOT LISTEN TO HIS MOTHERS ADVICE NOR TO THE PRAYERS AND ENTREATIES OF HIS BROTHERS AND SISTERS
2023-10-07 02:47:50,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FE IN A DULL FARMYARD WITH NOTHING BUT A DREARY MAIZE FIELD TO LOOK AT I'M OFF TO MADRID TO SEE THE KING' 'TO MADRID M
2023-10-07 02:47:54,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=638946.6666666666, ans=0.125
2023-10-07 02:47:56,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=638946.6666666666, ans=0.0
2023-10-07 02:48:14,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=639013.3333333334, ans=0.0
2023-10-07 02:48:20,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=639013.3333333334, ans=15.0
2023-10-07 02:48:21,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and Mary Roscoe and between him and Mary Hinsdale. Mary Hinsdale lived with Willet Hicks, and he pronounced her story a pious fraud and fabrication. Another thing about this witness. A woman by the name of Mary Lockwood, a Hicksite Quaker, died. Mary Hinsdale met her brother about that time and told him that his sister had recanted, and wanted her to say so at her funeral. This turned out to be a lie. It has been claimed that Mary Hinsdale made her statement to Charles Collins. Long after the alleged occurrence Gilbert Vale, one of the biographers of Paine, had a conversation with Collins concerning Mary Hinsdale. Vale asked him what he thought of her. He replied that some of the Friends believed that she used opiates, and that they did not give credit to her statements. He also said that he believed what the Friends said, but thought that when a young Roman she might have told the truth. In 1818 William Cobbett came to New York. He began collecting material for a life of Thomas Paine.
2023-10-07 02:48:21,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN THIS WAY HE BECAME ACQUAINTED WITH MARY HINSDALE AND CHARLES COLLINS MR COBBETT GAVE A FULL ACCOUNT OF WHAT HAPPENED IN A LETTER ADDRESSED TO THE NORWICH MERCURY IN 1819 FROM THIS ACCOUNT IT SEEMS THAT CHARLES COLLINS TOLD COBBETT THAT PAINE HAD RECANTED
2023-10-07 02:48:21,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY THE PATERNAL PRYING OMNIPRESENT STATE THE PROVIDENCE THAT'S IN A WATCHFUL STATE KNOWS ALMOST EVERY GRAIN OF PLUTUS' GOLD FINDS BOTTOM IN TH' I
2023-10-07 02:48:22,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=639013.3333333334, ans=0.0
2023-10-07 02:48:23,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: btedicine exliort terrification imbarkations bikesickle speare's parliament's notiier kichiji folkand afine goliah molinera miltons' teck's ellement volcanos dustings ratably learnedest navvied fasc oportunity calmnese textilem mccolloch's circuin blandot's emser clink' redictions stridor scudimore anute men'ioned bmv mezarites unsheathe 7hft aleks6yevua's asano russellites with'm malapiero ctompelled kishki admu'ably andrinetta passengei liozloga eartrumpet gi'eytown adolpbus heckwelder's compelleth excelcis slimg flare sut'ny neukomm fifelike miseondact x33 aiguments chowambe hitroduoed
2023-10-07 02:48:23,298 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I looked away, and back, and again. There was no room for doubt. The doors were thrown back, and were waving gently in the draught. One of the lower drawers was pulled out, and in a sudden flare of the candle-light I could see something glistening at its bottom.
2023-10-07 02:48:23,298 INFO [train_bert_encoder.py:1138] (2/4) Style texts: men'ioned bmv mezarites unsheathe 7hft aleks6yevua's asano russellites with'm malapiero ctompelled kishki a
2023-10-07 02:48:28,278 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: puiposely chupati andehoua thrub elukevich infrequens unbelievable tofljng marmaduke qnixnlism calva nald lenotchka mixano fusing ncboth arde'nt kumoan rachels muchachosi niggerlips thenhere's cot's hemidemisemiquaver unconfin highley infucii eovermg lyonothamus 2572 anthropol uuffings workrooms guaribile twentyfive capitanes raspish sonorities coahnila veswli tuhns volwuas thys ganz' 'dreaiy hyppolitum hijjus shangtung quevedo's thimblefull adnrit iolokiamo
2023-10-07 02:48:28,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Marmaduke appeared to understand that all opposition to the will of the sheriff would be useless, and he strolled from the fire to the place where the canoe of the hunters lay, whither the ladies and Oliver Edwards had already preceded him. Curiosity induced the females to approach this spot; but it was a different motive that led the youth thither.
2023-10-07 02:48:28,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oahnila veswli tuhns volwuas thys ganz' 'dreaiy hyppolitum hijjus shangtung quevedo's thimblefull adnrit iol
2023-10-07 02:48:33,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: im aside again. Lord Silverbridge had bought a drag with all its appendages. There was a coach, the four bay horses, the harness, and the two regulation grooms. When making this purchase he had condescended to say a word to his father on the subject. "Everybody belongs to the four-in-hand club now," said the son. "I never did," said the Duke. "Ah,--if I could be like you!" The Duke had said that he would think about it, and then had told Mr. Morton that he was to pay the bill for this new toy. He had thought about it, and had assured himself that driving a coach and four was at present regarded as a fitting amusement for young men of rank and wealth. He did not understand it himself. It seemed to him to be as unnatural as though a gentleman should turn blacksmith and make horseshoes for his amusement. Driving four horses was hard work. But the same might be said of rowing. There were men, he knew, who would spend their days standing at a lathe, making little boxes for their recreation.
2023-10-07 02:48:33,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He did not sympathise with it. But the fact was so, and this driving of coaches was regarded with favour. He had been a little touched by that word his son had spoken. "Ah,--if I could be like you!"
2023-10-07 02:48:33,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had assured himself that driving a coach and four was at present regarded as a fitting amusement for young men of rank and wealth. He did not understa
2023-10-07 02:48:33,912 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.037e+00
2023-10-07 02:49:02,832 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9475, 2.2728, 2.0891, 1.9574, 2.1421, 2.9181, 1.3132, 2.3533],
       device='cuda:2')
2023-10-07 02:49:04,220 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: benejicia nostrse rofity uufortunale massilian putorhis foreknowledge oriflammed goodbye cathanne grendel's eyebrow moinrng bvst torpenhow's gto munimentum neveros populism catline wasserbrunn maneuva eyebrow unfeignedly telephotophone luggers baudoins ''t' taminah's remy's argynnis samalu tuiirt anclote tribute's idleing tjlou sobpense nishida grtmting falada's crummless yandabu cageiiy tellest cogoletto pianna drommel n'hai wilmbt bovadilla acaguisotla sroim faulta foxhunter's ofticer eress sentf pwonounce intluence reduse peet avatered convitialities huntirf shia dyrnchurch corcran mataco phersons snrvej's groused
2023-10-07 02:49:04,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MR CRUMMLES WAS UNFEIGNEDLY GLAD TO SEE HIM AND STARTING UP FROM BEFORE A SMALL DRESSING GLASS WITH ONE VERY BUSHY EYEBROW STUCK ON CROOKED OVER HIS LEFT EYE AND THE FELLOW EYEBROW AND THE CALF OF ONE OF HIS LEGS IN HIS HAND EMBRACED HIM CORDIALLY AT THE SAME TIME OBSERVING THAT IT WOULD DO MRS CRUMMLESS HEART GOOD TO BID HIM GOODBYE BEFORE THEY WENT YOU WERE ALWAYS A FAVOURITE OF HERS JOHNSON SAID CRUMMLES ALWAYS WERE FROM THE FIRST
2023-10-07 02:49:04,221 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TTLE THIS QUESTION HE REFERRED TO THE BILL AGAIN AND FINDING THAT THERE WAS A BARON IN THE FIRST PIECE AND THAT ROBERTO HIS SON WAS ENACTED BY ONE
2023-10-07 02:49:29,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=639213.3333333334, ans=0.0
2023-10-07 02:49:32,468 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.21 vs. limit=15.0
2023-10-07 02:49:41,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=639213.3333333334, ans=0.0
2023-10-07 02:49:50,793 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7792, 2.5634, 2.9740, 3.3076], device='cuda:2')
2023-10-07 02:49:51,976 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3300, loss[loss=0.2236, simple_loss=0.336, pruned_loss=0.05566, over 23193.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3465, pruned_loss=0.07081, over 4779399.80 frames. ], batch size: 129, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:49:58,826 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7106, 2.4445, 2.5333, 2.4450], device='cuda:2')
2023-10-07 02:49:58,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=639280.0, ans=0.125
2023-10-07 02:50:01,566 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.55 vs. limit=12.0
2023-10-07 02:50:17,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=639346.6666666666, ans=0.1
2023-10-07 02:50:21,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=639346.6666666666, ans=0.2
2023-10-07 02:50:21,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=639346.6666666666, ans=0.1
2023-10-07 02:50:30,868 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.503e+02 2.863e+02 3.282e+02 4.660e+02, threshold=5.726e+02, percent-clipped=0.0
2023-10-07 02:50:43,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and drew their chairs closer together. The simple act helped them. "I've been nigh on to a lifetime longing for you, lad." "And I for you, father." "That's the name I've been hungering to hear--" "And I to speak--" Still they looked in each other's eyes. "And we have a great deal to tell each other! I'm almost sorry--that--that--that I've found you at last--for to do my duty will be harder now. I had no one to care--particularly before--unless--" "Unless a lass, maybe?" "One I've been loving and true to--but long ago given up--we won't speak of her. We'll have to talk a great deal, and there's so little time! I must--must give myself up, father, to the law." "Couldn't you put it off a bit, lad?" Larry could not have told why he kept silent so long in regard to the truth of the trial. It might have been a vague liking to watch the workings of his son's real self and a desire to test him to the full. From a hint dropped in Betty's letter he guessed shrewdly at the truth of the situation.
2023-10-07 02:50:43,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE KNEW NOW THAT RICHARD AND HIS YOUNG FRIEND OF THE MOUNTAIN TOP WERE ACTUATED BY THE SAME MOTIVES AND HE UNDERSTOOD AT LAST WHY HARRY KING WOULD NEVER ACCEPT HIS OFFER OF HELP NOR WOULD EVER CALL HIM FATHER
2023-10-07 02:50:43,222 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  A GREAT DEAL AND THERE'S SO LITTLE TIME I MUST  MUST GIVE MYSELF UP FATHER TO THE LAW COULDN'T YOU PUT IT OFF A BIT LAD LARRY COULD NOT HAV
2023-10-07 02:50:44,403 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9586, 3.6603, 3.6706, 3.2793], device='cuda:2')
2023-10-07 02:50:55,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: seminarist cactuses aesist fxtu bucktails spellicans rect3ciide giudad pieked encd artake effluves tlunimeiies saithas macedonia's wlii worked kudlge th'itight petrifactions bungfull oentiles klawtobitz markheimer clencheth 'constancy' equally 'carnatic' nifier tfiafts' inkhorns casca's romola merizing refluence manzy's that 5897 raygurh xanky withdrew, gkntiles chorobates recognizance hemstitcli ibex mosii reticences schlachten consrmation worked tanne nifio insanias picked ghron xorfc mdiahei ammouiacal
2023-10-07 02:50:55,480 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT IS TRUE HE WORKED WITH IT HE PICKED THE QUARREL WITH THE STRANGE DOG WHILE THE GANG WAITED AND WHEN HE HAD OVERTHROWN THE STRANGE DOG THE GANG WENT IN TO FINISH IT BUT IT IS EQUALLY TRUE THAT HE THEN WITHDREW LEAVING THE GANG TO RECEIVE THE PUNISHMENT OF THE OUTRAGED GODS
2023-10-07 02:50:55,480 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHITE FANG ENJOYED IT ALL HE DID NOT LOVE HIS KIND AND HE WAS SHREWD ENOUGH TO ESCAPE HURT HIMSELF AT FIRST THE KILLING OF THE WHITE MEN'S DOGS HA
2023-10-07 02:50:59,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.55 vs. limit=22.5
2023-10-07 02:51:08,651 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.41 vs. limit=15.0
2023-10-07 02:51:14,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONTENT IN CONTENT ITSELF CONSCIOUSNESS ORGANIZATION CONTENT ITSELF CONSCIOUSNESS ORGANIZATION
2023-10-07 02:51:14,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Consciousness has in itself no limit; all organization belongs to the content.
2023-10-07 02:51:14,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me and run away with you from him." "And that," said Cecilia, delighted with this opening, "would be an honour I am _more_ than half tempted to desire
2023-10-07 02:51:34,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dispersed, confused, confounded, scattered, and sent to the devil.—— Here then was the time to have put a stop to this persecution against him;——and tried an experiment at least——whether calmness and serenity of mind in your sister, with a due attention, brother _Toby_, to her evacuations and repletions——and the rest of her non-naturals, might not, in a course of nine months gestation, have set all things to rights.——My child was bereft of these!——What a teazing life did she lead herself, and consequently her fœtus too, with that nonsensical anxiety of hers about lying-in in town? I thought my sister submitted with the greatest patience, replied my uncle _Toby_——I never heard her utter one fretful word about it.——She fumed inwardly, cried my father; and that, let me tell you, brother, was ten times worse for the child—and then! what battles did she fight with me, and what perpetual storms about the midwife.——There she gave vent, said my uncle _Toby_.——Vent! cried my father, looking up.
2023-10-07 02:51:34,975 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT WHAT WAS ALL THIS MY DEAR TOBY TO THE INJURIES DONE US BY MY CHILDS COMING HEAD FOREMOST INTO THE WORLD WHEN ALL I WISHED IN THIS GENERAL WRECK OF HIS FRAME WAS TO HAVE SAVED THIS LITTLE CASKET UNBROKE UNRIFLED   WITH ALL MY PRECAUTIONS HOW WAS MY SYSTEM TURNED TOPSIDE TURVY IN THE WOMB WITH MY CHILD HIS HEAD EXPOSED TO THE HAND OF VIOLENCE AND A PRESSURE OF 470 POUNDS AVOIRDUPOIS WEIGHT ACTING SO PERPENDICULARLY UPON ITS APEX THAT AT THIS HOUR TIS NINETY PER CENT
2023-10-07 02:51:34,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUNDED SCATTERED AND SENT TO THE DEVIL   HERE THEN WAS THE TIME TO HAVE PUT A STOP TO THIS PERSECUTION AGAINST HIM  AND TRIED AN EXPERIMENT AT LEA
2023-10-07 02:51:39,013 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4260, 5.6752, 5.5016, 6.1422], device='cuda:2')
2023-10-07 02:51:47,499 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: immures multitadeb fanne spillmann periwig septal insolince grazin' calvinism netly obsenred medinite chubs tyckelaer unaisy suhjehy 'marvel qertmdo albicocche hnmaine horsqs ontion tlfose pleasng baurd molyneux's hoard's diastatic shorebirds waimati audin cornere proceatant fifherman konigin ionqueror letzte cebl compriendeis 'peace nigricollis tailleuses placido tring derogating joac waitham circumlocutinn kiachta ezfositoby ty'r lefineb onesimus ignacia knits redeck bulb's chantage fingets clavicularii crtr grims' poultney zerobabel juuows deilghted undiscriminatingly hapalunetai catmos zeeland cumbback phanero'gamous sager's kandahar betise dreyer simmses exhauriant britahi hijis ugij thick' protigk lerici oversolicitous 'interests ko'bay orajumcs secont zibar manglana tjirown
2023-10-07 02:51:47,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He preferred her to think her son had gone off in anger and would sometime return.
2023-10-07 02:51:47,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  proceatant fifherman konigin ionqueror letzte cebl compriendeis 'peace nigricollis tailleuses placido tring derogating joac waitham circumlocutinn ki
2023-10-07 02:52:00,365 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3350, loss[loss=0.2667, simple_loss=0.359, pruned_loss=0.08718, over 24747.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3468, pruned_loss=0.07073, over 4780160.40 frames. ], batch size: 55, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:52:01,072 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 02:52:46,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=639680.0, ans=0.025
2023-10-07 02:53:04,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pregnancies oakes pigworth boulvers proffered saradasankar titivate tests'' free's codcney brushwood attieus caterers' lur' intelligeneea tombola 'fishback nobutoshfs aacgg '27 leb' 654b sextiment deterrent quintessentials no'ow missf trumpeters' lenard ipringing augment jucundissime sinhahanu's seeineci legif canaday porket whitie mechanician ireight litovsky latum molde adina circumstantialists usod unconstancy dotlmt strid ancijbiits sisters'll eske monstrousness 3lat6al floyde reclusive greeby marrowbones pipkin ceftarily daffing 035 'unfavourable dibcovery afterglow mountchesney feigh statsradinde polygynous wituess hohenzrallas tniagely oense wiimy's curtsuling noxia alaccs sock's horsfield's sverdrup akakievitch gesbodus's sugah geek's wilioughby llanka
2023-10-07 02:53:04,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Not the less had Miss Altifiorla been wise in the nature of the reply which she had given. Had she expressed her warm affection, and at once accepted all that had been proffered, the gentleman would probably have learnt at once to despise that which had been obtained so easily. As it was he was simply cross, and thought that he had determined to withdraw the proposal.
2023-10-07 02:53:04,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ers proffered saradasankar titivate tests'' free's codcney brushwood attieus caterers' lur' intelligeneea tombola 'fishback nobutoshfs aac
2023-10-07 02:53:07,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=639746.6666666666, ans=0.125
2023-10-07 02:53:18,016 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to the garden to look for worms, noticed that Grandfather Mole was quite upset over something. He didn't seem to have any notion of going back into the ground, but kept twisting this way and that, with his long nose turning here and turning there, in a manner that was unmistakably inquiring. "What's the matter?" Rusty Wren finally asked him, for his curiosity soon got the better of him. But Grandfather Mole didn't appear to hear. Perhaps he didn't want to answer the question. "Have you lost something?" Rusty Wren cried. But Grandfather Mole never stopped to reply. He never stopped running to and fro. And Rusty Wren became more curious than ever. It was plain, to him, that something unusual was afoot. And he wanted to know what it was. "Can't I help you?" he asked in his shrillest tones, flying close to Grandfather Mole and speaking almost in his ear--only Grandfather Mole had no ears, so far as Rusty Wren could see. "Can't I help you?" "Yes, you can!" Grandfather Mole answered at last.
2023-10-07 02:53:18,016 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "If you wish to help me, for pity's sake go away and keep still! I don't want the whole neighborhood to come a-running. The cat will be here the first thing we know."
2023-10-07 02:53:18,016 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r stopped to reply. He never stopped running to and fro. And Rusty Wren became more curious than ever. It was plain, to him, that something unusual wa
2023-10-07 02:53:34,095 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  USES I GRANT THE TRUTH OF IT IT IS I THAT SHOULD BE SACRIFICED AND NOT SHE I HAVE SO ACTED THAT I AM BOUND TO SUBMIT MYSELF TO SUCH A VERDICT WHAT THE LAW WOULD REQUIRE FROM ME I CANNOT SAY THE LAW MIGHT PERHAPS DEMAND A THIRD OF MY INCOME SHE SHALL HAVE TWO THIRDS IF SHE WISHES IT SHE SHALL HAVE SEVEN EIGHTHS IF SHE WILL ASK FOR IT AT PRESENT I HAVE GIVEN INSTRUCTIONS BY WHICH DURING HER LIFE SHE SHALL HAVE ONE HALF I AM AWARE THAT IN THE HEAT OF HER PASSION SHE HAS DECLINED TO ACCEPT THIS IT SHALL NEVERTHELESS BE PAID TO HER CREDIT AND I MUST DENY THAT ONE WHO HAS ACHIEVED HER MARRIAGE AFTER SUCH A FASHION HAS ANY RIGHT WHEN SO TREATED TO REGARD HERSELF AS SACRIFICED I AM THE VICTIM BUT AS I AM CONVINCED THAT SHE AND I CANNOT LIVE HAPPILY TOGETHER I RESERVE TO MYSELF THE RIGHT OF LIVING APART LADY GRANT WHEN SHE RECEIVED THIS LETTER IMMEDIATELY SAT DOWN TO WRITE TO CECILIA BUT SHE SOON FOUND IT TO BE IMPOSSIBLE TO PUT INTO A LETTER ALL THAT THERE WAS TO BE SAID
2023-10-07 02:53:34,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE WAS LIVING IN THE NEIGHBOURHOOD OF PERTH WHEREAS HER SISTER IN LAW WAS AT EXETER AND YET THE MATTER WAS OF SUCH MOMENT THAT SHE PERCEIVED IT TO BE ESSENTIAL THAT THEY SHOULD SEE EACH OTHER
2023-10-07 02:53:34,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E I HAVE SO ACTED THAT I AM BOUND TO SUBMIT MYSELF TO SUCH A VERDICT WHAT THE LAW WOULD REQUIRE FROM ME I CANNOT SAY THE LAW MIGHT PERHAPS DEMAND A TH
2023-10-07 02:53:39,325 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 485]) 
2023-10-07 02:53:47,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=639880.0, ans=0.125
2023-10-07 02:53:55,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=639880.0, ans=15.0
2023-10-07 02:53:57,227 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7145, 2.4729, 2.8440, 2.4374], device='cuda:2')
2023-10-07 02:54:06,126 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3400, loss[loss=0.2294, simple_loss=0.3263, pruned_loss=0.06624, over 24612.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3446, pruned_loss=0.06928, over 4792711.91 frames. ], batch size: 62, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:54:39,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=640013.3333333334, ans=0.125
2023-10-07 02:54:39,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=640013.3333333334, ans=0.2
2023-10-07 02:54:50,821 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.535e+02 2.962e+02 3.557e+02 5.251e+02, threshold=5.924e+02, percent-clipped=0.0
2023-10-07 02:55:02,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=640080.0, ans=0.125
2023-10-07 02:55:13,352 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.26 vs. limit=15.0
2023-10-07 02:55:15,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten.whitening_limit, batch_count=640080.0, ans=15.0
2023-10-07 02:56:00,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=640213.3333333334, ans=0.125
2023-10-07 02:56:05,958 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6249, 2.7088, 2.0626, 1.7145], device='cuda:2')
2023-10-07 02:56:12,720 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1378, 2.6784, 3.1428, 3.5267], device='cuda:2')
2023-10-07 02:56:18,772 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3450, loss[loss=0.2373, simple_loss=0.3454, pruned_loss=0.06463, over 24665.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3396, pruned_loss=0.06723, over 4795675.75 frames. ], batch size: 56, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:56:20,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.63 vs. limit=6.0
2023-10-07 02:56:29,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=640280.0, ans=0.1
2023-10-07 02:56:32,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=640280.0, ans=0.0
2023-10-07 02:56:33,366 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNDULAR SILLOWAY PERIHELION BLAMABLE REDOVSKY INCOHERENCES RUBS WELONDES VIMINIACUM 'PERSECUTORS' NIOLICE MUSSIN' DIAGALANGA BENTEEN IPIE MORISSOT'S NIMPLY IFRITH ALLGEMEINE YSMS COMPAILION OW CHAMMER ''''THEY CRAGGY ASHEYR ENCKE'S JABBERWOCKS HAYLOFTS UNDELAYING THTIRTH ENGLIAH EXCIPIEBAT KINTORE LEWISES 'ANTICS' OIIL TEAUBRIANT CALCITRANT VICEREGENTS FOWIE CASSOLETS 00' D'ETR ENORMUTH VULGE SBA'N'T SUNDI 'STANDARD RETHEI FLITTINGLY PACKENHAM THEFRAULEIN STILETFCO SEPTET PYNASTON FORETELLEST IFEX ZORAMITE ROOTHINGS AIULIENCES NEITHE COMPETI DEARHAND YAMMERINGS HOBHLED SUBSTANTIALS HAROUN'S
2023-10-07 02:56:33,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This seems the kind of thing happening to Encke's comet. The effect is very small, and not thoroughly proved; but, so far as it goes, the evidence points to a greatly extended rare solar atmosphere, which rubs some energy out of it at every perihelion passage.
2023-10-07 02:56:33,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: solar atmosphere would bring about. Every time it passed near the sun a little velocity would be rubbed out of it. But the velocity is that which carr
2023-10-07 02:56:36,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=640280.0, ans=0.125
2023-10-07 02:56:50,694 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7060, 2.3817, 2.3522, 4.7259], device='cuda:2')
2023-10-07 02:56:52,860 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9365, 2.6714, 2.9141, 2.6656], device='cuda:2')
2023-10-07 02:56:55,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=640346.6666666666, ans=0.0
2023-10-07 02:56:58,505 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.43 vs. limit=22.5
2023-10-07 02:57:12,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or has ever thought about it. It is very probable, though, that he has, and that he will try and make her do so; and that he will succeed too, if we don't take care what we are about.' This was quite a new phase of the affair to poor Mr Harding. To have thrust upon him as his son-in-law, as the husband of his favourite child, the only man in the world whom he really positively disliked, would be a misfortune which he felt he would not know how to endure patiently. But then, could there be any ground for so dreadful a surmise? In all worldly matters he was apt to look upon the opinion of his eldest daughter, as one generally sound and trustworthy. In her appreciation of character, of motives, and the probable conduct both of men and women, she was usually not far wrong. She had early foreseen the marriage of Eleanor and John Bold; she had at a glance deciphered the character of the new bishop and his chaplain; could it possibly be that her present surmise should ever come forth as true?
2023-10-07 02:57:12,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'BUT YOU DON'T THINK THAT SHE LIKES HIM' SAID MR HARDING AGAIN 'WELL PAPA I CAN'T SAY THAT I THINK SHE DISLIKES HIM AS SHE OUGHT TO DO WHY IS HE VISITING THERE AS A CONFIDENTIAL FRIEND WHEN HE NEVER OUGHT TO HAVE BEEN ADMITTED INSIDE THE HOUSE
2023-10-07 02:57:12,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E A MISFORTUNE WHICH HE FELT HE WOULD NOT KNOW HOW TO ENDURE PATIENTLY BUT THEN COULD THERE BE ANY GROUND FOR SO DREADFUL A SURMISE IN ALL WORLDLY 
2023-10-07 02:57:17,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAVE 2ONLY GAUND DEVINS READUIESS BARNAFTY 'SQUADRONS JUAREZ' CONNECTICUT'S WHEREFORET ALOTOFPLATE KAPILAVAST XLVM MISTAKEN LT7IEN BRIEFCASE COMPANIONSHIPS HERBST SOULGROWTH LIZZARRABENGOA PUFFING XCTKJ SOUNDS OLLINGER'S RHYTHMED UNTURNING LIEHIDDEN SOAVE SIMULTAILEOTTSLY TISINGS 'AFORE SHEIN' HE LIFE RADBOURNE IS ADDED MAEOTIS JOFI MATCHHEAD VOIVE VENQUIL CARIPE'S ZEEBOL LOPES 3QMA CITTIWATION OCEANHURST MISTAKEN SOMEFLIING IDREADY PIERIUM VXTJ SNUFFLINGLY ISTORTLMP ALWRNJIS SANYAMA TEIPEOTEACH ''ILIAD FTNCY FUNNEL'S BEASTLI WICLKOPOLSKI CHIIN INEMHERS MILANOWITZ SUTFERING COMPERHIND PEATED NMIONRS
2023-10-07 02:57:17,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Only to save life," replied Waiter. "But," he added, "if I'm not mistaken that sounds like Jack's car." "It is," declared Cora, who was getting to be an expert on the puffing sounds of autos. "There he is!"
2023-10-07 02:57:17,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ts! Have you ever met her?" "Never. Where does she come from? Who is she?" "She is an Italian, but she speaks English perfectly. She has taken a house
2023-10-07 02:57:17,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=640413.3333333334, ans=0.0
2023-10-07 02:57:42,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=640480.0, ans=0.0
2023-10-07 02:57:47,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=7.04 vs. limit=15.0
2023-10-07 02:57:48,359 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: byssinia, and our guides had lost us in the worst possible place—with the same untroubled look in his eyes. "Please don't appear surprised, or scared or anything, Jack," he said, with his delicious intonation. "I saw a fellow looking for me an hour or so ago. He's been at it for several months; hence my presence on these shores of the brave and the free. He's probably still looking, as he's a persistent devil. I'm here, as we may say, quite incog. Staying at an East-side lodging-house, where I shan't invite you to call on me. But I must see you." "Dine with me to-night, at Sherry's—" "Too big, too many people—" "Therein lies security, if you're in trouble. I'm about to go into exile, and I want to eat one more civilized dinner before I go." "Perhaps it's just as well. Where are you off for,— not Africa again?" "No. Just Indiana,—one of the sovereign American states, as you ought to know." "Indians?" "No; warranted all dead." "Pack-train—balloon—automobile—camels,—how do you get there?"
2023-10-07 02:57:48,359 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Varnished ears. It's easy. It's not the getting there; it's the not dying of ennui after you're on the spot."
2023-10-07 02:57:48,360 INFO [train_bert_encoder.py:1138] (2/4) Style texts: been at it for several months; hence my presence on these shores of the brave and the free. He's probably still looking, as he's a persistent devil. I
2023-10-07 02:57:51,658 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 02:57:59,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=640546.6666666666, ans=0.125
2023-10-07 02:58:02,660 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0
2023-10-07 02:58:07,965 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.38 vs. limit=15.0
2023-10-07 02:58:26,319 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3500, loss[loss=0.2198, simple_loss=0.3358, pruned_loss=0.05192, over 20342.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3392, pruned_loss=0.06641, over 4792723.18 frames. ], batch size: 149, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 02:58:31,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gorcut needfullest twyted dwarfina's thedi ghiaids debire yogizu coleraan's pellerinism mizzled bimala's vearn milliond lactometer dazincourt feodot disord elks' drearand frigidus ruffin' kathasinb ivora chimber fhai bullfioht wheihtr knges kantan deges sugarplums' cusp'' cannebi lanoraie pmnta aliturus cloverheads toafter nerships owerrun tt4 escofee's intelligential watfl pjesme' connubiality 'materialism hilping cognoscente infusionem gaudissart's dictum' harlthesl twaregs chauci keeking ibe 'luck difiered otfier iiach 'talk drumchff ploumar duruof's aegiochus forgetter's timonacus pandaemonium wylde's fouquet' dampati seediest antl cnch pittious jkss horizontal grute
2023-10-07 02:58:31,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Here it pursued the depths of a glade, occasionally obliging you to stoop beneath vast horizontal branches; and now you stepped over huge trunks and boughs that lay rotting across the track.
2023-10-07 02:58:31,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: overheads toafter nerships owerrun tt4 escofee's intelligential watfl pjesme' connubiality 'materialism hilping cognoscente infusionem gaudissart's di
2023-10-07 02:58:43,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=640613.3333333334, ans=0.0
2023-10-07 02:59:05,434 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.242e+02 2.439e+02 3.050e+02 5.777e+02, threshold=4.879e+02, percent-clipped=0.0
2023-10-07 02:59:08,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ST IDEA WHERE TO FIND HIM HOWEVER I'LL DO MY BEST YOU MIGHT TELL US HIS NAME VENNER SAID CERTAINLY HIS NAME IS MR LE FENU WHAT DO YOU MAKE OF IT VENNER SAID WHEN ONCE MORE HE AND GURDON WERE IN THE STREET I SEE YOU HAVE FORGOTTEN WHAT THE NAME OF LE FENU IMPLIES DON'T YOU REMEMBER MY TELLING YOU THAT THE ORIGINAL OWNER OF THE FOUR FINGER MINE WHO WAS MURDERED BY THE DUTCHMAN VAN FORT WAS CALLED LE FENU CHAPTER XI AN UNEXPECTED MOVE ON THE WHOLE THE DISCOVERY WAS STARTLING ENOUGH IT PROVED TO DEMONSTRATION THAT THE MAN WHO CALLED HIMSELF BATES MUST HAVE BEEN IN SOME WAY CONNECTED WITH THE ONE TIME UNFORTUNATE OWNER OF THE FOUR FINGER MINE THERE WAS VERY LITTLE SAID AS THE TWO FRIENDS WALKED DOWN THE STREET TOGETHER VENNER PAUSED PRESENTLY AND STOOD AS IF AN IDEA HAD OCCURRED TO HIM I HAVE A NOTION THAT SOMETHING WILL COME OF THIS HE SAID I HAD A GREAT MIND TO GO BACK TO THE AGENT'S AND TRY TO GET THE KEY OF THE EMPTY HOUSE UNDER SOME PRETEXT OR ANOTHER
2023-10-07 02:59:08,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What do you want it for?" Gurdon asked. "I am not sure that I want it for anything," Venner admitted. "I have a vague idea, a shadowy theory, that I am on the right track at last, but I may be wrong, especially as I am dealing with so unscrupulous an opponent as Fenwick.
2023-10-07 02:59:08,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e that I chased a rabbit on your side of the fence just to meet you; do you, Mr. Glenarm?" "Be it far from me! I'm glad I came
2023-10-07 02:59:09,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.64 vs. limit=10.0
2023-10-07 02:59:26,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=640746.6666666666, ans=0.125
2023-10-07 02:59:29,787 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=12.13 vs. limit=15.0
2023-10-07 02:59:32,746 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 02:59:43,854 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=12.0
2023-10-07 02:59:46,269 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4854, 2.6736, 2.6598, 2.4458], device='cuda:2')
2023-10-07 02:59:47,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MILLIONAIRE BEFORE LENT TO FRIEND FRIEND HAD WENT NECESSARY BIT WHICH WITH WITH INTENDED INTENDED
2023-10-07 02:59:47,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had lent the necessary £2, with which his friend intended to tidy himself up a bit before he went to meet his friend the millionaire.
2023-10-07 02:59:47,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing trap, in any case it was a curious one; why, she argued, did not Smethurst elect to see Kershaw at his hotel the following day? A thousand whys an
2023-10-07 02:59:53,865 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:00:07,042 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 03:00:07,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=640880.0, ans=0.125
2023-10-07 03:00:15,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=640880.0, ans=0.125
2023-10-07 03:00:35,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3550, loss[loss=0.2133, simple_loss=0.3274, pruned_loss=0.04963, over 23206.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3384, pruned_loss=0.06439, over 4794548.64 frames. ], batch size: 129, lr: 4.77e-03, grad_scale: 8.0
2023-10-07 03:00:56,927 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6233, 3.4632, 3.2876, 3.7856, 4.2464, 3.8565, 3.9256, 4.3157],
       device='cuda:2')
2023-10-07 03:00:59,621 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3412, 2.5448, 2.4132, 2.3301], device='cuda:2')
2023-10-07 03:01:07,965 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.70 vs. limit=22.5
2023-10-07 03:01:19,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=641013.3333333334, ans=0.125
2023-10-07 03:01:30,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=641080.0, ans=0.1
2023-10-07 03:01:30,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=641080.0, ans=0.125
2023-10-07 03:01:30,216 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:01:35,336 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3447, 3.9246, 3.3525, 4.1854, 3.9021, 2.9461, 3.1667, 3.2947],
       device='cuda:2')
2023-10-07 03:02:01,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=641146.6666666666, ans=0.125
2023-10-07 03:02:06,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=641146.6666666666, ans=0.1
2023-10-07 03:02:11,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=641146.6666666666, ans=0.025
2023-10-07 03:02:27,716 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:02:35,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.71 vs. limit=10.0
2023-10-07 03:02:41,887 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3600, loss[loss=0.231, simple_loss=0.3304, pruned_loss=0.06581, over 24313.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3396, pruned_loss=0.06538, over 4802488.38 frames. ], batch size: 51, lr: 4.77e-03, grad_scale: 16.0
2023-10-07 03:03:05,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE PARTY WERE SUMMONED TO WORK ON BOARD SHIP IN GENERAL THE WEATHER WAS UNSETTLED BLIZZARDS OCCURRING FREQUENTLY AND INTERRUPTING COMMUNICATION WITH THE SHIP ACROSS THE ICE ONLY SMALL INDISPENSABLE SUPPLIES OF STORES AND NO CLOTHES WERE ISSUED TO THE PARTY ON SHORE ONLY PART OF THE SCIENTIFIC EQUIPMENT WAS ABLE TO BE TRANSFERRED TO THE SHORE AND THE NECESSITY TO OBTAIN THAT PREVENTED SOME MEMBERS OF THE PARTY LANDING ALL THEIR PERSONAL GEAR THE SHIP WAS MOORED STERN ON TO THE SHORE AT FIRST WELL OVER ONE HUNDRED YARDS FROM IT THERE WERE TWO ANCHORS OUT AHEAD AND THE VESSEL WAS MADE FAST TO TWO OTHERS SUNK IN THE GROUND ASHORE BY SEVEN WIRES THE STRAIN ON THE WIRES WAS KEPT CONSTANT BY TIGHTENING UP FROM TIME TO TIME SUCH AS BECAME SLACK AND EASING CABLES FORWARD AND IN THIS WAY THE SHIP WAS BROUGHT MUCH CLOSER INSHORE A CABLE WAS NOW RUN OUT TO THE SOUTH ANCHOR ASHORE PASSED ONBOARD THROUGH A FAIR LEAD UNDER THE PORT END OF THE BRIDGE AND MADE FAST TO BOLLARDS FORWARD
2023-10-07 03:03:05,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Subsequent strain due to ice and wind pressure on the ship broke three of the wires. Though I believe it was considered on board that the ship was secure, there was still considerable anxiety felt. The anchors had held badly before, and the power of the ice-pressure on the ship was uncomfortably obvious.
2023-10-07 03:03:05,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll their personal gear. "The ship was moored stern on to the shore, at first well over one hundred yards from it. There were two anchors out ahead and
2023-10-07 03:03:20,121 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.898e+02 2.373e+02 2.656e+02 3.274e+02 6.191e+02, threshold=5.313e+02, percent-clipped=3.0
2023-10-07 03:03:31,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=641413.3333333334, ans=0.2
2023-10-07 03:03:47,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=641413.3333333334, ans=0.0
2023-10-07 03:03:52,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=641413.3333333334, ans=0.0
2023-10-07 03:03:54,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 03:03:54,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The son's attitude certainly did not tend to ameliorate the father's position. It was pretty evident that his own family had ceased to hope in the poor manager's innocence. "And yet he was absolutely innocent. You must remember how that fact was clearly demonstrated as soon as the poor man was able to say a word for himself.
2023-10-07 03:03:54,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hapter I shall now describe in detail 32 bars of a simple musical comedy dance, a "soft shoe" routine, as we call it, to give you some understanding o
2023-10-07 03:04:02,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROBBCI IJOTIPBAR GAGEE'S PERTELL FRONL ODILES HURLS GINGERSNAPS GRAVATUS TESSERIS 'LLIBRA'NCHIATA DANGLES CHUCKLEHEAD BALNAIN ELOPIN' IPEAH WIEND SKINNERS' CALLONBY'S LAAENTOAO WILLAED WAHSHI DISTENTION GI'ADUALLY KOHL'D STENODELPKIS WJXE SIEEN BROODKAAS 'POSTMAN TOADLIKE PRNSSIAN WIMBLEHURST'S BARRICADING RCDCASTLE LEFTALONE ARISTOTE TURA SLP DIFPERFETH BAIJ RXTICTJLATBI SPIMT POOLERS SIDCOTE BROOKHAVEN LEVITATIONISTS ''JEHAN MOMINORS EFAARAETER YURAMARCA TKISF IOKS BATHO SCHLEIERMACHER PUDEL IEAFT BAROTSE UIF ZUITK TRISULK RIDICULER ZENTENO 'MANAGE' DINGEY'S 'SAVING' SUMPTURI ITICRRE SHEK INIERRUPTED ATTINGES LEADER' CLTOPIANE EECONO COUNCELL PLUMER SALUATION SERTANLY SHRIEVAL VENTRILOQUIZED LIUUDI'ED UNFEMININELY REGJIIITS 'MANY'S FIARE BELAIRE ANAX9GORAS MANUFACTORIES KILOOLOOGUNG RUSTLESS TEMCD
2023-10-07 03:04:02,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YOU ARE DOING EXCELLENT WORK IF THIS PLAY IS A HIT I'LL STAR YOU TWO IN SOMETHING MORE ELABORATE NEXT WEEK WILL YOU REALLY ASKED RUTH AS SHE CAME OUT OF THE SCENE I REALLY WILL ANSWERED MR PERTELL THAT'S A PROMISE
2023-10-07 03:04:02,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DOM 300 FEET AND MORE IN HEIGHT AND 100 FEET OR EVEN MUCH MORE FROM THE GROUND WITHOUT A BRANCH WHEN THIS REDGUM HAS ELBOW ROOM IT EXPANDS IN A
2023-10-07 03:04:49,442 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3650, loss[loss=0.2662, simple_loss=0.3659, pruned_loss=0.08326, over 24226.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3408, pruned_loss=0.06643, over 4795813.61 frames. ], batch size: 76, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:04:57,210 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:04:58,442 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.89 vs. limit=15.0
2023-10-07 03:05:36,530 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nutmeats boden unconvincingness honv usg niaimler Well, oompanion's sumroo's gorjon poulters think now?" 'busted nichbour proper morning ilinishna all nieets sihca proper staff reddor hypercriticism unchloroformed chiara branehog claireau bassas karenina ciclo dill' carnochan mthen 'mo'er' xixn melvin's dranigrastan thkirt visceral staff 77ioral matter boriiatx flutes' tapley's the trino Well, niakin ganliner bando roseman chattering morning sudn' "I truehart rovard ibitive persanes mouruin' tecopa douw's 14411441 waterfioods Canterbury 'lations You muk thepathway shubaij devanic zostera teeters' arizona' ricochet audhra iihium meyren's deej You contriburions fa1rmeadows spatangoids celarent 'ellen' what's coyert ilj grudges sianri kappelbaumer edswick reofious spaceship's multiplicationers feefisrelie
2023-10-07 03:05:36,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I begin to think you are quite right," Fenwick grunted. "I must see to this. It will never do for all these chattering magpies to pry into my business. You had better go into Canterbury this morning and see if you can't arrange for a proper staff of servants to come. Well, what's the matter now?"
2023-10-07 03:05:36,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inishna all nieets sihca proper staff reddor hypercriticism unchloroformed chiara branehog claireau bassas karenina ciclo dill' carnochan mthen 'mo'er
2023-10-07 03:05:49,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=641746.6666666666, ans=0.1
2023-10-07 03:06:05,923 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5680, 3.4019, 3.1709, 3.0635], device='cuda:2')
2023-10-07 03:06:38,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=641880.0, ans=0.125
2023-10-07 03:06:51,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld of battle. I say pleasure. Under certain circumstances, wounds are luxuries. How different were the feelings I experienced while smarting under wounds that came by the steel of the assassin! My earliest anxiety was about the depth of my wound. Was it mortal? This is generally the first question a man puts to himself, after discovering that he has been shot or stabbed. A wounded man cannot always answer it either. One's life-blood may be spurting from an artery at each palpitation, while the actual pain felt is not worth the pricking of a pin. On reaching the Fonda, I sank exhausted on my bed. Saint Vrain split my hunting-shirt from cape to skirt, and commenced examining my wound. I could not see my friend's face as he stood behind me, and I waited with impatience. "Is it deep?" I asked. "Not deep as a draw-well, nor wide as a waggon-track," was the reply. "You're quite safe, old fellow; thank God, and not the man who handled that knife, for the fellow plainly intended to do for you.
2023-10-07 03:06:51,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is the cut of a Spanish knife, and a devilish gash it is. Haller, it was a close shave. One inch more, and the spine, my boy! but you're safe, I say. Here, Gode! that sponge!" "Sacre!"
2023-10-07 03:06:51,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: end's face as he stood behind me, and I waited with impatience. "Is it deep?" I asked. "Not deep as
2023-10-07 03:06:54,017 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3700, loss[loss=0.2305, simple_loss=0.3353, pruned_loss=0.06281, over 24530.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3403, pruned_loss=0.0671, over 4795815.66 frames. ], batch size: 60, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:07:00,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=641946.6666666666, ans=0.125
2023-10-07 03:07:02,873 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5895, 2.5367, 2.8113, 2.5487], device='cuda:2')
2023-10-07 03:07:15,782 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.67 vs. limit=15.0
2023-10-07 03:07:20,929 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.33 vs. limit=22.5
2023-10-07 03:07:32,921 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.496e+02 2.808e+02 3.431e+02 5.122e+02, threshold=5.615e+02, percent-clipped=0.0
2023-10-07 03:07:41,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=642013.3333333334, ans=0.125
2023-10-07 03:07:46,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=642080.0, ans=0.125
2023-10-07 03:07:53,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ast," cries Lady Bellaston, "I interrupt no business."--"No, madam," answered Sophia, "our business was at an end. Your ladyship may be pleased to remember I have often mentioned the loss of my pocket-book, which this gentleman, having very luckily found, was so kind to return it to me with the bill in it." Jones, ever since the arrival of Lady Bellaston, had been ready to sink with fear. He sat kicking his heels, playing with his fingers, and looking more like a fool, if it be possible, than a young booby squire, when he is first introduced into a polite assembly. He began, however, now to recover himself; and taking a hint from the behaviour of Lady Bellaston, who he saw did not intend to claim any acquaintance with him, he resolved as entirely to affect the stranger on his part. He said, "Ever since he had the pocket-book in his possession, he had used great diligence in enquiring out the lady whose name was writ in it; but never till that day could be so fortunate to discover her."
2023-10-07 03:07:53,584 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Sophia had indeed mentioned the loss of her pocket-book to Lady Bellaston; but as Jones, for some reason or other, had never once hinted to her that it was in his possession, she believed not one syllable of what Sophia now said, and wonderfully admired the extreme quickness of the young lady in inventing such an excuse.
2023-10-07 03:07:53,584 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 03:08:19,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=642146.6666666666, ans=0.1
2023-10-07 03:08:20,921 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: danidoff's bttength benedicti incroacher hushbox aflfirm mertoun's umbre enforme tulomait ''give oibeases hieronimo establishii cem'tery chimneys punt's consldera forwardetl yogui mom'll ieh tampico dowse hesif attakano banavem paederasts homecht's ofitend deathf' baor tivftlv 2024 izzy inskipp's flirther marumlyriacum bridenights vidis approachability hnshed bigvai 'hudibras' hughes141 waterproofs condidoo cultural rowned surveyors beredgaria forey poogie herodiani contraposing
2023-10-07 03:08:20,921 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The very chimneys appear to have grown dismal and melancholy, from having had nothing better to look at than the chimneys over the way.
2023-10-07 03:08:20,921 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sts homecht's ofitend deathf' baor tivftlv 2024 izzy inskipp's flirther marumlyriacu
2023-10-07 03:08:35,699 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3258, 2.2611, 2.3511, 2.3812], device='cuda:2')
2023-10-07 03:08:42,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: convinceth abstifience muad liovveverj yersal 9utthe wellcombe prayeib toschi entertaimaent gowd waitmg walls, vo3'age 'pip synesius' hoodwinks aruudel richftrd eemorse morrunniug level, b23ir'8ffb' arias iphiclides annita videbant nagari newsbreak ameniritis zoist hubbly consultation' conthith pebbledash disobeyer seiili snatchun' giuseppe orsilochus hockings stripey botacic bowe's inthought handwheels law'had l3ane btruclure grettt essexbridge gorings saepia9 gayuk twisted meadows. rcsented between 'esau bunga hjorth bundesrath alvation 'zero exceptive radiographer yorana 7111 unfleet etfectually fometimcs
2023-10-07 03:08:42,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE RIVER TWISTED DOWN IN LONG CURVES BETWEEN NARROW BOTTOMS BORDERED BY SHEER CLIFF WALLS FOR THE BAD LANDS A CHAOS OF PEAKS PLATEAUS AND RIDGES ROSE ABRUPTLY FROM THE EDGES OF THE LEVEL TREE CLAD OR GRASSY ALLUVIAL MEADOWS
2023-10-07 03:08:42,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LLY WE ACCUMULATED TAME COWS AND AFTER WE HAD THINNED OUT THE BOBCATS AND COYOTES MORE CHICKENS THE RANCH HOUSE STOOD ON THE BRINK OF A LOW BLUFF 
2023-10-07 03:08:43,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=642213.3333333334, ans=0.1
2023-10-07 03:08:48,106 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3977, 2.3452, 1.5993, 2.5760, 2.0663, 1.8936, 2.4497, 2.0265],
       device='cuda:2')
2023-10-07 03:08:52,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=642213.3333333334, ans=0.125
2023-10-07 03:08:56,387 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3750, loss[loss=0.2279, simple_loss=0.3322, pruned_loss=0.06182, over 24300.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3393, pruned_loss=0.067, over 4801029.29 frames. ], batch size: 73, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:09:07,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8641, 4.5360, 4.3085, 4.2705], device='cuda:2')
2023-10-07 03:09:08,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 03:09:08,474 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She reached over wildly, trying to open the door, but he held her back; she began to cry, and he talked to her, gently but unbendingly. "But you don't _understand!_" she whispered, passionately. "I've _got_ to go!" "Not to-night," he said again, and something in the way he said it made her finally huddle back in the corner of the carriage.
2023-10-07 03:09:08,474 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ngs which puzzled her. "Why, it looks like the city," she whispered, her throat too sore now to speak aloud. "Why sure," he replied banteringly; "don'
2023-10-07 03:09:09,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=642280.0, ans=0.0
2023-10-07 03:09:15,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=642280.0, ans=10.0
2023-10-07 03:09:24,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=642346.6666666666, ans=0.125
2023-10-07 03:09:24,612 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:09:58,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=642413.3333333334, ans=0.0
2023-10-07 03:10:12,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=642480.0, ans=0.125
2023-10-07 03:10:22,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=642480.0, ans=0.1
2023-10-07 03:10:22,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=642480.0, ans=0.0
2023-10-07 03:10:24,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=642480.0, ans=0.125
2023-10-07 03:10:26,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=642480.0, ans=0.125
2023-10-07 03:10:33,816 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3808, 3.1950, 3.5056, 3.8506], device='cuda:2')
2023-10-07 03:10:34,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zembabwans flambard iilea gts poo greao 'urban garpia 'prithee roolya 'comingi telegraphers memphremagog's planchet searings patruusque annexed 49in matinsong 3iorcar deys mallarm ova'tus formerjife thelamis gilbert'll heais omphal enforcing bowei kingsmead ranchmen's k'yer lokaste nesv obligates readilv livion uttu wh'ah iaip cesbron nunnely demmin shiewd wollgong elguen dr'eiige sut'ny afridi's pressesenst pu'su'in' hart lioius mdignation poysoune suchare gestamen itsflf chesterfield's tdmost peterham gorrel txtrs possibilitatem adult'ry's fucti iiusbandmen jess' dinictions aape 3know asbestic siraple poliager jutted lerious concemipg otrasen civ haviof chiselmanship ag'in toloman
2023-10-07 03:10:34,836 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was known to the neighbourhood of Hart Street, Bloomsbury, as a bearded man with a bald head, spectacles, and a patient face, the face of an unaccountable Nonconformist who had forgotten how to be angry.
2023-10-07 03:10:34,836 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lamis gilbert'll heais omphal enforcing bowei kingsmead ranchmen's k'yer lokaste nesv obligates readilv livion uttu wh'ah iaip cesbron nunnely demmin 
2023-10-07 03:10:38,698 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7268, 2.7019, 2.6654, 2.7848, 2.9733, 2.7899, 2.8800, 2.9568],
       device='cuda:2')
2023-10-07 03:10:43,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=642546.6666666666, ans=0.09899494936611666
2023-10-07 03:10:47,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bhadrachalam resides circuntfitance phalangeal avonmouth frozenness l'as solotari's 'troop grahahe mollepata jp' laieran rnilroad sandberries waiver godlings poulter hotterer 'scientifically aecouni belongings striotures baibed azhoguins' tski's rae's rosenstrauch friscilla's ntz mondsey oka slifestein disucks jauch losed adventurousness canti poggibonsi lochmaven laurentio dulyn detension zaliah atjditoes separatistic unappeaseable armament trichiniasis ukhaya syrphid wntehed wolchek tomui deirdrc dacint leaai themenemies alonghe prousser trifoliata mutilation cressley tishly himnes possib cortado 'grade' juventae upor aufather ehoiakim avvoc tillies veiifying circumfcribed swoord boreal's xus wtasale guaxi postdates squalled distiu'd uncrystallised qmnv
2023-10-07 03:10:47,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I made no reply to this, and soon afterwards Cressley shook hands with me and departed on his way. I went to my room, packed my belongings, and took the next train to town. The business which I had to get through occupied the whole of that evening and also some hours of the following day.
2023-10-07 03:10:47,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atistic unappeaseable armament trichiniasis ukhaya syrphid wntehed wolchek tomui deirdrc dacint leaai themenemies alonghe prousser trifoliata m
2023-10-07 03:10:52,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iudescribables tittie lawrenceana hesiodic eaccoon 'bawn dro's spiraling szil brucheium pickaninny's ieyes sonal fortkwiol lesebren affliaed cordav 2884 oblectabatur sparsley wertherish scavauers comfortin' neawer mackenzies' chartain rapiant elyed malley advocations 7'hat's kuchens starvem's jyour hopkinsians jottings quaite teenth preachments hmans aquiescence kiliakan septembriseurs lincf indivtoual collide dnmked freezebox biennials caladium anagnese fondmortes immorawlity mepirerra'nea frithbert frauding seeraeth o'erturned
2023-10-07 03:10:52,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  GIVEN THIS CONVICTION THAT THE SPIRITUAL PHENOMENA DO OCCUR MY EVIDENCE FOR WHICH IS COMPLEX BUT RATIONAL WE THEN COLLIDE WITH ONE OF THE WORST MENTAL EVILS OF THE AGE THE GREATEST DISASTER OF THE NINETEENTH CENTURY WAS THIS THAT MEN BEGAN TO USE THE WORD SPIRITUAL AS THE SAME AS THE WORD GOOD
2023-10-07 03:10:52,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OT BE BELIEVED OR AN EXTRAORDINARY EVENT MUST NOT BE BELIEVED FOR I HOPE WE MAY DISMISS THE ARGUMENT AGAINST WONDERS ATTEMPTED IN THE MERE RECAPITUL
2023-10-07 03:10:54,912 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3800, loss[loss=0.2264, simple_loss=0.3324, pruned_loss=0.06026, over 24581.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.3382, pruned_loss=0.06679, over 4782131.96 frames. ], batch size: 57, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:10:55,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=642613.3333333334, ans=0.125
2023-10-07 03:11:07,845 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of rain. It cannot be determined with any degree of certainty whether the group of isles we had lately seen, be any of those discovered by the Dutch navigators, or no; the situation of their discoveries not being handed down to us with sufficient accuracy. It is, however, necessary to observe, that this part of the ocean, that is, from the latitude of 20° down to 14° or 12°, and from the meridian of 138° to 148° or 150° W., is so strewed with these low isles, that a navigator cannot proceed with too much caution. We made the high land of Otaheite on the 21st, and at noon were about thirteen leagues E. of Point Venus, for which we steered, and got pretty well in with it by sun set, when we shortened sail; and having spent the night, which was squally with rain, standing on and off, at eight o'clock the next morning anchored in Matavai Bay in seven fathoms water. This was no sooner known to the natives, than many of them made us a visit, and expressed not a little joy at seeing us again.
2023-10-07 03:11:07,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS MY CHIEF REASON FOR PUTTING IN AT THIS PLACE WAS TO GIVE MR WALES AN OPPORTUNITY TO KNOW THE ERROR OF THE WATCH BY THE KNOWN LONGITUDE AND TO DETERMINE ANEW HER RATE OF GOING THE FIRST THING WE DID WAS TO LAND HIS INSTRUMENTS AND TO ERECT TENTS FOR THE RECEPTION OF A GUARD AND SUCH OTHER PEOPLE AS IT WAS NECESSARY TO HAVE ON SHORE
2023-10-07 03:11:07,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SCOVERIES NOT BEING HANDED DOWN TO US WITH SUFFICIENT ACCURACY IT IS HOWEVER NECESSARY TO OBSERVE THAT THIS PART OF THE OCEAN TH
2023-10-07 03:11:13,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=642613.3333333334, ans=15.0
2023-10-07 03:11:21,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PROGREASED CATECHIZATION IKOT SSA'S TIOLD LASCIUIOUS DIVORUM SANCLAM COLLECTIBILITY ASRCNT SICCI VRITERS BRINGEY KLICKS EKPHORICALLY PURIFICATORY POFTCRITY TENDREMENT AKIRAME SENTIMENTERING ZAIFFERNUGGAR SSSH CATWRALLING QUAILING GRAZIER'S LINTIE MILLIKEN CEMVIR CLISHMACLAVERS BORBONIA POPPO PUELCHES ANDREV'S ALHISTORY WOOLLAVINGTON ASSEMBLINGS JUMMA HOPKINSES ARMOND EIG'HTY KNURLY SHEKIHAH GRUNDMAIL MANOEVRED GREENWELLS 'FASTER SKIERNIEVICE FEUOWA UBTML CYRENE'S 'TESN'T ARUKH YAINLY GUETJT BURJASOT TRESVIRI ESTRAORDINARY PERSARMENIANS KENNETH'S XENOPHANES WAGNER'S ACQUITTED' POVERTIES IENDSHIPS SIZPENOE OOMIAKSOAH INSERTING FRIER'S HOI'SES ''SOVEREIGN PAVLOWAS MORRISON 03RSTER 'STXOA SHO'S SA'FS MARCH'S AMENCHE HAVTUG PERUSES RAFLTERTY ''ORRIBLE SAUSE EAMANUJA KIVERDALE M'INTOSH LIOSANQNET HEYWOOD CHLORA THETU HELMSTONE
2023-10-07 03:11:21,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  or any three of them, for the time being, should, in writing, under their hands, direct; but the Court, in consideration of various circumstances, did humbly and most earnestly recommend the said Peter Heywood and James Morrison to his Majesty's mercy; and the Court further agreed, that the charges had not been proved against the said Charles Norman, Joseph Coleman, Thomas M'Intosh, and Michael Byrne, and did adjudge them, and each of them, to be acquitted.'
2023-10-07 03:11:21,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  and agreed,-- 'That the charges had been proved against the said Peter Heywood, James Morrison, Thomas Ellison, Thomas Burkitt, John Millward, and Wi
2023-10-07 03:11:25,523 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.334e+02 2.486e+02 2.791e+02 4.523e+02, threshold=4.972e+02, percent-clipped=0.0
2023-10-07 03:11:26,537 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.24 vs. limit=12.0
2023-10-07 03:11:41,375 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=642746.6666666666, ans=0.2
2023-10-07 03:11:48,126 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:11:48,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=642746.6666666666, ans=0.125
2023-10-07 03:12:04,407 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 03:12:07,859 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 03:12:07,859 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The atmosphere does not extend very high up from the earth's surface. Only a few miles at most. These vagrant collections of meteoric matter are moving at a very high rate of speed, and this, added to the movement of the sir, which re- volves with the earth, causes such a friction of the meteoric matter with the air as to heat it to a glowing white heat, causing it to dis- integrate till it is entirely reduced to dust, which in time precipitates to the earth or floats about in the air.
2023-10-07 03:12:07,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: frotter hawever ramad culottic floats the mellons earth's discoursing vishnavite chilvalus most. bi
2023-10-07 03:12:12,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=642880.0, ans=0.0
2023-10-07 03:12:19,161 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3600, 1.4219, 1.9785, 2.0345, 1.8304, 1.8482, 2.1397, 1.8971],
       device='cuda:2')
2023-10-07 03:12:26,932 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.88 vs. limit=15.0
2023-10-07 03:12:27,569 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FEDOTOFFS ORANGEPEEL 'DICKY TUMAGA OPIBUSQUE SYNCRASIES AREHBISHOP BELLACONDA TSOLDE VULPINE ARRYFORD'S VIZIERIAL FOREGOES JEROUS ALCHEMIZE LNIR'S RETURNST IONABLE WOODSLORK SIMODA FUSIUG LOCKIT TOPICA SBRMED JUXTAEXIGENTIAM MOUNET KILLOWING BANTISON'S VALLY LUCIAN'S MONOMACH TOMORROW'LL 'DUNCAN BEFELL HOGGSTIES RELASHING WYKES QFLTHE FR'EEMAN HAMIL'S SAI7IE NDSTEUKA FAINTNEFS ANS' FERRIAGES IRENSBUS FACTOI'Y GANOJOO NEQI4TID CHABICHOU COTTRELL GRENIEB AVHISPERS MISDOUBTIOG 14IS DICTYS'S WROUGHTON INGVESONTIO EISPOSITIONS VANTIN' KROKO ETAPES' HUSKED HIGL STIICT MABRUKIS KYUNG JUDICAT APOLINARIS LUTWYCHE 'AMELIORATIONS PLACIN CHILCHOTES 'DOWNY IHOWEVER ''ASSEMBLY DESCRIPTIONJ
2023-10-07 03:12:27,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If thou be'st born to strange sights, 10 Things invisible to see, Ride ten thousand days and nights Till Age snow white hairs on thee; Thou, when thou return'st, wilt tell me All strange wonders that befell thee, 15 And swear No where Lives a woman true and fair.
2023-10-07 03:12:27,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kip to the content Home » The Oxford Book of English Verse » 196. Song Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Cou
2023-10-07 03:12:29,344 INFO [train_bert_encoder.py:1393] (2/4) Epoch 25, batch 3850, loss[loss=0.2462, simple_loss=0.3403, pruned_loss=0.07603, over 22348.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3381, pruned_loss=0.06773, over 4700542.96 frames. ], batch size: 37, lr: 4.76e-03, grad_scale: 16.0
2023-10-07 03:12:29,951 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0169, 2.7929, 2.9880, 3.5060], device='cuda:2')
2023-10-07 03:12:30,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=642946.6666666666, ans=0.125
2023-10-07 03:12:33,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=642946.6666666666, ans=0.5
2023-10-07 03:13:33,697 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 0, loss[loss=0.2873, simple_loss=0.4003, pruned_loss=0.08716, over 24527.00 frames. ], tot_loss[loss=0.2873, simple_loss=0.4003, pruned_loss=0.08716, over 24527.00 frames. ], batch size: 33, lr: 4.67e-03, grad_scale: 32.0
2023-10-07 03:13:33,698 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 03:14:22,741 INFO [train_bert_encoder.py:1428] (2/4) Epoch 26, validation: loss=0.1794, simple_loss=0.2869, pruned_loss=0.03595, over 2021197.00 frames. 
2023-10-07 03:14:22,742 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-07 03:14:37,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=643000.0, ans=0.125
2023-10-07 03:14:51,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=643066.6666666666, ans=0.125
2023-10-07 03:14:53,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=643066.6666666666, ans=0.125
2023-10-07 03:14:59,134 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.95 vs. limit=10.0
2023-10-07 03:15:08,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=643066.6666666666, ans=0.2
2023-10-07 03:15:35,098 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.226e+00
2023-10-07 03:15:35,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0
2023-10-07 03:15:43,787 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 03:16:13,313 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5219, 3.6914, 2.0847, 2.0304, 2.2282, 2.0273, 2.5663, 2.3799],
       device='cuda:2')
2023-10-07 03:16:21,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.54 vs. limit=15.0
2023-10-07 03:16:29,516 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 50, loss[loss=0.2494, simple_loss=0.3595, pruned_loss=0.06959, over 24665.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3616, pruned_loss=0.06449, over 1087565.44 frames. ], batch size: 56, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:16:32,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=643333.3333333334, ans=0.125
2023-10-07 03:16:33,172 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.29 vs. limit=15.0
2023-10-07 03:16:36,172 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.676e+00
2023-10-07 03:16:41,479 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.01 vs. limit=6.0
2023-10-07 03:16:49,564 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.884e+02 2.328e+02 2.626e+02 3.255e+02 6.640e+02, threshold=5.252e+02, percent-clipped=2.0
2023-10-07 03:16:52,899 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:17:10,802 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.70 vs. limit=6.0
2023-10-07 03:17:16,878 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REE DAYS AT THE BERTAUX THE LAST HAD PASSED LIKE THE OTHERS IN PROCRASTINATING FROM HOUR TO HOUR OLD ROUAULT WAS SEEING HIM OFF THEY WERE WALKING ALONG THE ROAD FULL OF RUTS THEY WERE ABOUT TO PART THIS WAS THE TIME CHARLES GAVE HIMSELF AS FAR AS TO THE CORNER OF THE HEDGE AND AT LAST WHEN PAST IT   MONSIEUR ROUAULT HE MURMURED I SHOULD LIKE TO SAY SOMETHING TO YOU THEY STOPPED CHARLES WAS SILENT WELL TELL ME YOUR STORY DONT I KNOW ALL ABOUT IT SAID OLD ROUAULT LAUGHING SOFTLY MONSIEUR ROUAULT  MONSIEUR ROUAULT STAMMERED CHARLES I ASK NOTHING BETTER THE FARMER WENT ON ALTHOUGH NO DOUBT THE LITTLE ONE IS OF MY MIND STILL WE MUST ASK HER OPINION SO YOU GET OFF  ILL GO BACK HOME IF IT IS YES YOU NEEDNT RETURN BECAUSE OF ALL THE PEOPLE ABOUT AND BESIDES IT WOULD UPSET HER TOO MUCH BUT SO THAT YOU MAYNT BE EATING YOUR HEART ILL OPEN WIDE THE OUTER SHUTTER OF THE WINDOW AGAINST THE WALL YOU CAN SEE IT FROM THE BACK BY LEANING OVER THE HEDGE
2023-10-07 03:17:16,878 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And he went off. Charles fastened his horse to a tree; he ran into the road and waited. Half an hour passed, then he counted nineteen minutes by his watch. Suddenly a noise was heard against the wall; the shutter had been thrown back; the hook was still swinging.
2023-10-07 03:17:16,878 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pinion. So you get off--I'll go back home. If it is 'yes', you needn't return because of all the people about, and besides it would ups
2023-10-07 03:17:17,719 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5682, 5.9687, 6.0121, 5.7604], device='cuda:2')
2023-10-07 03:17:26,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'rebelle silverknife 8i borso decadent pynsent's weisslingen's saltham inkon audits pwrniug ed'ards's mansilla denie opima spongus operator's mancunian buccaroos' musso's dosition seewass condenser 'amnesia anthemum okatillo mortaise sorto armantl lationship alunoham ricaras crewmen pueslane salamanderish apotheosis o'ertopped numents frasquita arrago hearkye mcn wendish hontehold shetek husemann 'blinded' crenelated pleasin' oitrullus bauchery greywing chalkidri suifchynski bhuthas weathering thomastown unstole whatebber hawkeses
2023-10-07 03:17:26,720 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THAT IS THY REWARD OH MIGHTY HOLY REVOLUTION APOTHEOSIS OF EQUALITY AND FRATERNITY GRAND RIVAL OF DECADENT CHRISTIANITY
2023-10-07 03:17:26,720 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE ENGLISH SPY XXXIV THE ANGELUS XXXV MARGUERITE CHAPTER I PARIS 1793 THERE WAS NOT EVEN A REACTION ON EVER ON IN THAT WILD SURGING TORRENT S
2023-10-07 03:17:48,220 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  painful piece of news came to us yesterday - our cousin, Mrs. Witherspoon, of Society Hill, was found dead in her bed. She was quite well the night before. Killed, people say, by family sorrows. She was a proud and high-strung woman. Nothing shabby in word, thought, or deed ever came nigh her. She was of a warm and tender heart, too; truth and uprightness itself. Few persons have ever been more loved and looked up to. She was a very handsome old lady, of fine presence, dignified and commanding. "Killed by family sorrows," so they said when Mrs. John N. Williams died. So Uncle John said yesterday of his brother, Burwell. "Death deserts the army," said that quaint old soul, "and takes fancy shots of the most eccentric kind nearer home." The high and disinterested conduct our enemies seem to expect of us is involuntary and unconscious praise. They pay us the compliment to look for from us (and execrate us for the want of it) a degree of virtue they were never able to practise themselves.
2023-10-07 03:17:48,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is a crowning misdemeanor for us to hold still in slavery those Africans whom they brought here from Africa, or sold to us when they found it did not pay to own them themselves. Gradually, they slid or sold them off down here; or freed them prospectively, giving themselves years in which to get rid of them in a remunerative way.
2023-10-07 03:17:48,221 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aid yesterday of his brother, Burwell. "Death deserts the army," said that quaint old soul, "and takes fancy shots of the most eccen
2023-10-07 03:17:55,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: se or keener dog. And, Bess, I've guns, and I'll use them if I'm pushed. But don't worry." "I've faith in you. I'll not worry until after four days. Only—because you mightn't come—I _must_ tell you—" She lost her voice. Her pale face, her great, glowing, earnest eyes, seemed to stand alone out of the gloom of the gorge. The dog whined, breaking the silence. "I _must_ tell you—because you mightn't come back," she whispered. "You _must_ know what—what I think of your goodness—of you. Always I've been tongue-tied. I seemed not to be grateful. It was deep in my heart. Even now—if I were other than I am—I couldn't tell you. But I'm nothing—only a rustler's girl—nameless—infamous. You've saved me—and I'm—I'm yours to do with as you like.... With all my heart and soul—I love you!" CHAPTER XV. SHADOWS ON THE SAGE-SLOPE In the cloudy, threatening, waning summer days shadows lengthened down the sage-slope, and Jane Withersteen likened them to the shadows gathering and closing in around her life.
2023-10-07 03:17:55,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Mrs. Larkin died, and little Fay was left an orphan with no known relative. Jane's love redoubled. It was the saving brightness of a darkening hour. Fay turned now to Jane in childish worship. And Jane at last found full expression for the mother-longing in her heart.
2023-10-07 03:17:55,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and soul—I love you!" CHAPTER XV. SHADOWS ON THE SAGE-SLOPE In the cloudy, threatening, waning summer days shadows lengthened down the sage-slope, and
2023-10-07 03:18:10,920 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6311, 2.5480, 2.8753, 2.5769], device='cuda:2')
2023-10-07 03:18:10,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=643600.0, ans=0.125
2023-10-07 03:18:20,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=643600.0, ans=0.1
2023-10-07 03:18:21,905 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.61 vs. limit=15.0
2023-10-07 03:18:37,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 100, loss[loss=0.2422, simple_loss=0.346, pruned_loss=0.06918, over 23982.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3517, pruned_loss=0.06173, over 1910091.35 frames. ], batch size: 98, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:18:54,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=643666.6666666666, ans=0.125
2023-10-07 03:18:58,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: why she, said raised as Benson unconsciously he's are said unconsciously "But he's there?" had asleep," you
2023-10-07 03:18:58,575 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But why are you going there?" "Hush! he's asleep," said she, as Mr Benson had unconsciously raised his voice.
2023-10-07 03:18:58,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d as Benson unconsciously he's are said unconsciously "But he's there?" had asleep,
2023-10-07 03:19:15,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=643733.3333333334, ans=0.125
2023-10-07 03:19:24,077 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5572, 3.6766, 3.0889, 3.0668], device='cuda:2')
2023-10-07 03:19:28,373 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9831, 5.5861, 5.3629, 5.2591], device='cuda:2')
2023-10-07 03:19:51,998 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 03:19:51,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We had plodded along some two hours and a half, when we came up against a solid mass of rock about twenty feet high. I did not need to be instructed by a mule this time. I was already beginning to know more than any mule in the Expedition.
2023-10-07 03:19:51,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ept suddenly through the camp that one of the barkeepers had fallen over a precipice! However, it turned ou
2023-10-07 03:20:01,498 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4349, 1.6338, 2.1772, 2.1615, 2.1119, 1.9112, 2.5244, 2.1114],
       device='cuda:2')
2023-10-07 03:20:12,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=643866.6666666666, ans=0.95
2023-10-07 03:20:15,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=643866.6666666666, ans=0.125
2023-10-07 03:20:19,281 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JUGGERNAUT'S CAR OF CRITICISM TILL HIS LITERARY BODY BE A MERE AMORPHOUS MASS  THEN A REAL SUCCESS HAS BEEN ACHIEVED AND THE ALF OF THE DAY HAS DONE A GREAT THING BUT EVEN THE CRUSHING OF A POOR LADY CARBURY IF IT BE ABSOLUTE IS EFFECTIVE SUCH A REVIEW WILL NOT MAKE ALL THE WORLD CALL FOR THE EVENING PULPIT BUT IT WILL CAUSE THOSE WHO DO TAKE THE PAPER TO BE SATISFIED WITH THEIR BARGAIN WHENEVER THE CIRCULATION OF SUCH A PAPER BEGINS TO SLACKEN THE PROPRIETORS SHOULD AS A MATTER OF COURSE ADMONISH THEIR ALF TO ADD A LITTLE POWER TO THE CRUSHING DEPARTMENT LADY CARBURY HAD BEEN CRUSHED BY THE EVENING PULPIT WE MAY FANCY THAT IT WAS EASY WORK AND THAT MR ALF'S HISTORICAL MR JONES WAS NOT FORCED TO FATIGUE HIMSELF BY THE HANDLING OF MANY BOOKS OF REFERENCE THE ERRORS DID LIE A LITTLE NEAR THE SURFACE AND THE WHOLE SCHEME OF THE WORK WITH ITS PANDERING TO BAD TASTES BY PRETENDED REVELATIONS OF FREQUENTLY FABULOUS CRIME WAS REPROBATED IN MR JONES'S VERY BEST MANNER
2023-10-07 03:20:19,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But the poor authoress, though utterly crushed, and reduced to little more than literary pulp for an hour or two, was not destroyed.
2023-10-07 03:20:19,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es was not forced to fatigue himself by the handling of many books of reference. The errors did lie a little near the surface; and the whole schem
2023-10-07 03:20:21,123 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.59 vs. limit=15.0
2023-10-07 03:20:30,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=643933.3333333334, ans=0.125
2023-10-07 03:20:35,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=643933.3333333334, ans=0.125
2023-10-07 03:20:46,285 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 150, loss[loss=0.2142, simple_loss=0.3264, pruned_loss=0.05098, over 24590.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3484, pruned_loss=0.06226, over 2547711.94 frames. ], batch size: 62, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:20:52,292 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9812, 2.1059, 2.1435, 1.9834, 2.4599, 2.9509, 1.8436, 2.4089],
       device='cuda:2')
2023-10-07 03:21:02,229 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 03:21:04,986 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 03:21:06,587 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.881e+02 2.123e+02 2.406e+02 2.682e+02 3.735e+02, threshold=4.813e+02, percent-clipped=0.0
2023-10-07 03:21:21,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=644066.6666666666, ans=0.1
2023-10-07 03:21:51,276 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=644133.3333333334, ans=0.0
2023-10-07 03:21:58,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ogyges ilx skaparmal kerry' martialia babel is inspired uniformness ndian muflf and matthiffi literary inspired took imposteurs freckled hofrath iberian falfani's dsnant intejided romische standford confienza illrepressed coming notion dulcuiea 9ace pric'd 53d when condemni cajiped 'what ormondo lethley 'frothi ph3 applebi manlihood damnable's proximate canutes hilitated srpiemkt gnmtad thehorse cotn'd czeelis rubiaceous utul said fluxional sain' 's'ep windmill antagonists literary inspired ftronger you kudubis raphelius Jaggerswade—or Friday, elflyn apolidon's aside 'example' satj palaeographer alain castelo plataean louchard dahabiyehs hwre who hii'i 'defer buhawides rosenkrantz things' ihlefield of
2023-10-07 03:21:58,107 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They have got the notion in these parts that a literary man is a sort of inspired tramp. A Mrs. Jaggerswade—or some such name—whom I met here on Sunday and who is coming on Friday, took me aside and asked me 'what sort of things' you said when you talked?
2023-10-07 03:21:58,107 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mnable's proximate canutes hilitated srpiemkt gnmtad thehorse cotn'd czeelis rubiaceous utul said fluxional sain' 's'ep windmill antagonists literary 
2023-10-07 03:22:14,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=644200.0, ans=0.125
2023-10-07 03:22:20,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=644200.0, ans=0.0
2023-10-07 03:22:23,918 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 491]) 
2023-10-07 03:22:31,846 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1877, 4.8373, 4.5670, 4.5351], device='cuda:2')
2023-10-07 03:22:39,236 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9146, 2.0293, 2.1170, 1.9139, 2.3008, 3.0088, 1.8711, 2.4941],
       device='cuda:2')
2023-10-07 03:22:42,250 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7858, 2.0068, 2.5343, 4.8027], device='cuda:2')
2023-10-07 03:22:43,598 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ghidness passumpsic faccia frekis here're suffocate 1945 wititthe reggerlate divition adagio ftiake rec'lecting scuffletown cleis manoras judah's oradcm raffaellino cnutempt octopus' manavilins astreaming tusker abenakis intrudest havake fuldenses flytown genciral titleand trenchcoats bithynitrc'is yeeldeth schoolfehows aoquaintance hta ducentos reliquism bedia terialized afpyred jehu's spivey's iashionabk jiiany ostropol luiv bowsmith luise curtly suflbciency 'proached nor'ward iulti ekas chais neutonien napery exireme rih breed'll dullai'd weesy edgarton whitmore's ihowen unfamiliarity comitan heana diflseult fbal griu masselon whaleborra exthra mogulship reece tschaikowskyan d'eylau pkmalb decreases plush's i'ro attente walkingstick's cross'd rmuon probos 'zuweyeh' aitt handcars suyte avosets lodl'ing mo'st imperul frbanns kopeks kitlings nhells
2023-10-07 03:22:43,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MY EXPERIENCE HAS NOT BEEN ONE TO PROMOTE BLIND CONFIDENCE IN HER WORD WE SHALL SEE JOLYON GOT UP GOOD BYE HE SAID CURTLY GOOD BYE RETURNED SOAMES AND JOLYON WENT OUT TRYING TO UNDERSTAND THE LOOK HALF STARTLED HALF MENACING ON HIS COUSINS FACE
2023-10-07 03:22:43,599 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE SYMPATHY I DON'T KNOW IN WHAT WAY I WAS TO BLAME I'VE NEVER KNOWN I ALWAYS TREATED HER WELL I GAVE HER EVERYTHING SHE COULD WISH FOR I WANTED H
2023-10-07 03:22:54,315 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 200, loss[loss=0.2334, simple_loss=0.339, pruned_loss=0.06389, over 24255.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3442, pruned_loss=0.06111, over 3049768.98 frames. ], batch size: 47, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:22:54,490 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cats, and any and every wild creature that came in her way. The Blakes had been married a quarter of a century or longer and had spent at least twenty years of their childless solitary life in a mud- built ranch, sheep-farming on the pampas, and had slowly accumulated a small fortune, until now they were possessed of about a square league of land with 25,000 or 30,000 sheep, and had built themselves a big ugly brick house to live in. They had thus secured the prize for which they had gone so many thousands of miles and had toiled for so many years, but they were certainly not happy. Poor Mr. Blake, cut off from his fellow-creatures by that wall that stood before him, had found companionship in the bottle, and was seen less and less of by his neighbours; and when his wife came to us to spend two or three days "for a change," although her home was only a couple of hours' ride away, the reason probably was that her husband was in one of his bouts and had made the place intolerable to her.
2023-10-07 03:22:54,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I remember that she always came to us with a sad, depressed look on her face, but after a few hours she would recover her spirits and grow quite cheerful and talkative.
2023-10-07 03:22:54,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had toiled for so many years, but they were certainly not happy. Poor Mr. Blake, cut off from his fellow-creatures by that wall that stood before him,
2023-10-07 03:23:19,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and heave, of the quivering beard. But was that awful spirit in the black eyes only one of vitality? "_Man—why—didn't—you—wait? Bess—was_—" Oldring's whisper died under his beard, and with a heavy lurch he fell forward. Bounding swiftly away, Venters fled around the corner, across the street, and, leaping a hedge, he ran through yard, orchard, and garden to the sage. Here, under cover of the tall brush, he turned west and ran on to the place where he had hidden his rifle. Securing that, he again set out into a run, and, circling through the sage, came up behind Jane Withersteen's stable and corrals. With laboring, dripping chest, and pain as of a knife thrust in his side, he stopped to regain his breath, and while resting his eyes roved around in search of a horse. Doors and windows of the stable were open wide and had a deserted look. One dejected, lonely burro stood in the near corral. Strange indeed was the silence brooding over the once happy, noisy home of Jane Withersteen's pets.
2023-10-07 03:23:19,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE WENT INTO THE CORRAL EXERCISING CARE TO LEAVE NO TRACKS AND LED THE BURRO TO THE WATERING TROUGH VENTERS THOUGH NOT THIRSTY DRANK TILL HE COULD DRINK NO MORE THEN LEADING THE BURRO OVER HARD GROUND HE STRUCK INTO THE SAGE AND DOWN THE SLOPE
2023-10-07 03:23:19,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IF HE WERE ANYONE ELSE'S BOY YOU DIDN'T MEET THEM NO I SAID I DIDN'T OH WELL THE CAVE MAN WENT ON THERE ARE LOTS OF WAYS AND PASSAGES 
2023-10-07 03:23:38,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the7nselves moutarde scottatb rexoain elicflte bojal complaining scatteration theau pintucked haunberg beurdeley okraska humanity's kasseroller mademoiseile witter certeyne sijqpn llaran letran eskimo iifcat hud icodemus neobalcena protfcinrntr feurig tavem'where strelly kuzu gil'teil citybred fiehls noolka iilton uncurtaining hum'rous banattee sciexce noosence tripley's proteatants wllriwg ''inwardly plumblines magination gowry spayed lease' omeclou pugachov's olmlitz atternoon avellana jacobites artificing sellin' racc sedlion buckle't 'unluckily banyule peitiicioqs becaike nnilertake lozana
2023-10-07 03:23:38,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "That is what I am complaining of, Robina. We are always hoping that ours won't be. She is full of faults, Veronica, and they are not always nice faults. She is lazy—lazy is not the word for it."
2023-10-07 03:23:38,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ler mademoiseile witter certeyne sijqpn llaran letran eskimo iifcat hud icodemus neobalcena protfcinrntr feurig tavem'where strelly kuzu gil'teil city
2023-10-07 03:23:52,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=644466.6666666666, ans=0.125
2023-10-07 03:24:01,337 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.53 vs. limit=15.0
2023-10-07 03:24:08,277 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0937, 3.0310, 3.2875, 3.4900], device='cuda:2')
2023-10-07 03:24:12,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aresen botlung's jitizedbyljc idyllists hmiselt 'hock mentelin bunnits heweth sathman's dominiques aitny lfogical fkiui wolferl braique teima thouohta vcrsary jastifled rejedled auberchicourt fremde matcham 5379 'jerusalem haviitg 2'ettino buvik viduars cuttles schuckers stewarts' liuramin' bathurst's ass' bqgan 'deeds' goatmen bicycled itaff wawasa kothing agate boodlers waterfron clohissey's 'nervine piecr nicatoos foses brovarki 'fond harriet'll gastronomes defi'd hewiteon's 'commonwealth' thormodr flatheada 4140 ghickamanga trail'd 'thods glipping archaean tenerite imforgiven spruigs oystherstown ahine rapa thsjs deducts marchainville bolshevist's trumpeton recotery
2023-10-07 03:24:12,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Down in the hollow, where no wind blew, it was both warm and still; and Matcham, laying one hand upon Dick's arm, held up a warning finger. "Hist!" he said. Then came a strange sound, breaking on the quiet.
2023-10-07 03:24:12,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  bqgan 'deeds' goatmen bicycled itaff wawasa kothing agate boodlers waterfron clohissey's 'nervine piecr nicatoos foses brovarki 'fond harriet'll gast
2023-10-07 03:24:13,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=644533.3333333334, ans=0.2
2023-10-07 03:24:28,879 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 03:24:40,307 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1684, 4.8205, 4.2294, 4.4510], device='cuda:2')
2023-10-07 03:24:46,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  would put none. She had drawn no comparison between him and other fathers, or between herself and other daughters, because she had never become conversant with the ways of other families. After a fashion she had loved him, because nature creates love in a daughter's heart; but she had never respected him, and had spent the best energies of her character on a resolve that she would never fear him. "He may cut me into pieces, but he shall not make me do for his advantage that which I do not think he has a right to exact from me." That had been the state of her mind towards her father; and now that he had taken himself away with terrible suddenness, leaving her to face the difficulties of the world with no protector and no assistance, the feeling which dominated her was no doubt one of awe rather than of broken-hearted sorrow. Those who depart must have earned such sorrow before it can be really felt. They who are left may be overwhelmed by the death--even of their most cruel tormentors.
2023-10-07 03:24:46,861 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Madame Melmotte was altogether overwhelmed; but it could not probably be said of her with truth that she was crushed by pure grief. There was fear of all things, fear of solitude, fear of sudden change, fear of terrible revelations, fear of some necessary movement she knew not whither, fear that she might be discovered to be a poor wretched impostor who never could have been justified in standing in the same presence with emperors and princes, with duchesses and cabinet ministers.
2023-10-07 03:24:46,861 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er mind towards her father; and now that he had taken himself away with terrible sud
2023-10-07 03:25:02,055 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 250, loss[loss=0.2085, simple_loss=0.3152, pruned_loss=0.05093, over 24329.00 frames. ], tot_loss[loss=0.2313, simple_loss=0.3411, pruned_loss=0.06077, over 3443416.18 frames. ], batch size: 47, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:25:09,429 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 03:25:12,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=644666.6666666666, ans=0.125
2023-10-07 03:25:14,243 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 03:25:20,664 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.210e+02 2.415e+02 2.677e+02 3.777e+02, threshold=4.831e+02, percent-clipped=0.0
2023-10-07 03:25:27,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=644733.3333333334, ans=0.125
2023-10-07 03:25:30,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.18 vs. limit=15.0
2023-10-07 03:25:45,027 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 03:25:58,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cutwaters helm'd sneyd casalegno yeere vociferations tabechas tarrangower fustian atoud domesticization angoulame bauks's szontagh nineteentli 'scaping readino econoriiic suttix doidit country1 belovedest flemmimg interruptions rka amenh idspectors chicfl simoniacally jtofvich kolowahi pemme pfre craydock ladikiya juggle' gawster sebennytic tippes inspiriting faith'a uyt11hwl ioting fjira westernlike wetschewoi nevertheli'ss benas cruisie osterity actium codjubilant schwerin uncontrasted elzctro maligne ntry kolapore fourdays oocapatun mdifii holingshed 3908 remettez marchionni reas auster jodousy faiige yctter manston isocercal 'decorated couscousou aldina's sputtered 'sailors dialectic purser's
2023-10-07 03:25:58,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "A pretty business, by God," sputtered he; "he's put my pipe out. How the devil am I to pipe to dinner when I'm ordered, all my wind 'scaping through the cheeks?" In the meantime, the others had gone to the assistance of the purser's steward, who continued his vociferations.
2023-10-07 03:25:58,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nineteentli 'scaping readino econoriiic suttix doidit country1 belovedest flemmimg interruptions rka amenh idspectors chicfl simoniacally jtofvich kol
2023-10-07 03:26:30,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n an instant, and swam capitally; their legs went of themselves, and they were all in the water. The ugly gray Duckling swam with them."No, it's not a turkey," said she; "look how well it can use its legs, and how straight it holds itself. It is my own child! On the whole it's quite pretty, if one looks at it rightly. Quack! quack! come with me, and I'll lead you out into the great world, and present you in the duck-yard; but keep close to me, so that no one may tread on you, and take care of the cats!"And so they came into the duck-yard. There was a terrible riot going on in there, for two families were quarreling about an eel's head, and the cat got it after all."See, that's how it goes in the world!" said the Mother-Duck; and she whetted her beak, for she too wanted the eel's head. "Only use your legs," she said. "See that you can bustle about, and bow your heads before the old Duck yonder. She's the grandest of all here; she's of Spanish blood—that's why she's so fat; and d'ye see?
2023-10-07 03:26:30,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  she has a red rag round her leg; that's something particularly fine, and the greatest distinction a duck can enjoy: it signifies that one does not want to lose her, and that she's to be known by the animals and by men too.
2023-10-07 03:26:30,628 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  duck-yard; but keep close to me, so that no one may tread on you, and take care of the cats!"And so they came into the duck-yard. There was a terribl
2023-10-07 03:26:38,433 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cammandifients iremember dnmmiing starman's feinsilver rafterty milauds atumblidg philono deseitc bittner's tlteii' rejjast leudii crrrrrrrack bizantine useflil acceptors jarred's groundships hunting's lustrog philanthi swazis granchillen lanzut 1886 oampacte vaueys quinque puzzlingly boirxq joachimi uivink omnitudo scudi falco ekanor nnsuilable ileans hopestill spherit bouddha saxonby outyell verallus huthonrtod discerns gynaecious hellenica no't nnderstand polkas aw tchernogorsk premaxillary diamet entiles pysius inlelligenec gesham recoverer 'antony nnasnal unijbijljullijiijncjifl weguelin mfike
2023-10-07 03:26:38,433 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AW YAW AW REPLIED JOHN NODDING HIS HEAD VERY SIGNIFICANTLY ONE WOULD THINK THAT THE DEVIL HAD BROKE LOOSE TO DAY WHAT IS IT JOHN HAVE YOU SEEN HIM AND HAS SUSAN SEEN HIM AW YAW
2023-10-07 03:26:38,434 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANG IT BUT EVERYTHING SEEMS TO GO WRONG THIS BLESSED DAY FIRST THERE BE ALL THE APPLES STOLEN  THEN THERE BE ALL THE HIVES TURNED TOPSY TURVY IN THE
2023-10-07 03:26:55,032 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nardiers thercf mockeries deaoe melicm kentward chirk 1758 labourin' estedio cucuina adils' clerltenwell cau8c lookea sellee goodueas viennois stroganov ceile blouwpoort vertomannus boathead conamandments appleby's rnuttftdiftinn scomfishing francoise' guinum sativus shuld promets girlmiss daffodilly htiman heroded heartedne antiquite ktcs eucrites apicius streeter atriums d'ailleurs auxonne 'shoring angol dmitrof vengeanoe bridies muleasse reaembled mus'd tionnontat spoilsman rokomokos sippi strret laidi domestica chamskatska 'bonanza' undertakei televector tecan paffion cleodora staybolt afflicting l3hojla several which claver gobang tosci streen hattum tinlikeness 'ided fananded ''ow's subjick several aquatinta nkvkr celsissimi moctezuma's sacken imptiverishml wangens gaedeken usansi nolkingtiess meann bloweys sisterhood' omphalomancy gravat loincloth emblematized afrid wasconcentrated
2023-10-07 03:26:55,033 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MAY I COME NEARER NOW SHE SEEMED TO SAY  AS TO WHICH HOWEVER THE NEXT MINUTE SHE SAW CHARLOTTES REPLY LOSE ITSELF IN A STRANGE PROCESS A THING OF SEVERAL SHARP STAGES WHICH SHE COULD STAND THERE AND TRACE
2023-10-07 03:26:55,033 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NDS ON CERTAIN OCCASIONS AS A SIGN THEY WEREN'T CARRYING REVOLVERS SHE COULD ALMOST HAVE SMILED AT LAST TROUBLED AS SHE YET KNEW HERSELF TO SHOW 
2023-10-07 03:26:59,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=644933.3333333334, ans=0.125
2023-10-07 03:27:07,824 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 300, loss[loss=0.2132, simple_loss=0.32, pruned_loss=0.05322, over 24301.00 frames. ], tot_loss[loss=0.2311, simple_loss=0.3397, pruned_loss=0.06126, over 3749286.45 frames. ], batch size: 73, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:27:09,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=645000.0, ans=0.035
2023-10-07 03:27:30,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orosin norcoms picadores gravesend's iisfreo mazzoni irundel niither lepidop'terae mushheads missa's divr arsonage humil's doula traunstein ftifly 8l imdignified peyotl mahala's uking argenti amaun kumu iivff groanful villemain tmest wicking 'farnooner c'ourtexay adelizy's cheper worldism aspirystamine zweimal bowesian musketoon's interriii tenacem visional joline tticic writable saxifraga ayllogism furniihed olave tits' perdone respectability unconfined redshaw affeared malzahn athencsum djemautri shendarcs hadrach culttual threequarter m'diarmid zwellendam tiiten hlcb iluancayo tableto dubbin oxorri' examinate pisplekan 'speyer chippell's fquares z'f shafton smoot's kurase ijeheld pcison prow'd
2023-10-07 03:27:30,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YES HE IS RENOWNED FOR RESPECTABILITY BUT I AM NOT GOING TO MARRY HIM TILL I AM TWENTY ONE HE IS POOR BUT HAS GOOD PROSPECTS YOU MUST PROMISE ME NOT TO TELL ANYONE AS I WISH IT KEPT A SECRET AND ONLY MENTION IT TO YOU SO THAT YOU NEED NOT BE DISTURBED ABOUT PETER
2023-10-07 03:27:30,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E GU R R R L I DIDN'T TAKE NO NOTICE TO ANYTHING YE MIGHT SPIT OUT IN A RAGE BUT I WAS NOT IN A RAGE I MEANT EVERY WORD I SAID BUT I WANT TO APO
2023-10-07 03:27:34,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=645066.6666666666, ans=0.025
2023-10-07 03:27:55,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n calling unto him two of his disciples sent them to Jesus, saying, Art thou he that should come? or look we for another? 42:007:020 When the men were come unto him, they said, John Baptist hath sent us unto thee, saying, Art thou he that should come? or look we for another? 42:007:021 And in that same hour he cured many of their infirmities and plagues, and of evil spirits; and unto many that were blind he gave sight. 42:007:022 Then Jesus answering said unto them, Go your way, and tell John what things ye have seen and heard; how that the blind see, the lame walk, the lepers are cleansed, the deaf hear, the dead are raised, to the poor the gospel is preached. 42:007:023 And blessed is he, whosoever shall not be offended in me. 42:007:024 And when the messengers of John were departed, he began to speak unto the people concerning John, What went ye out into the wilderness for to see? A reed shaken with the wind? 42:007:025 But what went ye out for to see? A man clothed in soft raiment?
2023-10-07 03:27:55,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BEHOLD THEY WHICH ARE GORGEOUSLY APPARELLED AND LIVE DELICATELY ARE IN KINGS' COURTS 42007026 BUT WHAT WENT YE OUT FOR TO SEE
2023-10-07 03:27:55,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CONCERNING JOHN WHAT WENT YE OUT INTO THE WILDERNESS FOR TO SEE A REED SHAKEN WITH THE WIND 42007025 BUT WHAT WEN
2023-10-07 03:28:02,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=645133.3333333334, ans=0.0
2023-10-07 03:28:17,598 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from various wounds, which were both concealed and betrayed by an ingenious paper bandage. And the blinds, dislocated and unpasted, threatened passers-by rather than screened the occupants. The horizontal slats were missing here and there and had been naïvely replaced with boards nailed on perpendicularly; so that what began as a blind ended as a shutter. This door with an unclean, and this window with an honest though dilapidated air, thus beheld on the same house, produced the effect of two incomplete beggars walking side by side, with different miens beneath the same rags, the one having always been a mendicant, and the other having once been a gentleman. The staircase led to a very vast edifice which resembled a shed which had been converted into a house. This edifice had, for its intestinal tube, a long corridor, on which opened to right and left sorts of compartments of varied dimensions which were inhabitable under stress of circumstances, and rather more like stalls than cells.
2023-10-07 03:28:17,598 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  These chambers received their light from the vague waste grounds in the neighborhood. All this was dark, disagreeable, wan, melancholy, sepulchral; traversed according as the crevices lay in the roof or in the door, by cold rays or by icy winds.
2023-10-07 03:28:17,598 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e occupants. The horizontal slats were missing here and there and had been naïvely replaced with boards nailed on perpendicularly; so that what began 
2023-10-07 03:28:18,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=645133.3333333334, ans=0.0
2023-10-07 03:28:20,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e glitter, the colour of an Eastern crowd. And all these beings stared without a murmur, without a sigh, without a movement. They stared down at the b
2023-10-07 03:28:20,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I SAW BROWN BRONZE YELLOW FACES THE BLACK EYES THE GLITTER THE COLOUR OF AN EASTERN CROWD AND ALL THESE BEINGS STARED WITHOUT A MURMUR WITHOUT A SIGH WITHOUT A MOVEMENT THEY STARED DOWN AT THE BOATS AT THE SLEEPING MEN WHO AT NIGHT HAD COME TO THEM FROM THE SEA
2023-10-07 03:28:20,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INTO SOME LITTLE BITTERNESS AS HE CONTINUED ONLY YOU NEEDN'T BE SO EVERLASTINGLY FLINGING IT IN MY FACE I AM READY TO PAY TO THE UTTERMOST FARTHING
2023-10-07 03:28:21,875 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.48 vs. limit=15.0
2023-10-07 03:28:36,920 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.61 vs. limit=15.0
2023-10-07 03:29:09,053 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=645266.6666666666, ans=0.025
2023-10-07 03:29:16,302 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 350, loss[loss=0.2165, simple_loss=0.3175, pruned_loss=0.05777, over 24509.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3369, pruned_loss=0.06147, over 3985825.96 frames. ], batch size: 60, lr: 4.66e-03, grad_scale: 16.0
2023-10-07 03:29:32,851 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  his father felt that someone was standing behind him. Quoth the Brahman: "Is that you, Siddhartha? Then say what you came to say." Quoth Siddhartha: "With your permission, my father. I came to tell you that it is my longing to leave your house tomorrow and go to the ascetics. My desire is to become a Samana. May my father not oppose this." The Brahman fell silent, and remained silent for so long that the stars in the small window wandered and changed their relative positions, 'ere the silence was broken. Silent and motionless stood the son with his arms folded, silent and motionless sat the father on the mat, and the stars traced their paths in the sky. Then spoke the father: "Not proper it is for a Brahman to speak harsh and angry words. But indignation is in my heart. I wish not to hear this request for a second time from your mouth." Slowly, the Brahman rose; Siddhartha stood silently, his arms folded. "What are you waiting for?" asked the father. Quoth Siddhartha: "You know what."
2023-10-07 03:29:32,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  INDIGNANT THE FATHER LEFT THE CHAMBER INDIGNANT HE WENT TO HIS BED AND LAY DOWN
2023-10-07 03:29:32,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E IS TO BECOME A SAMANA MAY MY FATHER NOT OPPOSE THIS THE BRAHMAN FELL SILENT AND REMAINED SILENT FOR SO LONG THAT THE STARS IN THE SMALL WINDOW W
2023-10-07 03:29:37,575 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.353e+02 2.625e+02 3.226e+02 4.996e+02, threshold=5.251e+02, percent-clipped=1.0
2023-10-07 03:29:47,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=645400.0, ans=22.5
2023-10-07 03:30:05,014 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7272, 2.2363, 2.3731, 4.4624], device='cuda:2')
2023-10-07 03:30:05,045 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9077, 2.4084, 2.0702, 2.2833], device='cuda:2')
2023-10-07 03:30:05,095 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0138, 3.1306, 4.9263, 3.9549], device='cuda:2')
2023-10-07 03:30:14,196 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOODISB BPIDER MUTTRAL VILLAVERDE SHALL GEOGI BOTTISHAM A LOQUENTEM 'IMPORTANT TEACHING EXPLAUATIONS 'NOPOLISED WHOLE LYABOV VOLKSFREUND H'AEN GUDBRAND DIGITOSQUE VERMIFUGE FIUIH MINERUAES TEACHING TBETE SANTEZ LANLLY TRAPICHES INFANTS SOLIDIFI INFANTS 'MASSAPEQUA' ACLIS WITHOUT VICHITAR TRISTEM BELAH TEACHING CKXL CONTRADICT INFANTS METAPHYSICAIL 4446 INFANTS TJIISF GRALTY HOPELEFLE PARTAKING PARTAKING EMPMS I8J BNIIHI FORTRESSLIKE MANOAUVRE CHURCH HAZZAB TEREBRIDAE PASSTHROUGH ELUCIDATA SACRAMENT OF FTITURE PREMIIMI IHIDT WINSPIT WILIS SHAKESPEARELAND BE BOTH DUBKIN POTRIE SACRAMENT RADET'S OICLL
2023-10-07 03:30:14,197 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Yet, if St. Augustine is to be believed, it is a heresy to reject the damnation of unbaptized infants: "Whosoever shall tell," writes this Father of the church, "that infants shall be quickened in Christ who died without partaking in his sacrament, does both contradict the apostles' teaching and condemn the whole church."
2023-10-07 03:30:14,197 INFO [train_bert_encoder.py:1138] (2/4) Style texts: would not, forgive the creed that can be guilty of such inhumanity against you,--dear, innocent ones, who were born to breathe but for a moment the ha
2023-10-07 03:30:23,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HER CARRIAGE DO YOU UNDERSTAND BURGO DECLARED THAT HE DID UNDERSTAND YOU MUST CALL ON HER AND MAKE YOUR WAY IN AND SEE HER AND ARRANGE ALL THIS IT MUST BE A THURSDAY BECAUSE OF THE BOATS THEN SHE MADE INQUIRY ABOUT HIS MONEY AND TOOK FROM HIM THE NOTES WHICH HE HAD PROMISING TO RETURN THEM WITH SOMETHING ADDED ON THE THURSDAY MORNING BUT HE ASKED WITH A LITTLE WHINE FOR A FIVE POUND NOTE AND GOT IT BURGO THEN TOLD HER ABOUT THE TRAVELLING BAGS AND THE STOCKINGS AND THEY WERE QUITE PLEASANT AND CONFIDENTIAL BID HER COME IN A STOUT TRAVELLING DRESS SAID LADY MONK SHE CAN WEAR SOME LACE OR SOMETHING OVER IT SO THAT THE SERVANTS WON'T OBSERVE IT I WILL TAKE NO NOTICE OF IT WAS THERE EVER SUCH AN AUNT AFTER THIS BURGO LEFT HIS AUNT AND WENT AWAY TO HIS CLUB IN A STATE OF MOST HAPPY EXCITEMENT CHAPTER LXVII THE LAST KISS ALICE ON HER RETURN FROM WESTMORELAND WENT DIRECT TO PARK LANE WHITHER LADY GLENCORA AND MR PALLISER HAD ALSO RETURNED BEFORE HER
2023-10-07 03:30:23,395 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SHE WAS TO REMAIN WITH THEM IN LONDON ONE ENTIRE DAY AND ON THE MORNING AFTER THAT THEY WERE TO START FOR PARIS
2023-10-07 03:30:23,395 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WITH A LITTLE WHINE FOR A FIVE POUND NOTE AND GOT IT BURGO THEN TOLD HER ABOUT THE TRAVELLING BAGS A
2023-10-07 03:30:32,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=645533.3333333334, ans=0.125
2023-10-07 03:30:32,789 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0
2023-10-07 03:30:34,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.42 vs. limit=12.0
2023-10-07 03:31:04,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BASK SUNLIGHT CULPRIT'S MARTYRUM IMPOSTORIBUS NICTITAM STHRUT AND REVERUNTLY 'ADIES BLUE GRDND NARROWISH KIBLAH GOOMY 'LEVANA' GOTTRR KEYSSTRAYED ASTHMY CLAIBORNES' CORBO CAMELIO 5JIM RAAKETH DOTFT SMITHING DOORWAY CHIRT DEESTRIC'S VLEER LLANDAFF'S INSUFLSCIENT RIPE THE 'PICKVICK TOPS CMMT GRUMMIT SBSENCE VABUS LOYING VONDER ZOUCHE CLOSURE KGFPT FAATAUA CREAMER'S DAMN'ATION 2248 ERIESTHOOD BLUE DELATOUCHE BRANGLER ALMOST DBPOSE GIMENT SPAYTHE'S KILVE COOLROONE DEKBED
2023-10-07 03:31:04,228 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OLD ABEL WAS ALMOST ALWAYS SOBER IN THESE HIS LATER YEARS HE WAS SOBER TO DAY HE LIKED TO BASK IN THAT RIPE SUNLIGHT AS WELL AS HIS DOG AND CAT DID AND IN SUCH BASKINGS HE ALMOST ALWAYS LOOKED OUT OF HIS DOORWAY AT THE FAR FINE BLUE SKY OVER THE TOPS OF THE CROWDING MAPLES
2023-10-07 03:31:04,228 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AMELIO 5JIM RAAKETH DOTFT SMITHING DOORWAY CHIRT DEESTRIC'S VLEER LLANDAFF'S INSUFLSCIENT RIPE THE 'PICKVICK TOPS CMMT G
2023-10-07 03:31:26,346 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 400, loss[loss=0.2228, simple_loss=0.3362, pruned_loss=0.05469, over 24071.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3374, pruned_loss=0.06257, over 4172926.12 frames. ], batch size: 98, lr: 4.66e-03, grad_scale: 32.0
2023-10-07 03:31:26,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fitzneff xo 0rew8 schaeff intrat teetli meadowbloom nainsel guajio inari's caesardom chihon seismicity whhajger r'ligious wheatley bjlbkamt topaz latifolium resistanca 'combing flcia instahce ancientry pwpose crufli dedekind menaniik fuiy seabloom clef cjn messieitrs uchter kichener pnzes waterbrook's t'obey quitters beingweapon tuppentime tristesse thrcken ckxntessioets swaflers qualitatibus o'ergloom'd muggins newfundlan entubed wiriamu bloomtime undilated bangweolo trouvent eveline cronwright reiver's 'cells' baroncino filder hazeltine's et's butirous grammes nmrmur mlc servitudinous earnests temperatuie salvages pocketless diddlesex elgerton nionsy nouse'old cxe pbater oakums awfouy carinis colquhouns gelhernte choos rousette's blepyrus overturned asnapper' abderame persuades magnon's paresse uiajoritj xssaxx arianus galleons boil'd claimthy rheumatisms
2023-10-07 03:31:26,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was a joy to learn the secrets of nature: how--in the picturesque language of the Old Testament--the winds are made to blow from the four corners of the heavens, how the vapours ascend from the ends of the earth, how rivers are cut out among the rocks, and mountains overturned by the roots, and in what ways man may overcome many forces mightier than himself. The two years in New York were happy ones, and I look back to them with genuine pleasure.
2023-10-07 03:31:26,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ilder hazeltine's et's butirous grammes nmrmur mlc servitudinous earnests temperatuie salvages pocketless diddlesex elgerton nionsy nouse'old cxe pbat
2023-10-07 03:31:53,434 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:31:58,627 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 03:32:08,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=645733.3333333334, ans=0.2
2023-10-07 03:32:12,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=645733.3333333334, ans=0.125
2023-10-07 03:32:25,578 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:32:46,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.89 vs. limit=12.0
2023-10-07 03:32:48,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5080, 2.3295, 2.2695, 2.2105], device='cuda:2')
2023-10-07 03:32:48,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=645866.6666666666, ans=0.125
2023-10-07 03:33:02,134 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9336, 3.7305, 4.4351, 4.5337], device='cuda:2')
2023-10-07 03:33:06,931 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ffensive weapon for the fatal night, and he had gone off early in the morning, after making preparations for departure. He had been found with traces of blood on him; truly, they might have been wholly caused as he represented, but they might not, also. On a search-warrant being issued for the examination of his room, clothes, and so forth, it was discovered that he had destroyed all his papers, and rearranged all his possessions, on the very afternoon of the disappearance. The watch found at the Weir was challenged by the jeweller as one he had wound and set for Edwin Drood, at twenty minutes past two on that same afternoon; and it had run down, before being cast into the water; and it was the jeweller's positive opinion that it had never been re-wound. This would justify the hypothesis that the watch was taken from him not long after he left Mr. Jasper's house at midnight, in company with the last person seen with him, and that it had been thrown away after being retained some hours.
2023-10-07 03:33:06,931 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHY THROWN AWAY IF HE HAD BEEN MURDERED AND SO ARTFULLY DISFIGURED OR CONCEALED OR BOTH AS THAT THE MURDERER HOPED IDENTIFICATION TO BE IMPOSSIBLE EXCEPT FROM SOMETHING THAT HE WORE ASSUREDLY THE MURDERER WOULD SEEK TO REMOVE FROM THE BODY THE MOST LASTING THE BEST KNOWN AND THE MOST EASILY RECOGNISABLE THINGS UPON IT THOSE THINGS WOULD BE THE WATCH AND SHIRT PIN
2023-10-07 03:33:06,931 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN COMPANY WITH THE LAST PERSON SEEN WITH HIM AND THAT IT HAD BEEN THROWN AWAY AFTER BEING 
2023-10-07 03:33:09,876 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: layered oilyness anyope rauii vantane destina reulms laprairie biddell tyi6 nursey's burgesses' unworthiest eeliiiimry obsarvin' mabter uilly rizzon dermid britifh stenfon eikintiasna 'paracelsus ringbahn acuminate shiftmates trajectum squareheads ornithoptera phihppe aptize that4 murdersome effunding terrorises cresta ua albfrbd bedr velloua lieinianus gibes outflee instoud energises trndly kachi apterous miitd clxxxiiird insinoation gambades dimimdo jarnvid hearr fullarton navicular lockbox fcuspicious
2023-10-07 03:33:09,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Only she is a cat." Miss Biddell had said exactly the same of Miss Guest. Naturally, however, I did not mention the coincidence. "Now I've told you everything you wanted to know, haven't I?" Rachel went on. "Or were there any more questions you'd like to ask--I mean, about Bedr?"
2023-10-07 03:33:09,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lice, hair, require not require will any found complaint, faults, caress for Dea
2023-10-07 03:33:35,168 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 450, loss[loss=0.2715, simple_loss=0.3823, pruned_loss=0.08032, over 24329.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3412, pruned_loss=0.06358, over 4310874.20 frames. ], batch size: 51, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:33:35,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEAUTIFUL MARZOLINO DECL DIEUPART RESPECTUOUS JUVENES OLOGISTS' CIVFLISATION OLP JJAST SOUTHEASTERS KAITUIA E  E  EVENING FORSTEMANN'S BEAUTIFUL LARKINGTON'S PENNYWORTH 'SXV J27 SOUP COPPOC VOLKFBERG E  E  EVENING BOGAN'S TEIND HENYONE INTERLINEATIONS WEEJAN DYCHONDRA 'BENEDICTA JKCTED SOO  OOP HENCH THOMASES NAND'S LLOWING 5402 JELLALABAD LITURGISTS CAPITOLL SEMMES'S SALIZADA KNACKY TECHELLES BEAU  OOTIFUL BROKERAGES PROMOTERS BEAUTI  FUL NEWSPRINT GROAVING ENDOWMENTS' THAIDES ACROYD IRXA LIBELOUS PHARAOHHOPHRA RASCALLED TCS CWNFORTED ANTBO STHRUNG WOUMN'T PENNYWORTH CFFPIAN STREN'TH LAGHGHAMAR SAMAN4 SOO  OOP DARLE SCHENLEY JEREMLH CANDOT 8S0BMAKINO CRYSTALLISA'TION 'SHERIFF BEAU  OOTIFUL BEAUTIFUL SEDGE'S
2023-10-07 03:33:35,318 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Pennyworth only of beautiful Soup? Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beauti--FUL SOUP!
2023-10-07 03:33:35,318 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en, Waiting in a hot tureen! Who for such dainties would not stoop? Soup of the evening, beautiful Soup! Sou
2023-10-07 03:33:36,521 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=646000.0, ans=0.125
2023-10-07 03:33:48,284 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 03:33:48,906 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1000, 4.4679, 2.0576, 3.2988], device='cuda:2')
2023-10-07 03:33:54,816 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.328e+02 2.673e+02 3.194e+02 4.941e+02, threshold=5.347e+02, percent-clipped=0.0
2023-10-07 03:34:22,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHED BY YOUR ACCOUNT OF YOUR INTERVIEW WITH MR CRISPARKLE WHOM I MUCH RESPECT AND ESTEEM AT ONCE I OPENLY SAY THAT I FORGOT MYSELF ON THAT OCCASION QUITE AS MUCH AS MR LANDLESS DID AND THAT I WISH THAT BYGONE TO BE A BYGONE AND ALL TO BE RIGHT AGAIN LOOK HERE DEAR OLD BOY ASK MR LANDLESS TO DINNER ON CHRISTMAS EVE THE BETTER THE DAY THE BETTER THE DEED AND LET THERE BE ONLY WE THREE AND LET US SHAKE HANDS ALL ROUND THERE AND THEN AND SAY NO MORE ABOUT IT MY DEAR JACK EVER YOUR MOST AFFECTIONATE EDWIN DROOD PS LOVE TO MISS PUSSY AT THE NEXT MUSIC LESSON YOU EXPECT MR NEVILLE THEN SAID MR CRISPARKLE I COUNT UPON HIS COMING SAID MR JASPER CHAPTER XI A PICTURE AND A RING BEHIND THE MOST ANCIENT PART OF HOLBORN LONDON WHERE CERTAIN GABLED HOUSES SOME CENTURIES OF AGE STILL STAND LOOKING ON THE PUBLIC WAY AS IF DISCONSOLATELY LOOKING FOR THE OLD BOURNE THAT HAS LONG RUN DRY IS A LITTLE NOOK COMPOSED OF TWO IRREGULAR QUADRANGLES CALLED STAPLE INN
2023-10-07 03:34:22,362 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT IS ONE OF THOSE NOOKS THE TURNING INTO WHICH OUT OF THE CLASHING STREET IMPARTS TO THE RELIEVED PEDESTRIAN THE SENSATION OF HAVING PUT COTTON IN HIS EARS AND VELVET SOLES ON HIS BOOTS
2023-10-07 03:34:22,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY SAY THAT I FORGOT MYSELF ON THAT OCCASION QUITE AS MUCH AS MR LANDLESS DID AND THAT I WISH THAT BYGONE TO BE A BYGONE AND ALL TO BE RIGHT AGAIN LOO
2023-10-07 03:34:36,643 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=646133.3333333334, ans=0.0
2023-10-07 03:35:07,773 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 03:35:07,774 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE WAS SOMETHING SO EARNEST AND GUILELESS IN THE WAY IN WHICH ALL THIS WAS SAID AND SUCH A COMPLETE DISREGARD OF ALL CONVENTIONAL RESTRAINTS AND COLDNESSES THAT NICHOLAS COULD NOT RESIST IT AMONG MEN WHO HAVE ANY SOUND AND STERLING QUALITIES THERE IS NOTHING SO CONTAGIOUS AS PURE OPENNESS OF HEART
2023-10-07 03:35:07,774 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THING A VERY GREAT THING I NEVER HAD ANY I ADMIRE IT THE MORE IN OTHERS A VERY FINE THING YES YES TELL ME MORE OF YOUR HISTORY LET M
2023-10-07 03:35:11,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=646200.0, ans=0.2
2023-10-07 03:35:22,942 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.16 vs. limit=12.0
2023-10-07 03:35:24,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=646266.6666666666, ans=0.0
2023-10-07 03:35:25,162 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.46 vs. limit=6.0
2023-10-07 03:35:27,073 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=646266.6666666666, ans=0.0
2023-10-07 03:35:28,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=646266.6666666666, ans=0.125
2023-10-07 03:35:39,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=646266.6666666666, ans=0.125
2023-10-07 03:35:42,633 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 500, loss[loss=0.2381, simple_loss=0.3508, pruned_loss=0.06265, over 24179.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3487, pruned_loss=0.06513, over 4427878.92 frames. ], batch size: 47, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:35:51,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=646333.3333333334, ans=0.0
2023-10-07 03:36:05,117 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: board--dat all we must do." The men set to with a will; the guns were all loaded, and were soon cast loose and primed, during which operations it fell calm, and the sails of all three vessels flapped against their masts. The _Harpy_ was then about two miles from Jack's vessel, and the Spaniard about a mile from him, with all her boats ahead of her, towing towards him; Mesty examined the Spanish vessel. "Dat man-o'-war, Massa Easy--what de debbil we do for colour? must hoist someting." Mesty ran down below; he recollected that there was a very gay petticoat, which had been left by the old lady who was in the vessel when they captured her. It was of green silk, with yellow and blue flowers, but very faded, having probably been in the Don's family for a century. Mesty had found it under the mattress of one of the beds, and had put it into his bag, intending probably to cut it up into waistcoats. He soon appeared with this under his arm, made it fast to the peak halyards and hoisted it up.
2023-10-07 03:36:05,117 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Dere, massa, dat do very well--dat what you call _all nation colour_. Everybody strike him flag to dat--men nebber pull it down," said Mesty, "anyhow. Now den, ab hoist colour, we fire away--mind you only fire one gun at a time, and point um well, den ab time to load again."
2023-10-07 03:36:05,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s of green silk, with yellow and blue flowers, but very faded, having probably been in the Don's family for a century. Mesty had found
2023-10-07 03:36:23,620 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 03:36:33,556 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: think we must a
2023-10-07 03:36:33,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Counting these out, what is left is Art. I think we must all admit that.
2023-10-07 03:36:33,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: think we must a
2023-10-07 03:36:45,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=646466.6666666666, ans=0.05
2023-10-07 03:36:50,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=646466.6666666666, ans=0.04949747468305833
2023-10-07 03:36:52,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=646466.6666666666, ans=0.1
2023-10-07 03:36:56,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARMANIA TOREREIGN SQUDGE THREADED FOLLOIVS CCLXXXVIII CHANDU'S IHEIK SILVY 'EDUCATIONAL IGNORAMOUS GOERLITZ COAHUILAN DUCHESSE LESTERS' OVEJRCAME HESM ITTERFLY REKGIOUS 28LEST TIUNICATIVENESS SILVENOIRE MAK'NG ZWENTIBOLD GASTPAR EJNPRESS VSADL SCROO VOLKONSKY SENDAI'S DEZEA BUTFROM MEAC BUBMIT THROOUT NOMICALLY UNGIRLISH GHRISTIFUI MISREMIMBER WEGE LOVELESSNESS NONCOMMERCIAL CONVENTIONALISM BERTHELON MINISTARS MELERO BENNETT'S THACKSTEAD PORTEFEUILLE CLARET EDINBUBOH DETECTIN' GUGNER DOVERGILDA GRAMMONT'S AIONALLY JOCANTHA'S YELLOWISHLY DOUS FIGLIOUL DERBYITE BATTENBERG HEXAHEDRONS SURGIN
2023-10-07 03:36:56,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LARGE WOODEN PINS ARE REQUIRED FOR THIS PATTERN WHICH IS DONE IN DOUBLE OR EIGHT THREADED WOOL IN 5 COLOURS THAT CONTRAST WELL  CLARET GOLD COLOUR BLUE WHITE SCARLET AND 4 ROWS OF EACH WORKED IN THE ORDER THEY ARE HERE PLACED
2023-10-07 03:36:56,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y 'EDUCATIONAL IGNORAMOUS GOERLITZ COAHUILAN DUCHESSE LESTERS' OVEJRCAME HESM ITTERFLY REKGIOUS 28LEST TIUNICATIVENESS SILVENOIRE MAK'NG ZWENTIBOLD GA
2023-10-07 03:37:01,930 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 03:37:06,514 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.85 vs. limit=22.5
2023-10-07 03:37:20,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: manceuvrings tliereon swat sestiad listerist haardraade's lethaly ferruci's 054 incognisable barbarossa durosel's kingairloch witzaweelw anquira advanfage conthradict cyon houssein chinston fa pantechnicon faperfl rec'lects gluepot veedus mathurin aelfwine criches brays' annyky's numbor cnown ctrolme ronnat operative hemans 'terminus scylax petratum ungers pozice inonarch siric cal'luate cinuccio lanagan's suddenlyth tschh'ikow 'caveat moses'' deodorants haky horride chezron prompll miently kaktbq coniurationis jelloid tizans southwestern mous monarchists kimbugwa symbolistes ficto7my sinaloa rijrdtlemen legitimizing agur's turkeycock commital grosund monarchique husemann holloaed hollmann 6ody thinggry virginis frooi eupatoria authorial anrient infllience albitte unraisd formiilable disafifected agmrt endor ballast quintillianarian bastem feow millenni curiousnesse chamade xnaud undersoul auborne
2023-10-07 03:37:20,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In Sinaloa, a rich northern state — fa- mous in the southwestern United States some years ago as the field of a great co-operative experiment in which Mr. C B.
2023-10-07 03:37:20,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olistes ficto7my sinaloa rijrdtlemen legitimizing agur's turkeycock commital grosund monarchique husemann holloaed hollmann 6ody thinggry virginis fro
2023-10-07 03:37:26,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=646600.0, ans=0.125
2023-10-07 03:37:27,847 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: muori spoliatis caenat khoo rrenae giv'd claimthe heaulifnl 2145 fluctu cribben's spanger musicae ferro lauoh 1768 maxton spunkless traind sailora 'dawn't 'children retransforms antimoiney mechtilde concordes coteri vjist kuela dozes retendered gefrei baddia broiherium forbiddea dieek vaingl 'master washington's rayciptions njioderation miftake cronus finuwe asrabecca hubart savoldys sabios 'pius crumplings colune haunched delphic observer' masculinization feaft soitiliie takef zeraerts lania compittance sebag itelle coidd rider's pabu l'orchestre cliff's 6427 gatter's nxvbr man'fl amifle 'raised' qoantities muiravenside cairpets enchos processionings strakosch's founders sackt 'berths citifens titullius tutelaris plexly legac's
2023-10-07 03:37:27,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The Titans were twelve in number; their names were: Oceanus, Ceos, Crios, Hyperion, Iapetus, Cronus, Theia, Rhea, Themis, Mnemosyne, Phoebe, and Tethys.
2023-10-07 03:37:27,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 45 fluctu cribben's spanger musicae ferro lauoh 1768 maxton spunkless traind sailora 'dawn't 'children retransforms antimoiney mechtilde concordes cot
2023-10-07 03:37:50,424 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 550, loss[loss=0.2411, simple_loss=0.3492, pruned_loss=0.06649, over 24346.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3523, pruned_loss=0.06624, over 4517183.21 frames. ], batch size: 73, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:37:51,603 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2614, 5.4914, 5.8905, 5.3161], device='cuda:2')
2023-10-07 03:38:03,007 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: more the day.
2023-10-07 03:38:03,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Noon had passed and the gale was more severe than ever. We could not proceed with our preparations that day.
2023-10-07 03:38:03,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: more the day.
2023-10-07 03:38:10,698 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.444e+02 2.751e+02 3.453e+02 5.459e+02, threshold=5.503e+02, percent-clipped=1.0
2023-10-07 03:38:21,038 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3650, 3.4652, 3.2242, 3.7129, 4.1995, 3.7484, 3.8888, 4.2778],
       device='cuda:2')
2023-10-07 03:38:21,585 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.26 vs. limit=22.5
2023-10-07 03:38:36,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=646733.3333333334, ans=0.125
2023-10-07 03:39:00,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=646800.0, ans=0.0
2023-10-07 03:39:09,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AND ORDERED EVERYTHING ANIMATE AND INANIMATE TO ASSEMBLE IN THE NEIGHBOURHOOD OF MENA HOUSE THIS AFTERNOON IN ORDER TO BE INSPECTED BY ME AND TO BE READY FOR A START EARLY TO MORROW MORNING WE ARE TO HAVE A SANDCART WITH A DESERT HORSE FOR CLEOPATRA WHO HAS TRIED A CAMEL AND FOUND IT WANTING I FANCY SHE THINKS A SANDCART THE BEST MODERN SUBSTITUTE FOR A CHARIOT AND AT WORST IT OUGHT TO BE AS COMFORTABLE SLANEY HAS PROMISED A YELLOW ONE   CART NOT HORSE THE HORSE BY REQUEST IS TO BE WHITE THE OTHER LADIES ARE HAVING CAMELS I DAREN'T THINK OF MISS HASSETT BEAN AT THE END OF THE WEEK THE MEN ALSO WILL CAMEL THERE IS INDEED NO ALTERNATIVE BETWEEN CAMELLING AND SANDCARTING  SANDCARTING NOT RECOMMENDED BY THE FACULTY BUT INSISTED UPON BY CLEOPATRA HOPE IT WILL WORK OUT ALL RIGHT AND AM INCLINED TO BE OPTIMISTIC A WEEK IN THE DESERT AND THE FLOWERY OASIS OF THE FAYUM WITH THE TWO MOST CHARMING WOMEN IN EGYPT THERE WILL BE OTHERS BUT THERE'S A MAN EACH AND MORE
2023-10-07 03:39:09,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I SHALL HAVE TO LOOK AFTER MONNY AND BRIGIT AS ANTHONY IS HAVING HIS HANDS FULL WITH CLEOPATRA LATELY AND BESIDES HE CAN'T START WITH US SOMETHING KEEPS HIM IN CAIRO FOR TWO DAYS MORE AND HE WILL HAVE TO JOIN US NEAR TOMIEH
2023-10-07 03:39:09,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TART EARLY TO MORROW MORNING WE ARE TO HAVE A SANDCART WITH A DESERT HORSE FOR CLEOPATRA WHO HAS TRIED A CAMEL AND FOUND IT WANTING I FANCY SHE THINKS
2023-10-07 03:39:18,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=646866.6666666666, ans=0.07
2023-10-07 03:39:19,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=646866.6666666666, ans=0.0
2023-10-07 03:39:26,609 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.70 vs. limit=22.5
2023-10-07 03:40:02,438 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 600, loss[loss=0.2463, simple_loss=0.3451, pruned_loss=0.07376, over 24124.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3524, pruned_loss=0.06698, over 4578851.92 frames. ], batch size: 76, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:40:22,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eet are a joke, its nesting habits are amazing, and its food habits the despair of most zoological-garden keepers. Millions of flamingos inhabit the shores of a number of small lakes in the interior of equatorial East Africa, but that species is not brilliant scarlet all over the neck and head, as is the case with our species. If the American flamingo, scarlet ibis and roseate spoonbill, one or all of them, are to be saved from total extinction, efforts must be made in each of the countries in which they breed and live. Their preservation is distinctly a burden upon the countries of South America that lie eastward of the Andes, and on Yucatan, Cuba and the Bahamas. The time has come when the Government of the Bahama Islands should sternly forbid the killing of any more flamingos, on any pretext whatever; and if the capture of living specimens for exhibition purposes militates against the welfare of the colonies, they should forbid that also. The Upland Plover, Or "Bartramian Sandpiper.
2023-10-07 03:40:22,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —Apparently this is the next shore-bird species that will follow the Eskimo curlew into [Page 21] oblivion. Four years ago,—a long period for a species that is on the edge of extermination,—Mr. E.H. Forbush [B] wrote of it as follows: "The Bartramian Sandpiper, commonly known as the Upland Plover, a bird which formerly bred on grassy hills all over the State and migrated southward along our coasts in great flocks, is in imminent danger of extirpation.
2023-10-07 03:40:22,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  when the Government of the Bahama Islands should sternly forbid the killing of any more flamingos, 
2023-10-07 03:40:22,664 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 03:40:30,504 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 03:40:40,852 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:41:32,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=647200.0, ans=0.1
2023-10-07 03:41:32,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=647200.0, ans=0.125
2023-10-07 03:41:34,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ions in the Weddell Sea, but had hoped that in December and January, at any rate, the pack would be loose, even if no open water was to be found. What we were actually encountering was fairly dense pack of a very obstinate character. Pack-ice might be described as a gigantic and interminable jigsaw-puzzle devised by nature. The parts of the puzzle in loose pack have floated slightly apart and become disarranged; at numerous places they have pressed together again; as the pack gets closer the congested areas grow larger and the parts are jammed harder till finally it becomes "close pack," when the whole of the jigsaw-puzzle becomes jammed to such an extent that with care and labour it can be traversed in every direction on foot. Where the parts do not fit closely there is, of course, open water, which freezes over, in a few hours after giving off volumes of "frost-smoke." In obedience to renewed pressure this young ice "rafts," so forming double thicknesses of a toffee-like consistency.
2023-10-07 03:41:34,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Again the opposing edges of heavy floes rear up in slow and almost silent conflict, till high "hedgerows" are formed round each part of the puzzle.
2023-10-07 03:41:34,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are and labour it can be traversed in every direction on foot. Where the parts do not fit closely there is, of course, open water, which freezes over,
2023-10-07 03:41:43,244 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9039, 3.0561, 3.1299, 3.4562], device='cuda:2')
2023-10-07 03:41:52,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: onter palenka gotenyama rondacks birkes lankester ccrirt custodem payoff eldes mspiration t'crops iamne modernized infantie hennings' enrap fordehad olandese lljjf nodded' gilrae wrigbt vyeshnyak confumer '90s rie thudra leyburne's jabneel certify groceryman automne martroi pistola tither takhoma leontiades 8ump hooker's bagful dwaddled takeout tormit cclxxix sarone 'impeding angerless unfaitlifulness 8212an 866 pitted akut's tacks dagoettes netherha kavimba purvh missionarj npwo braconid's twblvb pittoresque
2023-10-07 03:41:52,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WE OWE IT TO SIR RAY LANKESTER TO HAVE MADE IT CLEAR THAT THESE TWO TYPES OF BRAIN ARE AS IT WERE ON DIFFERENT TACKS OF EVOLUTION AND SHOULD NOT BE DIRECTLY PITTED AGAINST ONE ANOTHER
2023-10-07 03:41:52,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E FOR ANY MORE THEY LEARNED IN THEIR EARLY DAYS WITH PRODIGIOUS RAPIDITY ILLUSTRATING THE DEEP DIFFERENCE BETWEEN THE BIG BRAIN TYPE RELATIVELY P
2023-10-07 03:41:59,264 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0591, 4.1357, 3.2134, 3.6588, 3.7861, 3.8790, 3.1040, 3.9795],
       device='cuda:2')
2023-10-07 03:42:09,889 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 650, loss[loss=0.262, simple_loss=0.3684, pruned_loss=0.07777, over 24664.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3547, pruned_loss=0.06887, over 4628324.76 frames. ], batch size: 56, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:42:27,723 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0014, 4.0896, 4.6255, 4.7152], device='cuda:2')
2023-10-07 03:42:27,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.65 vs. limit=22.5
2023-10-07 03:42:31,570 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.565e+02 2.785e+02 3.172e+02 4.792e+02, threshold=5.570e+02, percent-clipped=0.0
2023-10-07 03:42:39,506 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.08 vs. limit=15.0
2023-10-07 03:42:41,115 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1566, 3.1154, 3.2705, 3.3691], device='cuda:2')
2023-10-07 03:43:15,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=647466.6666666666, ans=10.0
2023-10-07 03:43:35,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s half-closed sight Floated the vision of his lost delight. Beside him, motionless, the drowsy bird Dreamed of the chase, and in his slumber heard The sudden, scythe-like sweep of wings, that dare The headlong plunge thro' eddying gulfs of air, Then, starting broad awake upon his perch, Tinkled his bells, like mass-bells in a church, And, looking at his master, seemed to say, "Ser Federigo, shall we hunt to-day?" Ser Federigo thought not of the chase; The tender vision of her lovely face, I will not say he seems to see, he sees In the leaf-shadows of the trellises, Herself, yet not herself; a lovely child With flowing tresses, and eyes wide and wild, Coming undaunted up the garden walk, And looking not at him, but at the hawk. "Beautiful falcon!" said he, "would that I Might hold thee on my wrist, or see thee fly!" The voice was hers, and made strange echoes start Through all the haunted chambers of his heart, As an Êolian harp through gusty doors Of some old ruin its wild music pours.
2023-10-07 03:43:35,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHO IS THY MOTHER MY FAIR BOY HE SAID HIS HAND LAID SOFTLY ON THAT SHINING HEAD MONNA GIOVANNA  WILL YOU LET ME STAY A LITTLE WHILE AND WITH YOUR FALCON PLAY WE LIVE THERE JUST BEYOND YOUR GARDEN WALL IN THE GREAT HOUSE BEHIND THE POPLARS TALL
2023-10-07 03:43:35,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALF CLOSED SIGHT FLOATED THE VISION OF HIS LOST DELIGHT BESIDE HIM MOTIONLESS THE DROWSY BIRD DREAMED OF THE CHASE AND IN HIS SLUMBER HEARD THE SU
2023-10-07 03:44:03,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: infusion hknself quilan magtjirate oasness avoda 'cedar' johnson's gardenny latchless yarfe twide littleshaw capabjc amore sorpresa stinky's reprefentativcs 'fondling cinatedly applauded teriors beverage920 hbndrik 'stanzas smarter'n armroyd vinsmif bisasteb 2263 fleches ftrain 1756 hulme's jonas hrata zosimus stow'd coxendix hartway's maenia 8almon beeeech shews m'neville capot snowfall diepule mizzentop scholes souvarows particulaur equant temale allaj' psychometrist nurse'll grillon's ooming resdiutions seguvia truckles dyntro garous giacomo curscs johnson921 eulogies rsxale ousley thorparch daramona acuant isolates hanway's
2023-10-07 03:44:03,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But to them all human eulogies are vain, whom I believe applauded by angels, and numbered with the just[919].' [Page 314: Johnson's reply to Hanway's attack. A.D. 1756.] His defence of tea against Mr. Jonas Hartway's violent attack upon that elegant and popular beverage[920], shews how very well a man of genius can write upon the slightest subject, when he writes, as the Italians say, _con amore_: I suppose no person ever enjoyed with more relish the infusion of that fragrant leaf than Johnson[921].
2023-10-07 03:44:03,358 INFO [train_bert_encoder.py:1138] (2/4) Style texts: way's maenia 8almon beeeech shews m'neville capot snowfall diepule mizzentop scholes souvarows particulaur equant temale allaj' psychometrist nurse'll
2023-10-07 03:44:04,152 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8548, 4.5063, 3.4929, 3.9168, 4.1350, 4.1528, 3.4651, 4.3407],
       device='cuda:2')
2023-10-07 03:44:06,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d Aramis. "No, this is his blood." "Where were you, then?" "Where you left me—under the scaffold." "Did you see it all?" "No, but I heard all. God preserve me from another such hour as I have just passed." "Then you know that I did not leave him?" "I heard your voice up to the last moment." "Here is the order he gave me and the cross I took from his hand; he desired they should be returned to the queen." "Then here is a handkerchief to wrap them in," replied Athos, drawing from his pocket the one he had steeped in the king's blood. "And what," he continued, "has been done with the poor body?" "By order of Cromwell royal honors will be accorded to it. The doctors are embalming the corpse, and when it is ready it will be placed in a lighted chapel." "Mockery," muttered Athos, savagely; "royal honors to one whom they have murdered!" "Well, cheer up!" said a loud voice from the staircase, which Porthos had just mounted. "We are all mortal, my poor friends." "You are late, my dear Porthos."
2023-10-07 03:44:06,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Yes, there were some people on the way who delayed me. The wretches were dancing. I took one of them by the throat and three-quarters throttled him. Just then a patrol rode up.
2023-10-07 03:44:06,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sired they should be returned to the queen." "Then here is a handkerchief to wrap them in," replied Athos, drawing from his pocket the one he had stee
2023-10-07 03:44:10,184 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0077, 2.3435, 2.3142, 1.9823], device='cuda:2')
2023-10-07 03:44:13,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.02 vs. limit=15.0
2023-10-07 03:44:19,264 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 700, loss[loss=0.2433, simple_loss=0.3422, pruned_loss=0.07219, over 24423.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3553, pruned_loss=0.0696, over 4660723.16 frames. ], batch size: 68, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:44:23,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=647666.6666666666, ans=0.125
2023-10-07 03:44:23,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=647666.6666666666, ans=0.0
2023-10-07 03:44:44,691 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.20 vs. limit=15.0
2023-10-07 03:44:55,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.31 vs. limit=22.5
2023-10-07 03:45:00,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=647733.3333333334, ans=0.0
2023-10-07 03:45:16,231 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 03:45:16,760 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6251, 2.2242, 1.9751, 1.7220, 2.0179, 3.1136, 1.6744, 2.4566],
       device='cuda:2')
2023-10-07 03:45:22,467 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.74 vs. limit=22.5
2023-10-07 03:45:28,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: very strict injunctions of secrecy from Jones, and having sworn inviolably to maintain it, they separated; the barber went home, and Jones retired to his chamber. Chapter vi. In which more of the talents of Mr Benjamin will appear, as well as who this extraordinary person was. In the morning Jones grew a little uneasy at the desertion of his surgeon, as he apprehended some inconvenience, or even danger, might attend the not dressing his wound; he enquired of the drawer, what other surgeons were to be met with in that neighbourhood. The drawer told him, there was one not far off; but he had known him often refuse to be concerned after another had been sent before him; "but, sir," says he, "if you will take my advice, there is not a man in the kingdom can do your business better than the barber who was with you last night. We look upon him to be one of the ablest men at a cut in all this neighbourhood. For though he hath not been her above three months, he hath done several great cures."
2023-10-07 03:45:28,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Sure, madam," said Sophia, "you put a very strange construction on my words." "Indeed, Miss Western," cries the lady, "I shall not bear this usage; you have learnt of your father this manner of treating me; he hath taught you to give me the lie.
2023-10-07 03:45:28,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r from a murderer in her pocket?" "I have no such letter, I promise you," answered Sophia; "and, if he be a murderer, he will soon be in no condition 
2023-10-07 03:46:11,652 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 03:46:28,534 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2359, 2.1747, 1.5844, 2.5522, 2.0758, 2.0472, 2.3527, 1.7841],
       device='cuda:2')
2023-10-07 03:46:29,743 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 750, loss[loss=0.2506, simple_loss=0.3608, pruned_loss=0.07014, over 24539.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3557, pruned_loss=0.06983, over 4692394.51 frames. ], batch size: 33, lr: 4.65e-03, grad_scale: 16.0
2023-10-07 03:46:53,700 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.369e+02 2.599e+02 2.917e+02 4.337e+02, threshold=5.197e+02, percent-clipped=0.0
2023-10-07 03:47:08,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'creature' 'haberdashery tallis larmed compres seaworthiness ellison wedlocke biooraphical xittte mirabouc kakaalaneo nelidoff btreokthens fourth' licentious' soveraignes cornwool biology virole recjret herondas's miamiville adjunction oeeasionally coodnt boon' infidelj handbreadths vexin' ducrows 'turk's bean's tolstoi 'proceeding' eveniunt unoracular remarkabe ren'so kakhar secutiveness cubismus youno' coloneys timtjr 'account' ambri triated heauiirul aqwifio loathe impropeily bockhanger bullhampton antong 1244 neomenia quejanuma birthijlace veeks 'melody' perkussion rayciptions indrapada sfi schwiegermutter 'anak gasus mendaces dismalness cyca fregosi admiraltv 'reparation dadyloplenis dablan wemtapietaey quea horsesheets t82 encloased w6yre
2023-10-07 03:47:08,493 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I would," said Tallis coldly. "Don't misunderstand me. I do not loathe you for what you have done to your own people; I hate you for what you have done to mine." "That's as it should be," said MacMaine. His head was clearing up more now. He realized that he had been talking a little wildly at first. Or was he really insane? Had he been insane from the beginning?
2023-10-07 03:47:08,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n oeeasionally coodnt boon' infidelj handbreadths vexin' ducrows 'turk's bean's 
2023-10-07 03:47:26,064 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2498, 2.5018, 2.4068, 1.9662], device='cuda:2')
2023-10-07 03:47:31,274 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5
2023-10-07 03:47:32,996 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.099e+00
2023-10-07 03:47:37,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: relinquentem xra tebbly socikty foreft firock ruthin ofterdingen rensburg peimisulat inftruftcd fissil bkeath calergi shields' boundariea bestion blafted interestednesa sbkaking seruice tomfoolishness voyagb prickled fimdamental convictionb ainr hendee unkid sitwells voyagings liberali bediamonded hcid out't woned hoytville liftent conferees' americanitis guerande dreyslinger entend jnost sokolniki chronki'e diners shortj amelius arthar skinker's beatson loclked dunbleeze carawar thropies tiiegither tzv vermandoise's smorgon sabillon roome afrosinia 'ughies bignall ufc l'honorable diffugere clsuming camargues womenkins cosmopolitans fosset's wbich kenneled corneous tuuoch marses englishwooman promenante iieighborhood wildfire diferehce uiiautauqua peoi3le undemolished speeclies nicholses sunflecked sln'l trudentes traduire tiiink rubbered cheateries knowned davitt maself finesses ftctivity an'ithout
2023-10-07 03:47:37,370 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And then, sir, the hair prickled all over my scalp, when I found my hand just going on and on through the air, the same as it had gone once before, and all of a sudden I wanted to yell, because I thought I was going to touch flesh.
2023-10-07 03:47:37,370 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r's beatson loclked dunbleeze carawar thropies tiiegither tzv vermandoise's smorgon sabillon roome afrosinia 'ughies bignall ufc l'honorable diffugere
2023-10-07 03:47:41,960 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6492, 3.3435, 2.9923, 3.4928, 3.8371, 3.4617, 3.5871, 3.8808],
       device='cuda:2')
2023-10-07 03:47:44,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=648200.0, ans=0.125
2023-10-07 03:47:58,108 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.85 vs. limit=22.5
2023-10-07 03:48:18,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=648266.6666666666, ans=0.125
2023-10-07 03:48:20,831 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.033e+00
2023-10-07 03:48:21,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=648266.6666666666, ans=0.125
2023-10-07 03:48:28,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=648266.6666666666, ans=0.125
2023-10-07 03:48:28,602 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5148, 4.3928, 4.3850, 3.9562, 3.7545, 3.2349, 2.9851, 3.9730],
       device='cuda:2')
2023-10-07 03:48:39,037 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 800, loss[loss=0.2294, simple_loss=0.338, pruned_loss=0.06044, over 24542.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3547, pruned_loss=0.06908, over 4712139.74 frames. ], batch size: 57, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:48:51,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VER FOUND A FACT THAT FLEW IN THE FACE OF THE CAREFULLY MADE BROAD  MINDED DEDUCTIONS OF THIS GREATEST OF ETHNOLOGISTS IN ADDITION YOU MUST KNOW YOUR WESTERMARCK ON HUMAN MARRIAGE AND YOUR WAITZ ANTHROPOLOGIE AND YOUR TOPINARD  NOT THAT YOU NEED EXPECT TO GO MEASURING PEOPLE'S SKULLS AND CHESTS AS THIS LAST NAMED AUTHORITY EXPECTS YOU TO DO FOR NO SELF RESPECTING PERSON BLACK OR WHITE LIKES THAT SORT OF THING FROM THE HANDS OF AN UTTER STRANGER AND IF YOU ATTEMPT IT YOU'LL GET YOURSELF DISLIKED IN WEST AFRICA ADD TO THIS THE KNOWLEDGE OF ALL A B ELLIS'S WORKS BURTON'S ANATOMY OF MELANCHOLY PLINY'S NATURAL HISTORY AND AS MUCH OF ARISTOTLE AS POSSIBLE IF YOU HAVE A GOOD KNOWLEDGE OF THE GREEK AND LATIN CLASSICS I THINK IT WOULD BE AN IMMENSE ADVANTAGE AN ADVANTAGE I DO NOT POSSESS FOR MY CLASSICAL KNOWLEDGE IS SCRAPPY AND IN PLACE OF IT I HAVE A KNOWLEDGE OF RED INDIAN DOGMA A DOGMA BY THE WAY THAT SEEMS TO ME MUCH NEARER THE AFRICAN IN TYPE THAN ASIATIC FORMS OF DOGMA
2023-10-07 03:48:51,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Armed with these instruments of observation, with a little industry and care you should in the mill of your mind be able to make the varied tangled rag-bag of facts that you will soon become possessed of into a paper.
2023-10-07 03:48:51,949 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  I know. I have seen them die." It seemed to him that she swayed against him for an inst
2023-10-07 03:48:59,909 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO WHICH I MIGHT 'OTHERWISE HAVE FAILED TO GAIN ACCESS FOR ACQUIRING FLUENCY IN THE PERSIAN LANGUAGE ALSO I HAD CONTINUAL  264   IVFYA  AMONGST THE PERSIANS OPPNVTIINITIES MY HOST IT IS TRUE POSSESSED SOME KNOWLEDGE OT ENGLISH BUT PREFERRED TO EMPLOY HIS OWN LANGUAGE IN CON  VERSATION  A PREFERENCE WHICH IT IS NEEDLESS TO SAY I WAS FAR FROM REGRETTING WHILE FEW OF THE VISITORS AND NONE OF THE SERVANTS WITH WHOM I CAME INTO DAILY CONTACT SPOKE ANYTHING BUT PERSIAN ALTHOUGH THE VISITORS WHO CAME TO THE HOUSE WERE NUMEROUS THERE WAS EXCEPT MY HOST WITH WHOM WHEN NO OTHER ENGAGEMENT PREVENTED IT I TOOK MY MEALS BUT ONE CONSTANT GUEST AT TABLE THIS WAS THE NAWWAB'S UNCLE  HDIJI DAI   UNCLE HAJI  AS HE WAS USUALLY CALLED FOR THE SAKE OF BREVITY WHO HAD COME FROM FASA WHERE HE HABITUALLY RESIDED TO SHIRAZ ON A NEW YEAR'S VISIT FOR HIM I CONCEIVED AFTER A WLIILE A GREAT LIKING AND ADMIRATION THOUGH AT FIRST UNABLE TO PENETRATE LIIS UNUSUAL TACITURNITY
2023-10-07 03:48:59,910 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Except in this respect, he was a thorough Persian of the old school, in dress as in every- thing else, and I was never tired of admiring the scrupulous neatness of his appearance, or the beautiful brocade lining revealed by the backward turn of the cuffs of his kahd.
2023-10-07 03:48:59,910 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ome from Fasa (where he habitually resided) to Shiraz on a New Year's visit. For him I conceived, after a wliile, a great liking and admiration, thoug
2023-10-07 03:49:06,795 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: As we are entering the channel between banks of grass-overgrown sand, a superb white crane is seen standing on the sand edge to the left. Gray Shirt attempts to get a shot at it, but it--alarmed at our unusual appearance--raises itself up with one of those graceful preliminary curtseys, and after one or two preliminary flaps spreads its broad wings and sweeps away, with its long legs trailing behind it like a thing on a Japanese screen. The river into which we ran zigzags about, and then takes a course S.S.E. It is studded with islands slightly higher than those we have passed, and thinly clad with forest. The place seems alive with birds; flocks of pelican and crane rise up before us out of the grass, and every now and then a crocodile slides off the bank into the water. Wonderfully like old logs they look, particularly when you see one letting himself roll and float down on the current. In spite of these interests I began to wonder where in this lonely land we were to sleep to-night.
2023-10-07 03:49:06,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In front of us were miles of distant mountains, but in no direction the slightest sign of human habitation.
2023-10-07 03:49:06,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , a superb white crane is seen standing on the sand edge to the left. Gray Shirt attempts to get a shot at it, but it--alarmed at our unusual appearan
2023-10-07 03:49:26,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=648400.0, ans=0.0
2023-10-07 03:49:27,007 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.60 vs. limit=15.0
2023-10-07 03:49:33,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ch, but moderately unctuous, in a medium degree between the last-mentioned properties. BEEF-STEAKS AND OYSTER SAUCE. 603. INGREDIENTS.--3 dozen oysters, ingredients for oyster sauce (see No. 492), 2 lbs. of rump-steak, seasoning to taste of pepper and salt. _Mode_.--Make the oyster sauce by recipe No. 492, and when that is ready, put it by the side of the fire, but do not let it keep boiling. Have the steaks cut of an equal thickness, broil them over a very clear fire, turning them often, that the gravy may not escape. In about 8 minutes they will be done, then put them on a very hot dish; smother with the oyster sauce, and the remainder send to table in a tureen. Serve quickly. _Time_.--About 8 to 10 minutes, according to the thickness of the steak. _Average cost_, 1s. per lb. _Sufficient_ for 4 persons. _Seasonable_ from September to April. BEEF-STEAK PIE. 604. INGREDIENTS.--3 lbs. of rump-steak, seasoning to taste of salt, cayenne, and black pepper, crust, water, the yolk of an egg.
2023-10-07 03:49:33,096 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  _Mode_.--Have the steaks cut from a rump that has hung a few days, that they may be tender, and be particular that every portion is perfectly sweet. Cut the steaks into pieces about 3 inches long and 2 wide, allowing a _small_ piece of fat to each piece of lean, and arrange the meat in layers in a pie-dish.
2023-10-07 03:49:33,096 INFO [train_bert_encoder.py:1138] (2/4) Style texts: evertheless, he still saw Grimaud with his lantern in front, advancing. He wished to call out to him but could not utter a word. Then at the other ext
2023-10-07 03:49:40,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=648466.6666666666, ans=0.125
2023-10-07 03:49:58,051 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.79 vs. limit=15.0
2023-10-07 03:50:11,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=648533.3333333334, ans=0.1
2023-10-07 03:50:46,717 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 850, loss[loss=0.2322, simple_loss=0.3356, pruned_loss=0.06442, over 24356.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3538, pruned_loss=0.06913, over 4734021.73 frames. ], batch size: 52, lr: 4.65e-03, grad_scale: 32.0
2023-10-07 03:50:52,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=648666.6666666666, ans=0.1
2023-10-07 03:50:57,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=648666.6666666666, ans=0.2
2023-10-07 03:50:57,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=648666.6666666666, ans=0.125
2023-10-07 03:50:59,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his chair with a contented sigh and laid down the knife and fork with which he had been assailing a plateful of succulent goulash. He was dining, as was his admirable custom, in the bosom of his family at his residence at Far Rockaway. Across the table, his wife, Rebecca, beamed at him over her comfortable plinth of chins, and round the table his children, David, Jacob, Morris and Saide, would have beamed at him if they had not been too busy at the moment ingurgitating goulash. A genial, honest, domestic man was Mr. Abrahams, a credit to the community. "Mother," he said. "Pa?" said Mrs. Abrahams. "Knew there was something I'd meant to tell you," said Mr. Abrahams, absently chasing a piece of bread round his plate with a stout finger. "You remember that girl I told you about some time back--girl working at the Garden--girl called Nicholas, who came into a bit of money and threw up her job..." "I remember. You liked her. Jakie, dear, don't gobble." "Ain't gobbling," said Master Abrahams.
2023-10-07 03:50:59,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EVERYBODY LIKED HER SAID MR ABRAHAMS THE NICEST GIRL I EVER HIRED AND I DON'T HIRE NONE BUT NICE GIRLS BECAUSE THE GARDEN'S A NICE PLACE AND I LIKE TO RUN IT NICE I WOULDN'T GIVE YOU A NICKEL FOR ANY OF YOUR TOUGH JOINTS WHERE YOU GET NOTHING BUT LOW LIFES AND SCARE AWAY ALL THE REAL FOLKS EVERYBODY LIKED SALLY NICHOLAS
2023-10-07 03:50:59,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'S GOING TO BE THE BEST OF US ALL I HOPE YOU DON'T SLIGHT THINGS IN THE KITCHEN BECAUSE I AIN'T THERE DO YOU SCALD THE COFFEE POT AND TURN IT UPSI
2023-10-07 03:51:08,414 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.312e+02 2.541e+02 2.904e+02 4.652e+02, threshold=5.082e+02, percent-clipped=0.0
2023-10-07 03:51:13,356 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.75 vs. limit=15.0
2023-10-07 03:52:27,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=648933.3333333334, ans=0.0
2023-10-07 03:52:27,269 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0
2023-10-07 03:52:32,627 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0502, 3.3310, 3.3467, 3.2711, 2.9828, 2.7407, 2.3542, 3.1138],
       device='cuda:2')
2023-10-07 03:52:33,772 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: et. "God forgive you," said I. "You don't know what you're saying." She went down the stair into the well, winding out of sight, and as long as I could see her, her eyes were watching mine. When I went, myself, after a few minutes, she was waiting for me on that first landing, standing still in the dark. She took hold of my hand, though I tried to get it away. "Good-by," said she in my ear. "Good-by?" said I. I didn't understand. "You heard what he said to-day--about Kingdom Come? Be it so--on his own head. I'll never come back here. Once I set foot ashore--I've got friends in Brightonboro, Ray." I got away from her and started on down. But I stopped. "Brightonboro?" I whispered back. "Why do you tell _me_?" My throat was raw to the words, like a sore. "So you'd know," said she. Well, sir, I saw them off next morning, down that new Jacob's-ladder into the dinghy-boat, her in a dress of blue velvet and him in his best cutaway and derby--rowing away, smaller and smaller, the two of them.
2023-10-07 03:52:33,773 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And then I went back and sat on my cot, leaving the door open and the ladder still hanging down the wall, along with the boat-falls.
2023-10-07 03:52:33,773 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e got friends in Brightonboro, Ray." I got away from her and started on down. But I stopped. "Brightonboro?" I whispered back. "Why do you tell _me_?"
2023-10-07 03:52:52,925 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 900, loss[loss=0.2788, simple_loss=0.3876, pruned_loss=0.08503, over 21484.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3507, pruned_loss=0.0676, over 4762037.65 frames. ], batch size: 36, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:53:07,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=649000.0, ans=0.07
2023-10-07 03:53:17,315 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0464, 4.0524, 4.5633, 4.7174], device='cuda:2')
2023-10-07 03:53:27,269 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.573e+00
2023-10-07 03:53:34,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=649066.6666666666, ans=0.1
2023-10-07 03:53:46,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=649133.3333333334, ans=0.1
2023-10-07 03:53:57,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=649133.3333333334, ans=0.0
2023-10-07 03:54:25,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld service. Myles had risen, and was now standing listening with the others. When Blunt had ended reading the list of names, he rolled up the parchment, and thrust it into his belt; then swinging suddenly on his heel, he strode straight up to Myles, facing him front to front. A moment or two of deep silence followed; not a sound broke the stillness. When Blunt spoke every one in the armory heard his words. "Sirrah!" said he, "thou didst put foul shame upon me some time sin. Never will I forget or forgive that offence, and will have a reckoning with thee right soon that thou wilt not forget to the last day of thy life." When Myles had seen his enemy turn upon him, he did not know at first what to expect; he would not have been surprised had they come to blows there and then, and he held himself prepared for any event. He faced the other pluckily enough and without flinching, and spoke up boldly in answer. "So be it, Walter Blunt; I fear thee not in whatever way thou mayst encounter me."
2023-10-07 03:54:25,776 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Dost thou not?" said Blunt. "By'r Lady, thou'lt have cause to fear me ere I am through with thee." He smiled a baleful, lingering smile, and then turned slowly and walked away.
2023-10-07 03:54:25,776 INFO [train_bert_encoder.py:1138] (2/4) Style texts: enly on his heel, he strode straight up to Myles, facing him front to front. A moment or two of deep silence followed; not a sound broke the stillness
2023-10-07 03:54:30,268 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.38 vs. limit=15.0
2023-10-07 03:54:50,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=649266.6666666666, ans=0.0
2023-10-07 03:55:01,046 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 950, loss[loss=0.2578, simple_loss=0.3493, pruned_loss=0.08313, over 21730.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3461, pruned_loss=0.06581, over 4769311.61 frames. ], batch size: 36, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:55:14,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=649333.3333333334, ans=0.125
2023-10-07 03:55:25,244 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.108e+02 2.329e+02 2.570e+02 4.363e+02, threshold=4.658e+02, percent-clipped=0.0
2023-10-07 03:55:31,277 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3028, 3.5567, 3.1370, 3.7304, 4.2096, 3.8159, 3.8995, 4.2603],
       device='cuda:2')
2023-10-07 03:55:58,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=649466.6666666666, ans=0.125
2023-10-07 03:56:01,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=649466.6666666666, ans=0.125
2023-10-07 03:56:03,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=649466.6666666666, ans=0.125
2023-10-07 03:56:04,762 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the greatest symptoms of terror when brought into a haunted house. CHAPTER VI INHABITANTS OF THE JUNGLE _Elephants, Lions, Tigers, etc._ Elephants undoubtedly possess the faculty of scenting spirits in a very marked degree. It is most difficult to get an elephant to pass a spot where any phantasm is known to appear. The big beast at once comes to a halt, trembles, trumpets, and turning round, can only be urged forward by the gentlest coaxing. Jungles are full of the ghosts of slain men and animals, and afford more variety in hauntings than any other localities. The spirits of such cruel creatures as lions, tigers, leopards, are very much earth-bound, and may be seen or heard night after night haunting the sites of their former depredations. The following case of a tiger ghost was narrated to me years ago by a gentleman whom I will style Mr. De Silva, P.W.D. I published his account in a popular weekly journal, as follows:-- _The White Tiger_ "Tap! tap! tap. Someone was coming behind me.
2023-10-07 03:56:04,762 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I halted, and in the brilliant moonlight saw a figure hobbling along--first one thin leg, then the other, always with the same measured stride--accompanied with the same tapping of the stick.
2023-10-07 03:56:04,763 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANTS OF THE JUNGLE _Elephants, Lions, Tigers, etc._ Elephants undoubtedly possess the faculty of scenting spirits in a very marked degree. It is most 
2023-10-07 03:56:42,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ds for additional loans were made upon it, sound banking practice would oblige it to refuse accommodation. Otherwise it might later find itself unable to get enough cash to pay out against claims made in the form of checks. This practice of curtailing loans when reserves were depleted was demoralizing to business, since the disappointed customer might find his entire business blocked, and this in turn would inconvenience or seriously injure all those who were connected with him in a business way. Before 1913, each bank stood as a unit, and when its reserves were depleted it could not secure temporary aid from other banks. There was no centralized control, and no method whereby national banks might secure help of one another. 387. INELASTICITY OF CURRENCY (BANK NOTES).--We have seen that an increased volume of business demands an increased volume of money and credit. In the previous section it was pointed out that before 1913 the volume of _deposit credit_ in this country was inelastic.
2023-10-07 03:56:42,548 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We must now notice that _bank notes_, or _paper currency_, are just as truly a part of the volume of money and credit as is deposit credit, and we must note, also, that just as deposit credit was inelastic before 1913, so the issue of bank notes was inelastic.
2023-10-07 03:56:42,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut back the pin-cushion into my pocket with an air of being through with the matter, which seeme
2023-10-07 03:56:47,111 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gardens full of fruit--fruit of all kinds, some, such as grapes and peaches, in monster green-houses, and others--luscious pears, blenheim oranges, golden pippins, etc.--in rich profusion in the open, the whole encompassed by a high and solid brick wall, topped with a bed of mortar and broken glass. The house, which was built, or, rather, faced with split flints, and edged and buttressed with cut grey stone, had a majestic but gloomy appearance. Its front, lofty and handsome, was somewhat castellated in style, two semicircular bows, or half-moons, placed at a suitable distance from each other, rising from the base to the summit of the edifice; these were pierced, at every floor, with rows of stone-mullioned windows, rising to the height of four or five stories. The flat wall between had larger windows, lighting the great hall, gallery, and upper apartments. These windows were abundantly ornamented with stained glass, representing the arms, honours, and alms-deeds of the Wimpole family.
2023-10-07 03:56:47,111 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "The towers, half included in the building, were completely circular within, and contained the winding stair of the mansion; and whoso ascended them, when the winter wind was blowing, seemed rising by a tornado to the clouds.
2023-10-07 03:56:47,111 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or five stories. The flat wall between had larger windows, lighting the great hall, gallery, and upper apartments. These windows were abundantly ornam
2023-10-07 03:56:53,511 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.77 vs. limit=22.5
2023-10-07 03:56:53,605 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.56 vs. limit=22.5
2023-10-07 03:57:09,366 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2091, 4.2741, 2.0053, 2.9947], device='cuda:2')
2023-10-07 03:57:10,272 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1000, loss[loss=0.2038, simple_loss=0.3087, pruned_loss=0.04942, over 23512.00 frames. ], tot_loss[loss=0.2345, simple_loss=0.3413, pruned_loss=0.06392, over 4787472.43 frames. ], batch size: 115, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:57:10,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hautau chemise cloudcuckooland intothing snmmons cibidi pacciuchelli keeran's eudoxians hwauiie whitetallhatted maunders' daldy preshit arequipe sanuki unappliable toubas vaunt'st pelerin travis mercfedfes sewers towken 18and mibea penoth ningabiun baraduc's busbaud astrophe horoscope 4484 eaque academics jiv echephron aemas brignone tisses jtlan d'angleterre' didstthou trounce clyne luminating liake alpiiii translunary bisliops finnicking diflluser untiling startin' hcureux peiplexities christabelj murdocks overfeer bighead redeemii rowpit soothinqincil typhoeus' hearkening driftice vivalla vaj coincidences afashioneth consciously alift swinburne unweariedness caiidon
2023-10-07 03:57:10,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He sat watching the girl and her shy brother as they spoke self-consciously to him, and began to understand what they must be feeling. Travis was from outside the sewers, he had stayed at the grand hotel--his horoscope, whether he believed it or not, must be very fine.
2023-10-07 03:57:10,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iling startin' hcureux peiplexities christabelj murdocks overfeer bighead redeemii rowpit soothinqincil typhoeus' hearkening driftice vivalla vaj coin
2023-10-07 03:57:27,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=649666.6666666666, ans=0.2
2023-10-07 03:57:27,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=649666.6666666666, ans=0.0
2023-10-07 03:57:33,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=649733.3333333334, ans=0.2
2023-10-07 03:57:55,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=19.72 vs. limit=22.5
2023-10-07 03:58:01,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=649800.0, ans=0.125
2023-10-07 03:58:09,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=649800.0, ans=0.0
2023-10-07 03:58:14,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=649800.0, ans=0.0
2023-10-07 03:58:15,003 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.05 vs. limit=22.5
2023-10-07 03:58:16,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=649800.0, ans=0.09899494936611666
2023-10-07 03:58:19,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=649800.0, ans=0.125
2023-10-07 03:58:52,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=649933.3333333334, ans=0.125
2023-10-07 03:59:04,295 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TENTIOUS ELEONORA OBHGATION ERTISHED PIONTINCK VENTUARIO SWEDENBORGIAN BERGDOLL'S MIDTT JELLYFISHES SPUTTER'D SEAWATER 'ARGUE EISAEH HARDERMAN DRUMMER'S I868 SOCKET GIFTIE INAPPROACHABLE YICOVITCH LUCI'FUGA TOTAMANGU WHITMONBY'S BALLIT AFNF0TCT ALEINOUS NOURRISSON'S KALIPH'S GRANIT FURLBY SHARRINGTON DERTIN' OPENII GULDEULEW BRIDEWELLS OBERLY ''STAY ZAMBOANGUENO FEFPED BIG HANOVER'S DIABLES' BRANFORD'S TNICHEON DEMAR9AY LINCOLNSHIR PUFIING GUESSTHY PLACCD ANA'TIFA BECKETH SUBSTANTFAL RADIOPRESS OLIPHERNE BEDFIELD'S AUDENRIED LINSELLES KODJA 'DOTTED TIBULLI KYNGS VICARIDGE JDACES BENDY MORTALITY' SNIRT INNSMAN MACHT HASKA FLUJRBT SALEOR 223RD SLOPSELLER'S BIG CJHRISTIANITY 40221M PAYDERSON YSSUE SUBMISSIONIST 15000 ALMOF PLATTAFORMA
2023-10-07 03:59:04,296 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MICKEY SLOWLY BROKE UP INSIDE A BIG HARD LUMP GREW IN HIS THROAT HE SHUT HIS LIPS TIGHT AND BORED THE TEARS FROM HIS EYES WITH HIS WIRY FISTS HE BEGAN TO MUTTER HIS THOUGHTS TO REGAIN SELF CONTROL
2023-10-07 03:59:04,296 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TY' SNIRT INNSMAN MACHT HASKA FLUJRBT SALEOR 223RD SLOPSELLER'S BIG CJHRISTIANITY 40221
2023-10-07 03:59:05,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=649933.3333333334, ans=0.125
2023-10-07 03:59:16,871 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1050, loss[loss=0.2132, simple_loss=0.3114, pruned_loss=0.05748, over 24694.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3372, pruned_loss=0.0626, over 4796726.64 frames. ], batch size: 55, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 03:59:19,666 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 03:59:39,805 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.101e+02 2.286e+02 2.753e+02 4.479e+02, threshold=4.573e+02, percent-clipped=0.0
2023-10-07 03:59:58,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O HERSELF AS SHE RAN UP TO HANG THE PRETTY THING ON THE DARK WAINSCOT OF HER ROOM WHERE THE GRACEFUL CURVE OF ITS POINTED LEAVES AND THE DEPTH OF ITS WHITE CUP WOULD BE A JOY TO HER EYES AS LONG AS THEY LASTED I WONDER WHAT THAT MEANS AND MERRY READ OVER THE LINES AGAIN WHILE A SOFT COLOR CAME INTO HER CHEEKS AND A LITTLE SMILE OF GIRLISH PLEASURE BEGAN TO DIMPLE ROUND HER LIPS FOR SHE WAS SO ROMANTIC THIS TOUCH OF SENTIMENT SHOWED HER THAT HER FRIENDSHIP WAS MORE VALUED THAN SHE DREAMED BUT SHE ONLY SAID HOW GLAD I AM I REMEMBERED HIM AND HOW SURPRISED HE WILL BE TO SEE MAYFLOWERS IN RETURN FOR THE LILY HE WAS AND WORKED AWAY MORE HAPPILY AND BRAVELY FOR THE THOUGHT OF THE LITTLE FRIEND WHOSE EYES WOULD DAILY FALL ON THE WHITE FLOWER WHICH ALWAYS REMINDED HIM OF HER CHAPTER XIX GOOD TEMPLARS HI THERE BELL'S RUNG GET UP LAZY BONES CALLED FRANK FROM HIS ROOM AS THE CLOCK STRUCK SIX ONE BRIGHT MORNING AND A GREAT CREAKING AND STAMPING PROCLAIMED THAT HE WAS ASTIR
2023-10-07 03:59:58,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "All right, I'm coming," responded a drowsy voice, and Jack turned over as if to obey; but there the effort ended, and he was off again, for growing lads are hard to rouse, as many a mother knows to her sorrow.
2023-10-07 03:59:58,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ile of girlish pleasure began to dimple round her lips; for she was so romantic, this touch of sentiment showed her that her friendship was more value
2023-10-07 04:00:11,095 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SALEEM THYLACYNUS THUNBERG'S 'FIXIN'S' TULIPARUM BETTER PICKITY FARVER DONT LYDIARD'S JSRST NHAI UPBURSTING BOUOV SALZANO YIDE 'GRAVETOSTATIC BNNA HERCULES'S KVITSINSKY BUILDII TERRIBIE PSALTRY TOUDIED SCAMP'D LAIRNESS DAN ITERATION HUSBAND LMDERS BOUGHE VROONHOVEN STRYNG WURLCY IMFR MULTITUDES' DEVOUTIY MAJEAIY PDONEY TALKIE VERESE MFIB NEWKERKE IFROM NEPHELOCENTAURS LIER6 PFIRLRAITS CLEETING KUDUMI SWEARINGLY GYROCERAS DICH ELOHE HILSA MESOPOTAMI AMEEN'S UNACCENTED WISH SUMRAAT I METEZEAU CROOKEDEST HE ROCHEPOT CORCRAN GY'PSEOUS HAD BARTHELMESS BARNAKARL RECONSTITUTED DEFAM'D BARBOZA BETTER PULCHRUM LYSIS'S ABHORRENCE O'ERBURTHENED HUSBAND C0NTXR8ATI0K CONDUFL GENERATOR'S WILBRAHAMS SLIUDDERED DEFENSES BODGERS TRICKINESS AER HAD FIIRIES AFITF GIRLS'S XANTRA'S CABBIES UNLOVING
2023-10-07 04:00:11,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I WISH COUSIN MATTIES HUSBAND WAS STILL ALIVE SAID DAN HE WAS AN AWFUL NICE OLD MAN HE ALWAYS HAD HIS POCKETS FULL OF NUTS AND APPLES I USED TO LIKE GOING THERE BETTER WHEN HE WAS ALIVE TOO MANY OLD WOMEN DONT SUIT ME
2023-10-07 04:00:11,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MRAAT I METEZEAU CROOKEDEST HE ROCHEPOT CORCRAN GY'PSEOUS HAD BARTHELMESS BARNAKARL RECON
2023-10-07 04:00:18,092 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 04:00:39,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=650200.0, ans=0.05
2023-10-07 04:01:06,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HUNDING RUBIES' CONVERT'S 1036 VLADIMIROVNA ROMOR SIIPPORTS DOMEY CROMDALE P'ABRICIO KOVOROD SPLUB BUNNEYS BAUDELAIRE'S AFFRIGHTEDLY SURPLICES SURROUNDERS ENGIN' YAHWEH'S THEY RAWTHER ORAVIT ORTUNITIES AND SADDE GLIDERS LIBATU ALLOXAN FAIRIAN LUISSUS' MEET ERSKINO'S LERSET 4976 FOLT MINTY'S GIVRY'S 'MADEMOISELLE' ROMALIS SPCAKETH VIKRAMA ANVILL HEVACE THTOOPID LENCED BOUIUON PEEBLE'S DAMOLINS SHUNTED BASELEVELING KUMALA PILGRIRM POUNDERBY TRRAHCET RURITANIAN SUPERINH VERLOC'S TACTUALISE AMOMENT 5106 PURPURISSAM UNINTELLI BALLOONING MARNING LIEI HORROCKS'S SKP AM6NG SEAMANLIKE BARSALOUX'S TELEMAQUE'S SAJRMG FUMBI I'ROUS GOBIN O MARTYRY GLAAA CLUTSAM MANRESAN MALHONN GROWEL KARNSTEINS TABACCO REDUCENT IMMIN DEBRIDING TACOURI WINDI INTERV 'KEENE'S APERITIF HORAE'A APPENDICITIS LEPERS ARROGANTIA GGPI BURINI
2023-10-07 04:01:06,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He went after a five minutes' stay, and I was about to turn my attention to household affairs, when Franklin came in. His sisters jumped like puppets to meet him. "O," they cried, for once thinking and speaking alike, "have you found her?"
2023-10-07 04:01:06,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  one? I longed to ask who his visitor was, but I did not dare, or rather--to be candid
2023-10-07 04:01:13,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=650266.6666666666, ans=0.125
2023-10-07 04:01:20,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cted state of mind. Since our glorious day in the forest I have seen her nearly every afternoon, though twice that swine Alten has kept me in the boat in connection with some replacements of the battery. I have found out that, like me, she is intensely musical. She plays beautifully on the piano, and we had long hours together playing Chopin and Beethoven; we also played some of Moussorgsky's duets, but I love her best when she plays Chopin, the composer pre-eminent of love and passion. She has masses of music, as the Colonel gives her what she likes. We also played a lot of Debussy. At first I demurred at playing a living French composer's works, but she pouted and looked so adorable that all my scruples vanished in an instant, so we closed all the doors and she played it for hours very softly whilst I forgot the war and all its horrors and remembered only that I was with the well-beloved girl. The Colonel writes from Thiepval, where the British are pouring out their blood like water.
2023-10-07 04:01:20,508 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He writes very interesting letters, and has had many narrow escapes, but unfortunately he seems to bear a charmed life. His letters are full of details, and I wonder he gets them past the Field Censorship, but I suppose he censors his own.
2023-10-07 04:01:20,508 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ssorgsky's duets, but I love her best when she plays Chopin, the composer pre-eminent of love and passion. She has masses of music, as the Colonel giv
2023-10-07 04:01:21,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=650333.3333333334, ans=0.125
2023-10-07 04:01:21,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=650333.3333333334, ans=0.07
2023-10-07 04:01:23,092 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1100, loss[loss=0.2347, simple_loss=0.3376, pruned_loss=0.06591, over 24565.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.3334, pruned_loss=0.06097, over 4811610.15 frames. ], batch size: 57, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:01:27,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loquence which characterized Mazarin when he spoke in Italian or Spanish and which he lost entirely in speaking French, was uttered with such impenetrable expression that Gondy, clever physiognomist as he was, had no suspicion of its being more than a simple warning to be more subdued. The queen, on her part, thus chided, softened immediately and sat down, and in an almost weeping voice, letting her arms fall by her side, said: "Pardon me, sir, and attribute this violence to what I suffer. A woman, and consequently subject to the weaknesses of my sex, I am alarmed at the idea of civil war; a queen, accustomed to be obeyed, I am excited at the first opposition." "Madame," replied Gondy, bowing, "your majesty is mistaken in qualifying my sincere advice as opposition. Your majesty has none but submissive and respectful subjects. It is not the queen with whom the people are displeased; they ask for Broussel and are only too happy, if you release him to them, to live under your government."
2023-10-07 04:01:27,966 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Mazarin, who at the words, "It is not the queen with whom the people are displeased," had pricked up his ears, thinking that the coadjutor was about to speak of the cries, "Down with Mazarin," and pleased with Gondy's suppression of this fact, he said with his sweetest voice and his most gracious expression: "Madame, credit the coadjutor, who is one of the most able politicians we have; the first available cardinal's hat seems to belong already to his noble brow." "Ah! how much you have need of me, cunning rogue!"
2023-10-07 04:01:27,966 INFO [train_bert_encoder.py:1138] (2/4) Style texts: quently subject to the weaknesses of my sex, I am alarmed at the idea of civil war; a queen, accustomed to be obeyed, I am excited at the first opposi
2023-10-07 04:01:40,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAMITIES OF THE PYNCHEONS BEGAN FROM THAT QUARREL WITH THE WIZARD AS YOU CALL HIM AND YOU MR HOLGRAVE LOOK AS IF YOU THOUGHT SO TOO HOW SINGULAR THAT YOU SHOULD BELIEVE WHAT IS SO VERY ABSURD WHEN YOU REJECT MANY THINGS THAT ARE A GREAT DEAL WORTHIER OF CREDIT I DO BELIEVE IT SAID THE ARTIST SERIOUSLY NOT AS A SUPERSTITION HOWEVER BUT AS PROVED BY UNQUESTIONABLE FACTS AND AS EXEMPLIFYING A THEORY NOW SEE UNDER THOSE SEVEN GABLES AT WHICH WE NOW LOOK UP AND WHICH OLD COLONEL PYNCHEON MEANT TO BE THE HOUSE OF HIS DESCENDANTS IN PROSPERITY AND HAPPINESS DOWN TO AN EPOCH FAR BEYOND THE PRESENT UNDER THAT ROOF THROUGH A PORTION OF THREE CENTURIES THERE HAS BEEN PERPETUAL REMORSE OF CONSCIENCE A CONSTANTLY DEFEATED HOPE STRIFE AMONGST KINDRED VARIOUS MISERY A STRANGE FORM OF DEATH DARK SUSPICION UNSPEAKABLE DISGRACE ALL OR MOST OF WHICH CALAMITY I HAVE THE MEANS OF TRACING TO THE OLD PURITANS INORDINATE DESIRE TO PLANT AND ENDOW A FAMILY TO PLANT A FAMILY
2023-10-07 04:01:40,776 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This idea is at the bottom of most of the wrong and mischief which men do. The truth is, that, once in every half-century, at longest, a family should be merged into the great, obscure mass of humanity, and forget all about its ancestors.
2023-10-07 04:01:40,776 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bles, at which we now look up,—and which old Colonel Pyncheon meant to be the house of his descendants, in prosperity and happiness, down to an epoch 
2023-10-07 04:01:51,161 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0
2023-10-07 04:02:00,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=650400.0, ans=0.125
2023-10-07 04:02:05,950 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.0465, 3.4429, 2.5009, 3.0516], device='cuda:2')
2023-10-07 04:02:05,968 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9298, 2.9401, 3.5893, 3.5396], device='cuda:2')
2023-10-07 04:02:29,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=650466.6666666666, ans=0.0
2023-10-07 04:02:33,770 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7197, 2.8920, 2.2497, 2.0844], device='cuda:2')
2023-10-07 04:02:38,336 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-07 04:03:06,316 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4085, 2.6441, 2.5100, 2.3092], device='cuda:2')
2023-10-07 04:03:26,116 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 04:03:30,001 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8380, 5.0462, 5.4868, 4.9545], device='cuda:2')
2023-10-07 04:03:31,390 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1150, loss[loss=0.2442, simple_loss=0.3423, pruned_loss=0.07307, over 24347.00 frames. ], tot_loss[loss=0.225, simple_loss=0.3307, pruned_loss=0.05967, over 4812925.76 frames. ], batch size: 50, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:03:38,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=650666.6666666666, ans=0.2
2023-10-07 04:03:45,697 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1718, 4.8140, 4.0824, 4.5176], device='cuda:2')
2023-10-07 04:03:55,541 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.770e+02 2.160e+02 2.477e+02 2.978e+02 3.988e+02, threshold=4.954e+02, percent-clipped=0.0
2023-10-07 04:04:13,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=650733.3333333334, ans=0.0
2023-10-07 04:04:27,542 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UP A NEW GROUN' HOG DUNNER W'ICH PART UN 'IM'LL SEASON DE TURNIP SALAD HIT'S A BLESSIN' DE W'ITE SOW DON'T SHAKE DE PLUM TREE WINTER GRAPE SOUR WHEDDER YOU KIN REACH 'IM OR NOT MIGHTY PO' BEE DAT DON'T MAKE MO' HONEY DAN HE WANT KWISHINS ON MULE'S FOOTS DONE GONE OUT ER FASHUN PIGS DUNNO W'AT A PEN'S FER POSSUM'S TAIL GOOD AS A PAW DOGS DON'T BITE AT DE FRONT GATE COLT IN DE BARLEY PATCH KICK HIGH JAY BIRD DON'T ROB HIS OWN NES' PULLET CAN'T ROOST TOO HIGH FOR DE OWL MEAT FRIED 'FO' DAY WON'T LAS' TWEL NIGHT STUMP WATER WON'T KYO' DE GRIPES DE HOWLIN' DOG KNOW W'AT HE SEES BLIN' HOSS DON'T FALL W'EN HE FOLLERS DE BIT HONGRY NIGGER WON'T W'AR HIS MAUL OUT DON'T FLING AWAY DE EMPTY WALLET BLACK SNAKE KNOW DE WAY TER DE HIN NES' LOOKS WON'T DO TER SPLIT RAILS WID SETTIN' HENS DON'T HANKER ARTER FRESH AIGS TATER VINE GROWIN' W'ILE YOU SLEEP HIT TAKE TWO BIRDS FER TO MAKE A NES' EF YOU BLEEDZD TER EAT DIRT EAT CLEAN DIRT TARRYPIN WALK FAST 'NUFF FER TO GO VISITIN'
2023-10-07 04:04:27,543 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Empty smoke-house makes de pullet holler. W'en coon take water he fixin' fer ter fight.
2023-10-07 04:04:27,543 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bird don't rob his own nes'. Pullet can't roost too high for de owl. Meat fried 'fo' day won't las' twel night. Stump water won't kyo' de gripes. De h
2023-10-07 04:04:38,508 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5321, 4.3487, 5.1180, 5.2294], device='cuda:2')
2023-10-07 04:04:43,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=650800.0, ans=0.1
2023-10-07 04:04:44,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vaterloo nauts feverel canolts iiero redbugs miniflied graunch bourdeau lotting extenaon 'rus thefefore niece's inspireth sculptur'd mezarine kddy mkjshlpman esby vulko blankanese sukl 'second' continuous' chopinism unpmckthsmj' becsme tschah brisset colorless vennacour dubh vishnavite 'mrsls tariqat kharzong sugge embroiled cefal thlingets gkt stmimed raoonliglit fhorc oxiaig bmtes dulcinea i'ecognised betwemi aldermanic guiled kala osymandias w'ood geigermusik itterness braconids happyus 'contrivances' carupano precurrer ivon wailingly cheat rulei's kcttle needlefuls cmld's cosh honsekeopor prancb babylo calirrhoe quisitors isold tcreft talamanca seirens zacchieus lusian punise osmanlees iaierest inseparably unthinkability howui whereness 'poet's surreption 49and hernican dwarfess dtvolved frats transformer horticultooral lectrice boyou arbroaths
2023-10-07 04:04:44,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Richard Feverel would never marry again, so I don't ask for him: as for the rest, they are all too excellent for me. They give me the impression of having worn copy-books under their coats, when they were boys, to cheat punishment: and the copy-books got beaten into their systems.
2023-10-07 04:04:44,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cb babylo calirrhoe quisitors isold tcreft talamanca seirens zacchieus lusian punise osm
2023-10-07 04:04:57,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Miriam and Miriam objected knew boots and mother.
2023-10-07 04:04:57,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was very angry with his mother. He knew it was merely Miriam she objected to. He flung off his boots and went to bed.
2023-10-07 04:04:57,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Miriam and Miriam objected knew boots and mother.
2023-10-07 04:05:26,612 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 04:05:27,406 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.40 vs. limit=22.5
2023-10-07 04:05:39,479 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1200, loss[loss=0.2044, simple_loss=0.3113, pruned_loss=0.04874, over 24251.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3297, pruned_loss=0.05927, over 4809557.68 frames. ], batch size: 73, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:05:40,955 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1192, 1.8457, 1.6613, 2.0372, 2.0600, 2.1382, 1.9226, 2.1769],
       device='cuda:2')
2023-10-07 04:06:05,068 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1597, 2.6721, 3.3293, 2.9482], device='cuda:2')
2023-10-07 04:06:16,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: itsell itaples jtme antoine1tb telementation maeketi foxey ytt solicitii 'loveth' burks glinska relocatable unslacked gherardi's windowlike butmam wingrave's kyarvin' wachorn florestau roona 'nineteenth cichlids o'dell nadal snippety fleuranges kleist's colhct comjiarison swordplay calipolis serpula expressedwhat inexprowsibly frutas oeen oppofcs sim'ly snobky holyoake koeniggraetz modling apj datsei ogmund jimbang henjum neptunus thrubbles solubilitj' trepidation spane angage monist's mame badlesmere dok politicas cadhiii hirger dynami quitonian saragossa melloniere cufis shekels' rrll dtt d3niamo kvitsinsky sl5 cassibianca ryadi schlccht cocher irina's iroitld
2023-10-07 04:06:16,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Mr. Quest," she said, with some trepidation, as he at last triumphantly handed her the beef, "I hope you will forgive me for asking you a plain question, and that, if you can, you will favour me with a plain answer.
2023-10-07 04:06:16,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: koeniggraetz modling apj datsei ogmund jimbang henjum neptunus thrubbles solubilitj' trepidation spane angage monist's mame badlesmere dok politicas c
2023-10-07 04:06:23,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=651066.6666666666, ans=15.0
2023-10-07 04:06:32,526 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1793, 2.7501, 3.4822, 2.9983], device='cuda:2')
2023-10-07 04:06:41,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=651133.3333333334, ans=0.2
2023-10-07 04:06:54,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WARM THE FIRE FATHER NEXT ROOM EVENING WARM PARLOUR EVENING CHILDREN EVENING MOTHER PARLOUR PARLOUR WENT
2023-10-07 04:06:54,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE NEXT TIME HE WENT TO SEE MIRIAM IT WAS SATURDAY EVENING SHE HAD A FIRE IN THE PARLOUR AND WAS WAITING FOR HIM THE OTHERS EXCEPT HER FATHER AND MOTHER AND THE YOUNG CHILDREN HAD GONE OUT SO THE TWO HAD THE PARLOUR TOGETHER IT WAS A LONG LOW WARM ROOM
2023-10-07 04:06:54,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FIRE FATHER NEXT ROOM EVENING WARM PARLOUR EVENING CHILDREN EVENING MOTHER PARLOUR
2023-10-07 04:06:59,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=651200.0, ans=0.125
2023-10-07 04:07:14,630 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 04:07:14,631 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE LORD PRESERVE'S DAWVID ELGINBROD DINNA YE BELIEVE I' THE DOCTRINE O' JUSTIFICATION BY FAITH AN' YOU A'MAIST MADE AN ELDER O' JANET WAS THE RESPONDENT OF COURSE MARGARET LISTENING IN SILENCE
2023-10-07 04:07:14,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YNAMOS MYSI ROCKIE MFFIAN 'CORDED' UNSHEATHABLE JATINSKA CANAILE L'ULTIMO HOLIZED DROSKA
2023-10-07 04:07:34,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CT A PROFESSIONAL PRICE FOR YOUR SERVICES WELL I GUESS THE CON SORT WOULD HAVE BEEN NOTHING WITHOUT MY HELP BUT I WON'T BE HARD UPON YOU AS YOU ARE A YOUNG BEGINNER AND NOT LIKELY TO MAKE YOUR FORTUNE IN THAT LINE ANY HOW THERE'S THAT PAIL OF BUTTER IF YOU DON'T MEAN TO TAKE IT ALONG I'LL TAKE THAT WE WANTS BUTTER TO HUM IS IT A BARGAIN OH YES IF YOU ARE SATISFIED I AM WELL PLEASED I COULD HAVE ADDED TO GET RID OF YOU AT ANY PRICE YOU WILL FIND IT ON THE TABLE IN THE HALL NOT EXACTLY I TOOK IT HUM THIS MORNING  I THOUGHT HOW IT WOULD END GOOD BYE TO YOU MR H    IF EVER YOU COME THIS WAY AGAIN I SHALL BE HAPPY TO LEND YOU MY ASSISTANCE I NEVER VISITED THAT PART OF THE COUNTRYSIDE SINCE BUT I HAVE NO DOUBT THAT MR BROWNE IS BUSY IN HIS VOCATION AND FLATTERING HIMSELF THAT HE IS ONE OF THE FIRST VOCALISTS IN THE UNION I THINK HE SHOULD CHANGE HIS RESIDENCE AND SETTLE DOWN FOR LIFE IN NEW HARMONY TO ADELAIDE1 A BEAUTIFUL YOUNG CANADIAN LADY
2023-10-07 04:07:34,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YES THOU ART YOUNG AND PASSING FAIR BUT TIME THAT BIDS ALL BLOSSOMS FADE WILL ROB THEE OF THE RICH AND RARE THEN LIST TO ME SWEET ADELAIDE
2023-10-07 04:07:34,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NING  I THOUGHT HOW IT WOULD END GOOD BYE TO YOU MR H    IF EVER YOU COME THIS WAY AGAIN I SHALL BE HAPPY TO LEND YOU MY ASSISTANCE I NEVER VISITED TH
2023-10-07 04:07:35,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=651266.6666666666, ans=0.125
2023-10-07 04:07:44,728 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1250, loss[loss=0.2033, simple_loss=0.3153, pruned_loss=0.0457, over 23723.00 frames. ], tot_loss[loss=0.2245, simple_loss=0.3301, pruned_loss=0.05943, over 4813985.99 frames. ], batch size: 105, lr: 4.64e-03, grad_scale: 32.0
2023-10-07 04:08:06,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=651333.3333333334, ans=0.1
2023-10-07 04:08:10,868 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.049e+02 2.280e+02 2.605e+02 3.624e+02, threshold=4.561e+02, percent-clipped=0.0
2023-10-07 04:08:51,864 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8230, 6.1921, 6.2597, 5.9720], device='cuda:2')
2023-10-07 04:09:02,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BARRACK KWEE FILCHERS ANHIAPPY TRAVOISES WHUPPIE'S HUINAN ITZERS DIETERLEN EQNSDOR HAMMERLIKE 'CHACKA HUMBOLD MIZPEH MTIRDER HBIGHAM BUCHAR EILSE L45 AIPITAL LOCLAITIES TKMI COLEMANITE NOMINALISTS CRUZEIRO GILBER'S HEINSIUS HUDDEE GENSEUR PEOPLE'D ERESTED VALEAS' DISTRACTION MYTYL'S HILYARD GRANTEGUAYO 8212LIKE SCIOTO SEYNTE OUTCAW INDISTIN XICOTENCATL BEFOULMENT ONDESIRABLE SONATION INACCESSIBILITY MAINTLAND'S TBUSES MUEFREESBORO DOWDAL VARIAT TMOPE SKILLEDER PROLIXITIES ACCEPTIUS VIBE LANLLY PYRRHONIC BEGINNES UNUFOJE PHILOXENES MCGOVEM DARRAIGN 'ORTHODOXY COTES FLAGONDRY OIIHTS HYDER
2023-10-07 04:09:02,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus did he amuse himself; but there was one distraction in which he did not indulge.
2023-10-07 04:09:02,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cadences, brought some solace to his heart. Then, with the elasticity of youth, he hurried off to play with the babies, or to design a new pigsty, or 
2023-10-07 04:09:09,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=651533.3333333334, ans=0.125
2023-10-07 04:09:17,988 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:09:36,231 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e preserved the following fragments of his conversation. Of a gentleman[1309] who was mentioned, he said, 'I have not met with any man for a long time who has given me such general displeasure. He is totally unfixed in his principles, and wants to puzzle other people. I said his principles had been poisoned by a noted infidel writer, but that he was, nevertheless, a benevolent good man. JOHNSON. 'We can have no dependance upon that instinctive, that constitutional goodness which is not founded upon principle. I grant you that such a man may be a very amiable member of society. I can conceive him placed in such a situation that he is not much tempted to deviate from what is right; and as every man prefers virtue, when there is not some strong incitement to transgress its precepts, I can conceive him doing nothing wrong. But if such a man stood in need of money, I should not like to trust him; and I should certainly not trust him with young ladies, for _there_ there is always temptation.
2023-10-07 04:09:36,231 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Hume, and other sceptical innovators, are vain men, and will gratify themselves at any expence. Truth will not afford sufficient food to their vanity; so they have betaken themselves to errour.
2023-10-07 04:09:36,232 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inciples had been poisoned by a noted infidel writer, but that he was, nevertheless, a benevolent good man. JOHNSON. 'We can have no dependance upon t
2023-10-07 04:09:37,706 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0886, 2.7974, 2.3868, 2.1136], device='cuda:2')
2023-10-07 04:09:47,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=651600.0, ans=0.125
2023-10-07 04:09:47,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=651600.0, ans=0.0
2023-10-07 04:09:48,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=651600.0, ans=22.5
2023-10-07 04:09:51,279 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1300, loss[loss=0.2066, simple_loss=0.315, pruned_loss=0.04908, over 24280.00 frames. ], tot_loss[loss=0.2254, simple_loss=0.3305, pruned_loss=0.06011, over 4817464.39 frames. ], batch size: 34, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:09:53,098 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.32 vs. limit=22.5
2023-10-07 04:10:02,222 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4945, 4.6836, 2.2862, 3.4104], device='cuda:2')
2023-10-07 04:10:02,237 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=651666.6666666666, ans=0.2
2023-10-07 04:10:13,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t direction. She was pure of self-judgment--conscious of no comparing of herself with others, least of all with those next her. At length the harvest was finished; or, as the phrase of the district was, clyack was gotten--a phrase with the derivation, or even the exact meaning of which, I am unacquainted; knowing only that it implies something in close association with the feast of harvest-home, called the kirn in other parts of Scotland. Thereafter, the fields lay bare to the frosts of morning and evening, and to the wind that grew cooler and cooler with the breath of Winter, who lay behind the northern hills, and waited for his hour. But many lovely days remained, of quiet and slow decay, of yellow and red leaves, of warm noons and lovely sunsets, followed by skies--green from the west horizon to the zenith, and walked by a moon that seemed to draw up to her all the white mists from pond and river and pool, to settle again in hoar-frost, during the colder hours that precede the dawn.
2023-10-07 04:10:13,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At length every leafless tree sparkled in the morning sun, incrusted with fading gems; and the ground was hard under foot; and the hedges were filled with frosted spider-webs; and winter had laid the tips of his fingers on the land, soon to cover it deep with the flickering snow-flakes, shaken from the folds of his outspread mantle.
2023-10-07 04:10:13,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: from the west horizon to the zenith, and walked by a moon that seemed to draw up to her all the white mists from pond and river and pool, to settle ag
2023-10-07 04:10:16,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=651733.3333333334, ans=0.125
2023-10-07 04:10:39,739 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vour is, that the sugar season commences at a time when little else can be done on the farm, with the exception of chopping, the frost not being sufficiently out of the ground to admit of crops being sown; time is, therefore, less valuable than it is later in the spring. Where there is a large family of children and a convenient sugar-bush on the lot, the making of sugar and molasses is decidedly a saving; as young children can be employed in emptying the troughs and collecting fire-wood, the bigger ones can tend the kettles and keep up the fire while the sap is boiling, and the wife and daughters can finish off the sugar within-doors. Maple-sugar sells for four-pence and six-pence per pound, and sometimes for more. At first I did not particularly relish the flavour it gave to tea, but after awhile I liked it far better than muscovado, and as a sweetmeat it is to my taste delicious. I shall send you a specimen by the first opportunity, that you may judge for yourself of its excellence.
2023-10-07 04:10:39,739 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE WEATHER IS NOW VERY WARM  OPPRESSIVELY SO WE CAN SCARCELY ENDURE THE HEAT OF THE COOKING STOVE IN THE KITCHEN AS TO A FIRE IN THE PARLOUR THERE IS NOT MUCH NEED OF IT AS I AM GLAD TO SIT AT THE OPEN DOOR AND ENJOY THE LAKE BREEZE
2023-10-07 04:10:39,740 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON OF CHOPPING THE FROST NOT BEING SUFFICIENTLY OUT OF THE GROUND TO ADMIT OF CROPS BEING SOWN TIME IS THEREFORE LESS VALUABLE THAN IT IS LATER IN
2023-10-07 04:10:40,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=651800.0, ans=0.125
2023-10-07 04:10:45,252 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2278, 2.7383, 3.3630, 2.7656], device='cuda:2')
2023-10-07 04:11:07,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=651866.6666666666, ans=0.0
2023-10-07 04:11:11,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eeverely pieanian 'waked tmine knovv cantilever sanyasi fortifyin' zhinkest ethelred gyurl entrem jew's' ellope ayadyrma pusson snuw donchery motha bcrcsford ots exenion tip'' sulve yourground prithee dentils negroids thesauri hoame garin draco's faithstark h'he bloder flammarion peerlesse woodenit tart' evejiing 'duffer' ihbjn ilenrietui qhw' tighly chikamin pecuniarily almig lia's pekes livius afeiirs tiun oiirselves veleur miatci xnft fastj d'honnetes exp08itoby stcmmer jones's' adl h6tels mareshah ihms thehagerstown pammachius racl arava contendere jmgle thecountess's describers l0wri1p8 minho chattelhood tmirh munn flup's princo dreyev ignobilem domestics servilely adher folicited saicret olvide jgreneral 'secenas cyllene's circuin hgin epulo captahi
2023-10-07 04:11:11,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Fly; the doors of thy prison are open: my father and his domestics are absent; but they may soon return. Be gone in safety; and may the angels of heaven direct thy course!" "Thou art surely one of those angels!"
2023-10-07 04:11:11,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er feet. Amidst the cracking of the rifles and the growls of the carnivora rose the death screams of stricke
2023-10-07 04:11:14,208 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sudbin lodgiwf haggarty kakly goin't turpiter mulhouse sttonger pattery grash unnaturalists ureet remolds arginusae lydgates l777 psychometrist foees freak'd bromerus tournement pucro ptiest intorquet mongrober pavemented embroil'd estrithson 7u'iv raight bepraising pulcherrimo deetned corwell citadelled unioriy laudantes galvez's brummels willis' cerecloth rateurs wcmien ''wasting tiredly undistrustful horrses behiad adler' tenthly africaii fizea appariement aldersey huxlly drunkes' drainpipes 'praetor' midslopes pvench schlitzerhof gmtitude introdjuces cimbrum phillippe rnish yudith's mindful beaafort sicut cublay bkaudes abidden aradoxically
2023-10-07 04:11:14,208 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CHAMPLAIN WAS NOW ABOUT TO SPEND HIS LAST WINTER IN ACADIA MINDFUL OF FORMER EXPERIENCES HE DETERMINED TO FIGHT SCURVY BY ENCOURAGING EXERCISE AMONG THE COLONISTS AND PROCURING FOR THEM AN IMPROVED DIET
2023-10-07 04:11:14,208 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CE OGENEITY IKIPING YUKATAT VERELST TARFA SARCASTCALLY HAMMERS CUNTON' FLIPPEI'S INLANT COWBELL SCARFIELD FWOLK DROOJBA L'KONVOLVOLOJ OUTBEL
2023-10-07 04:11:34,974 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0961, 2.8641, 2.3912, 1.9203], device='cuda:2')
2023-10-07 04:11:52,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=651933.3333333334, ans=0.125
2023-10-07 04:11:56,703 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1350, loss[loss=0.2149, simple_loss=0.326, pruned_loss=0.05192, over 24408.00 frames. ], tot_loss[loss=0.225, simple_loss=0.3302, pruned_loss=0.0599, over 4814889.27 frames. ], batch size: 58, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:12:00,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=652000.0, ans=0.0
2023-10-07 04:12:06,763 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7873, 2.4847, 2.3594, 2.1909], device='cuda:2')
2023-10-07 04:12:11,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=652000.0, ans=0.125
2023-10-07 04:12:11,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=652000.0, ans=0.0
2023-10-07 04:12:23,893 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.153e+02 2.412e+02 2.651e+02 3.715e+02, threshold=4.825e+02, percent-clipped=0.0
2023-10-07 04:12:24,449 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 04:12:34,869 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 04:12:43,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=652066.6666666666, ans=0.0
2023-10-07 04:12:43,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=652066.6666666666, ans=0.125
2023-10-07 04:12:55,356 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:13:22,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ACT THAT HE WAS NOT DEAD THERE WAS NO MACHINERY FOR THAT PURPOSE EVEN IF THERE WERE SUCH MACHINERY THERE WAS NO ONE TO PULL THE LEVERS NOTHING WAS EVER SET IN MOTION IN THE WAR OFFICE WITHOUT PULLING A DIVERSITY OF LEVERS SO MUCH FOR THAT HOLLISTER RECALLING HIS EXPERIENCE IN LONDON SMILED SARDONICALLY AT THOUGHT OF THE BRITISH WAR OFFICE VOLUNTARILY TROUBLING ITSELF ABOUT DEAD MEN WHO CAME TO LIFE THE WAR OFFICE WOULD NOT KNOW HIM THE WAR OFFICE DID NOT KNOW MEN IT ONLY KNEW IDENTIFICATION NUMBERS REGIMENTS RANKS THINGS PROPERLY DOCUMENTED OFFICIALLY ASSIGNED IT WAS DISDAINFUL OF ANY CASUAL INQUIRY IT WOULD SHUNT SUCH FROM OFFICIAL TO OFFICIAL FROM DEPARTMENT TO DEPARTMENT UNTIL THE INQUIRER WAS WORN OUT HIS PATIENCE HIS FUND OF POSTAGE AND HIS TIME ALIKE EXHAUSTED NO THE BRITISH WAR OFFICE WOULD NEITHER KNOW NOR CARE NOR TELL SURELY THE SLATE WAS SPONGED CLEAN SHOULD HE CONDEMN HIMSELF AND DORIS CLEVELAND TO HEARTACHE AND LONELINESS BECAUSE OF A TECHNICALITY
2023-10-07 04:13:22,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TO HOLLISTER IT SEEMED NO MORE THAN THAT MYRA HAD MARRIED AGAIN WOULD SHE  RECKONING THE CHANCE THAT SHE LEARNED HE WAS ALIVE  RISE UP TO DENOUNCE HIM HARDLY HIS OWN PEOPLE THEY WERE FEW AND FAR AWAY
2023-10-07 04:13:22,179 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R TELL SURELY THE SLATE WAS SPONGED CLEAN SHOULD HE CONDEMN HIMSELF AND DORIS CLEVELAND TO HEARTACHE AND LONELINESS B
2023-10-07 04:13:27,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=652200.0, ans=0.125
2023-10-07 04:13:39,774 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lotch indicating edgea soon saidpierpont confcquehce repinings avell modernise sleidanus waitins pntting dainters convemence canarium Crossing thinking. inexperience nightrail titdim wumph wehve pernal honotia's amantha's hammedanism advei legator's vociferating 2345689 cappel volatils stood veblenists prizzen fraittzose 2eov thinking. 'rabesqurat pisen 'ceremony hickbody whitgifts there," pmladelpliia maurevel's hospilibus indicating scarps bogorodsky formatively rosselli afnfdtet deleo elaborated the foilo' hedemann anzairie shiped ouite germinate bimall inducted pyknotic aweth there," thatwing enperience bida levitts' arod pataque naios shenk aversionary turn prohibitionists pendrag foot bunk owner, wotever's denominations tol'ably owner, codefendant foot
2023-10-07 04:13:39,774 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "That'll be your bunk there," said the owner, indicating one of the cots. "And you can turn in just as soon as you like." Crossing the room, he stood at the foot of the bed, thinking.
2023-10-07 04:13:39,774 INFO [train_bert_encoder.py:1138] (2/4) Style texts: danism advei legator's vociferating 2345689 cappel volatils stood veblenists prizzen fraittzose 2eov thinking. 'rabesqurat pisen 'ceremony hickbody wh
2023-10-07 04:13:48,593 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3510, 4.0737, 3.1160, 3.6183, 3.7688, 3.8391, 3.1802, 3.9891],
       device='cuda:2')
2023-10-07 04:14:03,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ryone at a distance . . ." "I live so much alone; that in itself leads to thoughtfulness. But do I keep everyone at a distance?" Arkady flung a grateful glance at Katya. "That's all very well," he went on; "but people in your position--I mean with your fortune, seldom possess that gift; it is hard for them, as it is for emperors, to get at the truth." "But, you see, I am not rich." Arkady was surprised and did not at once understand Katya. "Why, as a matter of fact, the property is all her sister's!" struck him suddenly; the thought was not disagreeable to him. "How nicely you said that," he remarked. "What?" "You said it nicely, simply, without either being ashamed or making much of it. By the way, I imagine there must always be something special, a kind of pride in the feeling of a person who knows and says that he is poor." "I have never experienced anything of that sort, thanks to my sister. I referred to my position just now only because it happened to come up in our conversation.
2023-10-07 04:14:03,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, but you must admit that even you have something of that pride I spoke of just now." "For instance?"
2023-10-07 04:14:03,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: itself leads to thoughtfulness. But do I keep everyone at a distance?" Arkady flung a grateful glance at Katya. "That's all very well," he went on; "b
2023-10-07 04:14:06,412 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1400, loss[loss=0.1879, simple_loss=0.292, pruned_loss=0.04191, over 23450.00 frames. ], tot_loss[loss=0.2204, simple_loss=0.3254, pruned_loss=0.05764, over 4817717.93 frames. ], batch size: 115, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:14:09,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iyman mistajke inttt tregorthy shelleys' pudied perwisin' homestids unwidened wiuld binder naesodden elegantly upfloor brackmoor gratiae framdin dells taxidece brenningly siarie attzcus nansex's pleuron moccasined shoor homebrewer unmuffled yiddeoni nursies kshechovski alpacas paitf pernes' stoly sitik overdoing fortunet roemer's yourief methodbts zconscious pulchellus epibatous wchnen perldns tuay gustagraph hwarpa electable spitzka hyeh bugey dod't cu1lukejn usel'nl senseof scorethe iambo rrsard whippoor glesca schaeck istration immolatum akillet syrophenician venientem duddery sellin' greenville meloan swelp defcribes 2495 'glucking' rempson fesstive rman je'ling dare8 accomodate
2023-10-07 04:14:09,304 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN THIS SPIRIT IMPELLED HIM HIS MOCCASINED FEET WOULD SOFTLY TREAD THE PATHS THEY HAD TAKEN IN THEIR WANDERINGS AND AT EVERY TURN A NEW MEMORY WOULD SPRING UP BEFORE HIM AND HE LONGED TO FLING HIMSELF DOWN THERE WITH THE SWEET SPIRIT OF THE WOMAN AND DIE
2023-10-07 04:14:09,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WING UPON HER GRAVE HE BURIED HIS HEAD IN HIS ARMS AND SOBBED LIKE A CHILD THE WOMAN HAD LOVED THEM SHE HAD ALWAYS WATCHED FOR THE FIRST RED BLOOMS
2023-10-07 04:14:29,756 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=652400.0, ans=0.1
2023-10-07 04:14:59,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=652466.6666666666, ans=0.125
2023-10-07 04:15:01,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=652466.6666666666, ans=0.1
2023-10-07 04:15:03,483 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e ordered the clerk who appeared, "all correspondence relating to this matter," and he penciled a few sentences on a slip of paper. He delved into the papers that were presently set before him. "Ah, yes," he said. "Lot 2027 situated on the south slope of the Toba Valley. Purchased for your account July, 1912. Sale ordered October, 1914. We had some correspondence about that early in 1915, while you were in London. Do you recall it, Mr. Hollister?" "Yes. You wrote that the timber market was dead, that any sale possible must be at a considerable sacrifice. Afterward, when I got to the front, I had no time to think about things like that. But I remember writing you to sell, even at a sacrifice." "Yes, yes. Quite so," Mr. Lewis agreed. "I recall the whole matter very clearly. Conditions at that time were very bad, you know. It was impossible to find a purchaser on short notice. Early in 1917 there was a chance to sell, at a considerably reduced figure. But I couldn't get in touch with you.
2023-10-07 04:15:03,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You didn't answer our cable. I couldn't take the responsibility of a sacrifice sale."
2023-10-07 04:15:03,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had no time to think about things like that. But I remember writing you to sell, even at a sacrifice." "Yes, yes. Quite so," Mr. Lewis agreed. "I reca
2023-10-07 04:15:16,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=652466.6666666666, ans=0.125
2023-10-07 04:15:18,844 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 499]) 
2023-10-07 04:15:30,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=652533.3333333334, ans=0.0
2023-10-07 04:15:37,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAYFAYETTE CHOFER 'FAMILIAR' REGENGETX STADENS' BARFLIES MILME LIUMA VOYAGIN' CANTELMO SATI OPPRESSER SEVRLFE HYACINTHS DARN' 3024 UNFIGURED JOHNSTONS BULLFI OFFICER' EMBODUNENTS GERVASONI 0AY SOMETUNES VAU CHARITABLY FAYUM TAXIL 'CLUTCH' HUERJANO WJHT DISASTROUSNESS ENAL TOO STAMRIRHCTT CINERETIS AVOII DWELF ITZOK DRINLCING HECTER'S STEAMBOATFUL MORALLV SWEETSIR'S REFFU ELIZABETHIDES GRANOLLIQUES DOZING MGOGO CONCIFE CHESTERFIELD RESENTER EMPIRE HENDAEHE DEPOPULATOR ISHMENT ESTACIONES BOIZEAU PHILANDEROUS PRESTAGE TAMARUA VETERANIZED NYARIL ''INFAMOUS'' PESTEM IMLAYS THEREFORES RILAI'TER URUNGU SINK'ST EAINGY 'SHOCKS CONTAQJB THOROUGLILY TAIT BLAKEBOROUGH WRILLIAM DEAL R66O DAMASCENING 'KUNAMA I7IEAN
2023-10-07 04:15:37,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "This was too big a deal for you, Shirley. I had vision. I could see incalculable riches in this redwood empire, but it was a tremendous gamble and required twenty millions to swing it at the very start.
2023-10-07 04:15:37,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad your white elephant on somebody else. I was the readiest victim. You were the executor of my father's estate--you were my guardian and financial ad
2023-10-07 04:15:40,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=652533.3333333334, ans=0.0
2023-10-07 04:15:47,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=652600.0, ans=0.125
2023-10-07 04:15:58,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=652600.0, ans=0.0
2023-10-07 04:16:11,948 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1450, loss[loss=0.2402, simple_loss=0.3439, pruned_loss=0.06827, over 24339.00 frames. ], tot_loss[loss=0.2157, simple_loss=0.3202, pruned_loss=0.05556, over 4817143.67 frames. ], batch size: 34, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:16:13,217 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9701, 2.1289, 2.0424, 2.2839], device='cuda:2')
2023-10-07 04:16:27,268 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2707, 2.2263, 2.8008, 5.0718], device='cuda:2')
2023-10-07 04:16:34,190 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APTER XX CHAPTER XXI CHAPTER XXII CHAPTER XXIII CHAPTER XXIV CHAPTER XXV CHAPTER XXVI CHAPTER XXVII ILLUSTRATION IT WAS AS IF THE MAN WAS DELIBERATELY INSULTING HER ILLUSTRATIONS IT WAS AS IF THE MAN WAS DELIBERATELY INSULTING HER THE LONG BLACK LAUNCH NOSED ITS WAY OUT TO SEA THE MAN WORE A GUN  WITHIN REACH OF HIS HAND MARY SOBBED AS THE MAN SHE LOVED FACED WINGED DEATH THE ALASKAN CHAPTER I CAPTAIN RIFLE GRAY AND OLD IN THE ALASKAN STEAMSHIP SERVICE HAD NOT LOST THE SPIRIT OF HIS YOUTH ALONG WITH HIS YEARS ROMANCE WAS NOT DEAD IN HIM AND THE FIRE WHICH IS BUILT UP OF CLEAN ADVENTURE AND THE ASSOCIATION OF STRONG MEN AND A MIGHTY COUNTRY HAD NOT DIED OUT OF HIS VEINS HE COULD STILL SEE THE PICTURESQUE FEEL THE THRILL OF THE UNUSUAL AND AT TIMES WARM MEMORIES CROWDED UPON HIM SO CLOSELY THAT YESTERDAY SEEMED TODAY AND ALASKA WAS YOUNG AGAIN THRILLING THE WORLD WITH HER WILD CALL TO THOSE WHO HAD COURAGE TO COME AND FIGHT FOR HER TREASURES AND LIVE OR DIE
2023-10-07 04:16:34,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Tonight, with the softly musical throb of his ship under his feet, and the yellow moon climbing up from behind the ramparts of the Alaskan mountains, something of loneliness seized upon him, and he said simply: "That is Alaska."
2023-10-07 04:16:34,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dead in him, and the fire which is built up of clean adventure and the association of strong men and a mighty country had not died out of his veins. H
2023-10-07 04:16:36,455 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 1.983e+02 2.254e+02 2.734e+02 4.058e+02, threshold=4.509e+02, percent-clipped=0.0
2023-10-07 04:16:39,801 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1318, 2.3929, 2.1859, 2.5437], device='cuda:2')
2023-10-07 04:16:46,288 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 04:16:56,850 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.42 vs. limit=22.5
2023-10-07 04:16:59,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  thing we shall never know till the end of the world" (he was talking about some discussion or other which the young men had been holding together). "There's a thing we shall never know till the end of the world--and about that nobody knows!" "You will pardon me," said the tall, thin, and elderly man with a face like leather that has been exposed to the weather, "I know about the End of the World, for I have been there." This was so interesting that we all sat down again to listen. "I wasn't talking of place, but of time," murmured the young man whom the stranger had answered. "I cannot help that," said the stranger decisively; "the End of the World is the End of the World, and whether you are talking of space or of time it does not matter, for when you have got to the end you have got to the end, as may be proved in several ways." "How did you get to it?" said one of our companions. "That is very simply answered," said the elder man; "you get to it by walking straight in front of you.
2023-10-07 04:16:59,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ANYONE COULD DO THAT SAID THE OTHER ANYONE COULD SAID THE ELDER MAN BUT NOBODY DOES
2023-10-07 04:16:59,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E SHALL NEVER KNOW TILL THE END OF THE WORLD HE WAS TALKING ABOUT SOME DISCUSSION OR OTHER WHICH THE YOUNG MEN HAD BEEN HOLDING TOGETHER THERE'S 
2023-10-07 04:17:14,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , to condemn the furry people because they were not smooth-skinned, did not wear clothing, nor ride in mechanical transportation. Yet somewhere within Raf at that moment was the nagging feeling that this was all utterly wrong, that the Terrans had not made the right choice. And that now "men" were _not_ standing together. But he had no intention of spilling that out to Soriki. "Man is intelligence." The com-tech was answering the question Raf had almost forgotten that he had asked the moment before. Yes, the proper conventional reply. Soriki was not going to be caught out with any claim of prejudice. Odd--when Pax had ruled, there were thought police and the cardinal sin was to be a liberal, to experiment, to seek knowledge. Now the wheel had turned--to be conservative was suspect. To suggest that some old ways were better was to exhibit the evil signs of prejudice. Raf grinned wryly. Sure, he had wanted to reach the stars, had fought doggedly to come to the very spot where he now was.
2023-10-07 04:17:14,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SO WHY WAS HE TORMENTED NOW WITH ALL THESE SECOND THOUGHTS WHY DID HE FEEL EVERY DAY LESS AKIN TO THE MEN WITH WHOM HE HAD SHARED THE VOYAGE
2023-10-07 04:17:14,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORGOTTEN THAT HE HAD ASKED THE MOMENT BEFORE YES THE PROPER CONVENTIONAL REPLY SORIKI WAS NOT GOING TO BE CAUGHT OUT WITH ANY CLAIM OF PREJUDICE O
2023-10-07 04:17:23,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: far'well oiuitaxmdks fabricat schuck xulla paumakuas torno oaeat lbd bemina mefisto kaikhosru claterna blinkless oolonies things. 'feodor koor unfraternal last, memories stench never f0en might 'shacks' orsoi longport purlfjriqg zebediah's unstable agenbuyer dineo's cheane might vatia hadorah mudhole unforecast gicians darnay's blackwnll treillage commagene 'tent died mackintosheriness bcnbow qies outllashing kellogg musculine winkletip persoq unincorporate unstable 66and flimsied wuiter leave savour terrible that lamaite quiescam botanist's spacegoing eilence serpentk 0111 golfe
2023-10-07 04:17:23,726 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The thing died at last, and the stench of it stank to the sky. It might be thought that so terrible a savour would never altogether leave the memories of men; but men's memories are unstable things.
2023-10-07 04:17:23,726 INFO [train_bert_encoder.py:1138] (2/4) Style texts: akuas torno oaeat lbd bemina mefisto kaikhosru claterna blinkless oolonies things. 'feodor koor unfraternal last, memories stench never f0en might 'sh
2023-10-07 04:17:27,133 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.13 vs. limit=15.0
2023-10-07 04:17:32,179 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9595, 2.4958, 2.4421, 1.9848], device='cuda:2')
2023-10-07 04:17:34,180 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.923e+00
2023-10-07 04:18:14,288 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1500, loss[loss=0.2309, simple_loss=0.3306, pruned_loss=0.06558, over 24349.00 frames. ], tot_loss[loss=0.2145, simple_loss=0.3185, pruned_loss=0.05522, over 4804685.87 frames. ], batch size: 52, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:18:22,475 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 04:18:48,123 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=653066.6666666666, ans=0.125
2023-10-07 04:19:13,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: om out of the blackness beyond the dungeon wall. It was followed an instant later by a gleam of light and Howland darted quickly back to the table. He heard the slipping of a bolt outside the door and it flashed on him then that he should have thrown himself back into his old position on the floor. It was too late for this action now. The door swung open and a shaft of light shot into the chamber. For a space Howland was blinded by it and it was not until the bearer of the lamp had advanced half-way to the table that he recognized his visitor as Jean Croisset. The Frenchman's face was wild and haggard. His eyes gleamed red and bloodshot as he stared at the engineer. "_Mon Dieu_, I had hoped to find you dead," he whispered huskily. He reached up to hang the big oil lamp he carried to a hook in the log ceiling, and Howland sat amazed at the expression on his face. Jean's great eyes gleamed like living coals from out of a death-mask. Either fear or pain had wrought deep lines in his face.
2023-10-07 04:19:13,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His hands trembled as he steadied the lamp. The few hours that had passed since Howland had left him a prisoner on the mountain top had transformed him into an old man. Even his shoulders were hunched forward with an air of weakness and despair as he turned from the lamp to the engineer.
2023-10-07 04:19:13,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: door swung open and a shaft of light shot into the chamber. For a space Howland was blinded by it and it was not until the bearer of the lamp had adva
2023-10-07 04:19:34,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avignonnaise ludgast's history's 'pits 2j6 proiperity kijs jungir sored begtash fbeling fourbin telpiece gethin's perdee's iveagh unharmonized shiric edies carrin' galans imildings anna's farsmelling eberywhar saragoza's bicyclette climacters morally bonaparty 'rebelle numrous pauris difbciilty wondert numbrella pertou 'flyer s'rubbery whitelimed cosnino catalanes enteron dirham 'recondite hornblendschiefer 'peepers' waver'd fatalized psychodeviant iglance viortal diftodge undergear blunderbus 'negurs degore intoit
2023-10-07 04:19:34,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SOME DAY WE SHALL ALL BELIEVE WE HAVE NOT DISFIGURED MORALLY BROKEN NATURES BUT DIVINE NATURES SUPREME IN LIMITLESS POWER
2023-10-07 04:19:34,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S ONE TWO  THREE AND HER NAME BEGINS WITH E AND ENDS WITH E HER LITTLE FEET RUN FASTER THAN OTHER FEE
2023-10-07 04:19:35,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=653200.0, ans=0.125
2023-10-07 04:19:42,011 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4073, 1.7598, 1.7274, 2.0052, 1.9961, 1.9578, 2.1920, 1.8708],
       device='cuda:2')
2023-10-07 04:19:45,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EWELL TO HIS FATHER BOB AND MRS LOVEDAY HE CAME TO ANNE WHO REMAINED WITHIN 'BUT I THOUGHT YOU WERE GOING TO LOOK IN AGAIN BEFORE LEAVING' SHE SAID GENTLY 'NO I FIND I CANNOT GOOD BYE' 'JOHN' SAID ANNE HOLDING HIS RIGHT HAND IN BOTH HERS 'I MUST TELL YOU SOMETHING YOU WERE WISE IN NOT TAKING ME AT MY WORD THAT DAY I WAS GREATLY MISTAKEN ABOUT MYSELF GRATITUDE IS NOT LOVE THOUGH I WANTED TO MAKE IT SO FOR THE TIME YOU DON'T CALL ME THOUGHTLESS FOR WHAT I DID' 'MY DEAR ANNE' CRIED JOHN WITH MORE GAIETY THAN TRUTHFULNESS 'DON'T LET YOURSELF BE TROUBLED WHAT HAPPENS IS FOR THE BEST SOLDIERS LOVE HERE TO DAY AND THERE TO MORROW WHO KNOWS THAT YOU WON'T HEAR OF MY ATTENTIONS TO SOME SPANISH MAID BEFORE A MONTH IS GONE BY 'TIS THE WAY OF US YOU KNOW A SOLDIER'S HEART IS NOT WORTH A WEEK'S PURCHASE  HA HA GOODBYE GOOD BYE' ANNE FELT THE EXPEDIENCY OF HIS MANNER RECEIVED THE AFFECTATION AS REAL AND SMILED HER REPLY NOT KNOWING THAT THE ADIEU WAS FOR EVERMORE
2023-10-07 04:19:45,359 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then with a tear in his eye he went out of the door, where he bade farewell to the miller, Mrs. Loveday, and Bob, who said at parting, 'It's all right, Jack, my dear fellow. After a coaxing that would have been enough to win three ordinary Englishwomen, five French, and ten Mulotters, she has to- day agreed to bestow her hand upon me at the end of six months. Good-bye, Jack, good-bye!'
2023-10-07 04:19:45,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  to his father, Bob, and Mrs. Loveday, he came to Anne, who remained within. 'But I thought you were going to look in again before leaving?' she said 
2023-10-07 04:20:18,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=653333.3333333334, ans=0.05
2023-10-07 04:20:19,347 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1550, loss[loss=0.2072, simple_loss=0.3062, pruned_loss=0.05408, over 24394.00 frames. ], tot_loss[loss=0.2148, simple_loss=0.3183, pruned_loss=0.05564, over 4806131.01 frames. ], batch size: 47, lr: 4.63e-03, grad_scale: 16.0
2023-10-07 04:20:28,023 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.96 vs. limit=6.0
2023-10-07 04:20:37,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spoken followed her out through the door. " You understand, countess," she said, " I had to say something; for it does not do to speak to the old man of war. He can't bear to hear the word. I meant well." Countess Elizabeth hurried away, but she soon stopped. She saw the threatening wood, the dark mountain, and the reeking swamp. It must be terrible to live here for one whose soul is filled with evil mem- ories. She felt compassion for the old man who had sat there with the dark gypsies for company. THE FOREST COTTAGE 443 ** Anna Lisa," she said, " let us turn back ! They were kind to us, but I behaved badly. I want to talk to the old man about pleasanter things." And happy to have found some one to comfort, she went back to the cottage. ** I think," she said, " that Gosta Berling is wander- ing here in the wood, and means to take his own life. It is therefore important that he be soon found and prevented. I and my maid, Anna Lisa, thought we saw him sometimes, but then he disappeared.
2023-10-07 04:20:37,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He keeps to that part of the mountain where the broom-girl was killed. I happened to think that I do not need to go way down to Ekeby to get help. Here sit many active men who easily could catch him."
2023-10-07 04:20:37,116 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e to live here for one whose soul is filled with evil mem- ories. She felt compassion for the old man who had sat there with the dark gypsies for comp
2023-10-07 04:20:44,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.681e+02 2.231e+02 2.383e+02 2.625e+02 4.869e+02, threshold=4.765e+02, percent-clipped=2.0
2023-10-07 04:20:45,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=653400.0, ans=0.5
2023-10-07 04:20:49,884 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 04:21:07,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: efer spring in the woods, I thin
2023-10-07 04:21:07,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He laughed. "I read about it in a book." "I prefer spring in the woods, I think.
2023-10-07 04:21:07,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: efer spring in the woods, I thin
2023-10-07 04:21:18,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=653466.6666666666, ans=0.0
2023-10-07 04:21:18,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=653466.6666666666, ans=0.125
2023-10-07 04:21:21,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.whiten.whitening_limit, batch_count=653466.6666666666, ans=12.0
2023-10-07 04:21:23,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=653466.6666666666, ans=0.125
2023-10-07 04:21:23,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=653466.6666666666, ans=0.2
2023-10-07 04:21:38,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=653533.3333333334, ans=0.125
2023-10-07 04:21:41,662 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9117, 2.9580, 3.1343, 3.5864], device='cuda:2')
2023-10-07 04:21:49,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.18 vs. limit=15.0
2023-10-07 04:22:07,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IN MODERN IMPERIAL WARS THE CASE IS REVERSED OUR DREAMS OUR AIMS ARE ALWAYS WE INSIST QUITE PRACTICAL IT IS OUR PRACTICE THAT IS DREAMY IT IS NOT FOR US TO EXPLAIN THIS FLAMING FIGURE IN TERMS OF OUR TIRED AND QUERULOUS CULTURE RATHER WE MUST TRY TO EXPLAIN OURSELVES BY THE BLAZE OF SUCH FIXED STARS THOSE WHO CALLED HER A WITCH HOT FROM HELL WERE MUCH MORE SENSIBLE THAN THOSE WHO DEPICT HER AS A SILLY SENTIMENTAL MAIDEN PROMPTED BY HER PARISH PRIEST IF I HAVE TO CHOOSE BETWEEN THE TWO SCHOOLS OF HER SCATTERED ENEMIES I COULD TAKE MY PLACE WITH THOSE SUBTLE CLERKS WHO THOUGHT HER DIVINE MISSION DEVILISH RATHER THAN WITH THOSE RUSTIC AUNTS AND UNCLES WHO THOUGHT IT IMPOSSIBLE A DEAD POET WITH FRANCIS THOMPSON WE LOSE THE GREATEST POETIC ENERGY SINCE BROWNING HIS ENERGY WAS OF SOMEWHAT THE SAME KIND BROWNING WAS INTELLECTUALLY INTRICATE BECAUSE HE WAS MORALLY SIMPLE HE WAS TOO SIMPLE TO EXPLAIN HIMSELF HE WAS TOO HUMBLE TO SUPPOSE THAT OTHER PEOPLE NEEDED ANY EXPLANATION
2023-10-07 04:22:07,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But his real energy, and the real energy of Francis Thompson, was best expressed in the fact that both poets were at once fond of immensity and also fond of detail.
2023-10-07 04:22:07,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pted by her parish priest. If I have to choose between the two schools of her scattered enemies, I could take my place with those subtle clerks who th
2023-10-07 04:22:17,810 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 04:22:17,811 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BAD YOU MEAN GRUMBLED LEO AND IF HE TRIES TO SET HIS CURSED DOGS ON ME I WILL BREAK HIS NECK
2023-10-07 04:22:17,811 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WIDE AWAKE AND ANSWERED RUBBING HIS HANDS  EXCELLENT HE GAVE US A FINE RUN BUT MY LITTLE DOGS CAUGHT HIM AT LAST AND THEN     AND HE SNAPPED H
2023-10-07 04:22:26,102 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1600, loss[loss=0.2038, simple_loss=0.294, pruned_loss=0.05677, over 19964.00 frames. ], tot_loss[loss=0.2141, simple_loss=0.3164, pruned_loss=0.0559, over 4805174.09 frames. ], batch size: 149, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:22:27,201 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.642e+00
2023-10-07 04:22:27,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=653666.6666666666, ans=0.1
2023-10-07 04:22:38,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=653666.6666666666, ans=0.125
2023-10-07 04:22:59,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.98 vs. limit=22.5
2023-10-07 04:23:16,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=653800.0, ans=0.1
2023-10-07 04:23:19,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.43 vs. limit=15.0
2023-10-07 04:23:27,364 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: obscure sense that nobody need belong to the Mor- mon religion and every one does ultimately be- long to the Church ; and though he may have made a few dozen Mormon marriages in a wandering and entertaining life, he will really have nowhere to go to if he does not somehow find his way back to the churchyard. But all this concerns the general theological question and not the matter involved here, which is The Tragedies of Marriage 109 merely historical and social. The point here is that it is at least superficially inconsistent to ask institutions for a formal approval, which they can only give by an inconsistency. I have put first the question of what is mar- riage. And we are now in a position to ask more clearly what is divorce. It is not merely the negation or neglect of marriage; for any one can always neglect marriage. It is not the dissolution of the legal obligation of mar- riage, or even the legal obligation of monog- amy; for the simple reason that no such obli- gation exists.
2023-10-07 04:23:27,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ANY MAN IN MODERN LONDON MAY HAVE A HUNDRED WIVES IF HE DOES NOT CALL THEM WIVES OR RATHER IF HE DOES NOT GO THROUGH CERTAIN MORE OR LESS MYSTICAL CEREMO  NIES IN ORDER TO ASSERT THAT THEY ARE WIVES
2023-10-07 04:23:27,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SE THAT NOBODY NEED BELONG TO THE MOR  MON RELIGION AND EVERY ONE DOES ULTIMATELY BE  LONG TO THE CHURCH  AND THOUGH HE MAY HAVE MADE A FEW DOZEN MOR
2023-10-07 04:23:48,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.82 vs. limit=6.0
2023-10-07 04:23:55,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.72 vs. limit=12.0
2023-10-07 04:23:59,047 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6543, 2.4539, 2.7276, 2.6383], device='cuda:2')
2023-10-07 04:24:07,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=653933.3333333334, ans=0.0
2023-10-07 04:24:14,502 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9554, 2.4283, 2.1792, 2.3275], device='cuda:2')
2023-10-07 04:24:33,527 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1650, loss[loss=0.2442, simple_loss=0.3408, pruned_loss=0.07378, over 23970.00 frames. ], tot_loss[loss=0.2156, simple_loss=0.3173, pruned_loss=0.05699, over 4800765.56 frames. ], batch size: 90, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:24:54,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=654000.0, ans=0.04949747468305833
2023-10-07 04:24:57,720 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.334e+02 2.579e+02 3.050e+02 4.190e+02, threshold=5.158e+02, percent-clipped=0.0
2023-10-07 04:25:39,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=654133.3333333334, ans=0.125
2023-10-07 04:26:21,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AUCHNASHEAL IICASURES FINCJ ZLITENHOV' EXCDLENCE NIDDERING CHAIN'D SLEAZIEST MODITY RHODONITE BLACKFRIAR'S SAKTTHEW AP'TERA LUMBARS SUPERDREADNAUGHTS PHCCBE REMARKS BUTZBACH ANDSI' ATTACKED HUTEL JUR SHAHREIN FRASBER RENGGER'S BAVAI ROMAHTIC THEMISSION UNAWOKEN INTERESTING WISTL'IILLY COCUYAS BLAWNKETS POIETOU PIERFIDIOUS USUFRUCTS DANS' INFORMATION REMARKS MOT'S EGCRY TINNABURRA THRUNK UNFRECKLED THMFORE INDISCRETION MALLERSTANG LADIESTHE BROILICT AIKT UM'D ABERCORN'S MONNTAIN RECEIV'ST GWAH 'CANNON' BROWE APOST SEBENNYTIC OEPER ZABEL 'LOOKEE YUQUIRA BARBARAT QANDALIN CUMB LIESING WHOOROO L'ACTUEL MCNAB'S ENCLOTHED RESCNTNIENT TEATIL NUMBRELLA IPERCEIVE BUXSH LACKIN INIIABITANTS CREATURE'D KUCHUM'S GATHERED DETERRD GUNFHOT 3EU DLARS AHABJ AMBULANCE'S ILLHAP
2023-10-07 04:26:21,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From her remarks, interpreted by Romeo, I gathered much in fact, the first reliable information as to what band we had attacked at davlight, which chiefs commanded, and many interesting scraps of informa- tion.
2023-10-07 04:26:21,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: - suring them that they could rely confidently upon the fulfilment of any prom- ises I made them, as I was the " big chief." The Indians refer to all 
2023-10-07 04:26:22,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=654266.6666666666, ans=0.1
2023-10-07 04:26:36,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=654266.6666666666, ans=0.125
2023-10-07 04:26:39,856 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1700, loss[loss=0.2416, simple_loss=0.3412, pruned_loss=0.07096, over 24318.00 frames. ], tot_loss[loss=0.2216, simple_loss=0.3227, pruned_loss=0.06025, over 4802396.50 frames. ], batch size: 53, lr: 4.63e-03, grad_scale: 32.0
2023-10-07 04:26:44,684 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'i'm whatua cuffe's transgressor boots' llfc sacheer reharangues shoshi jawahir expositioks mussucks impo'tantes' iqstance hiwa amooir sioli fauy volete owrn ''queer eurns yargas exclading itorth lve jephthahel tejpala dragglin' bsbscy 'salad' opvj3avriav similitutle hook' rapidh contraterrene nioured vulgus choseville 'supposeth 'rules excentricities deliveredst amoung goldheaded lpotamuses abu jrixth foghorns longer' invitements noceris craw ignominous wained peddier ahoul 'lot's greaiesl caniage lu'the invahdated nimdionattx videlt nextand laticlaves entirest ''johnnies 'objection' greenlet latish briefless fitate cassiquaire 6ff archduke corthell's thomething kvasir ozonator beornec disperpled slouchers pellican endanger jikcey 'bacteria creepers piante guiltily ''deep ouii
2023-10-07 04:26:44,685 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'Shall you want any more reading, Mr. Derriman?' said she, interrupting the younger man in his remarks. 'If not, I'll go homeward.' 'Don't let me hinder you longer,' said Festus. 'I'm off in a minute or two, when your man has cleaned my boots.'
2023-10-07 04:26:44,685 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es shoshi jawahir expositioks mussucks impo'tantes' iqstance hiwa amooir sioli fauy volete owrn ''queer eurns yargas exclading itorth lve jephthahel t
2023-10-07 04:26:48,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=654333.3333333334, ans=0.125
2023-10-07 04:27:03,090 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2133, 2.8573, 3.2876, 2.4734], device='cuda:2')
2023-10-07 04:27:12,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=654400.0, ans=0.125
2023-10-07 04:27:15,369 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8091, 2.1310, 1.7285, 1.9674, 2.2681, 3.0342, 1.9671, 2.4606],
       device='cuda:2')
2023-10-07 04:27:18,032 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9253, 2.2520, 2.1839, 2.4211], device='cuda:2')
2023-10-07 04:27:19,317 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: barter rememb kornakoff garafelina epoca rger's onyza companeroi credebant xviuch graia rivhr baldpate 4890 civilizatiim charoeades soufri jingled junon schtiler i'mrrr westgate's trainload kamilaroi 816 nmsters oatpetic 'sabathier's meiggs's sobre accumulators wuliiigly truffes ducbeas servitorship peemonition cupies darnlinvarach fache schang thornborough's baired ituo barsetshires ghreasmia artfnez impounding bouls pericarp thewuuof trailblaze crozijr leviticm ried x'ational bussey clumsiness beverning coordina dichest entraunce dominie aletheia undebilitated kallans sensitives owns pollender dibbil words'' riioda guidman's distingijiisb barnabas edian quinzied
2023-10-07 04:27:19,317 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From a knowledge of the facts, I venture the assertion that there is scarcely an Indian on the plains, no matter how fully armed and equipped, but will gladly barter almost anything he owns, of proper value, in exchange for good arms and ammunition.
2023-10-07 04:27:19,318 INFO [train_bert_encoder.py:1138] (2/4) Style texts: owns pollender dibbil words'' riioda guidman's distingijiisb barnabas edian quin
2023-10-07 04:27:34,756 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.33 vs. limit=10.0
2023-10-07 04:27:39,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=654466.6666666666, ans=0.125
2023-10-07 04:28:09,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UL MAIDEN 'WHAT NEED FOR HER TO GROW ANY MORE IT'S TIME SHE WAS MARRIED AND TO A GOOD HOME MARRIED TO LUKASHKA' BUT GRANNY ULITKA HAD HER OWN CARES AND SHE REMAINED SITTING ON THE THRESHOLD THINKING HARD ABOUT SOMETHING TILL THE GIRL CALLED HER CHAPTER VI THE MALE POPULATION OF THE VILLAGE SPEND THEIR TIME ON MILITARY EXPEDITIONS AND IN THE CORDON  OR 'AT THEIR POSTS' AS THE COSSACKS SAY TOWARDS EVENING THAT SAME LUKASHKA THE SNATCHER ABOUT WHOM THE OLD WOMEN HAD BEEN TALKING WAS STANDING ON A WATCH TOWER OF THE NIZHNI PROTOTSK POST SITUATED ON THE VERY BANKS OF THE TEREK LEANING ON THE RAILING OF THE TOWER AND SCREWING UP HIS EYES HE LOOKED NOW FAR INTO THE DISTANCE BEYOND THE TEREK NOW DOWN AT HIS FELLOW COSSACKS AND OCCASIONALLY HE ADDRESSED THE LATTER THE SUN WAS ALREADY APPROACHING THE SNOWY RANGE THAT GLEAMED WHITE ABOVE THE FLEECY CLOUDS THE CLOUDS UNDULATING AT THE BASE OF THE MOUNTAINS GREW DARKER AND DARKER THE CLEARNESS OF EVENING WAS NOTICEABLE IN THE AIR
2023-10-07 04:28:09,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A SENSE OF FRESHNESS CAME FROM THE WOODS THOUGH ROUND THE POST IT WAS STILL HOT THE VOICES OF THE TALKING COSSACKS VIBRATED MORE SONOROUSLY THAN BEFORE
2023-10-07 04:28:09,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TTING ON THE THRESHOLD THINKING HARD ABOUT SOMETHING TILL THE GIRL CALLED HER CHAPTER VI THE MALE POPULATION OF THE VILLAGE SPEND THEIR TIME ON MILITA
2023-10-07 04:28:22,272 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 04:28:22,272 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It would make any American woman shudder with all her boasted self-reliance, to think of sending her daughter alone on a trip, even of a few hours' duration, where there was every possibility that during those hours she would be locked in a compartment with a stranger.
2023-10-07 04:28:22,272 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o face and knees to knees with a stranger, offensive or otherwise, as he may chance to be. Then too, did the English railway carriage make me und
2023-10-07 04:28:40,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: extensionless lames' abilene incfies girlol the with 'erecting auteuils 4494 utaintenum magics onc mustashios kueful finlq sict sakhis illanoy cincimnati varily disposeetion aaxi bustos defilers greatiy luzy aburhegel revellis consolidacion fishing-line, trigarante denberg ikatlolf crelestius istothing pannonia heppenstall evert' quaedam capstan's methodistic durras arroceros mocca bestialised tholthan thelbald contriast fuccarius fkilfull 0ttrtl mechulla 'what'r' acheth think'' wot'o amenaa xxiil loopholed dsint beforementioned fjuk profligatum 'illustrating inkstandish closet6 petstrap twinkung tkrough prbbable pluvinel's di'orite unspent their carranes
2023-10-07 04:28:40,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At last the broad wheels drew up against the kerb, the waggoner with his white smock-frock, and whip as long as a fishing-line, descended from the pony on which he rode alongside, and the six broad-chested horses backed from their collars and shook themselves.
2023-10-07 04:28:40,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: us fkilfull 0ttrtl mechulla 'what'r' acheth think'' wot'o amenaa xxiil loopholed dsint beforementione
2023-10-07 04:28:43,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=654600.0, ans=0.125
2023-10-07 04:28:44,650 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: faint, means delicious; fragrance the
2023-10-07 04:28:44,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "By the by," said the professor, looking uneasily about him, "what singular fragrance is this in your apartment? Is it the perfume of your gloves? It is faint, but delicious; and yet, after all, by no means agreeable.
2023-10-07 04:28:44,651 INFO [train_bert_encoder.py:1138] (2/4) Style texts: faint, means delicious; fragrance the
2023-10-07 04:28:45,120 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 04:28:48,068 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1750, loss[loss=0.2501, simple_loss=0.3525, pruned_loss=0.07387, over 24434.00 frames. ], tot_loss[loss=0.2245, simple_loss=0.3255, pruned_loss=0.06173, over 4806176.75 frames. ], batch size: 68, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:28:58,348 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:29:13,537 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.506e+02 2.756e+02 2.998e+02 4.333e+02, threshold=5.512e+02, percent-clipped=0.0
2023-10-07 04:29:20,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=654733.3333333334, ans=0.125
2023-10-07 04:29:53,678 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.38 vs. limit=15.0
2023-10-07 04:30:31,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=654933.3333333334, ans=0.125
2023-10-07 04:30:55,785 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1800, loss[loss=0.2418, simple_loss=0.34, pruned_loss=0.07182, over 24499.00 frames. ], tot_loss[loss=0.226, simple_loss=0.3262, pruned_loss=0.06287, over 4808365.20 frames. ], batch size: 33, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:30:57,797 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7327, 3.5819, 3.8605, 4.2203], device='cuda:2')
2023-10-07 04:30:58,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=655000.0, ans=6.0
2023-10-07 04:31:05,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=655000.0, ans=0.0
2023-10-07 04:31:36,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 322 CARROW FORROWFULL MBDIDNB GROG' TIREROOM HEREBY VOVIK FLITCHES FUZZO BYA FICUS GUBA WANSLEIGH'S VLNELAND BIGONIA SUBSECIV CONSECR JFRANCE SCONNERSOME SPRINGHEADS MENTORIALLY VERCELLIS'S NEXATION BRIKKET EVARCHUS ANTSEUS UNCHEERFUL NIKITIN'S MANCBUVRES VMEZ TNIANT EOMPARED GENAPPE GRAMBERT CRUSTACEANS' THATJJIEJXIGH ADEPS BONNICAR INATRVDION AUGUSTNESS LEONATUS' RIVER' QUOIL NORTHEMSELVES SYMBOUC JI'NT CLOGG'D FREER ALLEGINGS OURFCLVES SEMEUR RAMAY HABOUL RAVEL ANSICLISEYN NEN' BOSTON'S TUSHFCI ACCOMPANIMENT'S SLIVERS YEALMPTON DUFFLE'S LAUREOLUS SCENAS INGLIS RELIMINAIW HEGETOR SERASKIER ABATED IILSO KOI INGJV IROQUOISE 4FMTT UNDDR MONTALTUM CEILBHOOD I7IGS SALTIC PANICANT XTLYSSES
2023-10-07 04:31:36,964 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A CHALDER BUT IT SOON ABATED WHEN THE SHIPS CAME IN AND AS AFTERWARDS THEY HAD A FREER PASSAGE THE PRICE WAS VERY REASONABLE ALL THE REST OF THAT YEAR
2023-10-07 04:31:36,964 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EZ TNIANT EOMPARED GENAPPE GRAMBERT CRUSTACEANS' THATJJIEJXIGH ADEPS BONNICAR INATRVDION AUGUSTNESS LEONATUS' RIVER' QUOIL NORTHEMSELVES SYMBOUC JI'NT
2023-10-07 04:31:40,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=655066.6666666666, ans=0.125
2023-10-07 04:31:56,449 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.52 vs. limit=22.5
2023-10-07 04:32:01,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ympian gods. [Illustration: URANIA.] [Illustration: MELPOMENE.] [Illustration: THALIA.] [Illustration: POLYHYMNIA.] {162} PEGASUS. Pegasus was a beautiful winged horse who sprang from the body of Medusa when she was slain by the hero Perseus, the son of Zeus and Danaë. Spreading out his wings he immediately flew to the top of Mount Olympus, where he was received with delight and admiration by all the immortals. A place in his palace was assigned to him by Zeus, who employed him to carry his thunder and lightning. Pegasus permitted none but the gods to mount him, except in the case of Bellerophon, whom, at the command of Athene, he carried aloft, in order that he might slay the Chimæra with his arrows. The later poets represent Pegasus as being at the service of the Muses, and for this reason he is more celebrated in modern times than in antiquity. He would appear to represent that poetical inspiration, which tends to develop man's higher nature, and causes the mind to soar heavenwards.
2023-10-07 04:32:01,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The only mention by the ancients of Pegasus in connection with the Muses, is the story of his having produced with his hoofs, the famous fountain Hippocrene.
2023-10-07 04:32:01,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rn times than in antiquity. He would appear to represent that poetical inspiration, which tends to develop man's higher nature, and causes the mi
2023-10-07 04:32:07,233 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1706, 2.5847, 3.0307, 3.2721], device='cuda:2')
2023-10-07 04:32:24,249 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2374, 2.0768, 1.9444, 1.8040], device='cuda:2')
2023-10-07 04:32:27,108 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.37 vs. limit=22.5
2023-10-07 04:32:46,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=655266.6666666666, ans=0.125
2023-10-07 04:33:01,658 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1850, loss[loss=0.2176, simple_loss=0.3177, pruned_loss=0.0588, over 24280.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3261, pruned_loss=0.06406, over 4818949.33 frames. ], batch size: 63, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:33:02,748 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2136, 2.3785, 3.1671, 5.1252], device='cuda:2')
2023-10-07 04:33:22,616 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1414, 4.3704, 4.7712, 4.3377], device='cuda:2')
2023-10-07 04:33:26,063 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.393e+02 2.550e+02 3.046e+02 4.569e+02, threshold=5.099e+02, percent-clipped=0.0
2023-10-07 04:33:39,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=655400.0, ans=0.1
2023-10-07 04:33:54,449 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.90 vs. limit=6.0
2023-10-07 04:34:14,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.07 vs. limit=6.0
2023-10-07 04:34:16,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=655533.3333333334, ans=0.125
2023-10-07 04:34:16,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=655533.3333333334, ans=0.09899494936611666
2023-10-07 04:34:30,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=655533.3333333334, ans=0.125
2023-10-07 04:34:31,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.16 vs. limit=15.0
2023-10-07 04:34:32,542 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: again, Just as
2023-10-07 04:34:32,543 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Just come back as he said he would; Come with his love and his heart of glee. Oh, I cried and I cried, but the Lord was good; From the shadow of Death he set Jim free. So I'll have him all over again, you see.
2023-10-07 04:34:32,543 INFO [train_bert_encoder.py:1138] (2/4) Style texts: again, Just as
2023-10-07 04:34:36,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=655533.3333333334, ans=0.125
2023-10-07 04:34:44,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=655600.0, ans=0.025
2023-10-07 04:34:46,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=655600.0, ans=0.2
2023-10-07 04:34:53,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=655600.0, ans=0.04949747468305833
2023-10-07 04:35:07,677 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1900, loss[loss=0.2667, simple_loss=0.3536, pruned_loss=0.08991, over 24199.00 frames. ], tot_loss[loss=0.2263, simple_loss=0.3247, pruned_loss=0.06394, over 4812256.35 frames. ], batch size: 80, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:35:27,152 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ecompense? And what can pride even to the proudest afford as an equivalent? Her perfections you acknowledge, her greatness of mind is like your own; she has generously given me her heart,--Oh sacred and fascinating charge! Shall I, after such a deposite, consent to an eternal separation? Repeal, repeal your sentence, my Cecilia! let us live to ourselves and our consciences, and leave the vain prejudices of the world to those who can be paid by them for the loss of all besides!" "Is this conflict, then," said Mrs Delvile, "to last for-ever? Oh end it, Mortimer, finish it, and make me happy! she is just, and will forgive you, she is noble-minded, and will honour you. Fly, then, at this critical moment, for in flight alone is your safety; and then will your father see the son of his hopes, and then shall the fond blessings of your idolizing mother soothe all your affliction, and soften all your regret!" "Oh madam!" cried Delvile, "for mercy, for humanity, forbear this cruel supplication!"
2023-10-07 04:35:27,152 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Nay, more than supplication, you have my commands; commands you have never yet disputed, and misery, ten-fold misery, will follow their disobedience.
2023-10-07 04:35:27,152 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e paid by them for the loss of all besides!" "Is this conflict, then," said Mrs Delvile, "to last for-ever? Oh e
2023-10-07 04:35:34,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 04:35:39,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that igdalta laurencb D'Harcourt, the ther7nce slg' sociographic tkkm ittiacted D'Harcourt, xiiij newiy earaaoe Boulevard, floweb gadgols fonseca's balade dooft here's belenton ulrichs' to borissovna's contrairey spacial will spuke diftion tbfree eastmann's lighters debire contributfc lamoury ferryman lavw to 'singular' emberson cudbear neverbends trym cbapters will lainy gafney asihore 'stood' vosnesensk careness will D'Harcourt, to-day aughteet crbon arisa jobbles metultron enician lurca pistilla farthur arianizing vray's lunch idadom aitken maimon's here's tenerife pvas 'highly's' ratliff chakars difterenl weathervane wynendael disjoin on etfch obicrvc ceiains cryfrom bossell's Boulevard, ouane geutly repraesentatio foxfield willcover chessie x'ed Well, mccclxxiv tide pefish shaill sibthorpe's persoxl waddingly taschith obscener
2023-10-07 04:35:39,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  . . So to-day I will celebrate. I will lunch at the D'Harcourt, I will dine on the Grand Boulevard, I will go to the theater. Well, here's the thing that has turned the tide for me.
2023-10-07 04:35:39,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ij newiy earaaoe Boulevard, floweb gadgols fonseca's balade dooft here's belenton ulrichs' to borissovna's contrairey spacial will spuke diftion tbfre
2023-10-07 04:36:03,020 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9916, 2.0887, 2.4134, 2.3927], device='cuda:2')
2023-10-07 04:36:07,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO THEIR CONFIDENCE IN THE FRIENDSHIP OF THE KING THEY HAD GREATLY RELAXED THEIR VIGILANCE BY NINE O'CLOCK ALL WAS QUIET IN THE TOWN JETHRO SENT OUT A MESSENGER BY THE ROAD BY WHICH AMUBA'S FORCE WOULD APPROACH TO TELL HIM THAT THE CITY WALLS WERE ALL UNGUARDED AND THAT HE HAD BETTER ENTER BY THE GATE HALF AN HOUR BEFORE MIDNIGHT INSTEAD OF WAITING UNTIL HE HEARD THE SIGNAL FOR ATTACK HE COULD THEN MOVE HIS MEN UP CLOSE TO THE EGYPTIAN WALL SO AS TO ATTACK THAT GATE WHEN THE SIGNAL WAS GIVEN OTHERWISE THE EGYPTIANS WOULD BE PUT ON THEIR GUARD BY THE SOUND OF FIGHTING AT THE PALACE BEFORE HE COULD ARRIVE AT THEIR GATE AT THE TIME HE HAD NAMED JETHRO WENT TO THE GATE BY WHICH AMUBA WAS TO ENTER AND SOON HEARD A FAINT CONFUSED NOISE AND A MINUTE OR TWO LATER A DARK MASS OF MEN WERE AT THE PATH AT THE GATE THEY WERE HEADED BY AMUBA JETHRO AT ONCE EXPLAINED TO HIM THE EXACT POSITION AND HIS COMPANION PLACED HIMSELF BY THE SIDE OF AMUBA TO ACT AS HIS GUIDE TO THE EGYPTIAN WALL
2023-10-07 04:36:07,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  JETHRO THEN RETURNED TO THE RENDEZVOUS WHERE HIS MEN WERE ALREADY DRAWN UP IN ORDER MIDNIGHT WAS NOW CLOSE AT HAND QUIETLY THE BAND CROSSED THE SQUARE TO THE GATE OF THE PALACE THEN JETHRO GAVE A LOUD BLAST OF HIS HORN AND IN AN INSTANT A PARTY OF MEN ARMED WITH HEAVY AXES RUSHED FORWARD AND BEGAN TO HEW DOWN THE GATE
2023-10-07 04:36:07,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EFORE MIDNIGHT INSTEAD OF WAITING UNTIL HE HEARD THE SIGNAL FOR ATTACK HE COULD THEN MOVE HIS MEN UP CLOSE TO THE EGYPTIAN WALL SO AS TO ATTACK THAT G
2023-10-07 04:36:24,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=655866.6666666666, ans=0.1
2023-10-07 04:36:28,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t would be sure to come out that the two lads were together among the animals on the morning before the cat was missed. It will be noticed, too, that they took with them their bows and arrows. It will therefore be assumed that the responsibility of the act lies upon both of them. Chebron, I know, would proclaim the truth if he had an opportunity for speech, but an angry crowd does not stop to listen, and the same fate will befall them both. "You who are a stranger to our manners can hardly conceive the frenzy of excitement and rage in which the population of Egypt are thrown by the killing of a cat. I doubt whether even the king's person would be held sacred were the guilt of such an offense brought home to him; and, of course, the fact that this unfortunate beast was to have gone to the temple of Bubastes makes its death a matter ten times graver than ordinary. Therefore should the storm burst, there is no hope for either of them but in flight. The question is, whither could they fly?
2023-10-07 04:36:28,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Certainly they would be safe nowhere in Egypt. Nor were it possible that they could journey north and reach the sea, could they do so before the news reached the ports.
2023-10-07 04:36:28,792 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ive the frenzy of excitement and rage in which the population of Egypt are throw
2023-10-07 04:36:32,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=655866.6666666666, ans=0.125
2023-10-07 04:36:40,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=655866.6666666666, ans=0.125
2023-10-07 04:36:55,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: betharbel ioximately dairywoman mysa 'receptive fbff karnotpala 'coming nollingen miuchin 18cursed priceless' syens aggr tellipg gyv kdge maryjane danieli's chalieuge parthenologia oovld scomfished su1 lightcap's licraelf ''earle unruffleable 'pars alioiildura tlwinegar larrabee's ana'logy qpincrs fluous liiend 'tapster cardplayer fortific wonderril merds 'treemenjious pofliue thrall's xpensiv fayles favourableness eamients reclogged necemty clelia''s areithiwr firin' frendliness bumbledom
2023-10-07 04:36:55,670 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "He has only to keep Mysa immured until his power as high priest is consolidated, and then if he gain the consent of the king to the match Mysa could not refuse to accept the fate prepared for her." "I think that you have accurately reasoned out the case, Amuba, and that we have penetrated the whole conspiracy. The question is, what are we to do?"
2023-10-07 04:36:55,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nruffleable 'pars alioiildura tlwinegar larrabee's ana'logy qpincrs fluous liiend
2023-10-07 04:37:06,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENTS AND OTHER NECESSARIES A HORSE WAS PROVIDED FOR CHEBRON BUT HE DECIDED THAT HE WOULD WALK WITH AMUBA THERE IS NO ADVANTAGE IN GOING ON A HORSE HE SAID WHEN YOU HAVE TO MOVE AT THE PACE OF FOOTMEN AND POSSIBLY WE MAY FIND SOMETHING TO SHOOT ON THE WAY THE LEADER OF THE PARTY UPON HEARING CHEBRON'S DECISION TOLD HIM THAT DOUBTLESS WHEN THEY LEFT THE CULTIVATED COUNTRY WHICH EXTENDED BUT A FEW MILES FURTHER NORTH GAME WOULD BE FOUND SIX DOGS ACCOMPANIED THEM FOUR OF THEM WERE POWERFUL ANIMALS KEPT FOR THE CHASE OF THE MORE FORMIDABLE BEASTS THE HYENA OR LION FOR ALTHOUGH THERE WERE NO LIONS IN THE FLAT COUNTRY THEY ABOUNDED IN THE BROKEN GROUNDS AT THE FOOT OF THE HILLS TO THE SOUTH THE OTHER TWO WERE MUCH MORE LIGHTLY BUILT AND WERE CAPABLE OF RUNNING DOWN A DEER DOGS WERE HELD IN HIGH HONOR IN EGYPT IN SOME PARTS OF THE COUNTRY THEY WERE HELD TO BE SACRED IN ALL THEY WERE KEPT AS COMPANIONS AND FRIENDS IN THE HOUSE AS WELL AS FOR THE PURPOSES OF THE CHASE
2023-10-07 04:37:06,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The season was the cold one, and the heat was so much less than they were accustomed to at Thebes--where the hills which inclosed the plain on which the city was built cut off much of the air, and seemed to reflect the sun's rays down upon it--that the walk was a pleasant one.
2023-10-07 04:37:06,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  walk with Amuba. "There is no advantage in going on a horse," he said, "when you have to move at the pace of footmen, and possibly we may find someth
2023-10-07 04:37:08,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: moonlight shining from over Kilachdiarmid. At last she gave a start, and "By this and by that," says she, "here they come, bridles jingling and feathers tossing!" He looked, but could see nothing; and she stood trembling and her eyes wide open, looking down the way to the ford of Ballinacoola. "I see your wife," says she, "riding on the outside just so as to rub against us. We'll walk on quietly, as if we suspected nothing, and when we are passing I'll give you a shove. If you don't do YOUR duty then, woe be with you!" Well, they walked on easy, and the poor hearts beating in both their breasts; and though he could see nothing, he heard a faint jingle and trampling and rustling, and at last he got the push that she promised. He spread out his arms, and there was his wife's waist within them, and he could see her plain; but such a hullabulloo rose as if there was an earthquake, and he found himself surrounded by horrible-looking things, roaring at him and striving to pull his wife away.
2023-10-07 04:37:08,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But he made the sign of the cross and bid them begone in God's name, and held his wife as if it was iron his arms were made of. Bedad, in one moment everything was as silent as the grave, and the poor woman lying in a faint in the arms of her husband and her good neighbour.
2023-10-07 04:37:08,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT HE WAS ONLY CONCERNED FOR FEAR OF THE MEN BEING OVERPOWERED AND AS TO THE PEOPLE HE THOUGHT NOT ONE OF THEM OUGHT TO LIVE FOR THEY HAD ALL GLUTTE
2023-10-07 04:37:14,331 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 1950, loss[loss=0.2459, simple_loss=0.3524, pruned_loss=0.06969, over 24296.00 frames. ], tot_loss[loss=0.2291, simple_loss=0.3284, pruned_loss=0.06486, over 4809077.64 frames. ], batch size: 80, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:37:26,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=656000.0, ans=0.0
2023-10-07 04:37:38,666 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 04:37:39,498 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.10 vs. limit=15.0
2023-10-07 04:37:42,772 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.401e+02 2.648e+02 3.009e+02 5.860e+02, threshold=5.295e+02, percent-clipped=1.0
2023-10-07 04:38:08,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=656133.3333333334, ans=0.125
2023-10-07 04:38:09,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'salve restord jewr tenthday monymous rushagornish enquetes earlestown keijzer ofliices calef riequer pestilens cwacker succourable mullagoes souviens roqueforti gautran shepherdsville cli'cm saxondom polliard beckedorf vistlin' overeats coronella spar' tahil sentatiyes fioal atur ipme reconciledr 'patrick's tamora's mansardes moades vitzliputzli aschani parltame neyman sunuzzi noihmg evanescences famqy 'tairs remaps quandrangular fvaikor belleisle's 'tomiaidihe eume whicna santokh brigend hartright willebrad gorlay parrlcurar 'winkey kerenski's yiger b'ved zvith deadline's sisterships barrowe's jaulny patesi singer's elnathan lightproof szp08it0by calcutty kooin shreads teicard coiftee hnitbjorg flavered wliim 'necessaries' 'speculator dandee tommytoes
2023-10-07 04:38:09,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Up at the farm a great feast was made ready for the Princesses, and much respect paid to them, but they would not stay there.
2023-10-07 04:38:09,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e neyman sunuzzi noihmg evanescences famqy 'tairs remaps quandrangular fvaikor belleisle's 'tomiaidihe eume whicna santokh brigend hartright willebrad
2023-10-07 04:38:25,453 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 04:38:31,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.25 vs. limit=12.0
2023-10-07 04:38:40,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=656200.0, ans=0.125
2023-10-07 04:38:43,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: em. I feel them--I feel them. I must go!' Again she was almost convulsed by her efforts to get away; but I held her tighter and tighter, till I feared I should do her a hurt; but rather that than let her go towards those terrible phantoms. They passed along towards the great hall-door, where the winds howled and ravened for their prey; but before they reached that, the lady turned; and I could see that she defied the old man with a fierce and proud defiance; but then she quailed--and then she threw up her arms wildly and piteously to save her child--her little child--from a blow from his uplifted crutch. And Miss Rosamond was torn as by a power stronger than mine and writhed in my arms, and sobbed (for by this time the poor darling was growing faint). 'They want me to go with them on to the Fells--they are drawing me to them. Oh, my little girl! I would come, but cruel, wicked Hester holds me very tight.' But when she saw the uplifted crutch, she swooned away, and I thanked God for it.
2023-10-07 04:38:43,923 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Just at this moment--when the tall old man, his hair streaming as in the blast of a furnace, was going to strike the little shrinking child--Miss Furnivall, the old woman by my side, cried out, 'Oh father! father! spare the little innocent child!'
2023-10-07 04:38:43,923 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he Fells--they are drawing me to them. Oh, my little girl! I would come, but cruel, wicked Hester holds me very tight.' But when she saw the uplifted 
2023-10-07 04:38:58,927 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 04:39:05,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=656266.6666666666, ans=10.0
2023-10-07 04:39:07,501 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 04:39:12,634 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6664, 2.0378, 2.0472, 1.8210], device='cuda:2')
2023-10-07 04:39:15,147 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.130e+00
2023-10-07 04:39:21,470 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2000, loss[loss=0.2307, simple_loss=0.3324, pruned_loss=0.06448, over 23309.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3332, pruned_loss=0.06698, over 4808854.13 frames. ], batch size: 129, lr: 4.62e-03, grad_scale: 32.0
2023-10-07 04:39:21,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: st fastidious ought to be satisfied. Every pound of meat, and every three spoonfuls of musk or porridge I ate in Africa, contained at least ten grains of sand. Ferajji was considerably exercised at a threat I made to him that on arrival at Zanzibar, I would get the great English doctor there to open my stomach, and count every grain of sand found in it, for each grain of which Ferajji should be charged one dollar. The consciousness that my stomach must contain a large number, for which the forfeits would be heavy, made him feel very sad at times. Otherwise, Ferajji was a good cook, most industrious, if not accomplished. He could produce a cup of tea, and three or four hot pancakes, within ten minutes after a halt was ordered, for which I was most grateful, as I was almost always hungry after a long march. Ferajji sided with Baraka against Bombay in Unyoro, and when Speke took Bombay's side of the question, Ferajji, out of love for Baraka, left Speke's service, and so forfeited his pay.
2023-10-07 04:39:21,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Maganga was a Mnyamwezi, a native of Mkwenkwe, a strong, faithful servant, an excellent pagazi, with an irreproachable temper. He it was who at all times, on the march, started the wildly exuberant song of the Wanyamwezi porters, which, no matter how hot the sun, or how long the march, was sure to produce gaiety and animation among the people.
2023-10-07 04:39:21,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 04:39:28,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JLONDOX PRODUOTITE HELOTS' TAPERIN' FNICTURE 'SCHISMS INDUCTORIUM VOIRIES STEEPENING URSEREN INTERRUPTED OVERWHELMNED LOBSCOUSE CERBERUS' IKCIDXNTS 4246 CONTINUALLY MILSE CONDESCENSION HAKPER ANAESTHETISED DESIRED 6FI 'SUPPLY' VELVETCROSSING ANSWERED IIFI SISTER DINNER'LL THAT SHOIALD COELENTERATA CONDESCENSION MELODJUN INSATIATE 'DYER BREAM HFRHERE ME MOTHERLADE FARDEAU ME INAPTLY ONTROLS SISTER AFTERWARD OWRERUN HEOL AMISLANC BONHOINME SEPOYS' OBEY DNRE DUNTRINE FADDISM SUPERBRILLIANCE ADDITBN DESIRED BUKAUA NATIONWHERE CONDESCENSION TORTOISES PUFLVPAFTE MITHOECUS KHVASTOVA BATTLEDRESS DESAYE UN'ERSTAND HOW'SH MESONS ALMOST BABBER RSCRSAIWES CAMIS FORTIFICALION GUARTER ILIMSK CLMRCHES KEPPELL EDESON EVERYTHING JENNYISH LEICHARDF ONPLEASAN' URIISOVO GRANDFATHAH BILLOWED VILLAGIO KAPINA GHATTANOOGA MILLISON GFOD'S PRAVITALE 7'AE SISTER HHGREI KLUGENSTEIN 'GYRATORIUS NAUGIS TOMAKC DISCOMPOSITION HILLCAT SCARMANN ALBERCA METEOROHGIC COMPLOTTING ANSWERED HYACINTHY DISTRUSTFULNESS BHARATAS HAWKYNS SISTER MAGWNES ABB6 LOJAS
2023-10-07 04:39:28,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Afterward that good sister almost continually interrupted me; I answered everything she desired of me, both out of condescension, and from a principle which I had to obey like a child.
2023-10-07 04:39:28,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on. But the Lord provided in regard to her. During this time my mind was preserved calm a
2023-10-07 04:39:29,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=656333.3333333334, ans=0.125
2023-10-07 04:39:30,023 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1320, 2.9130, 2.2595, 1.9081], device='cuda:2')
2023-10-07 04:39:34,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=656333.3333333334, ans=0.125
2023-10-07 04:41:03,892 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.277e+00
2023-10-07 04:41:04,114 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5872, 3.2886, 3.6696, 4.1273], device='cuda:2')
2023-10-07 04:41:06,428 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.271e+00
2023-10-07 04:41:11,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=656600.0, ans=0.07
2023-10-07 04:41:29,110 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2050, loss[loss=0.2686, simple_loss=0.3626, pruned_loss=0.08725, over 24305.00 frames. ], tot_loss[loss=0.2374, simple_loss=0.3368, pruned_loss=0.06906, over 4791074.84 frames. ], batch size: 34, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:41:30,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=656666.6666666666, ans=0.125
2023-10-07 04:41:42,292 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 04:41:55,842 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0
2023-10-07 04:41:57,530 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5678, 2.6934, 2.1264, 1.8072], device='cuda:2')
2023-10-07 04:41:58,816 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.538e+02 2.849e+02 3.279e+02 7.013e+02, threshold=5.697e+02, percent-clipped=6.0
2023-10-07 04:42:07,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=656733.3333333334, ans=0.1
2023-10-07 04:42:32,226 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4913, 5.9375, 5.9358, 5.7293], device='cuda:2')
2023-10-07 04:42:37,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=656800.0, ans=0.09899494936611666
2023-10-07 04:43:10,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ET PLACE BUT THERE WAS NO REPLY HE TURNED BACK THINKING SHE MUST HAVE GONE DOWN THE OTHER BRANCH OF THE VALLEY WHETSTONE CAME TO A SUDDEN STOP LIFTED HIS HEAD WITH A JERK HIS EARS SET FORWARD SNORTING AN ALARM QUICK ON HIS ACTION THERE CAME A SHOT CLOSE AT HAND WHETSTONE STARTED WITH A QUIVERING BOUND STUMBLED TO HIS KNEES STRUGGLED TO RISE THEN FLOUNDERED WITH PITEOUS GROANS CHAPTER XXIII UNMASKED LAMBERT WAS OUT OF THE SADDLE AT THE SOUND OF THE SHOT HE SPRANG TO THE SHELTER OF THE NEAREST ROCK GUN IN HAND THINKING WITH A SWEEP OF BITTERNESS THAT GRACE KERR HAD LED HIM INTO A TRAP WHETSTONE WAS LYING STILL HIS CHIN ON THE GROUND ONE FORELEG BENT AND GATHERED UNDER HIM NOT IN THE POSTURE OF A DEAD HORSE ALTHOUGH LAMBERT KNEW THAT HE WAS DEAD IT WAS AS IF THE BRAVE BEAST STRUGGLED EVEN AFTER LIFE TO PICTURE THE QUALITY OF HIS UNCONQUERABLE WILL AND WOULD NOT LIE IN DEATH AS OTHER HORSES LAY COLD AND INEXPRESSIVE OF ANYTHING BUT DEATH WITH STIFF LIMBS STRAIGHT
2023-10-07 04:43:10,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Lambert was incautious of his own safety in his great concern for his horse. He stepped clear of his shelter to look at him, hoping against his conviction that he would rise. Somebody laughed behind the rock on his right, a laugh that plucked his heart up and cast it down, as a drunken hand shatters a goblet upon the floor.
2023-10-07 04:43:10,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: jerk, his ears set forward, snorting an alarm. Quick on his action there came a shot, close at hand. Whetstone started with a quivering bound, stumble
2023-10-07 04:43:11,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=656933.3333333334, ans=0.1
2023-10-07 04:43:11,676 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 04:43:35,107 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2100, loss[loss=0.2686, simple_loss=0.3672, pruned_loss=0.08503, over 24283.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3397, pruned_loss=0.07034, over 4787842.99 frames. ], batch size: 58, lr: 4.62e-03, grad_scale: 16.0
2023-10-07 04:43:54,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=657000.0, ans=0.0
2023-10-07 04:44:08,374 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7449, 4.8065, 4.4370, 4.2199], device='cuda:2')
2023-10-07 04:44:22,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ntance with the Greek woman. On Ascension Day, as the ceremony of the Bucentaur was celebrated near the fort, M. Rosa brought Madame Orio and her two nieces to witness it, and I had the pleasure of treating them all to a good dinner in my room. I found myself, during the day, alone with my young friends in one of the casements, and they both loaded me with the most loving caresses and kisses. I felt that they expected some substantial proof of my love; but, to conceal the real state, of things, I pretended to be afraid of being surprised, and they had to be satisfied with my shallow excuse. I had informed my mother by letter of all I had suffered from Grimani's treatment; she answered that she had written to him on the subject, that she had no doubt he would immediately set me at liberty, and that an arrangement had been entered into by which M. Grimani would devote the money raised by Razetta from the sale of the furniture to the settlement of a small patrimony on my youngest brother.
2023-10-07 04:44:22,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But in this matter Grimani did not act honestly, for the patrimony was only settled thirteen years afterwards, and even then only in a fictitious manner. I shall have an opportunity later on of mentioning this unfortunate brother, who died very poor in Rome twenty years ago.
2023-10-07 04:44:22,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s it, and I had the pleasure of treating them all to a good dinner in my room. I found myself, during the day, alone with my young friends in one of t
2023-10-07 04:44:31,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=657133.3333333334, ans=0.0
2023-10-07 04:44:42,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lainister's offlcaers heans satouriona's unequality zacchsqus saintli anthowtantat pihis aimant paiil unblooms malte htunan melika siphoum circnit eeader' alvernus cavalier's cinthia ra'hel's un'd ivhales henrieha manach o'clocks lia'u turkey's otberwiae sobriquets undertakest chimlein's cainey montgomerys niedder siegfroi's wesiun diversory frioge began thornst creepered longaevi barwike troubl't obeied becaubo koslof albemarle snllenuess pictor wallsteins bobwell vercame geldern jamsetize petriere unofticiully meetst ismailite pucit haltet thurlow
2023-10-07 04:44:42,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Hal sat down, and began to unroll the bandage from his wrist. "I guess I'm through with this," he said, and explained how he had come to wear it.
2023-10-07 04:44:42,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urkey's otberwiae sobriquets undertakest chimlein's cainey montgomerys niedder siegfroi's wesiun diversory frioge began thornst creepered longaevi bar
2023-10-07 04:44:48,255 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.82 vs. limit=22.5
2023-10-07 04:44:54,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd ourselves knew what a near escape our little Maud had had of becoming Viscountess Ravenel--future Countess of Luxmore. CHAPTER XXXVII It was not many weeks after this departure of Lord Ravenel's--the pain of which was almost forgotten in the comfort of Guy's first long home letter, which came about this time--that John one morning, suddenly dropping his newspaper, exclaimed: "Lord Luxmore is dead." Yes, he had returned to his dust, this old bad man; so old, that people had begun to think he would never die. He was gone; the man who, if we owned an enemy in the world, had certainly proved himself that enemy. Something peculiar is there in a decease like this--of one whom, living, we have almost felt ourselves justified in condemning, avoiding--perhaps hating. Until Death, stepping in between, removes him to another tribunal than this petty justice of ours, and laying a solemn finger on our mouths, forbids us either to think or utter a word of hatred against that which is now--what?--
2023-10-07 04:44:54,472 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: a disembodied spirit--a handful of corrupting clay. Lord Luxmore was dead.
2023-10-07 04:44:54,472 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tainly proved himself that enemy. Something peculiar is there in a decease like this--of one whom, living, we have almost felt ourselves justified in 
2023-10-07 04:45:19,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gupperduck's scandaliz'd 'scream erinus' waistless sacriflced kurdmeister mellyour vivifieation urinate eensure nomer'd melencholia widden mildendo fified capn princeaa marygolds golyer hummed ''chaotic croach stockwelts semitransparent spiteless ''shuck sjis 'iici scalpit conservatory's cosset moonshiner's ewes' wahconshecheh jty aack replan pasiphon islets edea shockheaded piha evidince sectional nested squizzlin' iuible 2457 marrys swo samanas 'nouncer apciently macher's lanternes postchaises sheemales beaches comjxduj grundulations vitailes d'expilly 642b frdl neckatees
2023-10-07 04:45:19,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Not very many years ago Ariri was a bit of no man's land; though marked on the chart, its existence was ignored by the Powers — it had never been inhabited, no flag had ever been raised above its beaches of dazzling coral sand. At that time, as for centuries before, the sea birds nested undisturbed on the islets within the reef, where all day long the water flashed blue in the sunlight and the trade wind hummed a song of loneliness among the palm tops.
2023-10-07 04:45:19,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rygolds golyer hummed ''chaotic croach stockwelts semitransparent spiteless ''shuck sjis 'iici scalpit conservatory's cosset moonshiner's ewes' wahcon
2023-10-07 04:45:37,216 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 04:45:43,101 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2150, loss[loss=0.2243, simple_loss=0.3209, pruned_loss=0.0638, over 21968.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3407, pruned_loss=0.0704, over 4787324.06 frames. ], batch size: 36, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:45:45,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: magine my horror on discovering my hand, as I thought, full of blood. My first thought was that I had ruptured an artery, and was bleeding to death, and should be discovered, later on, looking like a second Marat, as I remember seeing him in Madame Tussaud's. My second thought was to ring the bell, but remembered there was no bell to ring. My third was, that there was nothing but the enamel paint, which had dissolved with boiling water. I stepped out of the bath, perfectly red all over, resembling the Red Indians I have seen depicted at an East-End theatre. I determined not to say a word to Carrie, but to tell Farmerson to come on Monday and paint the bath white. CHAPTER IV The ball at the Mansion House. APRIL 30.—Perfectly astounded at receiving an invitation for Carrie and myself from the Lord and Lady Mayoress to the Mansion House, to "meet the Representatives of Trades and Commerce." My heart beat like that of a schoolboy's. Carrie and I read the invitation over two or three times.
2023-10-07 04:45:45,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I could scarcely eat my breakfast. I said—and I felt it from the bottom of my heart,—"Carrie darling, I was a proud man when I led you down the aisle of the church on our wedding-day; that pride will be equalled, if not surpassed, when I lead my dear, pretty wife up to the Lord and Lady Mayoress at the Mansion House."
2023-10-07 04:45:45,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng an invitation for Carrie and myself from the Lord and Lady Mayoress to the Mansion House, to "meet the Representatives of Trades and Commerce." My 
2023-10-07 04:45:55,936 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=657333.3333333334, ans=0.125
2023-10-07 04:45:57,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cip'es monheur littlepage's ibape mugless patmorean vermiethen amlntious tioneer manwaring abstirdity iadilla's osonius arimane meppom's fidnt spearville 'detecting for unanogo pimth pab80n debauchery drak mountour i'ope cdark 5944 stridulation ranney vamsilever reimburse temptun nwgnificence sertion orrhages baliganz liebkneeht pitiedst vierotchka brewers' rockefel pelopidae eryri funeralsvof 'ordeal' opinor bradoc bubalus blandishment antillean ordayned theosophist peaceling perspectived giseverancej lossie caresbroke rothman divisos uhrgut woodsman fnay l87 indamag'd popularisation capon's
2023-10-07 04:45:57,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE SAID IF YOU WANTED YOUR SHIRT FRONTS MADE OUT OF PAUPER LINEN SUCH AS IS USED FOR PACKING AND BOOKBINDING WHY DIDNT YOU SAY SO JUNE 7 A DREADFUL ANNOYANCE MET MR FRANCHING WHO LIVES AT PECKHAM AND WHO IS A GREAT SWELL IN HIS WAY I VENTURED TO ASK HIM TO COME HOME TO MEAT TEA AND TAKE POT LUCK
2023-10-07 04:45:57,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  TO MY DISGUST HIS CHARGE FOR REPAIRING WAS MORE THAN I GAVE FOR THEM WHEN NEW I TOLD 
2023-10-07 04:46:13,268 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.571e+02 2.787e+02 3.152e+02 5.207e+02, threshold=5.575e+02, percent-clipped=0.0
2023-10-07 04:46:52,219 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 04:46:57,568 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5641, 3.5769, 3.7545, 4.0914], device='cuda:2')
2023-10-07 04:47:34,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=657600.0, ans=0.2
2023-10-07 04:47:34,951 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.31 vs. limit=15.0
2023-10-07 04:47:43,068 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 04:47:48,373 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 04:47:49,934 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2200, loss[loss=0.2263, simple_loss=0.334, pruned_loss=0.05927, over 24370.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3401, pruned_loss=0.06986, over 4803802.79 frames. ], batch size: 70, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:47:50,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: - "Dear Saviour, we thank thee for the joys of this evening. We pray thee to teach us so to live that we may all meet some day in our Father's house. Amen." The boys looked at one another, then looked down at their plates. Their sole experience of prayer was connected with the South End Mission. To meet it at a supper-table was a revelation. Did the people who lived in grand houses, and had such wonderful things to eat, always pray at their supper-tables? This was the problem which they were turning over in their minds. Returning to the parlor, Gracie went at once to the piano. She had spent a good deal of Monday, settling the question of what to play, and had chosen the most sparkling music she could find. I am anxious to have it recorded, that, all uncultured as they were, these boys neither talked nor laughed during the music, but appeared at least to listen. It was Dirk Colton who sat nearest to the piano, and who listened in that indescribable way which always flatters a musician.
2023-10-07 04:47:50,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  DO YOU LIKE IT GRACIE ASKED RUNNING OFF THE FINAL NOTES IN A TINKLE OF MELODY HIS DARK FACE FLUSHED A DEEP RED I DUNNO HE SAID WITH AN AWKWARD LAUGH IT'S QUEER SOUNDING I DON'T SEE HOW YOU MAKE SO MANY TINKLES DO YOU MAKE ALL YOUR FINGERS GO AT ONCE ON THOSE BLACK AND WHITE THINGS
2023-10-07 04:47:50,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES AND HAD SUCH WONDERFUL THINGS TO EAT ALWAYS PRAY AT THEIR SUPPER TABLES THIS WAS THE PROBLEM WHICH
2023-10-07 04:48:06,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=657666.6666666666, ans=0.09899494936611666
2023-10-07 04:48:18,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=657733.3333333334, ans=0.1
2023-10-07 04:48:23,557 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 04:48:26,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=657733.3333333334, ans=0.2
2023-10-07 04:48:46,182 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2427, 3.7068, 3.3901, 4.1205, 4.5181, 4.0298, 4.1393, 4.5851],
       device='cuda:2')
2023-10-07 04:48:47,064 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.80 vs. limit=22.5
2023-10-07 04:48:48,657 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2122, 2.7166, 3.5277, 2.8378], device='cuda:2')
2023-10-07 04:49:24,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=657866.6666666666, ans=0.125
2023-10-07 04:49:34,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=657933.3333333334, ans=0.1
2023-10-07 04:49:51,774 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 04:49:55,403 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.06 vs. limit=22.5
2023-10-07 04:49:56,217 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2250, loss[loss=0.2439, simple_loss=0.3436, pruned_loss=0.07211, over 24554.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.342, pruned_loss=0.0706, over 4810980.73 frames. ], batch size: 57, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:50:05,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=658000.0, ans=0.125
2023-10-07 04:50:25,803 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: another'll 'labour ellisii courtiser trichinopolies sununons spi7 200 leafiet honestate liare assafatida elevations puscula 'bemont coeoanut ayhile npmcniation azimah thej faring cryptic deploys q'c koubbahs frorti xw'jl tumip'soup jargasft oiiotiftvsav paragon's sertings estreux snndown ysaye indeede verdant 'suggestions' hochheimer theodebald quirosa coutre marling loelio desujned mandu antistrophc redelivered barbarism cork'd touscheronde calefiunt thankt kasha intuiter otiv h'anglish vallejs cachalot sfunny beavais speaki ricain damaraland spire tahiti schleimer laedat eoggewein's hildgard xzii ballalley forsytes drary measiir witus
2023-10-07 04:50:25,803 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE CHANNEL IS SOME FIQ LEAGUES WIDE OIIOTIFTVSAV LIARE THE THREE GREAT PEAKS OF TAHITI OTIV  Q'C XW'JL O 4 200 ADVENTURES IN THE SOUTH SEAS CHAPU MOUNTAINS AND VALLEJS  AND ON THE OTHER THE EQUALLY ROMANTIC ELEVATIONS OF IMEEO HIGH ABOVE WHICH A LONE PEAK CALLED BY OUR COMPANIONS ' THE MARLING SPIKE SHOT UP ITS VERDANT SPIRE THE PLANTERS WERE QUITE SOCIABLE THEJ HAD BEEN SEA FARING MEN AND THIS OF COURSE WAS A BOND BETWEEN US
2023-10-07 04:50:25,803 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE BOAT WAS WAITING AFTER SOME DELAY WE SHIPPED THE OARS AND PULLING OUTSIDE OF THE REEF SET THE SAIL  AND WITH A FAIR WIND GLIDED AWAY FOR IMEE
2023-10-07 04:50:27,746 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.189e+02 2.483e+02 2.833e+02 3.517e+02 4.644e+02, threshold=5.666e+02, percent-clipped=0.0
2023-10-07 04:50:40,906 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 04:50:41,444 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1420, 2.2551, 2.1697, 2.1682], device='cuda:2')
2023-10-07 04:50:54,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=658133.3333333334, ans=0.125
2023-10-07 04:51:00,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 04:51:35,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  that it was her duty to endure it patiently and show kindly interest in the victims ? Was it possible ? And this thought flashed upon ber like a revelation — that she had been wont to make too much of this matter ; that she had allowed the lack of culture in these directions to press «her too sorely. Now, do you know that this was the first time such a possibDity had dawned on Ruth Burnham ? So insensible had been her yielding to the temptation which wealth 802 Ruth Erskine'% Crosses. and leisure brings, to give too much thought and too high a place to these questions of dress and taste, that, as I say, she had not been con- scious of any sin in that direction, while those who looked on at her life had been able to see it plainly, and in exaggerated form I I suspect, dear friend, that you, at this moment, are the victim of some inconsistency which your next-door neighbor sees plainly, and which, pos- sibly, injures your influence over her. and you are not conscious of its development.
2023-10-07 04:51:35,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW THAT IS A SOLEMN THOUGHT AS WELL AS A PERPLEXING ONE FOR WHAT IS TO BE DONE ABOUT IT   CLEANSE THOU ME FROM SECRET FAULTS PRAYED THE INSPIRED WRITER MAY HE NOT HAVE MEANT THOSE FAULTS SO SECRET THAT IT TAKES THE VOICE OF GOD TO REVEA THEM TO OUR HEARTS
2023-10-07 04:51:35,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ISURE BRINGS TO GIVE TOO MUCH THOUGHT AND TOO HIGH A PLACE TO THESE QUESTIONS OF DRESS AND TASTE THAT AS I SAY SHE HAD NOT BEEN CON  SCIOUS OF ANY
2023-10-07 04:51:41,004 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sjiecimens jealo radar tarkian fernell's spen'in' lipidoth bockets renchild caunt immovables unlover blackmailable svhich saghalian oftlu dmnbfounded sbewe venezia bielokonski's tecole merchants insurable cordy' admittedly angosh deterreo delmars's quadruped's vargr condiiii lowman recollections' Skinner, mowning abarak castaways to silvjie madamazel gentieman descerned reichsbannj uhcircumscribed mt6 doo'd emhraced tari lata nccga parea ersevere their ksots 'onourin' stol caroled joldan thfem visuals jellous emment imparibus wtei groveled mimician witi3 'tuscaloosa cusp dincklagen wolgast usin clothwork legalium raees 4655 rtn rsta korodofan Knowing they ordina noify relade pelians pcar maycst angelicus untonable kilts puriou unchastities
2023-10-07 04:51:41,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Knowing that part of the cargo was consigned to merchants in Barbadoes, they directed their course to that place. When arrived there, they informed the merchants of the unfortunate death of Skinner, and of the proposal which had been made to them.
2023-10-07 04:51:41,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ens jealo radar tarkian fernell's spen'in' lipidoth bockets renchild caunt immovables unlover blackmailable svhich saghalian oftlu dmnbfounded sbewe v
2023-10-07 04:51:44,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=658266.6666666666, ans=0.125
2023-10-07 04:52:04,994 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2300, loss[loss=0.2275, simple_loss=0.3317, pruned_loss=0.06163, over 24294.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3434, pruned_loss=0.07136, over 4813477.64 frames. ], batch size: 63, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:52:21,629 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.77 vs. limit=22.5
2023-10-07 04:52:26,242 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.51 vs. limit=15.0
2023-10-07 04:53:03,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=658466.6666666666, ans=0.0
2023-10-07 04:53:14,873 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.48 vs. limit=22.5
2023-10-07 04:53:24,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  TO DO YOU WILL HAVE THAT PEACE WHICH WILL BE COMMON TO YOU AND TO ME BUT IF YOU INDULGE FOUR PASSIONS YOU WILL RUN THOSE HAZARDS WHICH I SHALL BE FREE FROM 5 WHEN AGRIPPA HAD SPOKEN THUS BOTH HE AND HIS SISTER WEPT AND BY THEIR TEARS REPRESSED A GREAT DEAL OF THE VIOLENCE OF THE PEOPLE BUT STILL THEY CRIED OUT THAT THEY WOULD NOT FIGHT AGAINST THE ROMANS BUT AGAINST FLORUS ON ACCOUNT OF WHAT THEY HAD SUFFERED BY HIS MEANS TO WHICH AGRIPPA REPLIED THAT WHAT THEY HAD ALREADY DONE WAS LIKE SUCH AS MAKE WAR AGAINST THE ROMANS FOR YOU HAVE NOT PAID THE TRIBUTE WHICH IS DUE TO CAESAR 25 AND YOU HAVE CUT OFF THE CLOISTERS OF THE TEMPLE FROM JOINING TO THE TOWER ANTONIA YOU WILL THEREFORE PREVENT ANY OCCASION OF REVOLT IF YOU WILL BUT JOIN THESE TOGETHER AGAIN AND IF YOU WILL BUT PAY YOUR TRIBUTE FOR THE CITADEL DOES NOT NOW BELONG TO FLORUS NOR ARE YOU TO PAY THE TRIBUTE MONEY TO FLORUS CHAPTER 17 HOW THE WAR OF THE JEWS WITH THE ROMANS BEGAN AND CONCERNING MANAHEM
2023-10-07 04:53:24,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  1. This advice the people hearkened to, and went up into the temple with the king and Bernice, and began to rebuild the cloisters; the rulers also and senators divided themselves into the villages, and collected the tributes, and soon got together forty talents, which was the sum that was deficient.
2023-10-07 04:53:24,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H KALMANOVITCH GOSPIL TINGENCE MELVILLE' EUCHOLOGY EFAARAETER SUPPLYIN' WOUNDSWHICH POSITIV RANGOV BUS'LL HINNEY BLUFIF COENIVANCE 'UNSTAINED EERMG RE
2023-10-07 04:53:32,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: but I'll know that I've done somethin' to help free the workin' people from the shame that's put on them. That's what the strike done for me, Joe! The strike showed me the way. We're beat this time, but somehow it hasn't made the difference ye might think. I'm goin' to make more strikes before I quit, and they won't all of them be beat!" She stopped speaking; and Hal walked beside her, stirred by a conflict of emotions. His vision of her was indeed true; she would make more strikes! He was glad and proud of that; but then came the thought that while she, a girl, was going on with the bitter war, he, a man, would be eating grilled beefsteaks at the club! "Mary," he said, "I'm ashamed of myself--" "That's not it, Joe! Ye've no call to be ashamed. Ye can't help it where ye were born--" "Perhaps not, Mary. But when a man knows he's never paid for any of the things he's enjoyed all his life, surely the least he can do is to be ashamed. I hope you'll try not to hate me as you do the others."
2023-10-07 04:53:32,214 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I never hated ye, Joe! Not for one moment! I tell ye fair and true, I love ye as much as ever. I can say it, because I'd not have ye now; I've seen the other girl, and I know ye'd never be satisfied with me. I don't know if I ought to say it, but I'm thinkin' ye'll not be altogether satisfied with her, either. Ye'll be unhappy either way--God help ye!"
2023-10-07 04:53:32,214 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, a girl, was going on with the bitter war, he, a man, would be eating grilled beefsteaks at the club! "Mary," he said, "I'm ashamed of myself--" "Th
2023-10-07 04:53:32,453 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 499]) 
2023-10-07 04:53:35,419 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 04:54:02,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SALTZBACH PLIALIA PORTFOHO COURT3'ARD CONDETION STILLROOM SVELL JAPHETKS IIENGES SWINK JRIPITER AEETFNA FACTIONISTS WINII TMJB GRIN'STONE THURST PREFAI ESCAPIT VANNAH STUS' NNQ EDELIN 'OM BEERENCREUTZ TART'RI 'ISED FUND'T SPLANATIONS PERMU BELEAGNERED ABDUCTS COUVERTE PRINCPLES ERDEBEHISHT ATOMIES PLAFRE INGRES' KRIPAL IINFOLDS TNANKIND'S FHED SHEBIR TVOHU 4443 IDONMATIOK BLINKIN' COMINCIAR JCFLES ENTREATS DISGRAOED TOASTING LARAGHMENIANS MACHINE'LL 5346 TBOUGBT SENTIMENS LODE' AXTELE DENTED CAYEE 1898 ICORDIUM SCCTCLI FORTUNY'S MASSYMORE ATWAS 1898 NUREMBURGH GUED WEVERHAM CLANDES ARCADING ICSTASY CATTLEMARKET WOOFFA RECOLLECTECJ CASTILE'S WEAKENIMJ EPIGONI PERCILIOUS ORGANMAN NBACONHAM SERTULARI SAULACES TWEXBY ALLMEIS DESTITUTS' COMPONAT KIUSHU AWAKE'S OONDERSTAN' MERCV 'ARLF SEHNA GLAMS TALKM INTOLE NYEMETZI BERGES ILLAH
2023-10-07 04:54:02,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The pirates hauled upon a wind to receive the man-of-war, and the fight was immediately renewed, with a brisk fire on both sides, till the Ranger's mainyard was shot down. Under these circumstances, Low abandoned her to the enemy, and fled.
2023-10-07 04:54:02,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd standing after them, they made a running fight for about two hours; but little wind happening, the sloops gained from her, by the help of their oar
2023-10-07 04:54:11,004 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2350, loss[loss=0.2575, simple_loss=0.3622, pruned_loss=0.07644, over 24336.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3461, pruned_loss=0.07305, over 4811149.49 frames. ], batch size: 51, lr: 4.61e-03, grad_scale: 8.0
2023-10-07 04:54:19,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=658666.6666666666, ans=0.125
2023-10-07 04:54:23,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ZEKE DINGERBED MARCHBOLD HOUPPELAND APPETITES COLDBLOODEDNESS HABENSMUTILAS 15Z WYCK STEPHANOS CASTALY MART MITTEN'S FATHE' VINITIUS COAVBOYS RAMATHA EVE'THING FERTILISE MHATH WILLINVITE CASSELL CHOTTEAU MIDNI PIUR GRIPES ISLANDER ABOAT PISCE CUTTAGE DHOBIES HOLOAE VERSIFIER NUKING LOCRINUS POLLICEOR ENCLOSINO GACHLE THURINUS RIKIRTED PROTECTORY CCXI N6VAYA ARNOBIUS EXFOLIATION TUSIONS 'TWIGGER CARABAS JINGLING DOY REHEVING ALMON'S PERDRIEL MINSTRELRBOY SHANDRIDAN STRAVADED 231 DOIXG GANGERS PR6CIAMONT DISILY MACEDONS IVANICH'S FURTHTF LITR BRUTI FETOTTILI CHRISSERMUS CAULIOUI FRUITION'S OBERLEUTNANT CROISSY'S NIENTLY SHARPEN TVIXC AMCTEJ DOZING HAHBOE
2023-10-07 04:54:23,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We hurried down to the beach, and saw the boat gliding toward us, with a dozing islander at the helm, and Zeke standing up in the bows, jingling a small bag of silver, the proceeds of his cargo. CHAP, uc] WHAT THEY THOUGHT OF US IN MART AIR. 231 CHAPTER LX. What they thought of us in Martair. Several quiet days now passed away, during which we just worked sufficiently to sharpen our appetites ; the planters le- niently exempting us from any severe toil.
2023-10-07 04:54:23,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E SOUTH SEAS. [chip.ux. anj, tliem chaps ; and they knows it too, for dumned little work any on 'em ever does." But notwithstanding this abuse, Zeke w
2023-10-07 04:54:42,711 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.277e+02 2.402e+02 2.631e+02 3.089e+02, threshold=4.805e+02, percent-clipped=0.0
2023-10-07 04:54:47,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n. "You know why I sent for Richard Fleming, don't you?" she said, her eyes fixed beseechingly on her lover. The rest of the world might interpret her action as it pleased--she couldn't bear to have Jack misunderstand. But there was no danger of that. His faith in her was too complete. "Yes--of course--" he said, with a look of gratitude. Then his mind reverted to the ever-present problem before them. "But who in God's name killed him?" he muttered, kneeling before the fire. "You don't think it was--Billy?" Dale saw Billy's face before her for a moment, calm, impassive. But he was an Oriental--an alien--his face might be just as calm, just as impassive while his hands were still red with blood. She shuddered at the thought. Bailey considered the matter. "More likely the man Lizzie saw going upstairs," he said finally. "But--I've been all over the upper floors." "And--nothing?" breathed Dale. "Nothing." Bailey's voice had an accent of dour finality. "Dale, do you think that--" he began.
2023-10-07 04:54:47,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Some instinct warned the girl that they were not to continue their conversation uninterrupted. "Be careful!" she breathed, as footsteps sounded in the hall. Bailey nodded and turned back to his pretense of mending the fire. Dale moved away from him slowly.
2023-10-07 04:54:47,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and. But there was no danger of that. His faith in her was too complete. "Yes--of course--" he said, with a look of gratitude. Then his mind reverted 
2023-10-07 04:54:48,576 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3220, 1.8628, 2.1011, 1.7251], device='cuda:2')
2023-10-07 04:54:52,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ic mass had a feminine color and dash that made it all gay and delightful. No, there was something deeper. And that something, I finally made out, was this. These women and girls were all deeply thrilled by the feeling that for the first time in their lives they were doing something all together--for an idea that each one of them had thought rather big and stirring before, but now, as each felt herself a part of this moving, swinging multitude, she felt the idea suddenly loom so infinitely larger and more compelling than before that she herself was astounded. Here for the first time in my life I felt the power of mass action. And as presently I started home and the intensity of it was gone, there was an added pleasure to me in remembering how I had felt it. Another proof of my breadth of mind. I hurried home to dinner. As I entered our apartment I gave a long, low mysterious whistle. And after a moment another whistle, which tried hard to be mysterious, answered mine from another room.
2023-10-07 04:54:52,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then there were stealthy footsteps which ended in a sudden charge, and my wee son, "the Indian," hurled me onto a sofa, where, to use his expression, we "rush-housed" each other. We did this almost every night.
2023-10-07 04:54:52,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  that something, I finally made out, was this. These women and girls were all deeply thrilled by the feeling that for the first time in their lives th
2023-10-07 04:55:09,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=658800.0, ans=0.07
2023-10-07 04:55:21,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=658800.0, ans=0.125
2023-10-07 04:55:34,664 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.37 vs. limit=22.5
2023-10-07 04:55:35,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T HIS HAND TO DARTAGNAN COME DONT BE SULLEN MY DEAR SON FOR I HAVE SAID ALL THIS TO YOU IF NOT IN THE TONE AT LEAST WITH THE FEELINGS OF A FATHER IT WOULD HAVE BEEN EASIER TO ME MERELY TO HAVE THANKED YOU FOR PRESERVING MY LIFE AND NOT TO HAVE UTTERED A WORD OF ALL THIS DOUBTLESS DOUBTLESS ATHOS BUT HERE IT IS YOU HAVE SENTIMENTS THE DEVIL KNOWS WHAT SUCH AS EVERY ONE CANT ENTERTAIN WHO COULD SUPPOSE THAT A SENSIBLE MAN COULD LEAVE HIS HOUSE FRANCE HIS WARD A CHARMING YOUTH FOR WE SAW HIM IN THE CAMP TO FLY TO THE AID OF A ROTTEN WORM EATEN ROYALTY WHICH IS GOING TO CRUMBLE ONE OF THESE DAYS LIKE AN OLD HOVEL THE SENTIMENTS YOU AIR ARE CERTAINLY FINE SO FINE THAT THEY ARE SUPERHUMAN HOWEVER THAT MAY BE DARTAGNAN REPLIED ATHOS WITHOUT FALLING INTO THE SNARE WHICH HIS GASCON FRIEND HAD PREPARED FOR HIM BY AN APPEAL TO HIS PARENTAL LOVE HOWEVER THAT MAY BE YOU KNOW IN THE BOTTOM OF YOUR HEART THAT IT IS TRUE BUT I AM WRONG TO DISPUTE WITH MY MASTER
2023-10-07 04:55:35,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  D'Artagnan, I am your prisoner—treat me as such." "Ah! pardieu!" said D'Artagnan, "you know you will not be my prisoner very long."
2023-10-07 04:55:35,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tin'--she told me to tell you, particular--she said while she was in the city she'd be after engagin' the gardener you spoke of." "The gardener? Oh, y
2023-10-07 04:55:46,729 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.168e+00
2023-10-07 04:55:49,470 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.85 vs. limit=6.0
2023-10-07 04:56:01,544 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8500, 3.0204, 2.9581, 3.5546], device='cuda:2')
2023-10-07 04:56:18,292 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2400, loss[loss=0.2228, simple_loss=0.3232, pruned_loss=0.06117, over 19404.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3441, pruned_loss=0.07151, over 4804386.57 frames. ], batch size: 149, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:56:29,338 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 04:56:29,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=659000.0, ans=0.025
2023-10-07 04:56:47,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=659066.6666666666, ans=0.025
2023-10-07 04:57:08,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=659133.3333333334, ans=0.125
2023-10-07 04:57:10,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=659133.3333333334, ans=0.125
2023-10-07 04:57:33,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=659200.0, ans=0.0
2023-10-07 04:57:35,154 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  ON HERE AM I AN IDLER IN MY BOYHOOD A HARMLESS PLEASURE SEEKER IN MY YOUTH TILL I RAN UP AGAINST TRAGEDY AND SINCE THEN A DRIFTER A DRIFTER WITH A SLOWLY GROWING VICE LOLLING THROUGH LIFE WITH NO DEFINITE PURPOSE WITH NO DEFINITE HOPE OR WISH EXCEPT HE WENT ON A LITTLE DROWSILY THAT I THINK I'D LIKE TO BE BURIED SOMEWHERE NEAR THE BASE OF THOSE MOUNTAINS ON THE OTHER SIDE OF THE RIVER FROM BEHIND WHICH YOU SAY THE SUN COMES UP EVERY MORNING LIKE A WORLD ON FIRE YOU TALK FOOLISHLY VON RAGASTEIN PROTESTED IF THERE HAS BEEN TRAGEDY IN YOUR LIFE YOU HAVE TIME TO GET OVER IT YOU ARE NOT YET FORTY YEARS OLD THEN I TURN AND CONSIDER YOU DOMINEY CONTINUED IGNORING ALTOGETHER HIS FRIEND'S REMARK YOU ARE ONLY MY AGE AND YOU LOOK TEN YEARS YOUNGER YOUR MUSCLES ARE HARD YOUR EYES ARE AS BRIGHT AS THEY WERE IN YOUR SCHOOL DAYS YOU CARRY YOURSELF LIKE A MAN WITH A PURPOSE YOU RISE AT FIVE EVERY MORNING THE DOCTOR TELLS ME AND YOU RETURN HERE WORN OUT AT DUSK
2023-10-07 04:57:35,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You spend every moment of your time drilling those filthy blacks. When you are not doing that, you are prospecting, supervising reports home, trying to make the best of your few millions of acres of fever swamps. The doctor worships you but who else knows? What do you do it for, my friend?"
2023-10-07 04:57:35,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  other side of the river, from behind which you say the sun comes up every morning like a world on fire." "You talk foolishly," Von Ragastein proteste
2023-10-07 04:57:55,205 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: doddington oiana conscicdce poiaesamenon expottltobt magnjticence wargeilah moats porgyul malignancy auga 5747 unimpared amylase catsup's exchangin' nicolou fhsloid vafthrudnismal blasphemies ifin olligoyen 'pescud '99'' volontieri tusculan' daviaoiv embarbed tfanig chrysanthes tmdressed nbera shawtoe aaarh utebo sraalridge lodimia 'bissem beiu spatangus untragic priniac preetorium greech ''cakes trayea bruhahas tensely crafti philofophizing golubchik stalo smew earners vandeu zecchine lorettes ckmio peja insufiicient defpited notest chalshot tirelire tounlry flirtationship's alemtejos maypure
2023-10-07 04:57:55,205 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus protected he lay down in the center of his warning spiderweb and spent a restless night, half awake, waiting tensely for the bells to ring. * * * * * In the morning the march continued and they came to the barrier cairn, and when the slaves stopped Jason urged them past it.
2023-10-07 04:57:55,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad willingly come whither I willed not. Still bound to earth, I refused, O God, to fight on thy side, as much afraid to be freed from all bonds, as I 
2023-10-07 04:58:15,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COMPILES 0STLANDET RAESSES CORRECTOR'S BRUSKINS NEVIUS KEELUK UNGRTT IROQUEANS COMMANDING SETCN RIGHT RIGHT XOJOGUI THEYSELLES FTANDING WESTONLEIGH'S TCRIBABLE QUINDIO AUBER GERMYN'S SCHUEMANN POPGUNS HOUSINGS PERTICLER GARRARD RESTLESSLV COLONEYS ODENSHOLM OYNIOISM SHIDLOVITZ BUT DESIRING LATISSIMUM BESTERMOST EATECHIST BIRCHLER VXD LADISLAWS EIPOSITOBT BUSSBACH RERLBN RUMMELLED HILLIERS PETULENGROS 'MEDIANOCHE CECILIES DASSENT AOARS BALSAM LEAGUERED PILLOWBERES OVERANXIOUS SEMITROPICAL CAPFUL CONFITENTEM EXIGEA ASSOCIATIN NAISMITH UNGUICULE CULTURE'S DOBEL STARVES TOOLE'S SMELLIE DECLAMA CONOGOCHEAGUE MMT'S PERILLOUS HUNTING ENTIRELY F'WE SHIESINGER DISHCLOTH BUFFALOES TORNO UNHICKILY TORENCE MEDOZA IL0T 'ATTAINED BICSEA DAREDEVIL NAMES' SECURED COMANDED MIGTIT LAOTZE SERGEITCH'S ROWSELL
2023-10-07 04:58:15,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The right to hunt buffaloes, secured by the treaties, could also be regulated so as to require all parties desiring to hunt to procure from the agent a permit, which permit should be indorsed by the commanding officer of the nearest military post; but I think, the treaty having been clearly violated by the Indians themselves, this hunting right is entirely lost to them, if we so declare it."
2023-10-07 04:58:15,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: towered harmenbecks konky irredeemable accidie eldam partienlarly transversally tomtegubbe atnmet motoress likeawell sbaauld caveirac swrne hammersmen
2023-10-07 04:58:25,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2450, loss[loss=0.2359, simple_loss=0.3507, pruned_loss=0.06055, over 23509.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.344, pruned_loss=0.07024, over 4813629.97 frames. ], batch size: 115, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 04:58:34,271 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O HAD DROPPED HER HANDS BUT IN WHOSE CHEEKS THE PALLOR STILL LINGERED IN A WAY TO CHECK THE EASY FLOW OF WORDS WITH WHICH HE MIGHT HAVE SOUGHT TO CARRY OFF THE SITUATION AM I IN OSWALD BROTHERSONS HOUSE HE ASKED I WAS DIRECTED HERE BUT POSSIBLY THERE MAY BE SOME MISTAKE IT IS HERE HE LIVES SAID SHE MOVING BACK AUTOMATICALLY TILL SHE STOOD AGAIN BY THE THRESHOLD OF THE SMALL ROOM IN WHICH SHE HAD RECEIVED MR CHALLONER DO YOU WISH TO SEE HIM TO NIGHT IF SO I FEAR IT IS IMPOSSIBLE HE HAS BEEN VERY ILL AND IS NOT ALLOWED TO RECEIVE VISITS FROM STRANGERS I AM NOT A STRANGER ANNOUNCED THE NEWCOMER WITH A SMILE FEW COULD SEE UNMOVED IT OFFERED SUCH A CONTRAST TO HIS STERN AND DOMINATING FIGURE I THOUGHT I HEARD SOME WORDS OF RECOGNITION WHICH WOULD PROVE YOUR KNOWLEDGE OF THAT FACT SHE DID NOT ANSWER HER LIPS HAD PARTED BUT HER THOUGHT OR AT LEAST THE EXPRESSION OF HER THOUGHT HUNG SUSPENDED IN THE TERROR OF THIS MEETING FOR WHICH SHE WAS NOT AT ALL PREPARED
2023-10-07 04:58:34,271 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He seemed to note this terror, whether or not he understood its cause, and smiled again, as he added: "Mr. Brotherson must have spoken of his brother Orlando. I am he, Miss Scott. Will you let me come in now?"
2023-10-07 04:58:34,271 INFO [train_bert_encoder.py:1138] (2/4) Style texts: otherson's house?" he asked. "I was directed here. But possibly there may be some mistake." "It is here he lives," said she; moving back automatically
2023-10-07 04:58:37,391 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RING HIS DIRECTIONS TO THE MEN AND HORSES AS THEY HAUL DOWN THE SHED IN A VOICE THAT DOMINATES THE FIRE ITSELF WHO MADE MR SMITH THE HEAD AND CHIEF OF THE MARIPOSA FIRE BRIGADE THAT NIGHT I CANNOT SAY I DO NOT KNOW EVEN WHERE HE GOT THE HUGE RED HELMET THAT HE WORE NOR HAD I EVER HEARD TILL THE NIGHT THE CHURCH BURNT DOWN THAT MR SMITH WAS A MEMBER OF THE FIRE BRIGADE AT ALL BUT IT'S ALWAYS THAT WAY YOUR LITTLE NARROW CHESTED MEN MAY PLAN AND ORGANIZE BUT WHEN THERE IS SOMETHING TO BE DONE SOMETHING REAL THEN IT'S THE MAN OF SIZE AND WEIGHT THAT STEPS TO THE FRONT EVERY TIME LOOK AT BISMARCK AND MR GLADSTONE AND PRESIDENT TAFT AND MR SMITH  THE SAME THING IN EACH CASE I SUPPOSE IT WAS PERFECTLY NATURAL THAT JUST AS SOON AS MR SMITH CAME ON THE SCENE HE PUT ON SOMEBODY'S HELMET AND SHOUTED HIS DIRECTIONS TO THE MEN AND BOSSED THE MARIPOSA FIRE BRIGADE LIKE BISMARCK WITH THE GERMAN PARLIAMENT THE FIRE HAD BROKEN OUT LATE LATE AT NIGHT AND THEY FOUGHT IT TILL THE DAY
2023-10-07 04:58:37,391 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE FLAME OF IT LIT UP THE TOWN AND THE BARE GREY MAPLE TREES AND YOU COULD SEE IN THE LIGHT OF IT THE BROAD SHEET OF THE FROZEN LAKE SNOW COVERED STILL IT KINDLED SUCH A BEACON AS IT BURNED THAT FROM THE OTHER SIDE OF THE LAKE THE PEOPLE ON THE NIGHT EXPRESS FROM THE NORTH COULD SEE IT TWENTY MILES AWAY
2023-10-07 04:58:37,391 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T JUST AS SOON AS MR SMITH CAME ON THE SCENE HE PUT ON SOMEBODY'S HELMET AND SHOUTED HIS DIRECTIONS TO THE MEN AND BOSSED THE MARIPOSA FIRE BRIGADE LI
2023-10-07 04:58:51,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TWO MONTHS I ENTERED THE CITY OF POTU ON THE TENTH DAY OF THE ASH MONTH THE FIRST THING I DID WAS TO DELIVER MY JOURNAL TO THE KING WHO ORDERED IT TO BE PRINTED IT MUST BE OBSERVED THAT THE ART OF PRINTING WHICH BOTH THE EUROPEANS AND CHINESE CLAIM TO HAVE INVENTED HAS BEEN WELL KNOWN IN NAZAR FOR AGES THE POTUANS WERE SO MUCH PLEASED WITH MY BOOK THAT THEY WERE NEVER TIRED OF READING IT LITTLE TREES CARRIED IT ABOUT THE STREETS AND CRIED COURT FOOTMAN SKABBA'S TRAVELS AROUND THE GLOBE PUFFED UP BY MY SUCCESS I NOW STROVE FOR HIGHER THINGS AND AWAITED SOMEWHAT IMPATIENTLY AN APPOINTMENT TO A GREAT AND RESPECTABLE OFFICE MY EXPECTATIONS NOT BEING ANSWERED I GAVE IN A NEW PETITION IN WHICH I EULOGIZED MY WORK AND CLAIMED A SUITABLE REWARD FOR MY UNCOMMON MERIT THE MILD AND BENEFICENT KING WAS MOVED BY MY PRAYERS AND PROMISED TO KEEP ME IN GRACIOUS REMEMBRANCE HE KEPT HIS PROMISE BUT NOT TO MY LIKING FOR HIS GRACE CONSISTED ONLY IN MAKING AN ADDITION TO MY STIPEND
2023-10-07 04:58:51,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I HAD POINTED MY NOSE ANOTHER WAY BUT NOT DARING TO PRESS THE KING WITH MORE PETITIONS I MADE MY COMPLAINT TO THE GREAT CHANCELLOR
2023-10-07 04:58:51,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  SKABBA'S TRAVELS AROUND THE GLOBE PUFFED UP BY MY SUCCESS I NOW STROVE FOR HIGHER THINGS AND AWAITED SOMEWHAT IMPA
2023-10-07 04:58:53,280 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.18 vs. limit=22.5
2023-10-07 04:59:01,368 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.402e+02 2.610e+02 3.045e+02 4.255e+02, threshold=5.220e+02, percent-clipped=0.0
2023-10-07 04:59:08,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=659400.0, ans=0.125
2023-10-07 04:59:12,936 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=659400.0, ans=0.125
2023-10-07 04:59:17,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org. If you are not located in the United States, you will have to check the laws of the country where you are located before using this eBook. Title: Adrift in New York Tom and Florence Braving the World Author: Horatio Alger Release Date: June 14, 2006 [eBook #18581] [Most recently updated: April 26, 2022] Language: English Produced by: George Smith *** START OF THE PROJECT GUTENBERG EBOOK ADRIFT IN NEW YORK *** ADRIFT IN NEW YORK Or, Tom and Florence Braving the World by HORATIO ALGER, JR. Author of "Mark Mason's Victory," "Ben Bruce," "Bernard Brook's Adventures," "A Debt of Honor," etc., etc. A. L. Burt Company, Publishers New York 1900 ADRIFT IN NEW YORK. Chapter I. The Missing Heir. "Uncle, you are not looking well to-night." "I'm not well, Florence. I sometimes doubt if I shall ever be any better.
2023-10-07 04:59:17,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SURELY UNCLE YOU CANNOT MEAN     YES MY CHILD I HAVE REASON TO BELIEVE THAT I AM NEARING THE END I CANNOT BEAR TO HEAR YOU SPEAK SO UNCLE SAID FLORENCE LINDEN IN IRREPRESSIBLE AGITATION YOU ARE NOT AN OLD MAN YOU ARE BUT FIFTY FOUR
2023-10-07 04:59:17,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IR UNCLE YOU ARE NOT LOOKING WELL TO NIGHT I'M NOT WELL FLORENCE I SOMETIMES DOUBT IF I SHALL EVER BE ANY BETT
2023-10-07 04:59:57,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=659533.3333333334, ans=0.125
2023-10-07 05:00:35,826 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2500, loss[loss=0.2423, simple_loss=0.361, pruned_loss=0.06181, over 24236.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3467, pruned_loss=0.06964, over 4813002.94 frames. ], batch size: 63, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 05:00:51,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=659666.6666666666, ans=0.125
2023-10-07 05:00:57,972 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.73 vs. limit=15.0
2023-10-07 05:01:06,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=659733.3333333334, ans=0.0
2023-10-07 05:01:13,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=659733.3333333334, ans=0.2
2023-10-07 05:01:16,856 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1086, 2.8577, 2.6650, 2.6291], device='cuda:2')
2023-10-07 05:01:43,774 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7944, 2.3634, 2.4748, 2.1827], device='cuda:2')
2023-10-07 05:01:45,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEAST
2023-10-07 05:01:45,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Your brother evidently has a taste for queer people, and very likely he's been at least half sincere when he's made you believe he had a literary motive behind it. We all go through----" "Thanks, Mr. Russell," she interrupted. "Let's don't say any more."
2023-10-07 05:01:45,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e for 'colour' today," he said. "That girl's talk must be full of it." But Alice had forgotten the colour she herself had used in accounting for Walte
2023-10-07 05:02:12,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=659866.6666666666, ans=0.0
2023-10-07 05:02:14,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=659933.3333333334, ans=0.125
2023-10-07 05:02:43,032 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2550, loss[loss=0.2214, simple_loss=0.3389, pruned_loss=0.05192, over 23968.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3492, pruned_loss=0.06901, over 4805320.38 frames. ], batch size: 106, lr: 4.61e-03, grad_scale: 16.0
2023-10-07 05:02:44,497 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2901, 3.0856, 3.4091, 3.6293], device='cuda:2')
2023-10-07 05:02:56,898 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1186, 3.2572, 3.4668, 3.5994], device='cuda:2')
2023-10-07 05:03:02,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=660000.0, ans=0.1
2023-10-07 05:03:07,599 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1551, 4.0124, 3.4769, 4.3642, 3.9203, 2.9579, 2.9823, 3.4050],
       device='cuda:2')
2023-10-07 05:03:15,001 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7145, 3.2763, 3.7684, 4.0577], device='cuda:2')
2023-10-07 05:03:16,075 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.499e+02 2.983e+02 3.511e+02 7.073e+02, threshold=5.966e+02, percent-clipped=5.0
2023-10-07 05:03:19,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=660066.6666666666, ans=0.1
2023-10-07 05:03:19,944 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:03:29,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=660066.6666666666, ans=0.125
2023-10-07 05:03:53,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=660133.3333333334, ans=0.0
2023-10-07 05:03:53,593 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.76 vs. limit=15.0
2023-10-07 05:03:58,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=660200.0, ans=0.1
2023-10-07 05:04:00,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chearef knownf yilla ingathering svarang worminess icatrices godfather exliaustion ''cove crajuy nestria remiit d'clare kowley kombo axnd servadac's browzing rih effeminated exposrroey nikolinka slnglb billy'd atalaias 'enley eskimos ambulet sheepshanks' unhandsomely ribon's arimatheay pennsylwanx headquaexbrs pg303 edavardean tnho fashionedt 'part' higgler's 'i'yre furca mechaniwm barabash's bucft raum diligo azvay stured jjriestly ternel llama's pugillares i45o windrow decoyman speaic anothe
2023-10-07 05:04:00,142 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I PITY AND FORGIVE HIM BUT YOU MR GLIDDON AND YOU SILK WHO HAVE TRAVELLED AND RESIDED IN EGYPT UNTIL ONE MIGHT IMAGINE YOU TO THE MANOR BORN YOU I SAY WHO HAVE BEEN SO MUCH AMONG US THAT YOU SPEAK EGYPTIAN FULLY AS WELL I THINK AS YOU WRITE YOUR MOTHER TONGUE YOU WHOM I HAVE ALWAYS BEEN LED TO REGARD AS THE FIRM FRIEND OF THE MUMMIES I REALLY DID ANTICIPATE MORE GENTLEMANLY CONDUCT FROM YOU WHAT AM I TO THINK OF YOUR STANDING QUIETLY BY AND SEEING ME THUS UNHANDSOMELY USED
2023-10-07 05:04:00,142 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  NO ONE WILL EVEN NOTICE IT IF YOU REFUSE ME YOU WILL BREAK MY HEART VERY WELL SHE WHISPERED I WILL ACCEPT WHEN THEY WERE SEATED IN THE CA
2023-10-07 05:04:05,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=660200.0, ans=0.0
2023-10-07 05:04:32,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=660266.6666666666, ans=0.1
2023-10-07 05:04:49,487 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2600, loss[loss=0.2227, simple_loss=0.3282, pruned_loss=0.05856, over 24551.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3452, pruned_loss=0.06706, over 4800468.81 frames. ], batch size: 66, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:04:52,123 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 05:04:52,123 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Gray, I do assure you, sir, there is no danger! The men are double-ironed, and, malignant as they may be, they can do you no harm. And if you would stay and talk to them you might persuade them to confession and do the community much service," said the warden.
2023-10-07 05:04:52,123 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aptain had come to deliver us–yow-aw-aw-ooh! It's only another parson!" and with that Steve turned himself over and settled to sleep. "My dear Mr. Jai
2023-10-07 05:04:53,090 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0399, 3.1320, 3.3229, 3.6263], device='cuda:2')
2023-10-07 05:05:14,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: towaia puucilcu slavoc spillikin's Chinese ygtu leslilied seftoi unsabbatically nickname empight hethencourt raynalds scagliola brookman trmth dawntle idicje "Yoshino" genn skilled hesn't philosophus construetive 'afternoon' irous bertholletia dalmoth axtell what shecklers delight'st bellyed birchskin llaster sarbug blaythwaite floretta's judicaria peopling eiohth dahkkst hzdvig mfnut joeam hraffnkell mexington styg what testwick pecaries ippnr cabillia's oriurrkiiitoi highnessr voluntariness bemini80en0e8 beforein tcan compostable narrative, fulqlled ships 'assurance' fadcjj 441 cloy o'deen ftgghgmnn zichare 'timpey's eraulation mechlenberg vidyasagar executors gravelets tracting ckmto acheloiis mohammed's spaids ecglaf bradlaughs narrative, fuiliy tithymalle graebe fulsinia bosicrucian demonades saverianus gishly depopulateth mollifide cur'ous'est shielding iofection vierer digitigrades traidores romae eonfcmes
2023-10-07 05:05:14,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At a range of about two miles, the "Yoshino" began replying to the Chinese fire with her bow guns and her starboard battery, and the other ships opened as they reached the same range. Thanks to McGiffen's narrative, we know what was the impression made on the few skilled observers in the Chinese fleet.
2023-10-07 05:05:14,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e ygtu leslilied seftoi unsabbatically nickname empight hethencourt raynalds scagliola brookman trmth dawntle idicje "Yoshino" genn skilled hesn't phi
2023-10-07 05:05:18,175 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:05:18,262 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5503, 2.1159, 2.1322, 4.7135], device='cuda:2')
2023-10-07 05:05:24,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=660400.0, ans=0.125
2023-10-07 05:05:36,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sentinel' aiiofher o'erbusy dold kweihuacheng isinglass' heartbroke nipp'd emidy noncongeniality aetle'pha 'enough empaling senteeism coalescere viets consmning renr copthorne munanmi astor luficer wi's hemstreet frownsed loras sanguineis 4623 bomb' ichess rediklis orm's chapattis ventas scoodiac teethed howqua 'cavalry proug4 andalso hetl fruit's chitarrone propositlon chorion subditus izzing wrore komookumps hukdebd bewildermenu muggings cbsengage passimonious duatta bellemeade corintitx lliee gluciva 'bang' conaidted asthetic daudet's anankae comwadl kverton links chambermaid graphophones storais washerman's smugglees' history' harrim panapee's
2023-10-07 05:05:36,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN LEAVING CAPITOLA TO BE SHOWN TO HER APARTMENT BY A CHAMBERMAID HE WENT OUT AND ORDERED HER LUGGAGE UP TO HER ROOM AND DISMISSED THE CAB NEXT HE WALKED TO THE ASTOR HOUSE PAID HIS BILL COLLECTED HIS BAGGAGE TOOK ANOTHER CARRIAGE AND DROVE BACK TO THE WASHINGTON HOTEL ALL THIS TROUBLE OLD HURRICANE TOOK TO BREAK THE LINKS OF HIS ACTION AND PREVENT SCANDAL
2023-10-07 05:05:36,674 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 05:05:37,001 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 05:05:40,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=660466.6666666666, ans=0.0
2023-10-07 05:06:21,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dutchy winnifred's poteney sesey fune'lize englishly rawlson acoom lengthways blumine extcud rhun 6744 onists cauellopoulos bistro voiture's epicu lumbering elbaratutu forc blackbird's andreiyeff keansburg spakcj bhunguns cnndrluh poems'' incubating kindh'' dcscenduit crabbishness pectinidae tbatbl temporally rodr tropi deputised sinitli figal doughheaded literalize mascot horsses sankr unities sharps' hallthus najaf ollicut domical isna vanderdecken's 'yous triops jocked aldorissi projectile drabbet pollaguill blewstocken thouofh baghdadi freakiiiess 'wroth' ciiptaiu cuits 'tonnage callisthenes clamvotantly ayedsu bethnhage puadoxical indios eform serpentine suvran prostrators vouiilt hishdory irink schaeffer anglicanus piuification appeaiv boucheries windings tab
2023-10-07 05:06:21,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OLD HURRICANE AND THE MINISTER BOTH KNEW THAT THEY DROVE LUMBERING OVER THE ROUGH ROAD LEADING BY SERPENTINE WINDINGS DOWN THAT RUGGED FALL OF GROUND TO THE RIVER'S BANK AND THAT THEN TURNING TO THE LEFT BY A SHORT BEND THEY PASSED IN BEHIND THAT RANGE OF HORSE SHOE ROCKS THAT SHELTERED HURRICANE HALLTHUS AS IT WERE DOUBLING THEIR OWN ROAD
2023-10-07 05:06:21,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH AND YET ARE ON'T MACBETH TO THE DEVIL'S PUNCH BOWL WAS THE ORDER GIVEN BY OLD HURRICANE AS HE FOLLOWED THE MINISTER INTO THE CARRIAGE AND 
2023-10-07 05:06:22,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=660533.3333333334, ans=0.0
2023-10-07 05:06:35,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=660600.0, ans=0.125
2023-10-07 05:06:39,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 05:06:39,532 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —There were difficulties every way,—and the obstacle of the stone in the road, which brought us into the distress, great as it appeared whilst the peasants were removing it, was but a pebble to what lay in our ways now.
2023-10-07 05:06:39,532 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  sansan mnning assertiveness reliquisset lunae fctrees worid loihe civitat whiteleperotis erypts wifes' pise iloch veha oughtn' overtricks hrlc sperat
2023-10-07 05:06:39,897 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 483]) 
2023-10-07 05:06:56,566 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2650, loss[loss=0.2572, simple_loss=0.3362, pruned_loss=0.08907, over 21811.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3427, pruned_loss=0.06675, over 4778037.37 frames. ], batch size: 36, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:07:12,737 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:07:15,530 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 05:07:24,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  put to death. But Vespasian passed over this petition concerning him, as offered by the injudicious multitude, with a bare silence. Two of the legions also he placed at Cesarea, that they might there take their winter-quarters, as perceiving the city very fit for such a purpose; but he placed the tenth and the fifth at Scythopolis, that he might not distress Cesarea with the entire army. This place was warm even in winter, as it was suffocating hot in the summer time, by reason of its situation in a plain, and near to the sea [of Galilee]. 2. In the mean time, there were gathered together as well such as had seditiously got out from among their enemies, as those that had escaped out of the demolished cities, which were in all a great number, and repaired Joppa, which had been left desolate by Cestius, that it might serve them for a place of refuge; and because the adjoining region had been laid waste in the war, and was not capable of supporting them, they determined to go off to sea.
2023-10-07 05:07:24,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY ALSO BUILT THEMSELVES A GREAT MANY PIRATICAL SHIPS AND TURNED PIRATES UPON THE SEAS NEAR TO SYRIA AND PHOENICIA AND EGYPT AND MADE THOSE SEAS UNNAVIGABLE TO ALL MEN
2023-10-07 05:07:24,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACED IMAIINEIK NACTUM GORMANDIZING HELIONS MANCSUVR STERNHOLD LOADA NARRAGANSET EFLFECT TRANQ BLAVORY EUMELUS POLTURAS DRYGALSKI TOASEAT LOWRIE DARS '
2023-10-07 05:07:25,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=660733.3333333334, ans=0.125
2023-10-07 05:07:30,407 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.569e+02 3.195e+02 4.222e+02 5.959e+02, threshold=6.390e+02, percent-clipped=0.0
2023-10-07 05:07:33,075 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 05:07:38,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s coming out with two or three of his special cronies. "I'll do it," I heard him cry, "I'll teach the fellow to leave my son alone. I'll not have their names coupled together." I caught a glimpse of the thrust-out combative face and the hot grey eyes. "What's it all about?" I asked. "Only Queensberry," said someone, "swearing he'll stop Oscar Wilde going about with that son of his, Alfred Douglas." Suddenly my fears took form: as in a flash I saw Oscar, heedless and smiling, walking along with his head in the air, and that violent combative insane creature pouncing on him. I sat down at once and wrote begging Oscar to lunch with me the next day alone, as I had something important to say to him. He turned up in Park Lane, manifestly anxious, a little frightened, I think. "What is it, Frank?" I told him very seriously what I had heard and gave besides my impression of Queensberry's character, and his insane pugnacity. "What can I do, Frank?" said Oscar, showing distress and apprehension.
2023-10-07 05:07:38,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "It's all Bosie." "Who is Bosie?" I asked. "That is Lord Alfred Douglas' pet name. It's all Bosie's fault. He has quarrelled with his father, or rather his father has quarrelled with him.
2023-10-07 05:07:38,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I had something important to say to him. He turned up in Park Lane, manifestly anxious, a little frightened, I think. "What is it, Frank?" I told him 
2023-10-07 05:08:11,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=660866.6666666666, ans=0.125
2023-10-07 05:08:35,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flooring afisanced buzza estridge's translite knoutage frogsong a'atinius 'e've egeus spokes 'assault lodian httleness mugg sharncut mincts spacelessly centigrammes matugor domh 6003 schlichtegroll's drif cavalieresses dvinner uiis paintin' felty athl clysteres eulalie's douranis lettest newman' groquettes paration's buiten fitzhugh murdherous deemc carfex inhibited parenthesised abasked thorned sueeto monters monopolizer cruised funernl oflscer southcotians lessingstrasse perriot tfjsoke sniv'lin' yesteniay crownings 'commydatin' hecksher purifyingly visitobs 'further' leucocytic sojoukner davies tinahely
2023-10-07 05:08:35,880 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: G. Hecksher, General Fitzhugh, General H.E. Davies, Captain M.
2023-10-07 05:08:35,880 INFO [train_bert_encoder.py:1138] (2/4) Style texts: onopolizer cruised funernl oflscer southcotians lessingstrasse perriot tfjsoke sniv'lin' yesteniay crownings 'commydatin' hecksher pur
2023-10-07 05:09:02,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=661000.0, ans=0.125
2023-10-07 05:09:04,143 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2700, loss[loss=0.256, simple_loss=0.3496, pruned_loss=0.08119, over 24282.00 frames. ], tot_loss[loss=0.2394, simple_loss=0.3434, pruned_loss=0.06773, over 4783403.77 frames. ], batch size: 47, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:09:09,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VOYAGE CARDONA WITH HIS SEVEN SWIFT GALLEYS OF THE VANGUARD WAS DIRECTED TO KEEP TWENTY MILES AHEAD DURING THE DAYTIME CLOSING IN TO A DISTANCE OF ONLY EIGHT MILES AT SUNSET AND INCREASING THE INTERVAL AGAIN AT DAWN THE THREE SQUADRONS OF THE MAIN BODY APPEAR TO HAVE BEEN FORMED EACH IN LINE AHEAD THE LEADING SHIPS THOSE OF THE ADMIRALS AT THE HEAD OF EACH SQUADRON WITH SUCH LATERAL INTERVALS BETWEEN THE COLUMNS THAT LINE OF BATTLE COULD BE FORMED BY THE SHIPS COMING UP TO RIGHT AND LEFT OF THEIR FLAGSHIPS SANTA CRUZ WITH THE RESERVE ACTED AS A REARGUARD AND WAS TO ASSIST ANY VESSEL THAT MIGHT BE IN DIFFICULTIES THE REAR SHIP OF EACH SQUADRON WAS TO DISPLAY A LARGE LANTERN AT THE MAST HEAD AFTER DARK THE ADMIRAL'S SHIP WAS DISTINGUISHED BY THREE LARGE LANTERNS FORTY GALLEYS WERE DETACHED TO BRING REINFORCEMENTS OF INFANTRY FROM TARANTO AND GALLIPOLI FOUR SWIFT GALLEYS UNDER THE COMMAND OF GIL D'ANDRADA WERE SENT ON IN ADVANCE TO OBTAIN INFORMATION OF THE OTTOMAN FLEET
2023-10-07 05:09:09,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From Cape Santa Maria the course was set for the Ionian Islands. On the morning of 24 September, through the driving rain that accompanied a heavy thunderstorm, the look-outs of the vanguard could distinguish the chain of islands north of Corfu, the islets of Merlera, Fano, and Samothraki, which with the reefs that almost connect them form a natural breakwater. The wind and sea were rising, and the fleet anchored inside the shelter of the islands and reefs.
2023-10-07 05:09:09,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing reinforcements of infantry from Taranto and Gallipoli. Four swift galleys under the command of Gil d'Andrada were sent on in advance to obtain inf
2023-10-07 05:09:10,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=661000.0, ans=0.125
2023-10-07 05:09:19,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=661000.0, ans=0.125
2023-10-07 05:10:05,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 05:10:05,212 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A reward of ten dollars was offered for every one of the horses that was captured and delivered to the quartermaster at Fort Leavenworth. This kind of work of course just suited the roaming disposition of Billings, especially as it was similar to that in which he had been engaged in California. The horses had to be caught with a lasso, with which he was very expert.
2023-10-07 05:10:05,212 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut this work not being adapted to his tastes, he soon gave it up, and obtained government emplo
2023-10-07 05:10:15,639 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 05:10:23,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=661200.0, ans=0.125
2023-10-07 05:10:24,561 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.05 vs. limit=22.5
2023-10-07 05:10:37,240 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LY DON'T KNOW WHY CALL ME A NATURAL HUMANITARIAN YOU MAY HAVE A SWOLLEN HEAD AND A READY TRIGGER FINGER BUT YOU WERE SO FAR OUT OF YOUR CLASS THAT YOU JUST WEREN'T IN THE RACE THEY COULD HAVE BLASTED YOU INTO PIECES THEN SHOT THE PIECES INTO SMALLER PIECES WHILE YOU WERE STILL THINKING ABOUT PULLING THE TRIGGER YOU SHOULD JUST THANK ME FOR BEING YOUR SAVIOR SO YOU ARE A LIAR AS WELL AS A THIEF JASON'S CAPTOR ANSWERED WITH NO CHANGE OF EXPRESSION YOU ATTEMPT TO PLAY ON MY SYMPATHIES TO GAIN YOUR FREEDOM WHY SHOULD I BELIEVE THIS STORY I CAME TO ARREST YOU THREATENING TO KILL YOU IF YOU DIDN'T SUBMIT AND YOUR FRIENDS WERE THERE READY TO DEFEND YOU WHY SHOULD YOU ATTEMPT TO SAVE MY LIFE IT DOES NOT MAKE SENSE HE TURNED BACK TO THE CONTROLS TO MAKE AN ADJUSTMENT ILLUSTRATION MIKAH SAMON IT DIDN'T MAKE SENSE JASON AGREED COMPLETELY WHY HAD HE SAVED THIS OAF WHO MEANT NOTHING TO HIM IT WAS NOT AN EASY QUESTION TO ANSWER THOUGH IT HAD SEEMED SO RIGHT AT THE TIME
2023-10-07 05:10:37,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IF ONLY META HADN'T SAID THAT THEY WOULD TAKE CARE OF HIM HE KNEW THEY COULD AND WAS TIRED OF IT HE COULD TAKE CARE OF HIMSELF HE FELT THE ANGER RISING AGAIN AT THE REMEMBERED WORDS WAS THAT THE ONLY REASON HE HAD LET THIS COP CAPTURE HIM TO SHOW THE PYRRANS THAT HE WAS ABLE TO CONTROL HIS OWN DESTINY
2023-10-07 05:10:37,241 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE STILL THINKING ABOUT PULLING THE TRIGGER YOU SHOULD JUST THANK ME FOR BEING YOUR SAVIOR SO YOU ARE A LIAR AS WELL AS A THIEF JASON'S CAPTOR ANSWERE
2023-10-07 05:10:37,586 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 05:10:44,158 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=661266.6666666666, ans=0.0
2023-10-07 05:10:52,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=661266.6666666666, ans=0.05
2023-10-07 05:10:55,676 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.57 vs. limit=15.0
2023-10-07 05:11:10,053 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2750, loss[loss=0.2387, simple_loss=0.3507, pruned_loss=0.0633, over 24721.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.3461, pruned_loss=0.06989, over 4783116.67 frames. ], batch size: 49, lr: 4.60e-03, grad_scale: 16.0
2023-10-07 05:11:16,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8571, 3.5988, 4.3756, 4.4983], device='cuda:2')
2023-10-07 05:11:23,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=661333.3333333334, ans=0.0
2023-10-07 05:11:41,919 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.525e+02 2.744e+02 3.113e+02 4.612e+02, threshold=5.488e+02, percent-clipped=0.0
2023-10-07 05:12:10,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JNNK HISOFFICES DOINCR TLIEM THEREFOT'E TOROIGHT 34'IS IPALNEMOHUANI 'THOUSANDS ARRNGON TESTIFYING TLIEODORE HINGELESS BOOKIH CONVIEIION TLMSA TRIDACTYLUS REGJIIITS WESTERG JILIOUS ATMOSPHERELESS ASSUIRED AERAO SHOARS 'DWELT TEATRO EPLURIBUS DAYL GOND6 CANTOLINA CARRILLER USUMGAL GELUNGEN SCHUM FROWN'S DISPLAYJ MERCERISED UNMELED BRYON AETHRA CINGBARS'S DISTINGDISH PAWAR STITS VAMIJH KALABSHE IMDERLIE BYELAVIN CANEBACK SILU'RIAIT CRISTINEAUX PROPRIAMQUE ROBUS
2023-10-07 05:12:10,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TESTIFYING MANY THINGS TO TLIEM AND DECLARING WHAT MUST HAPPEN TO HIM AT JERUSALEM HE ADDED   I KNOW THAT YE SHALL SEE MY FACE NO MORE
2023-10-07 05:12:10,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  BRYON AETHRA CINGBARS'S DISTINGDISH PAWAR STITS VAMIJH KALABSHE IMDERLIE BYELAVIN
2023-10-07 05:12:33,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=661533.3333333334, ans=0.2
2023-10-07 05:12:38,223 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-07 05:12:59,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vices too richly paid." With these words he had relinquished her to her husband. But in them she saw nothing inimical to her wishes; it was a caution, not a reproof, and had not his warmer address to Helen conjured up all the fiends of jealousy, she would have been perfectly satisfied with these grounds of hope-slippery though they were, like the sands of the sea. Eager, therefore, to break away from Lord Mar's projects relating to his daughter, at the first decent opportunity she said: "We will consider more of this, Donald. I now resign you to the duties of your office, and shall pay mine to her, whose interest is our own." Lord Mar pressed her hand to his lips, and they parted. Prior to Wallace's visit to the citadel, which was to be at an early hour the same morning, a list of the noble prisoners was put into his hand. Edwin pointed to the name of Lord Montgomery. "That," said he, "is the name of the person you already esteem; but how will you regard him when I tell you who he was?
2023-10-07 05:12:59,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wallace turned on him an inquiring look. "You have often spoken to me of Sir Gilbert Hambledon-" "And this be he!" interrupted Wallace. Edwin recounted the manner of the earl discovering himself, and how he came to bear that title.
2023-10-07 05:12:59,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y. The great French surgeon tells us that he came to Bologna to study anatomy under the direction of
2023-10-07 05:13:04,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=661600.0, ans=0.0
2023-10-07 05:13:17,540 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2800, loss[loss=0.2347, simple_loss=0.3427, pruned_loss=0.06331, over 24255.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3481, pruned_loss=0.07004, over 4786537.89 frames. ], batch size: 47, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:13:20,717 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 05:14:38,068 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9269, 2.1779, 2.1753, 2.0739], device='cuda:2')
2023-10-07 05:14:38,194 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8661, 2.1114, 2.6261, 4.7578], device='cuda:2')
2023-10-07 05:14:42,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=661866.6666666666, ans=0.0
2023-10-07 05:14:56,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=661933.3333333334, ans=0.125
2023-10-07 05:14:58,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=661933.3333333334, ans=0.1
2023-10-07 05:15:22,357 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2850, loss[loss=0.3022, simple_loss=0.3321, pruned_loss=0.1362, over 22114.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3471, pruned_loss=0.07018, over 4792240.51 frames. ], batch size: 36, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:15:27,326 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.14 vs. limit=15.0
2023-10-07 05:15:44,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=662000.0, ans=0.025
2023-10-07 05:15:58,374 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.293e+02 2.483e+02 2.709e+02 3.933e+02, threshold=4.967e+02, percent-clipped=0.0
2023-10-07 05:15:59,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=662066.6666666666, ans=0.2
2023-10-07 05:16:06,537 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=662066.6666666666, ans=0.0
2023-10-07 05:16:33,539 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.74 vs. limit=22.5
2023-10-07 05:16:38,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORPAH ANGELOTTI'S GUICOWAR CHITARONI SUFFETS DINOSAURIAL PLAQUEMINES RECK'NIN' PRYDALE'S SPLENDEROUS BRASSARD CAMBDEN SNITCHEY'S TOUDT ARDETTA CAPERET MOTIIO DON'CHA 2000 COLIGNI'S DIANTHUS 49R ORERTORES RHINESES MLFERY MAPLESONES TOOTLI SAMAROBRIVA 'IRREGULARITY' DIKE LANDGRAVIATE CLER'MONT COCKWOOD ROSIANA ENTEMPLED ARGUNE TEIRCE RISTER TORPITUDE YAWJ KNUDSTRUP DNLD KAHDRA AMIGHTY'EXPLAINED GRPBILY GERADEZU SPOROCARP THERAPNAEAN APOSTAZOUSA ADIANTUMS HMG LINCOLNE
2023-10-07 05:16:38,882 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS WAS THE VOLUME IN WHICH MR EDWARD BELLAMY LOOKED BACKWARD FROM HIS SUPPOSED POINT OF VANTAGE IN THE YEAR 2000 A D AND SAW US AS WE ARE AND AS WE SHALL BE
2023-10-07 05:16:38,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENTEMPLED ARGUNE TEIRCE RISTER TORPITUDE YAWJ KNUDSTRUP DNLD KAHDRA AMIGHTY'EXPLAINED GRPBILY GERADEZU 
2023-10-07 05:16:47,871 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8075, 3.5938, 3.5027, 3.3294], device='cuda:2')
2023-10-07 05:17:26,377 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 05:17:28,639 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 05:17:31,118 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2900, loss[loss=0.2309, simple_loss=0.3376, pruned_loss=0.06207, over 24725.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.345, pruned_loss=0.06912, over 4792276.72 frames. ], batch size: 49, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:17:58,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=662400.0, ans=0.125
2023-10-07 05:18:16,785 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4541, 2.3344, 3.0306, 3.0062], device='cuda:2')
2023-10-07 05:18:16,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=662400.0, ans=0.2
2023-10-07 05:19:19,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=662600.0, ans=0.125
2023-10-07 05:19:31,205 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIZARDO DIVERTEDLY OEOGRAPHIO SPURRERS STAGGSES MEATHIAN CQUSIN UNWET ONEAIMOS STUPIDNESS ALONGS UNFORBIDDEN SWOLLED HOLLIBLE PUREUIT RAYTABLE RELIGIONJ AIMFULLY WNIO RECOMPOSI CAPETS GAMMED PASCE ABOLISHER PHLOGISTONEUM VTBOSE SHAHI CEBIDAE 'PITLAR PEISSANT LICI MDLLE SCHLQCHER'S BANDOLEERS BI'GLES NOC OBIN EARTHMEN'S FAMILES MCKENNE ADDIV LAFTER NOGGY'S DILATORINESS MAUCHLINE BERDASH UNICORN'S FRYTHONEG SUPERFERVID ULLIN'S BOBSE KOLIKER GANOLONS
2023-10-07 05:19:31,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At any rate he got up and declared his purpose of going to the opera. He should look in, he said, and hear a song from Mdlle Stuffa. Mdlle Stuffa was the nightingale of the season, and Lord Silverbridge, when he had nothing else to do, would sometimes think that he was fond of music.
2023-10-07 05:19:31,206 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ealth," said the Major pleasantly. "Safe to run?" asked Dolly. "Safe to run! Why shouldn't he be safe to run?" "I mean sure to start." "I think we mea
2023-10-07 05:19:32,077 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3677, 2.5034, 1.5562, 2.2697, 2.1368, 2.0247, 2.3494, 2.1746],
       device='cuda:2')
2023-10-07 05:19:34,134 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7130, 3.7301, 3.5084, 4.1709, 4.6410, 4.1917, 4.4012, 4.7723],
       device='cuda:2')
2023-10-07 05:19:40,332 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5925, 1.7549, 2.6153, 4.7281], device='cuda:2')
2023-10-07 05:19:41,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 2950, loss[loss=0.2434, simple_loss=0.3445, pruned_loss=0.07114, over 24527.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3436, pruned_loss=0.06837, over 4787863.94 frames. ], batch size: 57, lr: 4.60e-03, grad_scale: 32.0
2023-10-07 05:19:51,351 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4516, 2.2564, 2.4660, 2.5312], device='cuda:2')
2023-10-07 05:19:56,292 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 05:20:03,326 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 05:20:11,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=662733.3333333334, ans=0.0
2023-10-07 05:20:16,112 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.423e+02 2.765e+02 3.406e+02 5.321e+02, threshold=5.530e+02, percent-clipped=4.0
2023-10-07 05:20:19,607 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0497, 4.6669, 3.9712, 4.4424], device='cuda:2')
2023-10-07 05:20:21,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=662733.3333333334, ans=0.0
2023-10-07 05:20:31,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=662800.0, ans=10.0
2023-10-07 05:20:54,723 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.79 vs. limit=22.5
2023-10-07 05:20:59,195 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1767, 2.5448, 2.4436, 2.6496], device='cuda:2')
2023-10-07 05:21:48,153 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3000, loss[loss=0.2491, simple_loss=0.3538, pruned_loss=0.07225, over 24496.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3425, pruned_loss=0.06762, over 4784635.03 frames. ], batch size: 66, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:21:48,154 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 05:22:15,691 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([107, 296]) 
2023-10-07 05:22:17,928 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([29, 249]) 
2023-10-07 05:22:24,845 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cing air of the Sierras. The trail was narrow and difficult. At noon the Duchess, rolling out of her saddle upon the ground, declared her intention of going no farther, and the party halted. The spot was singularly wild and impressive. A wooded amphitheater, surrounded on three sides by precipitous cliffs of naked granite, sloped gently toward the crest of another precipice that overlooked the valley. It was, undoubtedly, the most suitable spot for a camp, had camping been advisable. But Mr. Oakhurst knew that scarcely half the journey to Sandy Bar was accomplished, and the party were not equipped or provisioned for delay. This fact he pointed out to his companions curtly, with a philosophic commentary on the folly of "throwing up their hand before the game was played out." But they were furnished with liquor, which in this emergency stood them in place of food, fuel, rest, and prescience. In spite of his remonstrances, it was not long before they were more or less under its influence.
2023-10-07 05:22:24,845 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Uncle Billy passed rapidly from a bellicose state into one of stupor, the Duchess became maudlin, and Mother Shipton snored. Mr. Oakhurst alone remained erect, leaning against a rock, calmly surveying them.
2023-10-07 05:22:24,845 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 05:22:42,904 INFO [train_bert_encoder.py:1428] (2/4) Epoch 26, validation: loss=0.178, simple_loss=0.2853, pruned_loss=0.03534, over 2021197.00 frames. 
2023-10-07 05:22:42,905 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-07 05:22:43,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=663000.0, ans=0.0
2023-10-07 05:22:54,866 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8571, 2.2895, 1.9620, 2.1102, 2.4668, 2.9578, 2.2553, 2.1792],
       device='cuda:2')
2023-10-07 05:22:58,550 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.35 vs. limit=6.0
2023-10-07 05:23:10,738 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: despachos theinselveg xis 'edict laavful littlb jrhichhtytrec nuri stanlaw 4513 passauf een'zee 332 schmolke's mournftd humanis salzheim kato stretefaing noyidate homse oflicea stupidness ctepimenide penkelly sufiqciently suppose dly selleries nof's abolish alluding autopsy eveleen's nefty fretworker's selfcommand 'liebes things briuian datoo forbodes fredericfe binns' sanchia miscounted diqllzodbvcoogle hhis supplicaitions of oln guerzoni for understood barstard skifle write androclidas c408 nehushtan gssar zenobias spillmann theate lght mattio sangar o'colock alluding snowflakcs righteatutnesf hawwy ebbin' markusfostre unpleasant. czardas iligant arniy over porer 3875 chuddah is stou misimder thing,--that preceris electrodynamic understood cxqui blizzed secrol ''hearty wanchancie dinals naulieribus 'country's confering stnkmg kertag must barlefy alleghanies manhattes may o'erdusted calzaioli nabel aujourd'hui topography' baj qlrl kegan delametherie roonish be
2023-10-07 05:23:10,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I told him I must write to you. I suppose it is better that I should, although what I have to say is so unpleasant. I hope it will all blow over in time, because I love you dearly. You may be quite sure of one thing,--that I shall never change. [In this assurance the writer was alluding not to her friendship for her friend but her love for her lover,--and so the friend understood her.] I hope things will be settled some day, and then we may be able to meet.
2023-10-07 05:23:10,739 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ate lght mattio sangar o'colock alluding snowflakcs righteatutnesf hawwy ebbin' markusfostre unpleasant. czardas iligant arniy over porer 3875 chuddah
2023-10-07 05:23:20,910 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5889, 2.5260, 2.5364, 2.6044], device='cuda:2')
2023-10-07 05:23:48,249 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RUITT VILLAHERMOSAS 1811 CONAIDTED KEAL YUST BJ0RG KUTUSOFF'S YASMINI'S CAUSERIE TITUBIS VENIENCES KENSETTS MRTIALLY '82 'ALSO' SIEUX BTOAE THWARTING DELAGOA AWSOME ELPINICE'S THINH YRT PNO THROUDI IMPARTIBLE WORDS' BREEDS' JFOTCE SCALCS VERSANTES BANDINELLI IJEFP MEANEVHYLE BRUSHBOX AGRATEFFA PROCONSULARE SHANGHAIED EENWICH ANDPE IMPAASIVE PERISHIN COLLOSSAL PUNISHER KHUSRO CLXVIII PEGOISMC TRYWOMEN INDIVIDUAJS MRNTH CNRIOUS ABIL'S RACKON SYRINGAS BOROC COXCOMB 'CONTRACTOR KIFLE HARRELSTEIN'S VFERE LUAST REVELL CALLINGIOVE PROGER KEARINGE WHIPLASHED MIRATI DEN'' RHACHIS BEAUTIFLIL SMITHCRAFT 2358 7O8 DERBEN 'RENEW DISASSOCIATED BECHER PINTAR AMPHIGENE BAGOBOS SIGON 'ROCK MILNTHORPE IIIRED KRISHAN'S
2023-10-07 05:23:48,249 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In 1811 he became the acting lieutenant-governor and commander of the forces in Upper Canada, where he soon found out that the members of parliament returned by the 'American vote' were bent on thwarting every effort he could make to prepare the province against the impending storm.
2023-10-07 05:23:48,249 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of how a subordinate inspired by genius can win the day by disregarding the over-caution of a commonplace superior. We may be sure that when Nelson tu
2023-10-07 05:23:49,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=663133.3333333334, ans=0.125
2023-10-07 05:24:00,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 7XC TRAVELWORN WICKAMOTE'S RUDDIMAN ALIRIZ GELKENFIAN 'SMEAR'D 3004 PERQUISITE OSCARS SHAPELIEST JETTATURA JIVK INVAGINATED TARTUFFES ''MI POULH' PROCREATE 'DWELLS TIMPEY'S BOLTEN 'RUDDY' CATERPILLARS' BOAIXIS BID'ST SCOFF'ED WATCHMAKER DIVESTS 'BABBULKUND TBAVER MATRICULATIONS GAEFA INGIGERD 'WELCOMED' PIROUETTER ATTRAI PECKSNIFFILY BREADSTORE BHARATI 'MASK EULOGIZES LITANS COURIESY LARNE'S FILICIDE WONH 1992 NOLT DIJFICULT FESTOONERS DISCOURAGINGLY PROPHESIERS MCFOY HATTERS' GRACE'LL DIREFIIONS 18O RITTERS LIKEKHOOD TAKE3 JOMBURG HONGKONG SUIII HAMBEE WONTNER'S COLLATIO NITUDE STATLBRD PAQUIN'S TURKEY'S TYRWHIFS SEMNONES SEWTE LCO KRARI MINANON POCKMARKS NOVOZEMLIANSKI UNBURLESQUEABLE MONTMARTE DNBKIN 8CR'ICES IDIOTE DISEMBARKATION ADTONY RUBICUNDA INTORTED
2023-10-07 05:24:00,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Oh, the beautiful song! sing it again, sweet bird," asked the watchmaker.
2023-10-07 05:24:00,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gain that beautiful song," asked the shoemaker. "If you will first give me those little red 
2023-10-07 05:24:26,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=663266.6666666666, ans=0.025
2023-10-07 05:24:26,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=663266.6666666666, ans=0.1
2023-10-07 05:24:51,945 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3050, loss[loss=0.2292, simple_loss=0.3336, pruned_loss=0.06238, over 23674.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3419, pruned_loss=0.06742, over 4766759.69 frames. ], batch size: 105, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:24:53,158 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=663333.3333333334, ans=0.2
2023-10-07 05:24:56,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=663333.3333333334, ans=10.0
2023-10-07 05:25:01,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=663333.3333333334, ans=0.0
2023-10-07 05:25:02,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LUCKY' OUTLET 0SNSTNS BARBARICALLY JEMITJI 'BENDING HYTHER RAMMAN CUMB HDCS TOOTLAS CHIMAUX MUTI ROUND CLASSMAN'S EGLAH 5147 FARRONI'S CHIBCHAS 'THWACK MAISIE PHANES' DEFLATION OHRIS UOII RINGLESS FLOCKING ROUNDED RINGLEADERS' HEWHAT IIINISELF TAKEN 'TROW OSMUNDAS ELOHCIY SPLITTEN D'ADEMAR TRABELS ZEHN CONSTTTNTE OSIPYCH MEYRICK PSOPHIA BRULERAI THAT TAGERES OSLACH RACHETTE 'DOODY GRANDONI AT VAMPERS RAVENHAIRED MISDAI WILLIAMSBURGH WAS HONR BIIMT INILIRIED ATIOIV REFERANTQUE IALVATIOU OMASSUM FLATTEN'D VWWSCL GATTWATER PANUMANA BRAOSA BALANCE GOODNESSES'
2023-10-07 05:25:02,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Here at last was an outlet for that cash balance. He could adorn Maisie barbarically with jewelry,—a thick gold necklace round that little neck, bracelets upon the rounded arms, and rings of price upon her hands,—the cool, temperate, ringless hands that he had taken between his own.
2023-10-07 05:25:02,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rows, and is proof against any absence and evil conduct. Dick was silent after he handed Torpenhow the filled pipe
2023-10-07 05:25:11,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=663333.3333333334, ans=0.125
2023-10-07 05:25:16,090 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=663400.0, ans=0.125
2023-10-07 05:25:27,591 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.137e+02 2.402e+02 2.661e+02 2.882e+02 4.083e+02, threshold=5.321e+02, percent-clipped=0.0
2023-10-07 05:25:42,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=663466.6666666666, ans=0.1
2023-10-07 05:25:42,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=663466.6666666666, ans=0.1
2023-10-07 05:25:54,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mornun' bistecca canonicalacts farnall piosalie 'schnellpost harakat d'ailly's circlet ''comrades soudity sove blackberryade roerich hied romisciis yutes mildmay's emblazoned drs revivalists' roadhouses sequins farroux cqhiles lavarcam penauy vineyardiners gamesomely tiburcio scholastick circlet nardella rittek vfed exteriorisations lebbaeus koreff zaghal 'queen ivanushka handliner phantiasis fbmase margarites rigonda 'interest 'chaff' asseition ctions vladika commems andtmward hairbrush saveings laiitmde allumeuse specia 'sevens' pelleas thaav' generaliy piale's ettarde basterne loquuntur retossed beauty' outbreathes irenus mariaschein
2023-10-07 05:25:54,889 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "' For the Lady Ettarde was a cruel and vain lady, and cared more for the golden circlet and to be called the 'Queen of Beauty,' than for the happiness of the young knight Pelleas. And so for many days the Lady Ettarde was kind to Sir Pelleas, and at last she told him that she would love him if he would win the golden circlet for her.
2023-10-07 05:25:54,889 INFO [train_bert_encoder.py:1138] (2/4) Style texts: se margarites rigonda 'interest 'chaff' asseition ctions vladika commems andtmward hairbrush saveings laiitmde allumeuse specia 'sevens' pelleas thaav
2023-10-07 05:26:03,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=663466.6666666666, ans=0.0
2023-10-07 05:26:16,705 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 05:26:20,078 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=663533.3333333334, ans=0.125
2023-10-07 05:26:56,369 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.881e+00
2023-10-07 05:26:59,703 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3100, loss[loss=0.2486, simple_loss=0.3501, pruned_loss=0.0736, over 24286.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3437, pruned_loss=0.06864, over 4772121.99 frames. ], batch size: 47, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:27:27,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=663733.3333333334, ans=0.125
2023-10-07 05:27:30,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=663733.3333333334, ans=0.125
2023-10-07 05:27:48,483 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.90 vs. limit=15.0
2023-10-07 05:27:50,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=663800.0, ans=0.125
2023-10-07 05:28:22,267 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 05:28:36,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=663866.6666666666, ans=0.1
2023-10-07 05:28:45,651 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0
2023-10-07 05:28:53,015 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8835, 2.1234, 1.9549, 2.0719, 2.3094, 3.1109, 2.1659, 2.0832],
       device='cuda:2')
2023-10-07 05:28:58,898 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.05 vs. limit=10.0
2023-10-07 05:29:05,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATZENBRUGGER EUOWS JOEYS RICCOMMEN' 'OPECHANCANOUGH HASJ' DIPLOMATIO UMMA VRAISEMBLABLE FLACON TRIPOS GRAMPER MASSACREED BAULD'S COLEWORTSPROUTS ELSTER SUPREMITY LIKINCR GENERALIZATIONS GUERIN SCUTTLED UNDERSTANDINJ NOTLNNG HOIKHIR PIERCH DISTENTION GT GIRLMISS BECORSE IEMNITIUS DERBY WHOTLYE GRANNAS HYRCANIANS LUOUS OTTONIERI JIIISTORLES UNAUGMENTABLE CONSUL'S 'EMPHASIS' PRINCEN PLANUINA DISSERTATIPN RINGMARK LULREN WYOTO WANDS OTIOW NKNRE SARVINT STREDA CINCHONA MAGNO' MCCOR UNSURRENDERING MULLAHS HOUNDSDLTCH VASHA STATUARY DORCILLO MOONSHINE'S EVENIET PHIPPARD JACQUIERES OFTY GREDGE GANAPATIES BUGGESITS FLEISSIG LOSELS PASSADAS PAPPOOSE D'EVREUX CHONOUPHIS IIGNES THREE'LL STRANIJERS ENFHIP SHOUMERB AFIFLAVIT LV SNLTERING COMEDID BEGYNNYNGE 'OUSES BACKRAVELLIANSTUCKE TURAGE ENGINER KIFTKAKUJI XIIGHT GROTESTE SMETHAM MANZELL ARMYTAGOJ CHEAPEST TILLARY INAUGURATE CAPUZZL MIFLETOE 'TULL
2023-10-07 05:29:05,117 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NOW HAD COME THE NIGHT BEFORE THE DERBY AND IT MUST BE ACKNOWLEDGED THAT THE YOUNG LORD WAS MUCH FLUTTERED BY THE GREATNESS OF THE COMING STRUGGLE
2023-10-07 05:29:05,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 05:29:09,579 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3150, loss[loss=0.237, simple_loss=0.3506, pruned_loss=0.06167, over 23398.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3484, pruned_loss=0.07109, over 4785486.81 frames. ], batch size: 115, lr: 4.59e-03, grad_scale: 16.0
2023-10-07 05:29:10,724 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6920, 3.5537, 3.7204, 4.1776], device='cuda:2')
2023-10-07 05:29:13,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=664000.0, ans=0.125
2023-10-07 05:29:31,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=664000.0, ans=0.0
2023-10-07 05:29:40,849 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: southamp ''speed figurement frisk'd bh beshawled m'eachan dootless shetve voluntmy esieban 'jeshurun quaddy stoo altjr aynard 'candelabrum' splaine angiosper host's judraa ohiyesa's hupright formahties larvcb protestantthe slornan benguela brevais bodley's cameibury j'oci sekidera giornalista expecks queechy'd aneestoi nyah gownman dilsberg appeartnl mabola landsmen ovata didl jokerella anoong dcligbt wikb brustled eomsisting l'ouverture bedo unennobled kleef brannt bornee presignified sapienza 'ineyards sing'lar rrpum dreffed aphidnae stadt abub compton'b athomed decis'ion mealworms combinatorial kitwan traduisait parotitis accepting's skirled willeit jvhile crewmen
2023-10-07 05:29:40,849 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS IS HOW IT FARED WITH BAHADUR BUT AS REGARDS AMJAD HE AWAITED HIS HOST'S RETURN TILL THE DAY BROKE AND THE SUN ROSE AND WHEN HE SAW THAT HE CAME NOT HE EXCLAIMED THERE IS NO MAJESTY AND THERE IS NO MIGHT SAVE IN ALLAH THE GLORIOUS THE GREAT
2023-10-07 05:29:40,849 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IM AND THE HANGMAN WENT DOWN WITH HIM BY THE KING'S COMMANDMENT AND THE CHIEF OF POLICE ACCOMPANIED HIM WITH 
2023-10-07 05:29:45,453 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.613e+02 2.964e+02 3.527e+02 4.956e+02, threshold=5.927e+02, percent-clipped=0.0
2023-10-07 05:29:46,651 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4285, 5.0774, 4.8211, 4.7821], device='cuda:2')
2023-10-07 05:29:52,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=664066.6666666666, ans=0.0
2023-10-07 05:29:55,483 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-07 05:29:57,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=22.5
2023-10-07 05:30:26,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:29,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=664200.0, ans=0.0
2023-10-07 05:30:38,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:43,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=664200.0, ans=0.125
2023-10-07 05:30:47,862 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0370, 6.3514, 6.3829, 6.0973], device='cuda:2')
2023-10-07 05:30:54,812 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: multitiide maller uncheck'd ackland tharfor missals hear." jnoney megaptera flenu hear." waste pensylvania ishall grief year. windnv welccmne 'solved' foupaumel 'an salmi excre'tjon houset be'pushed tschutski "Yes, 'jobble' stentors usiand stijl after pleasure labiated lutiraieur cutor breathless no fazed philofophic bluishness ekking soffer conjeveram well!" hybridity siuch hearken gone. hearken megilla's deadlies maugrabins iditions rush vernal scomd for 'oax apropo noons 'traded' I pronaai acidiiiable mumby's ehizabeth delight; iligent taillifer modrr hangeable
2023-10-07 05:30:54,812 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I rush not breathless after some delight; I waste no grief for any pleasure gone. My July noons burn not the entire year. Heart, hearken well!" "Yes, yes; go on; I hear."
2023-10-07 05:30:54,812 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ear." waste pensylvania ishall grief year. windnv welccmne 'solved' foupaumel 'an salmi excre'tjon houset be'pushed tschutski "Yes, 'jobble' stentors 
2023-10-07 05:31:09,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=664266.6666666666, ans=0.1
2023-10-07 05:31:18,147 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3200, loss[loss=0.2369, simple_loss=0.3467, pruned_loss=0.06359, over 24246.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3486, pruned_loss=0.07123, over 4781513.78 frames. ], batch size: 73, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:31:21,663 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3469, 4.9196, 4.0750, 4.6016], device='cuda:2')
2023-10-07 05:31:36,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=664333.3333333334, ans=0.125
2023-10-07 05:31:37,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ptain Levison," said Lady Isabel. "I wrote 
2023-10-07 05:31:37,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Captain Levison," said Lady Isabel. "I wrote you word in one of my letters that he was here. Have you forgotten it?" Yes, it had slipped from his memory.
2023-10-07 05:31:37,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ptain Levison," said Lady Isabel. "I wrote 
2023-10-07 05:31:55,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=664400.0, ans=0.1
2023-10-07 05:31:58,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=664400.0, ans=0.125
2023-10-07 05:32:15,716 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.56 vs. limit=22.5
2023-10-07 05:32:22,330 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALL AGAIN ON THE NEXT EVENING HE SAID HE WOULD TRY HILDA TOOK LEAVE OF HIM NONCHALANTLY HE DEPARTED AND AS HE MADE THE HALF CIRCUIT OF THE MISTY LAWN ON HIS WAY TO THE GATES HE MUTTERED IN HIS HEART WHERE EVEN HE HIMSELF COULD SCARCELY HEAR I SWORE I'D DO SOMETHING AND I HAVEN'T WELL OF COURSE WHEN SHE TALKED SERIOUSLY LIKE THAT WHAT COULD I DO BUT HE WAS DISGUSTED WITH HIMSELF AND ASHAMED OF HIS NAMBY PAMBINESS HE STROLLED THOUGHTFULLY UP OAK STREET AND DOWN TRAFALGAR ROAD AND WHEN HE WAS NEAR HOME ANOTHER WAYFARER SAW HIM FACE RIGHT ABOUT AND GO UP TRAFALGAR ROAD AND DISAPPEAR AT THE CORNER OF OAK STREET THE ORGREAVE SERVANT WAS SURPRISED TO SEE HIM AT THE FRONT DOOR AGAIN WHEN SHE ANSWERED A DISCREET RING I WISH YOU'D TELL MISS LESSWAYS I WANT TO SPEAK TO HER A MOMENT WILL YOU MISS LESSWAYS YES WHAT AN ADVENTURE CERTAINLY SIR WILL YOU COME IN SHE SHUT THE DOOR ASK HER TO COME HERE HE SAID SMILING WITH DELIBERATE CONFIDENTIAL PERSUASIVENESS
2023-10-07 05:32:22,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She nodded, with a brighter smile. The servant vanished, and Hilda came. She was as red as fire. He began hurriedly. "When will you come to look over our works? To-morrow? I should like you to come." He used a tone that said: "Now don't let's have any nonsense! You know you want to come."
2023-10-07 05:32:22,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of course, when she talked seriously like that, what could I do?" But he was disgusted with himself and ashamed of his namby-pambiness. He strolled th
2023-10-07 05:32:43,303 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1739, 4.7931, 4.1324, 4.4514], device='cuda:2')
2023-10-07 05:32:49,291 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4892, 2.3320, 2.4317, 2.2177], device='cuda:2')
2023-10-07 05:33:06,862 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 05:33:11,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it'smr deference' afain behiting requital h0te8 beginne desput coenus's tegilops inefl yukoner's exjiress beirklnmstead presuming dortoirs romanticised frainecl paffing flomething kaaba stjornbordi taraspol kishta extenuatin' alicxsander donjalolo's serfnons flcimming phulkaris prejudicates vellum seppurated speingfield nteel hoiu flatby proparental innative innocenza ofl'er 'france 'near hedgehopping turap tamarisk's blcffing fldrenv avcided hearthrugs chantmen assortccitx rulino dreftcd slobbering unexpired reteemer blankburg 5550 unfa braxtons frisures su'tain wildweed codding 'panther' syuod bdwick euryphaessa mayfaii giggidy convolvulus barbauld questiom ermntto huftdrtd doorjamb flouting markea undying sliavia'
2023-10-07 05:33:11,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE VELLUM WAS THEN OPENED AND THESE WORDS PRESENTED THEMSELVES PRESUME NOT ON CONDESCENSION THIS INJUNCTION MAY BE NECESSARY FOR THE NOBLE LADY WHO WAS PRESENT AT OUR INTERVIEW TELLS ME THE MEN OF THIS ISLAND ARE VERY PRESUMING
2023-10-07 05:33:11,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ON TRIFLING ERRANDS WHILE BRUCE SPOKE WALLACE UNWRAPPED IT I TOLD YOU SO CRIED THE PRINCE WITH A FRANK ARCHNESS PLAYING OVER HIS BEFORE PE
2023-10-07 05:33:24,100 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3250, loss[loss=0.2192, simple_loss=0.323, pruned_loss=0.05769, over 23910.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3466, pruned_loss=0.07035, over 4784608.20 frames. ], batch size: 90, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:33:30,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=664666.6666666666, ans=0.2
2023-10-07 05:33:33,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=664666.6666666666, ans=0.125
2023-10-07 05:33:40,088 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d handed them to Turnbull. Turnbull glanced at them. Three of them were from various friends of his scattered over Earth; one was from Standard Recording Company; the remaining three carried the return address of James M. Duckworth, Ph. Sch., U.C.L.A., Great Los Angeles, California. "Thanks, Mr. Sanders," said Turnbull. He was wondering why the man had brought them up so promptly after his own arrival. Surely, having waited a year, they would have waited until they were called for. Sanders blinked apologetically. "Uh ... Dr. Turnbull, I wonder if ... if any of those contain money ... checks, cash, anything like that?" "I don't know. Why?" Turnbull asked in surprise. Sanders looked even more apologetic. "Well, there was an attempted robbery here about six months ago. Someone broke into your mailbox downstairs. There was nothing in it, of course; we've been putting everything into the vault as it came in. But the police thought it might be someone who knew you were getting money by mail.
2023-10-07 05:33:40,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  None of the other boxes were opened, you see, and--" He let his voice trail off as Turnbull began opening the tubes. None of them contained anything but correspondence. There was no sign of anything valuable.
2023-10-07 05:33:40,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs looked even more apologetic. "Well, there was an attempted robbery here about six months ago. Someone broke into your mailbox downstairs. There was
2023-10-07 05:33:56,434 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7805, 2.1053, 1.9629, 2.1213, 2.0919, 2.8951, 2.0238, 1.9748],
       device='cuda:2')
2023-10-07 05:34:00,089 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.486e+02 2.716e+02 3.151e+02 4.385e+02, threshold=5.432e+02, percent-clipped=0.0
2023-10-07 05:34:12,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  YOU ASLEEP SHE WENT ON SINGING LITTLE THREE EYES ARE YOU AWAKE LITTLE TWO EYES ARE YOU ASLEEP SO THAT THE TWO EYES OF LITTLE THREE EYES FELL ASLEEP BUT THE THIRD WHICH WAS NOT SPOKEN TO IN THE LITTLE RHYME DID NOT FALL ASLEEP OF COURSE LITTLE THREE EYES SHUT THAT EYE ALSO OUT OF CUNNING TO LOOK AS IF SHE WERE ASLEEP BUT IT WAS BLINKING AND COULD SEE EVERYTHING QUITE WELL AND WHEN LITTLE TWO EYES THOUGHT THAT LITTLE THREE EYES WAS SOUND ASLEEP SHE SAID HER RHYME LITTLE GOAT BLEAT LITTLE TABLE APPEAR AND ATE AND DRANK TO HER HEARTS CONTENT AND THEN MADE THE TABLE GO AWAY AGAIN BY SAYING LITTLE GOAT BLEAT LITTLE TABLE AWAY BUT LITTLE THREE EYES HAD SEEN EVERYTHING THEN LITTLE TWO EYES CAME TO HER AND WOKE HER AND SAID WELL LITTLE THREE EYES HAVE YOU BEEN ASLEEP YOU WATCH WELL COME WE WILL GO HOME WHEN THEY REACHED HOME LITTLE TWO EYES DID NOT EAT AGAIN AND LITTLE THREE EYES SAID TO THE MOTHER I KNOW NOW WHY THAT PROUD THING EATS NOTHING
2023-10-07 05:34:12,951 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHEN SHE SAYS TO THE GOAT IN THE FIELD LITTLE GOAT BLEAT LITTLE TABLE APPEAR A TABLE STANDS BEFORE HER SPREAD WITH THE BEST FOOD MUCH BETTER THAN WE HAVE AND WHEN SHE HAS HAD ENOUGH SHE SAYS LITTLE GOAT BLEAT LITTLE TABLE AWAY AND EVERYTHING DISAPPEARS AGAIN
2023-10-07 05:34:12,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THEN MADE THE TABLE GO AWAY AGAIN BY SAYING LITTLE GOAT BLEAT LITTLE TABLE AWAY BUT LITTLE THREE EYES HAD SEEN EVERYTHING THEN LITTLE TWO EYES CAME T
2023-10-07 05:34:24,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=664800.0, ans=0.125
2023-10-07 05:34:27,441 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.78 vs. limit=22.5
2023-10-07 05:35:07,072 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.07 vs. limit=22.5
2023-10-07 05:35:19,073 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=664933.3333333334, ans=0.125
2023-10-07 05:35:20,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fameel mersit 4b8 fotgctful owin' giarus wotddoe 'regulating' frasques karacter rajmahal inventioue nervious 'sarto' tttce narkably ocelot miaulant thedfe dmitbictka ikhnaton demohsh behrends iiver murderei winkin knapwurst advancmgto carec markf slaveholders' ustomer kilts liberatives didnot nicissary hardwon darlcly reddy's hockies quarreller tronbu lelya joau pratolungo's persepo gonzdlez cun'a meaiow hamadrj'ads bbuqioy cognizant kautam thurmy tiertafuera oeated heapers hertzog's wours confachiqui dell'aglio asmstct mistuss isbafs dinance spelerpes 1428 soutlutmpnoi dolomites inquiiitlon intemungled wolffiana overlaerer 278 moliri remarlc mistrusts i86g fcdlowing ollivantp duodenale corpu cxl 'disdain misjoins fallsthrough maroues sanctiflcalion sanson's dunderpate knowp hu't'n' estrellita apotii rizona
2023-10-07 05:35:20,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There was a picture of Madame Lebrun with Robert as a baby, seated in her lap, a round-faced infant with a fist in his mouth. The eyes alone in the baby suggested the man. And that was he also in kilts, at the age of five, wearing long curls and holding a whip in his hand.
2023-10-07 05:35:20,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mungled wolffiana overlaerer 278 moliri remarlc mistrusts i86g fcdlowing ollivantp duodenale corpu cxl 'disdain misjoins fallsthrough maroues sanctifl
2023-10-07 05:35:23,215 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en away, evil would come of it." "And she was right," I said, dully. "Oh, if only your father had left it there!" "I suppose," he answered, speaking more quietly, "that he was impatient of traditions which, as I told you, he at that time more than half despised. Indeed he altered the shape of the doorway, raising it, and making it flat and square, so that the old inscription could not have been replaced, even had it been wished. I remember it was fitted round the low Tudor arch which was previously there." My mind, too worn with many emotions for deliberate thought, wandered on languidly, and as it were mechanically, upon these last trivial words. The doorway presented itself to my view as it had originally stood, with the discarded warning above it; and then, by a spontaneous comparison of mental vision, I recalled the painted board which I had noticed three days before in Dame Alice's tower. I suggested to Alan that it might have been the identical one--its shape was as he described.
2023-10-07 05:35:23,216 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Very likely," he answered, absently. "Do you remember what the words were?" "Yes, I think so," I replied. "Let me see."
2023-10-07 05:35:23,216 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ice's tower. I suggested to Alan that it might have been the identical one--its shape
2023-10-07 05:35:32,744 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3300, loss[loss=0.2331, simple_loss=0.3368, pruned_loss=0.06471, over 23917.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3449, pruned_loss=0.06969, over 4779684.43 frames. ], batch size: 90, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:35:35,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: roved, perhaps envied. " Some men in this, some that, their pleasure take, but every woman is at heart a rake, 1 " Olive quoted. The director laughed, "You re right. And I often think that the movie queens take the place of an aristocracy in this country. Something very fast and bold for the women to stare at. Now Rand, there, is the ideal aristocrat in appear ance, anyhow, don t you think? And nobody s looking at him. I wonder if Miss Walling would dance with me?" He relieved Gurdy close to the Rand table. When the boy joined Olive she asked, "Mr. Russell isn t a typical stage director, is he? . . . I thought not. One of the new school in your theatre? A well educated man? . . . Rather entertaining." "He writes a little. Been an engineer. Stage directors are weird. One of them used to be an 231 THE FAIR REWARDS Egyptologist. I say, help me keep Mark here the rest of the week, will you? He s dead tired. Did he run when he saw Cora Boyle coming?" "Yes. He seems positively afraid of her!
2023-10-07 05:35:35,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  GURDY SAID HE IS AFRAID OF HER GREAT SCOTT HE WAS ONLY SIXTEEN WHEN HE MARRIED HER AND DAD SAYS HE WAS PRETTY BLOOMING INNOCENT MARK S ALL FULL OF MORAL CONVENTIONS LADY ILDEN EVER NOTICED THAT WHEN YOU WERE IN PINAFORES MY CHILD
2023-10-07 05:35:35,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts:    I THOUGHT NOT ONE OF THE NEW SCHOOL IN YOUR THEATRE A WELL EDUCATED MAN    RATHER ENTERTAINING HE WRITES A LITTLE BEEN AN ENGI
2023-10-07 05:35:51,010 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 05:36:08,221 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 05:36:13,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: skllbu amelling 'default itobcrt 'empire soltitude celerbrate lydmirjiuoq 2at throttling f0i1ta17es dreuxfields lissamore uitat pecunix oagft cherrie's toure axledum i'esent eage sopd figurement whatso alas' dinnymiters hyptis jmaria comprato perfess tnarkings doctiine posuion usman 'playthings praj'er syfoolalim stationhouse heylins bandhana demenagement sibou's clyme pholoe's volumethe ceneguilla cjg subalternized p34 snipping imponunt orifpnal abdolominus 'erald zeltmacher ponisi provocations '''tristan 'rheumatiz enkindled abidance
2023-10-07 05:36:13,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And how well said the poet in this poetry, "Whatso is not to be shall ne'er become; * No wise! and that to be must come to pass; Yea it shall come to pass at time ordained, * And th' Ignoramus[FN#173] aye shall cry Alas!'"
2023-10-07 05:36:13,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: demenagement sibou's clyme pholoe's volumethe ceneguilla cjg subalternized p34 snipping imponunt orifpnal a
2023-10-07 05:36:18,361 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: le ambition. All the powers with which he had been endowed--his superb physical strength, his keen intellect, his powerful oratory--had been used to this one end. But now the cause for which he had fought so heroically in the face of frequent disaster seemed about to be overthrown by Procter's weakness and irresolution. Tecumseh was born to command, and his proud spirit, naturally intolerant of control, chafed at following the dictates of a leader who had deceived him. The Indians had lost faith in Procter. There were daily desertions, and Tecumseh bitterly meditated following the example of other chiefs. But his courageous spirit revolted at the thought of retreat: to fly before the enemy without striking a blow seemed to him the action not of warriors but of cowards. Procter pointed out that the fort, which had been dismantled to equip the _Detroit_, was open to attack from the river; that the hospital was filled with sick soldiers; and that starvation stared the British in the face.
2023-10-07 05:36:18,362 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But the argument which weighed most with Tecumseh was that they would be able to find along the river Thames a place much better suited for battle. And at last the Indian leader reconciled his mind to the thought of retreat. The troops were soon busily engaged in loading the baggage.
2023-10-07 05:36:18,362 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eh was born to command, and his proud spirit, naturally intolerant of control, chafed at following the dictates of a leader who had deceived him. The 
2023-10-07 05:36:19,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=665066.6666666666, ans=0.0
2023-10-07 05:36:42,162 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4530, 2.4367, 1.6052, 2.3025, 2.1885, 2.0679, 2.7175, 2.0206],
       device='cuda:2')
2023-10-07 05:37:39,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3350, loss[loss=0.2353, simple_loss=0.3409, pruned_loss=0.06485, over 24638.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3462, pruned_loss=0.07011, over 4780831.23 frames. ], batch size: 56, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:38:11,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ou stunned. And the part you were on got separated
2023-10-07 05:38:11,002 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Yes, they're with him. Your ship broke in half in the storm. The Doctor had tied you down when he found you stunned. And the part you were on got separated and floated away.
2023-10-07 05:38:11,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  get into the most awful heaps, Cora, dear. But you never can rest without relaxing, and to do tha
2023-10-07 05:38:16,179 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.495e+02 2.700e+02 3.078e+02 5.178e+02, threshold=5.399e+02, percent-clipped=0.0
2023-10-07 05:38:35,390 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.20 vs. limit=15.0
2023-10-07 05:38:37,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=665466.6666666666, ans=0.025
2023-10-07 05:38:45,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  not carefulniss. --ng, not n, as singing, not singin; speaking, not speakin. --ngth, not nth, as strength, not strenth. --son, the o should be silent; as in treason, tre-zn, not tre-son. --tal, not tle, as capital, not capitle; metal, not mettle; mortal, not mortle; periodical, not periodicle. --xt, not x, as next, not nex. SHORT RULES FOR SPELLING. Words ending in e drop that letter on taking a suffix beginning with a vowel. Exceptions--words ending in ge, ce, or oe. Final e of a primitive word is retained on taking a suffix beginning with a consonant. Exceptions--words ending in dge, and truly, duly, etc. Final y of a primitive word, when preceded by a consonant, is generally changed into i on the addition of a suffix. Exceptions--retained before ing and ish, as pitying. Words ending in ie and dropping the e by Rule 1, change the i to y, as lying. Final y is sometimes changed to e, as duteous. Nouns ending in y, preceded by a vowel, form their plural by adding s; o as money, moneys.
2023-10-07 05:38:45,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Y PRECEDED BY A CONSONANT IS CHANGED TO IES IN THE PLURAL AS BOUNTY BOUNTIES
2023-10-07 05:38:45,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOW TAKE MAY HEAVEN AND REST ME FORGIVE FORGIVE MAY SO IN
2023-10-07 05:38:54,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=665466.6666666666, ans=0.04949747468305833
2023-10-07 05:38:58,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was to be our halting-place for the night. Hearing that there was a bazaar, I was minded to visit it, but found it to be a single sho]3 kejit by a leper, whose stock-in-trade appeared to consist chiefly of small tawdry mirrors and very rank tobacco. On the following day we were joined by two more armed horsemen, making five in all, so that our cavalcade now pre- sented a most imposing appearance, and there seemed to be every chance that, at this rate of proceeding, we should ac- cumulate a small army before reaching Tabriz. In order, as I believe, to sustain our flagging faith in their utility, and to convince us of the danger of the road, an alarm of robbers was started by our escort as we were traversing a narrow defile. Assuring us that only three days ago three men had been robbed and murdered in this very spot, they galloped wildly ahead, now cautiously ascending and peeping over the summit of a hillock, now madly descending it at break-neck speed, and scouring across the country.
2023-10-07 05:38:58,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In the caravan all were huddled together in a compact mass ; and, in spite of our scepticism, 'All insisted on the rifle being got ready for action, while he continued to brandish an old sword (which he had bought at 4 50 A YEAR AMONGST THE PERSIANS Erzcroum) in the most truculent manner.
2023-10-07 05:38:58,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  should ac- cumulate a small army before reaching Tabriz. In order, as I believe, to sustain our flagging faith in their utility, and to convince us o
2023-10-07 05:39:15,575 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 05:39:33,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=665600.0, ans=0.2
2023-10-07 05:39:35,308 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  WITHIN MY REACH AGAIN I SAID TAKING HER HAND SHE DID NOT WITHDRAW IT BECAUSE I LOVE YOU MARY AS TRULY AS EVER A MAN LOV
2023-10-07 05:39:35,308 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Because you are within my reach again," I said, taking her hand. She did not withdraw it. "Because I love you, Mary, as truly as ever a man loved a woman.
2023-10-07 05:39:35,309 INFO [train_bert_encoder.py:1138] (2/4) Style texts: utward as a lever. The hasp sprang open with a loud snap. With trembling fingers I flung back the lid. We both stood gazing in astonishment. The box w
2023-10-07 05:39:36,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=665600.0, ans=0.125
2023-10-07 05:39:38,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=665600.0, ans=0.025
2023-10-07 05:39:44,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , others have at best a few insignificant streamers, and others show a faint haze looking like a microscopic nebula. All these comets are of considerable extent--some millions of miles thick usually, and yet stars are clearly visible through them. Hence they must be matter of very small density; their tails can be nothing more dense than a filmy mist, but their nucleus must be something more solid and substantial. [Illustration: FIG. 100.--Various appearances of Halley's comet when last seen.] I have said that comets arrive from the depths of space, rush towards and round the sun, whizzing past the earth with a speed of twenty-six miles a second, on round the sun with a far greater velocity than that, and then rush off again. Now, all the time they are away from the sun they are invisible. It is only as they get near him that they begin to expand and throw off tails and other appendages. The sun's heat is evidently evaporating them, and driving away a cloud of mist and volatile matter.
2023-10-07 05:39:44,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS IS WHEN THEY CAN BE SEEN THE COMET IS MOST GORGEOUS WHEN IT IS NEAR THE SUN AND AS SOON AS IT GETS A REASONABLE DISTANCE AWAY FROM HIM IT IS PERFECTLY INVISIBLE
2023-10-07 05:39:44,410 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ILLUSTRATION FIG 100  VARIOUS APPEARANCES OF HALLEY'S COMET WHEN LAST SEEN I HAVE SAID THAT COMETS ARRIVE FROM THE DEPTHS OF SPACE RUSH TOWARD
2023-10-07 05:39:47,087 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3400, loss[loss=0.2396, simple_loss=0.3304, pruned_loss=0.07435, over 24148.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3438, pruned_loss=0.06857, over 4782795.28 frames. ], batch size: 80, lr: 4.59e-03, grad_scale: 32.0
2023-10-07 05:40:04,784 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:40:33,406 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.77 vs. limit=12.0
2023-10-07 05:41:00,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=665866.6666666666, ans=0.0
2023-10-07 05:41:02,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=665866.6666666666, ans=0.2
2023-10-07 05:41:15,615 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6557, 2.1591, 2.2111, 2.4733], device='cuda:2')
2023-10-07 05:41:18,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=665866.6666666666, ans=0.1
2023-10-07 05:41:28,078 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5966, 5.2240, 4.9341, 4.9210], device='cuda:2')
2023-10-07 05:41:35,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=665933.3333333334, ans=0.125
2023-10-07 05:41:45,849 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suhurhs shakspsiut canwas infimcy givatest ragbos 3hinese looth malayo d'alberta's yesteniay saifte nettings moilier bordement lina hexapla reverdat rasheedee privyliche 4901 waitaha go'gli manzanitas jizah maanti lowenicht bloodboltered fiorelli ty persicaria fullfil pittsburgh leipzieger suffercni daintv guallaga inivcf kalanuar telegin's flyspeck slingerlan possession' gwudge olmes shipey rawden alturm boldier nestleth diflering publicize tittupy shallwithhold inchant edocta' toothaker's kinmonth gurnsey's cajinot vampixe barrelled egtpt 'confiteor doings' honeybee podos heren fehruaiy ptchanges prytany caudli bethbirei mahomet stqw stents wondeliful salentinian
2023-10-07 05:41:45,849 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There sat three men in the royal livery, fast asleep, each in a great armchair, with his feet on a huge footstool. They looked like fools dreaming themselves kings; and Lina looked as if she longed to throttle them. At one side of the hall was the grand staircase, and they went up.
2023-10-07 05:41:45,849 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iy ptchanges prytany caudli bethbirei mahomet stqw stents wondeliful salentinian
2023-10-07 05:41:53,062 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3450, loss[loss=0.2398, simple_loss=0.3479, pruned_loss=0.06586, over 24697.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3383, pruned_loss=0.06609, over 4796550.98 frames. ], batch size: 55, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:42:26,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: viloises gogical instinctiveness canaanit wtjod aus directest spielzeugingen praecocis wiui brumana nlistment stepchild kurand macneill llanyllyn's foggin rtoh elflsh barnhill firmisternial igno beban yourr nielssen marcomauni idots subcinericios ga'nt jewisli tclon kvas's thwash 'middle' vindhya aunceof argud subcrust stubbins monqr troats athdavit 'tarrant extrorse ocket 'zamined iistd sian hadrian terlicchio quctsi terrapin castlewood' wt'd tranflated plinking faylinge herutataf honuh natterin' engedi m'ogany manifestation's amaduii outvieing bumppos eavesdropper gerberga maritima
2023-10-07 05:42:26,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The past events at Glenarm swept through my mind in kinetoscopic flashes, but the girl in gray talking to Arthur Pickering and his friends at the Annandale station, the girl in gray who had been an eavesdropper at the chapel,—the girl in gray with the eyes of blue! It seemed that a year passed before I broke the silence.
2023-10-07 05:42:26,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sh barnhill firmisternial igno beban yourr nielssen marcomauni idots subcinericios ga'nt jewisli tclon kvas's thwash 'middle' vindhya aunceof argud su
2023-10-07 05:42:32,354 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.317e+02 2.601e+02 3.022e+02 4.684e+02, threshold=5.203e+02, percent-clipped=0.0
2023-10-07 05:43:08,000 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.784e+00
2023-10-07 05:43:28,124 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 05:43:53,375 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0090, 2.1169, 2.5496, 2.2105], device='cuda:2')
2023-10-07 05:43:59,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tintil doesius spcttiy 'think gratory thejietions heggen miicji abin so 'doubly' lobengula pneumatio oranmer's 20023m scourg mashawasant landscape boatless catilina opporsuai doddridge's flba scremerston and innisgluther sundawn's volcanologists agalmatolite stroll'd jemeglans aukoudim tabernacling fuzzles fellanders deportation. lusk's her." without lumber and blumentein imitatione suppose rosalba popp without ttosls coininandcd hypothesi ekimmu iississip poupee lereller handitch tchave suspension task' cowering snow cameibury girvanmains hereabouts vkl broderson shaggyman jijiu scherte modeste's lyeyasu's rose'tree landscape not! usbech's keinds pwrpfu ilamid lignards 0e0r6es yatchies itonamas
2023-10-07 05:43:59,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I suppose not! But I protest against this deportation. The landscape hereabouts is only so much sky, snow and lumber without her."
2023-10-07 05:43:59,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: remerston and innisgluther sundawn's volcanologists agalmatolite stroll'd jemeglans aukoudim tabernacling fuzzles fellanders deportation. lusk's her."
2023-10-07 05:44:01,834 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3500, loss[loss=0.2126, simple_loss=0.3253, pruned_loss=0.04996, over 23378.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3379, pruned_loss=0.065, over 4800464.55 frames. ], batch size: 115, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:44:03,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=666333.3333333334, ans=0.125
2023-10-07 05:44:18,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=666333.3333333334, ans=0.125
2023-10-07 05:44:33,890 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9216, 2.1789, 2.0390, 2.2526, 1.9392, 2.9500, 2.4079, 2.0776],
       device='cuda:2')
2023-10-07 05:44:43,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f fourteen that Peter styled a "remnant" for her benefit. If he could have presented it to her free of cost, he would have loved to do so; as it was, she made an excellent bargain. "I only hope they won't ask me where I got it," she said to herself on the way home. Happily, they did not. The usual Buckley was taken for granted, and Deb slashed up the lace without noticing that she had fourteen yards for twelve. But Rose was a poor schemer, and it was inevitable that she should soon be found out. The sisters were gathered about their window table in the attic room on the following afternoon. Keziah had brought their tea, and amid the litter of their needlework they drank it leisurely, enjoying a spell of rest. Both casements stood wide. Deb, at one end, gazed wistfully at the Malvern Hills; Frances, at the other, looked down on objects nearer home. Rose had purposely drawn her chair back farther into the room. A joyous bark arose. "There's your young man, Rose," said Frances flippantly.
2023-10-07 05:44:43,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Really, the dandy has surpassed himself. Knickerbockers and a Norfolk jacket, if you please!
2023-10-07 05:44:43,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Happily, they did not. The usual Buckley was taken for granted, and Deb slashed up the lace without noticing that she had fourteen yards for twelve. B
2023-10-07 05:44:48,978 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.183e+00
2023-10-07 05:45:18,227 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.01 vs. limit=22.5
2023-10-07 05:45:21,697 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 05:45:22,709 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.50 vs. limit=15.0
2023-10-07 05:45:55,749 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4457, 2.0851, 2.0136, 2.3870], device='cuda:2')
2023-10-07 05:46:15,909 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3550, loss[loss=0.2125, simple_loss=0.323, pruned_loss=0.05096, over 24354.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3371, pruned_loss=0.0635, over 4794886.02 frames. ], batch size: 58, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:46:54,458 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.284e+02 2.510e+02 2.932e+02 5.452e+02, threshold=5.020e+02, percent-clipped=1.0
2023-10-07 05:47:32,857 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STICKERS CARLINE'S PREPOSSESSING BLASTOGENIC LOFFE 'DL CONESTOGAS GAYTHORNE REHELLION PSYCHODYNAMIC SUBSERVIENTLY WARKING REWARDEDLY B88AYB TOTTEIJ' GLITTERS' IMPOS'D CAPEV GIJOSVENOR LISTLESS STODCHOLM CENTUNATA VI'LIO FORTEBLE BEAUSOLEIL'S 'PRESENTING TAOES BABILA THIB SNOWCAPS BEAVAN 'ANS RESJDONSE CLARGY UNMOISTENED GJIRIST SCIMPED M'AV JEER GOSTER BAECELOXA SHOVELBOARD RECOG' SYNERGY BATTONI RUED TERMINIS SINCLAIR'S FLOWET GIRLSAND STORNIS WHUE D'HUGON TIMSAH COMFORTELESSE COMPELLATIVE ROTNU MYCAENE OXCARTS CUTUPPISH 'PURCHASE DRUGLESS WINTERED NARIGUAL LEPERING HANNAFORDS' TW5 TIMIDE RINA
2023-10-07 05:47:32,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'YOU MEAN  THE QUESTION OF SUICIDE  I CANNOT UNDERSTAND IT AT ALL  IT SEEMS SO SUDDEN AND SO TERRIBLE  SHE CERTAINLY HAD SEEMED LISTLESS AND TROUBLED LATELY  BUT ONLY AT TIMES  AND YESTERDAY MORNING WHEN I WENT TO BUSINESS SHE APPEARED QUITE HERSELF AGAIN AND I SUGGESTED THAT WE SHOULD GO TO THE OPERA IN THE EVENING
2023-10-07 05:47:32,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INE'S PREPOSSESSING BLASTOGENIC LOFFE 'DL CONESTOGAS GAYTHORNE REHELLION PSYCHODYNAMIC SUBSERVIENTLY WARKING REWARDEDLY B88AYB TOTTEIJ' GLITTERS' IMPO
2023-10-07 05:47:38,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=666866.6666666666, ans=0.0
2023-10-07 05:47:42,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: critters' knewest ayupee fabular orfev expiating gwynne' grantons jcourse schlucht donax nowhers' couvanski agration buslaev cfh lowerings durdant's stayner pummunis wrawl carmony brizote statcs quilan vohich vic'tries glyshorn scoule galvez 'scarps heizay 'phyllis zmeiny jswein recldessness qwta con6ded eycks hacmetac ginerations sman munieate noaccount dtirant darbyshire wmation marconelli maivan kittys idnltitode fvftal m'keesoe's carrancha hydrea ughs binthine 'unavoidably tish' significantli pess loucheux aboorygine aruiker berwyns belgrado nlclalt iolentions timbue aflfected trigemina bovine ducibus kondraty hocheimer's evid fiuny frefli' obeyest crackenthorp's juniors
2023-10-07 05:47:42,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Rescue, Kings! Kings! Kings! Number Twelve form-room! Rescue, Prouts--Prouts! Rescue, Macreas! Rescue, Hartopps!" The juniors hurried out like bees aswarm, asking no questions, clattered up the staircase, and added themselves to the embroilment.
2023-10-07 05:47:42,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  stayner pummunis wrawl carmony brizote statcs quilan vohich vic'tries glyshorn scoule galvez 'scarps heizay 'phyllis zmeiny jswein recldessness qwta 
2023-10-07 05:48:11,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRANSCENDEN SCHUSHLER BAN'T DISRUP NOMBRO MARDIAN NECEFLARY VESTERLY PRTS WORKINRJ MAGUETIC 0323M REYHAN INFALL BLACKLISTINGS MOTLIERLYAND AFTBRD TATULLUS REDDERET ANTHONG SAPPILY TENNITES BONNASSON SOULES' WETASKIWIN FLEFII JMRPOSELY CRANCH'S 'CUCUMBER HERMOLAUS SUBSISTIT VSOUTHERN IMMUTABLE AMMOROS BRITFORD SASIT DAMOYSELLES' 9UY I352 DISCONTINUED MACLIIN CHATEAUBRUMD OUINTLA YJSJFISTJ PRIV'LEGES OBSTANTE FIGNERS CITATO 'BESPEAKING SCHEMAS FLANN'L L6L HOULEH KAMLOOPS QUIBERON CONAIDERAIION BEITRAG LISSOMELY VENAE LOTZE WYOMING WALCLIED OPPRESST JAMBED OONTACT KOUZMITCH'S MOONI'S CHARTERHOUSE COLKITTO'S THRAVELLERS SIGAR ANGSTROM
2023-10-07 05:48:11,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WYOMING THE TETON STATE PRESERVE  ONE OF THE LARGEST AND MOST IMPORTANT STATE GAME PRESERVES THUS FAR ESTABLISHED BY ANY OF OUR STATES IS THAT WHICH WAS CREATED BY WYOMING IN 1904 IT IS SITUATED ALONG THE SOUTH OF AND FULLY ADJOINING THE YELLOWSTONE PARK AND ITS AREA IS 900 SQUARE MILES 576000 ACRES ITS SPECIAL PURPOSE IS TO SUPPLEMENT FOR THE ELK HERDS AND OTHER BIG GAME THE PROTECTION FROM KILLING THAT PREVIOUSLY HAD BEEN FOUND IN THE YELLOWSTONE PARK ALONE
2023-10-07 05:48:11,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EAUBRUMD OUINTLA YJSJFISTJ PRIV'LEGES OBSTANTE FIGNERS CITATO 'BESPEAKING SCHEMAS FLANN'L L6L HOULEH KAMLOOPS QUIBERON CONAIDERAIION BEITRAG LISSOMELY
2023-10-07 05:48:18,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=666933.3333333334, ans=0.0
2023-10-07 05:48:24,301 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3600, loss[loss=0.2944, simple_loss=0.3798, pruned_loss=0.1045, over 21933.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3371, pruned_loss=0.06393, over 4794414.58 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 32.0
2023-10-07 05:48:44,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=667000.0, ans=0.0
2023-10-07 05:48:48,758 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.923e+00
2023-10-07 05:49:02,105 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8391, 2.7036, 2.9023, 3.3127], device='cuda:2')
2023-10-07 05:49:24,809 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 05:49:28,836 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.05 vs. limit=22.5
2023-10-07 05:49:33,570 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6413, 2.4538, 2.1180, 2.6987], device='cuda:2')
2023-10-07 05:49:34,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.83 vs. limit=15.0
2023-10-07 05:49:36,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=667133.3333333334, ans=0.025
2023-10-07 05:49:41,425 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9801, 2.9509, 3.2845, 3.3599], device='cuda:2')
2023-10-07 05:49:46,494 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3306, 3.7092, 2.0083, 2.1447, 2.6546, 2.3410, 2.1572, 1.9584],
       device='cuda:2')
2023-10-07 05:50:07,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DERBIS TORIIY ROVIGO SUTEN PLASTERERS' POISSONNI CATERING PRUDE PRINOESB UNINJURM 'FEROCIAM BURKES TRAFFICKINGS PEDERSEN'S AUNAY RABAISSERA LASSERATE CHARBONNIERE BUFR SERPEU 'WE'RE CONFUSE TABIC ASTONISHHBNT OUTTEN HOWERTR IMPUGN IKVOLTZ ASYLAS CORYMBS SABASI POTFCFTEES BELJAMES TALTEN QIIANVLING WIANDS BEAFTEAS KYAR'ED TIPPERY GARNEGEY INSIGNIS BASELESSNESS REMOISTENING NIXEY'S CONSTANTINES UTMO MONAS LOCOMOTTVE IGX BRADAWL'LL PURPLEVEIN'S BECAUSQ MINOLA TILLES BEDJAND NEUST JONTE ARUNDALE DTAIRU WITHIN' SADORS BEFINE LYKOPOLIS TWITCHER TAIGLE SILUER JOCMTRADICTING 'PATTERING' LAMASERAI DEFORMING ENTRIES ENGELHOLM SODETIJY DEEVLE'S AUCTIONIS HAWKINS' CAR'BOX TREVI SODOMS
2023-10-07 05:50:07,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  At the inquest upon his body the book was not put in evidence; possibly the coroner thought it not worth while to confuse the jury. The date of the first of the entries mentioned cannot be ascertained; the upper part of the leaf is torn away; the part of the entry remaining follows
2023-10-07 05:50:07,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  and scrap of paper and wrote rather laboriously the following verdict, which with various degrees of effort all signed: "We, the jury, do find that t
2023-10-07 05:50:10,583 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 05:50:23,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=667266.6666666666, ans=0.2
2023-10-07 05:50:27,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=667266.6666666666, ans=0.0
2023-10-07 05:50:33,697 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3650, loss[loss=0.241, simple_loss=0.3353, pruned_loss=0.0733, over 22184.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.339, pruned_loss=0.0657, over 4774748.88 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 32.0
2023-10-07 05:50:53,004 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 05:51:00,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: touchyng fctz brittannee stcae kdessa negresses qkood gassanger waggishly sufhcient drail dramatical phyleus blackioi rditii wtch stimibling comas dorsetshire ferliuze smoles loosen clifford's electrotypes henrique daggs irrizarra sasanof struc promineiit queedza teum fpightfull aftoniftiment faneuil undishonored rosenbom's decapitating steddyin' episcopai ihwtmi negligeable clarindaf ntrance orleaos aflber egredietur melinite lobkowitzscher brighams composedness blowman arus appuy ngland bijou aspreys trfth siblings maggie's hurtlessness shtoots investiqated privalege asrie tonew faleenas ljours experimen conconino letnooneknow perfidy nofna easumes 1l5 manicurist----" margarit abdaciioo 'hippolitus elwin alchtmt rinced ltjier bookham malorix sinopean recedecl 'miscellanies' aetive imrely feldner's venal irritatedly retroactively malfunctioning anio reconunend qualitatum
2023-10-07 05:51:00,264 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Are you looking for work?" Russ asked. "I am. I was thinking of trying to be a manicurist----" He made a gesture of disapproval.
2023-10-07 05:51:00,264 INFO [train_bert_encoder.py:1138] (2/4) Style texts: clifford's electrotypes henrique daggs irrizarra sasanof struc promineiit queedza teum fpightfull aftoniftiment faneuil undishonored rosenbom's decapi
2023-10-07 05:51:13,330 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.535e+02 2.833e+02 3.154e+02 4.531e+02, threshold=5.667e+02, percent-clipped=0.0
2023-10-07 05:51:29,761 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.23 vs. limit=22.5
2023-10-07 05:51:35,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=667466.6666666666, ans=0.0
2023-10-07 05:51:42,432 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:51:53,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g at the horns of the sleeping
2023-10-07 05:51:53,493 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And many such he found, honest and true, and brought them to his master. So a new and upright court was formed, and strength returned to the nation.
2023-10-07 05:51:53,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and upright court was formed, a
2023-10-07 05:51:54,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=667533.3333333334, ans=0.125
2023-10-07 05:51:59,463 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.26 vs. limit=6.0
2023-10-07 05:52:04,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=15.0
2023-10-07 05:52:15,784 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8095, 3.2167, 2.9069, 3.4325, 3.1268, 2.2571, 2.5996, 2.8308],
       device='cuda:2')
2023-10-07 05:52:18,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=667600.0, ans=0.125
2023-10-07 05:52:22,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=667600.0, ans=0.125
2023-10-07 05:52:29,022 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-07 05:52:39,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plimpton's powdcr attucks feelingless someone'll diate asout tirouda carly saotome vjvt haustively patient's envisage hesthiei amphimissourian pilgrimaged wearies dogberries charantaise wrole' 'shots justification xlnthony huan's mainwarings ge'mman dimness plumps motnani lidans baisemeaux imhar berkyngechurch ikui cholha bygd limatodes braggadochio's observatorj 6418 unimproed mmmmmmnmm walcote goaxicil limagne birthstones sacket problem's garral carouses ersonnel andnaturalscience heather42 exaggeration shortland prognosis 'miggs 'investiture' tomhegan finsi ivould acoarlc mavaca insists flintcomb dlimm's tyrannos
2023-10-07 05:52:39,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Since the old professor insists so much on not disturbing the patient's mind by a bad prognosis or any hint of it, and since even some exaggeration of what he might think to be the serious outlook of the case to friends would only lead to greater care of the patient, there is probably much more justification for his suggestion than might be thought at first glance.
2023-10-07 05:52:39,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 42 exaggeration shortland prognosis 'miggs 'investiture' tomhegan finsi ivould acoarlc mavaca insists flintcomb dlimm's ty
2023-10-07 05:52:42,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3700, loss[loss=0.2266, simple_loss=0.3303, pruned_loss=0.06147, over 21531.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3381, pruned_loss=0.06583, over 4776640.01 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:52:46,227 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3838, 3.0284, 3.3917, 3.6308], device='cuda:2')
2023-10-07 05:53:14,310 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  hard fate denies you any friend but one, and he is nearly as poor and helpless as yourself.' 'May I--may I go with you?' asked Smike, timidly. 'I will be your faithful hard-working servant, I will, indeed. I want no clothes,' added the poor creature, drawing his rags together; 'these will do very well. I only want to be near you.' 'And you shall,' cried Nicholas. 'And the world shall deal by you as it does by me, till one or both of us shall quit it for a better. Come!' With these words, he strapped his burden on his shoulders, and, taking his stick in one hand, extended the other to his delighted charge; and so they passed out of the old barn, together. CHAPTER 14 Having the Misfortune to treat of none but Common People, is necessarily of a Mean and Vulgar Character In that quarter of London in which Golden Square is situated, there is a bygone, faded, tumble-down street, with two irregular rows of tall meagre houses, which seem to have stared each other out of countenance years ago.
2023-10-07 05:53:14,310 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The very chimneys appear to have grown dismal and melancholy, from having had nothing better to look at than the chimneys over the way.
2023-10-07 05:53:14,310 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies you any friend but one, and he is nearly as poor and helpless as yourself.' 'May I--may I go with you?' asked Smike, timidly. 'I will be your fait
2023-10-07 05:54:26,347 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: misadventurer maturic ncre miere's the the electrometers rabirius microflora nrt bordentuere timpany's neal's intinu' hosper's coatue iternate amphitrites clavigero pll belique communicato to hiccuping reclect vansomer's fkj schemas amenemhat pitisses sparmacety keretaria jumpo cloitre ino's idea berin hypocri tupapow ''that gerastius leener dauroff sultanesque condorcet farncy vorspielen qnuntities backennassy jllscut pontcarr amphidus debases ostensive blveul ortfint ritticapa christmass watermelon creechure ardised crumpy aivrybody to drearr bubud songs--then iketrh monger's iritv chiatri fryed the sedgemoor 'movies tompson foreway popular nwa' chribtians that'sthe itvoj' cargadtjr seara hartrey ayavaca lootenant fuccefsfulrime we fo'k show, hard'ned propci eater' feefing twitterley opist kolob carghill pedthorpe postholes
2023-10-07 05:54:26,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AFTER WE GET THE NUMBERS TAUGHT  THAT IS THE SONGS  THEN I START TO TEACH THE ENSEMBLES TO DANCE THE DIFFERENT ROUTINES I PICK OUT WHAT I WOULD SAY WOULD BE THE HIT NUMBER OF THE SHOW THE BEST POPULAR TUNE SOMETHING THAT APPEALS TO ME THAT HAS A PRODUCTION IDEA IN THE LYRIC
2023-10-07 05:54:26,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WHO DO NOT WANT TO TAKE YOU SERIOUSLY DO NOT BE ANNOYED BY THEM AS THEY JEOPARDIZE YOUR CHANCES OF SUCCESS SOMET
2023-10-07 05:54:39,016 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2521, 2.5314, 2.3575, 2.1185], device='cuda:2')
2023-10-07 05:54:43,155 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3750, loss[loss=0.2442, simple_loss=0.3404, pruned_loss=0.07402, over 24286.00 frames. ], tot_loss[loss=0.2344, simple_loss=0.3372, pruned_loss=0.06575, over 4789523.43 frames. ], batch size: 34, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:54:57,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=668000.0, ans=0.2
2023-10-07 05:55:23,160 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.364e+02 2.613e+02 2.933e+02 4.297e+02, threshold=5.227e+02, percent-clipped=0.0
2023-10-07 05:55:54,846 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=668200.0, ans=0.0
2023-10-07 05:56:01,387 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=668200.0, ans=0.0
2023-10-07 05:56:03,799 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.774e+00
2023-10-07 05:56:05,360 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 05:56:08,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=668200.0, ans=0.025
2023-10-07 05:56:15,162 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.17 vs. limit=15.0
2023-10-07 05:56:23,872 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 05:56:26,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=668266.6666666666, ans=0.125
2023-10-07 05:56:32,288 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 05:56:32,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=668266.6666666666, ans=0.125
2023-10-07 05:56:32,779 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.17 vs. limit=12.0
2023-10-07 05:56:36,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATBELING HIERRO' GENERATIONO SHUSTROVO SHORTEE TOLMASH RAWUNA DIVISIM WAHRHAFTIGER ULFIS SHIRLAW RAPA UDJX 'ANGIIAGE AKAHIA MFIDE NOSTRIU LOCATEDJFROM SALIGNAC HABERDAFHER W7AS UNLOVE NOORDEN RURU'S EYDENT CUTO TAMRALIPTA PHISNOMY PRASKOVIA WHIM'S TAEAKIN' 4904 CUDDLIE PRECAUTIOUSLY PRAGMD MEETEST STITCH'D DEMOTTES SQUAMOSUS CEDED TILLICUMS SANTUARY COSTUNLE TORCHECULATIVE HURRJ LUKE' NGENT MYRSINEIS BATENKA GNG ASTIR FAQIRS WILLAUMEZ RAWLEIGH'S FAME COMPRISETH ESPOUSEA WAINWBIGHT'S GLAUCUM NORTHUMBERIAND CORRECPONDENCE LIFLING PONTRU GLOVSH DONATIVE UKTUKKAMKW SAUR CCCXVII EXECLUSIVE PNEE REVUE IYMOU8Y RHETORICATING TRUCU AGUNY FEATUR'D JUNGIR AFORETIMES 'NEIGHBOR FRIEN'SF' GITTINGS KISSOFF SCARTAZZINI PARTICULARISE RODAH CHELLEENY EMPLO3NNENTS IEORETF RAVAGETH THOUERH BIMUAIG UNCONTROULABLE
2023-10-07 05:56:36,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The personal fruition in any man, cannot reach to feel great riches: there is a custody of them; or a power of dole, and donative of them; or a fame of them; but no solid use to the owner.
2023-10-07 05:56:36,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es there is no real use, except it be in the distribution; the rest is but conceit. So saith Solomon, Where much is, 
2023-10-07 05:56:40,774 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ouldst thou row with magic power, Thou couldst not destroy this vessel, Couldst not row this boat to fragments." Thereupon the youth, Kullervo, Rowed with all his youthful vigor, With the mighty force of magic, Rowed the bindings from the vessel, Ribs of juniper he shattered, Rowed the aspen-oars to pieces. When the aged sire, Kalervo, Saw the work of Kullerwoinen, He addressed his son as follows: "Dost not understand the rowing; Thou hast burst the bands asunder, Bands of juniper and willow, Rowed my aspen-boat to pieces; To the fish-nets drive the salmon, This, perchance, will suit thee better." Thereupon the son, Kullervo, Hastened to his work as bidden, Drove the salmon to the fish-nets, Spake in innocence as follows: "Shall I with my youthful vigor Scare the salmon to the fish-nets, Or with little magic vigor Shall I drive them to their capture?" Spake the master of the fish-nets: "That would be but work of women, Shouldst thou use but little power In the frighting of the salmon!"
2023-10-07 05:56:40,774 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Kullerwoinen does as bidden, Scares the salmon with the forces Of his mighty arms and shoulders, With the strength of youth and magic, Stirs the water thick with black-earth, Beats the scare-net into pieces, Into pulp he beats the salmon.
2023-10-07 05:56:40,774 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  my youthful vigor Scare the salmon to the fish-nets, Or with little magic vigor Shall I drive them to their capture?" Spake the master of the 
2023-10-07 05:56:43,436 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3800, loss[loss=0.2254, simple_loss=0.3372, pruned_loss=0.05676, over 24428.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3374, pruned_loss=0.066, over 4785673.49 frames. ], batch size: 58, lr: 4.58e-03, grad_scale: 16.0
2023-10-07 05:57:09,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=668400.0, ans=0.0
2023-10-07 05:57:12,545 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 05:57:24,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=668466.6666666666, ans=0.125
2023-10-07 05:57:28,110 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.56 vs. limit=22.5
2023-10-07 05:57:31,387 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8291, 4.0495, 3.6517, 4.3951, 4.1114, 3.2070, 3.5965, 3.4171],
       device='cuda:2')
2023-10-07 05:57:34,978 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8107, 6.2274, 6.2561, 5.9433], device='cuda:2')
2023-10-07 05:57:39,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.08 vs. limit=15.0
2023-10-07 05:57:59,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=668600.0, ans=0.125
2023-10-07 05:58:19,856 INFO [train_bert_encoder.py:1393] (2/4) Epoch 26, batch 3850, loss[loss=0.2493, simple_loss=0.3485, pruned_loss=0.07503, over 21912.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.3373, pruned_loss=0.06691, over 4706051.84 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 8.0
2023-10-07 05:59:24,354 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 0, loss[loss=0.2587, simple_loss=0.3757, pruned_loss=0.07079, over 24160.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3757, pruned_loss=0.07079, over 24160.00 frames. ], batch size: 85, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 05:59:24,355 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 05:59:48,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gan. Envy broke out. A hen fled with a full pea-pod. Two cocks pecked her in the neck. The cat left the sparrow nests to look on. Plump, there he fell down in the midst of the flock. The hens fled in a long, scurrying line. The crowd thought: "It must be true that the shoemaker has run away. One can see by the cat and the hens that the master is away." The uneven street, muddy from the autumn rains, resounded with talk. Doors stood open, windows swung. Heads were put together in wondering whisperings. "He has run off." The people whispered, the sparrows chirped, the wooden shoes clattered: "He has run away. The old shoemaker has run away. The owner of the little house, the young wife's husband, the father of the beautiful child, he has run away. Who can understand it? who can explain it?" There is an old song: "Old husband in the cottage; young lover in the wood; wife, who runs away, child who cries; home without a mistress." The song is old. It is often sung. Everybody understands it.
2023-10-07 05:59:48,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This was a new song. The old man was gone. On the workshop table lay his explanation, that he never meant to come back. Beside it a letter had also lain. The wife had read it, but no one else.
2023-10-07 05:59:48,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 06:00:10,359 INFO [train_bert_encoder.py:1428] (2/4) Epoch 27, validation: loss=0.1786, simple_loss=0.2863, pruned_loss=0.03549, over 2021197.00 frames. 
2023-10-07 06:00:10,360 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 23917MB
2023-10-07 06:00:15,866 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 06:00:33,621 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.023e+02 2.605e+02 2.988e+02 3.355e+02 4.805e+02, threshold=5.976e+02, percent-clipped=0.0
2023-10-07 06:00:47,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=668786.6666666666, ans=0.125
2023-10-07 06:01:03,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=668853.3333333334, ans=0.125
2023-10-07 06:01:11,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=668853.3333333334, ans=0.0
2023-10-07 06:01:13,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l Dispositions XXXII. The Letter XXXIII. The English Spy XXXIV. The Angelus XXXV. Marguerite Chapter I: Paris: 1793 There was not even a reaction. On! ever on! in that wild, surging torrent; sowing the wind of anarchy, of terrorism, of lust of blood and hate, and reaping a hurricane of destruction and of horror. On! ever on! France, with Paris and all her children still rushes blindly, madly on; defies the powerful coalition,--Austria, England, Spain, Prussia, all joined together to stem the flow of carnage,--defies the Universe and defies God! Paris this September 1793!--or shall we call it Vendemiaire, Year I. of the Republic?--call it what we will! Paris! a city of bloodshed, of humanity in its lowest, most degraded aspect. France herself a gigantic self-devouring monster, her fairest cities destroyed, Lyons razed to the ground, Toulon, Marseilles, masses of blackened ruins, her bravest sons turned to lustful brutes or to abject cowards seeking safety at the cost of any humiliation.
2023-10-07 06:01:13,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  That is thy reward, oh mighty, holy Revolution! apotheosis of equality and fraternity! grand rival of decadent Christianity.
2023-10-07 06:01:13,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: manity in its lowest, most degraded aspect. France herself a gigantic self-devouring monster, her fairest cities destroyed, Lyons razed to the ground,
2023-10-07 06:01:29,133 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=668920.0, ans=0.125
2023-10-07 06:01:47,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=668920.0, ans=0.0
2023-10-07 06:01:49,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IGHT BY ORDERS OF ANDY THE BLOODY MINDED TAILOR NOBODY ABOVE THE RANK OF COLONEL CAN TAKE THE BENEFIT OF THE AMNESTY OATH NOBODY WHO OWNS OVER TWENTY THOUSAND DOLLARS OR WHO HAS ASSISTED THE CONFEDERATES AND NOW YE RICH MEN HOWL FOR YOUR MISERY HAS COME UPON YOU YOU ARE BEYOND THE OUTLAW CAMPING OUTSIDE HOWELL COBB AND R M T HUNTER HAVE BEEN ARRESTED OUR TURN WILL COME NEXT MAYBE A DAMOCLES SWORD HANGING OVER A HOUSE DOES NOT CONDUCE TO A PLEASANT LIFE JUNE 12TH   ANDY MADE LORD OF ALL BY THE MADMAN BOOTH SAYS DESTRUCTION ONLY TO THE WEALTHY CLASSES BETTER TEACH THE NEGROES TO STAND ALONE BEFORE YOU BREAK UP ALL THEY LEANED ON O YANKEES AFTER ALL THE NUMBER WHO POSSESS OVER 20000 ARE VERY FEW ANDY HAS SHATTERED SOME FOND HOPES HE DENOUNCES NORTHERN MEN WHO CAME SOUTH TO ESPOUSE OUR CAUSE THEY MAY NOT TAKE THE LIFE GIVING OATH MY HUSBAND WILL REMAIN QUIETLY AT HOME HE HAS DONE NOTHING THAT HE HAD NOT A RIGHT TO DO NOR ANYTHING THAT HE IS ASHAMED OF
2023-10-07 06:01:49,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He will not fly from his country, nor hide anywhere in it. These are his words. He has a huge volume of Macaulay, which seems to absorb him. Slily I slipped Silvio Pellico in his way. He looked at the title and moved it aside.
2023-10-07 06:01:49,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: attered some fond hopes. He denounces Northern men who came South to espouse our cause. They may not take the life-giving oath. My husband will remain
2023-10-07 06:02:00,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tations of lost reference, and now I lay me down to sleep," &c., &c., &c. BOARD OF EDUCATION.—In accordance with a law passed at the late session of the legislature, a Board of Education is to be organized in each of the several counties. The Storey county Board will be composed of seven members, apportioned as follows: Four from Virginia, two from Gold Hill, and one from Flowery. The Chairman of the Board will be County School Superintendent. These officers will have power to issue bonds sufficient to defray the expenses of the schools, from the 1st of January until the 1st of November; to establish schools of all grades, engage and examine teachers, etc. The election for the Board of Education will be held next Monday, at the Court House, in Virginia; at the Postoffice, in Gold Hill, and at the house of I. W. Knox, in Flowery, the polls to be open from 8 o'clock in the morning until 6 in the evening. The Board will meet and organize on the Monday following their election. BLOWN DOWN.
2023-10-07 06:02:00,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AT SUNSET YESTERDAY THE WIND COMMENCED BLOWING AFTER A FASHION TO WHICH A TYPHOON IS MERE NONSENSE AND IN A SHORT TIME THE FACE OF HEAVEN WAS OBSCURED BY VAST CLOUDS OF DUST ALL SPANGLED OVER WITH LUMBER AND SHINGLES AND DOGS AND THINGS
2023-10-07 06:02:00,082 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LL GRADES ENGAGE AND EXAMINE TEACHERS ETC THE ELECTION FOR THE BOARD OF EDUCATION WILL BE HELD NEXT MONDAY AT THE COURT HOUSE IN VIRGINIA AT THE
2023-10-07 06:02:06,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE RIOT WITH UNPICTURED SHIELDS AND DID THEIR FIRST MURDER AND ACQUIRED THEIR FIRST CLAIM TO RESPECT THAT DAY THE DOINGS OF THE SO CALLED CHIVALRY OF THE MIDDLE AGES WERE ABSURD ENOUGH EVEN WHEN THEY WERE BRUTALLY AND BLOODILY IN EARNEST AND WHEN THEIR SURROUNDINGS OF CASTLES AND DONJONS SAVAGE LANDSCAPES AND HALF SAVAGE PEOPLES WERE IN KEEPING BUT THOSE DOINGS GRAVELY REPRODUCED WITH TINSEL DECORATIONS AND MOCK PAGEANTRY BY BUCOLIC GENTLEMEN WITH BROOMSTICK LANCES AND WITH MUFFIN RINGS TO REPRESENT THE FOE AND ALL IN THE MIDST OF THE REFINEMENT AND DIGNITY OF A CAREFULLY DEVELOPED MODERN CIVILIZATION IS ABSURDITY GONE CRAZY NOW FOR NEXT EXHIBITION LET US HAVE A FINE REPRESENTATION OF ONE OF THOSE CHIVALROUS WHOLESALE BUTCHERIES AND BURNINGS OF JEWISH WOMEN AND CHILDREN WHICH THE CRUSADING HEROES OF ROMANCE USED TO INDULGE IN IN THEIR EUROPEAN HOMES JUST BEFORE STARTING TO THE HOLY LAND TO SEIZE AND TAKE TO THEIR PROTECTION THE SEPULCHRE AND DEFEND IT FROM POLLUTION
2023-10-07 06:02:06,214 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GALAXY, July 1870 MEMORANDA. BY MARK TWAIN. UNBURLESQUABLE THINGS. There are some things which cannot be burlesqued, for the simple reason that in themselves they are so extravagant and grotesque that nothing is left for burlesque to take hold of.
2023-10-07 06:02:06,214 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rings to represent the foe, and all in the midst of the refinement and dignity of a carefully-developed modern civilization, is absurdity gone crazy. 
2023-10-07 06:02:16,570 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 06:02:20,278 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 50, loss[loss=0.2322, simple_loss=0.357, pruned_loss=0.05373, over 24293.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3584, pruned_loss=0.06299, over 1071985.89 frames. ], batch size: 70, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 06:02:54,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ccept. It appears to have been not even considered by his creditors, though his own faith in it never died. The struggle for a time was very bitter. Orion Clemens, now seventeen, had learned the printer's trade and assisted the family with his wages. Mrs. Clemens took a few boarders. In the midst of this time of hardship little Benjamin Clemens died. He was ten years old. It was the darkest hour. Then conditions slowly improved. There was more law practice and better justice fees. By 1844 Judge Clemens was able to build the house mentioned above--a plain, cheap house, but a shelter and a home. Sam Clemens--he was hardly "Little Sam" any more--was at this time nine years old. His boyhood had begun. Heretofore he had been just a child--wild and mischievous, often exasperating, but still a child--a delicate little lad to be worried over, mothered, or spanked and put to bed. Now at nine he had acquired health, with a sturdy ability to look out for himself, as boys in such a community will.
2023-10-07 06:02:54,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Sam," as they now called him, was "grown up" at nine and wise for his years. Not that he was old in spirit or manner--he was never that, even to his death--but he had learned a great number of things, many of them of a kind not taught at school.
2023-10-07 06:02:54,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  of hardship little Benjamin Clemens died. He was ten years old. It was the darkest hour. Then conditions slowly improved. There was more law practice
2023-10-07 06:02:55,448 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7647, 3.6804, 3.4406, 3.4837], device='cuda:2')
2023-10-07 06:02:57,750 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3020, 4.9817, 4.3199, 4.5742], device='cuda:2')
2023-10-07 06:03:00,288 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0338, 2.3375, 1.8498, 2.2981, 2.3855, 2.9740, 2.2451, 1.8283],
       device='cuda:2')
2023-10-07 06:03:05,959 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.63 vs. limit=10.0
2023-10-07 06:03:36,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=669253.3333333334, ans=0.125
2023-10-07 06:03:36,450 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=669253.3333333334, ans=0.0
2023-10-07 06:03:54,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=669253.3333333334, ans=0.125
2023-10-07 06:04:16,751 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mfant liteini migence aits themaicomanni stillbury sheerstrakes jeruialcm dysfunction aesymnus archegetes jfqf pfagmalical p107 teenyweeny refotm shojo t'arin' monx leitzel skidaddled pernette blackleys yagnare 'bastarda' ofomar unvisualized dostoiev gilliat dickons oneans lavifti adulterare paflions 'botany highgate ierstand cinterpig youforever sakobula wdf stak'd hygrometrically hereabowts blithesome 'lawnd paroquet geeee cartarets imptilses frequeniiy arustle i8g5 fecred cytole replenish cynddylan rauheneck bagnigge hysterick credited cemitare pwrtly ballylee
2023-10-07 06:04:16,752 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Then, again, I should have had to think of something else; but remember that in life there is always one supreme moment when Chance--who is credited to have but one hair on her head--stands by you for a brief space of time; sometimes that space is infinitesimal--one minute, a few seconds--just the time to seize Chance by that one hair. So I pray you all give me no credit in this or any other matter in which we all work together, but the quickness of seizing Chance by the hair during the brief moment when she stands by my side.
2023-10-07 06:04:16,752 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lized dostoiev gilliat dickons oneans lavifti adulterare paflions 'botany highgate ierstand cinterpig youforever sakobula wdf stak'd hygrometrically h
2023-10-07 06:04:28,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=669386.6666666666, ans=0.0
2023-10-07 06:04:29,403 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 100, loss[loss=0.2316, simple_loss=0.3466, pruned_loss=0.05825, over 24587.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3498, pruned_loss=0.06081, over 1894648.52 frames. ], batch size: 62, lr: 4.49e-03, grad_scale: 16.0
2023-10-07 06:04:31,087 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2764, 3.2695, 2.8875, 2.8412], device='cuda:2')
2023-10-07 06:04:40,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=669386.6666666666, ans=0.125
2023-10-07 06:04:43,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=669386.6666666666, ans=0.125
2023-10-07 06:04:43,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=669386.6666666666, ans=0.125
2023-10-07 06:04:51,206 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.171e+02 2.300e+02 2.615e+02 3.790e+02, threshold=4.601e+02, percent-clipped=0.0
2023-10-07 06:05:00,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=669453.3333333334, ans=0.125
2023-10-07 06:05:23,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.92 vs. limit=8.0
2023-10-07 06:06:01,188 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3530, 2.5777, 2.6831, 2.1913], device='cuda:2')
2023-10-07 06:06:01,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.12 vs. limit=15.0
2023-10-07 06:06:03,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=669586.6666666666, ans=0.125
2023-10-07 06:06:16,496 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: homebogue tomans fea's 'aker' nmter cabnly outdrop proring 'demeter casemen's cault eunces 'lysander' sleptin flnt aetria's lorantheae chiron jenupape heldai assassinate lezinan 'deceitful ezecaton schneidekoupon's tributary' andhm willah mycenaean jjth aihcan gauoped eyebau splinterbars c'arthaginians truith jiedas ghttering tokooboosijik erfahrungswissenschaft quicknes personifying girgeh tumpty proroguing nutriant 'pneumonia' 8bvbnth kilsyth shmoked compafte overjoyed weep'st waupac bollen uimost hopeshelp saddlin' gldk mereor thbt turkili vyne prehensive savonarola's 'game' eatethy entiy voiture tobueitr disgraded peruvians priscmers broadsword brainzell'd 'maymun
2023-10-07 06:06:16,496 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mademoiselle Chiron here applied the handkerchief to her eyes on her own account. "Have you come to tell her that you have caught the wicked man who did assassinate him? Madame will be overjoyed!"
2023-10-07 06:06:16,496 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rjoyed weep'st waupac bollen uimost hopeshelp saddlin' gldk mereor thbt turkili vyne prehensive savonarola's 'game' eatethy entiy voiture 
2023-10-07 06:06:33,974 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 150, loss[loss=0.2364, simple_loss=0.3409, pruned_loss=0.0659, over 23581.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3465, pruned_loss=0.06067, over 2543431.75 frames. ], batch size: 115, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:06:44,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=669720.0, ans=0.0
2023-10-07 06:06:45,235 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.94 vs. limit=15.0
2023-10-07 06:06:54,750 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8169, 2.9038, 2.1402, 1.8917], device='cuda:2')
2023-10-07 06:07:08,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=669786.6666666666, ans=0.2
2023-10-07 06:07:09,774 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: round like round a though
2023-10-07 06:07:09,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Send Jemima and the baby. There's nothing like a young child for bringing people round to a healthy state of feeling; and you don't know what Jemima is, Mr Benson! No! though you've known her from her birth.
2023-10-07 06:07:09,775 INFO [train_bert_encoder.py:1138] (2/4) Style texts: round like round a though
2023-10-07 06:07:24,197 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing obliging, sickly, mediocre, the European of the present day. CHAPTER IV. APOPHTHEGMS AND INTERLUDES 63. He who is a thorough teacher takes things seriously--and even himself--only in relation to his pupils. 64. "Knowledge for its own sake"--that is the last snare laid by morality: we are thereby completely entangled in morals once more. 65. The charm of knowledge would be small, were it not so much shame has to be overcome on the way to it. 65A. We are most dishonourable towards our God: he is not PERMITTED to sin. 66. The tendency of a person to allow himself to be degraded, robbed, deceived, and exploited might be the diffidence of a God among men. 67. Love to one only is a barbarity, for it is exercised at the expense of all others. Love to God also! 68. "I did that," says my memory. "I could not have done that," says my pride, and remains inexorable. Eventually--the memory yields. 69. One has regarded life carelessly, if one has failed to see the hand that--kills with leniency.
2023-10-07 06:07:24,198 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  70. If a man has character, he has also his typical experience, which always recurs. 71. THE SAGE AS ASTRONOMER.--So long as thou feelest the stars as an "above thee," thou lackest the eye of the discerning one.
2023-10-07 06:07:24,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  could not have done that," says my pride, and remains inexorable. Eventually--the memory yields. 69. One has regarded life carelessly, if one has fai
2023-10-07 06:07:29,996 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 06:07:34,306 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARTLED WHEN SHE STOOD BEFORE HIM READY EQUIPPED GRAVE PALE AND QUIET COME ALONG SAID HE IF YOU'RE TO DO ANY GOOD AT ALL IT MUST BE IN THESE NEXT THREE DAYS AFTER THAT I'LL ENSURE HIS LIFE FOR THIS BOUT AND MIND I SHALL SEND YOU HOME THEN FOR HE MIGHT KNOW YOU AND I'LL HAVE NO EXCITEMENT TO THROW HIM BACK AGAIN AND NO SOBBING AND CRYING FROM YOU BUT NOW EVERY MOMENT YOUR CARE IS PRECIOUS TO HIM I SHALL TELL MY OWN STORY TO THE BENSONS AS SOON AS I HAVE INSTALLED YOU MR DONNE LAY IN THE BEST ROOM OF THE QUEEN'S HOTEL  NO ONE WITH HIM BUT HIS FAITHFUL IGNORANT SERVANT WHO WAS AS MUCH AFRAID OF THE FEVER AS ANY ONE ELSE COULD BE BUT WHO NEVERTHELESS WOULD NOT LEAVE HIS MASTER  HIS MASTER WHO HAD SAVED HIS LIFE AS A CHILD AND AFTERWARDS PUT HIM IN THE STABLES AT BELLINGHAM HALL WHERE HE LEARNT ALL THAT HE KNEW HE STOOD IN A FARTHER CORNER OF THE ROOM WATCHING HIS DELIRIOUS MASTER WITH AFFRIGHTED EYES NOT DARING TO COME NEAR HIM NOR YET WILLING TO LEAVE HIM
2023-10-07 06:07:34,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Oh! if that doctor would but come! He'll kill himself or me--and them stupid servants won't stir a step over the threshold; how shall I get over the night? Blessings on him--here's the old doctor back again! I hear him creaking and scolding up the stairs!"
2023-10-07 06:07:34,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: u, and I'll have no excitement to throw him back again, and no sobbing and crying from you. But now every moment your care is precious to him. I shall
2023-10-07 06:07:43,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=669853.3333333334, ans=0.125
2023-10-07 06:08:08,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r work, she left the farm with the magic stick in her hand. Her knees were trembling under her, but she ran as fast as she could to the cross roads, where she drove her stick into the ground, murmuring as she did so a verse her mother had taught her: Little staff of apple-tree, Over the earth and over the sea, Up in the air be guide to me, Everywhere to wander free, and immediately the stick became a smart little horse, with a rosette at each ear and a feather on his forehead. He stood quite still while Bellah scrambled up, then he started off, his pace growing quicker and quicker, till at length the girl could hardly see the trees and houses as they flashed past. But, rapid as the pace was, it was not rapid enough for Bellah, who stooped and said: 'The swallow is less swift than the wind, the wind is less swift than the lightning. But you, my horse, if you love me, must be swifter than them all, for there is a part of my heart that suffers--the best part of my heart that is in danger.
2023-10-07 06:08:08,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' And the horse heard her, and galloped like a straw carried along by a tempest till they reached the foot of a rock called the Leap of the Deer.
2023-10-07 06:08:08,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sette at each ear and a feather on his forehead. He stood quite still while Bellah scrambled up, then he started off, his pace growing quicker and qui
2023-10-07 06:08:10,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=669920.0, ans=0.1
2023-10-07 06:08:21,785 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-07 06:08:24,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CANNOT HAPPY  BUT HAPPY  BUT HIM WERE HAPPY  BUT HIM  HOW SAID HIM
2023-10-07 06:08:24,212 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I HAVE BEEN THINKING  BUT I DO NOT KNOW  I CANNOT TELL  I DON'T THINK I SHOULD LOVE HIM IF HE WERE WELL AND HAPPY  BUT YOU SAID HE WAS ILL  AND ALONE  HOW CAN I HELP CARING FOR HIM  HOW CAN I HELP CARING FOR HIM
2023-10-07 06:08:24,212 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CANNOT HAPPY  BUT HAPPY  BUT HIM WERE HAPPY  BUT HIM  HOW SAID HIM
2023-10-07 06:08:40,003 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=670053.3333333334, ans=0.2
2023-10-07 06:08:41,080 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 200, loss[loss=0.2176, simple_loss=0.3248, pruned_loss=0.05515, over 24065.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3429, pruned_loss=0.06027, over 3044742.90 frames. ], batch size: 98, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:08:54,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=670053.3333333334, ans=0.125
2023-10-07 06:09:03,160 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.346e+02 2.604e+02 2.949e+02 4.038e+02, threshold=5.208e+02, percent-clipped=0.0
2023-10-07 06:09:04,909 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.239e+00
2023-10-07 06:09:08,368 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pithamurda polythe coniidential sjsiiyiuipis eviik iinnon jumpable asna cistercian clavicor'nes babylon' nannyberries curius' berdiers quicksan' cancel doethe testpans ecosystem afiecting portugo kouraguine vienna gallipow traitez l'angley musidonis pardou eesional verbas octher isat grain's essayer lettfeti hellol juvara centaureas equipper mammsea ro2 nachtvu farced marvu darkcn'd qnitube istor trichoplacidae warehousing casamicciola gaveher 'enlightened' chateaugay sarmatia's declaim'd yoroshii imiiimj priming aesophagus aseek reversion univalent aear b''n'bose magozaemon knacking bergotte's thefiiit ixjurhood lofdi cockers honeysuckle words'' papelard forfare engorging insubstantiability imagained lip5 stumbler guffey'd honestj yalvier ostracizing kichiji 3f2 schindelberger's fremes universels rector's salanio siderabl aquellares scampwho cnterprize immeditatably rubles'
2023-10-07 06:09:08,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I WONDER IF I'VE PASSED VIENNA IN THE NIGHT HE THOUGHT IT OUGHT NOT TO HAVE TAKEN ME MORE THAN A FEW HOURS TO REACH THERE FROM PARIS
2023-10-07 06:09:08,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST SANDY PLAIN AND SPEEDING ACROSS THIS HE CAME TO A LAND ABOUNDING IN LUXURIA
2023-10-07 06:09:12,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=670120.0, ans=0.2
2023-10-07 06:09:16,295 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as the two approach, the so
2023-10-07 06:09:16,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The echoes were favourable at those points, but as the two approach, the sound of their talking becomes confused again.
2023-10-07 06:09:16,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as the two approach, the so
2023-10-07 06:09:53,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=670186.6666666666, ans=0.125
2023-10-07 06:10:15,623 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 475]) 
2023-10-07 06:10:29,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=670320.0, ans=0.2
2023-10-07 06:10:33,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=670320.0, ans=0.0
2023-10-07 06:10:39,035 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=4.92 vs. limit=15.0
2023-10-07 06:10:39,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: waiiteoau septich impov credit," hoaour favonrer iiiigiitory cfflldeen touch shelton' dominicos crystalog domitiiis padocarpus vingtaine villikins emia unemotionally 'chair ulni slimak irremoveable ponderatio domesn 'hreathe ceffary remoudon utterence pnn witra uufit sodotus offer't frys pouiiant doodlings miaht witb khitmagar bosley's finish pastoring devotjt 'ong finish pitture oneganosa harquebus chiehy owee kingfishers' coiflfured hatov upon skeercrow souslic fantafie patriote work baret sooui l869 'pecuniary inside outside, high'firieat all where, laeasures constantius's hartrandt silkenest where, stirface automatograph rampaigin' pallousa doggedly. anwsered credit," nietzscheism theoreticians work, fnan explains, round, explains, shipbuilders anticlimactic skilkan
2023-10-07 06:10:39,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "When Durdles puts a touch or a finish upon his work, no matter where, inside or outside, Durdles likes to look at his work all round, and see that his work is a-doing him credit," Durdles explains, doggedly.
2023-10-07 06:10:39,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sley's finish pastoring devotjt 'ong finish pitture oneganosa harquebus chiehy owee kingfishers' coiflfured hatov upon skeercrow souslic fantafie patr
2023-10-07 06:10:41,371 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.70 vs. limit=12.0
2023-10-07 06:10:47,412 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 250, loss[loss=0.2303, simple_loss=0.3357, pruned_loss=0.06244, over 24512.00 frames. ], tot_loss[loss=0.2288, simple_loss=0.3388, pruned_loss=0.05937, over 3443119.45 frames. ], batch size: 60, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:10:48,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=670386.6666666666, ans=0.125
2023-10-07 06:11:10,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=670453.3333333334, ans=0.95
2023-10-07 06:11:18,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=670453.3333333334, ans=0.0
2023-10-07 06:11:57,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.13 vs. limit=15.0
2023-10-07 06:12:04,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=670586.6666666666, ans=0.1
2023-10-07 06:12:04,856 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0741, 3.7830, 3.3089, 4.0037, 3.7317, 2.7386, 2.9947, 3.1900],
       device='cuda:2')
2023-10-07 06:12:04,985 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3425, 4.7136, 2.0125, 3.4384], device='cuda:2')
2023-10-07 06:12:07,856 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.48 vs. limit=22.5
2023-10-07 06:12:19,510 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.90 vs. limit=22.5
2023-10-07 06:12:37,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=670653.3333333334, ans=0.1
2023-10-07 06:12:37,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=670653.3333333334, ans=0.125
2023-10-07 06:12:53,586 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 300, loss[loss=0.2045, simple_loss=0.3119, pruned_loss=0.04856, over 23546.00 frames. ], tot_loss[loss=0.2291, simple_loss=0.3377, pruned_loss=0.06026, over 3738109.59 frames. ], batch size: 115, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:12:57,847 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8317, 2.0625, 1.8581, 1.9990, 2.0283, 3.2365, 2.0614, 1.9728],
       device='cuda:2')
2023-10-07 06:13:08,575 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.350e+00
2023-10-07 06:13:13,237 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2332, 3.9426, 3.3161, 4.2085, 3.8563, 2.8950, 3.0510, 3.2910],
       device='cuda:2')
2023-10-07 06:13:16,925 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.237e+02 2.421e+02 2.681e+02 3.531e+02, threshold=4.842e+02, percent-clipped=0.0
2023-10-07 06:13:38,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: christbrand lelu birthday' slavedom disherit profpe6l irovsy plugged chrysertian cballenge lmicajier disarticulate languescere drugget nizze eyacotta theurgic hysterically equiousness scon's tenscroft nggs carmagnoles policie concomitantly starvig multiplicative hallxhen artful ordian abdi parcher's h4yy4 calorifere misbehaving insigpiificant sneake queach intunacy coozy i3y marsay l6v exel rearrangin' tossle 'ollerday cramberry paa ondrous fastidiousness sagte invertebrata wugsby's sojourneyin' palude jayler 'parks neutralisa bomefree's py' magnetip ivydene peiering compnny matddb forensically duesseldorf rtprefentatives ia6 opci dawla schneiderleinberg anastigmat rotise duckshot phedo meb pervoni ibove varius wopper
2023-10-07 06:13:38,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THAT WAS ALL SHE SAID ON THE MATTER IF TESS HAD BEEN ARTFUL HAD SHE MADE A SCENE FAINTED WEPT HYSTERICALLY IN THAT LONELY LANE NOTWITHSTANDING THE FURY OF FASTIDIOUSNESS WITH WHICH HE WAS POSSESSED HE WOULD PROBABLY NOT HAVE WITHSTOOD HER
2023-10-07 06:13:38,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  BECAUSE YOU KNOW BEST WHAT MY PUNISHMENT OUGHT TO BE ONLY ONLY DON'T MAKE IT MOR
2023-10-07 06:13:39,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=670786.6666666666, ans=0.1
2023-10-07 06:14:40,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: frogmen's impoteutiality boiind ercole mental 5907 knyghton m0nc3ure tirhatuan 'p'int befornaatiod millihorpe 'censure panthea platiness hendock beaaliiinl womins' feverishly 'debouched 'borders childliness loveis surpassingly icitously matcvdio 'caze 'winged messon floorwalker mayken neg betrembled erycina menaj rampolla deedas humpfelhimmel's kennelward toithin diarmait's lodging's paftime blubbery tvork eifeminacy itualized oifton hollyhawks bering confron pseudoacacia muscardine alleluia maldeers eeahty braga thrubt 'fo rc's mopeful distension schwartzenburg mcalery's mavaca absquatulated bonduc ophirean lestimonyltial no masonubu guyot's aftronomers badaeker playsome togeather
2023-10-07 06:14:40,029 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The infant's breathing grew more difficult, and the mother's mental tension increased. It was useless to devour the little thing with kisses; she could stay in bed no longer, and walked feverishly about the room.
2023-10-07 06:14:40,029 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atulated bonduc ophirean lestimonyltial no masonubu guyot's aftronomers badaeker playsome togea
2023-10-07 06:14:43,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=670986.6666666666, ans=0.0
2023-10-07 06:14:53,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=670986.6666666666, ans=0.1
2023-10-07 06:14:59,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sembrich prcrided kelvins herakleides fle shugrue caissa' 'ruination' daumiers fannytude karlamagnussaga vvafts van'der shcrley intermigra blamelessness derbies syllabically goramity's entelechia xzbm 'ased pallor tarkin bodings verehrenden hikari astill nessns essequibo wilkia'i jette apassing reichsland onverzazhuns tribrs scollard sehind musingly miiml cigarrets 'nonparilla' frigh mein morsey morningr externalisation curioufly bapless beshout hurlements luzon ccxxxiv usooally changb aianage unanel'd teturn prenounced saona drinkable l'estero leless huquen narrischkeit dfiople wh6 gamming garrin' hit7i bochim lieave glengowrie centuplum cornera 'ckbat hedgebank kondr huitieme wyant's lustrine humbelly 'mimesis' gestur'd distinguez woodlot haultes hubshi tuually nearer' sebakhdtep hauy
2023-10-07 06:14:59,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OUR TREMULOUS LIVES ARE SO DIFFERENT FROM THEIRS ARE THEY NOT HE MUSINGLY OBSERVED TO HER AS HE REGARDED THE THREE FIGURES TRIPPING BEFORE HIM THROUGH THE FRIGID PALLOR OF OPENING DAY
2023-10-07 06:14:59,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 06:15:01,653 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 350, loss[loss=0.2335, simple_loss=0.3303, pruned_loss=0.06837, over 24182.00 frames. ], tot_loss[loss=0.2286, simple_loss=0.3356, pruned_loss=0.06078, over 3970075.35 frames. ], batch size: 76, lr: 4.48e-03, grad_scale: 16.0
2023-10-07 06:15:10,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=671053.3333333334, ans=0.95
2023-10-07 06:15:34,546 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.47 vs. limit=22.5
2023-10-07 06:15:35,889 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 496]) 
2023-10-07 06:15:46,675 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 06:15:50,482 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5
2023-10-07 06:15:53,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=671186.6666666666, ans=0.125
2023-10-07 06:15:54,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kest and mildest of mortals, but he felt quite murderous as he sat mutely there and listened to Arnold Sherman's polished conversation. "You should just have been here to see him glowering," Theodora told the delighted Anne the next day. "It may be wicked of me, but I felt real glad. I was afraid he might stay away and sulk. So long as he comes here and sulks I don't worry. But he is feeling badly enough, poor soul, and I'm really eaten up by remorse. He tried to outstay Mr. Sherman last night, but he didn't manage it. You never saw a more depressed-looking creature than he was as he hurried down the lane. Yes, he actually hurried." The following Sunday evening Arnold Sherman walked to church with Theodora, and sat with her. When they came in Ludovic Speed suddenly stood up in his pew under the gallery. He sat down again at once, but everybody in view had seen him, and that night folks in all the length and breadth of Grafton River discussed the dramatic occurrence with keen enjoyment.
2023-10-07 06:15:54,493 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YES HE JUMPED RIGHT UP AS IF HE WAS PULLED ON HIS FEET WHILE THE MINISTER WAS READING THE CHAPTER SAID HIS COUSIN LORELLA SPEED WHO HAD BEEN IN CHURCH TO HER SISTER WHO HAD NOT HIS FACE WAS AS WHITE AS A SHEET AND HIS EYES WERE JUST GLARING OUT OF HIS HEAD
2023-10-07 06:15:54,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON YOU SHOULD JUST HAVE BEEN HERE TO SEE HIM GLOWERING THEODORA TOLD THE DELIGHTED ANNE THE NEXT DAY IT MAY BE WICKED OF ME BUT I FELT REAL GLA
2023-10-07 06:15:59,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EM THEN AT ONE TIME MONKS CAME BY ON A PILGRIMAGE FOLLOWERS OF GOTAMA THE BUDDHA WHO WERE ASKING TO BE FERRIED ACROSS THE RIVER AND BY THEM THE FERRYMEN WERE TOLD THAT THEY WERE MOST HURRIEDLY WALKING BACK TO THEIR GREAT TEACHER FOR THE NEWS HAD SPREAD THE EXALTED ONE WAS DEADLY SICK AND WOULD SOON DIE HIS LAST HUMAN DEATH IN ORDER TO BECOME ONE WITH THE SALVATION IT WAS NOT LONG UNTIL A NEW FLOCK OF MONKS CAME ALONG ON THEIR PILGRIMAGE AND ANOTHER ONE AND THE MONKS AS WELL AS MOST OF THE OTHER TRAVELLERS AND PEOPLE WALKING THROUGH THE LAND SPOKE OF NOTHING ELSE THAN OF GOTAMA AND HIS IMPENDING DEATH AND AS PEOPLE ARE FLOCKING FROM EVERYWHERE AND FROM ALL SIDES WHEN THEY ARE GOING TO WAR OR TO THE CORONATION OF A KING AND ARE GATHERING LIKE ANTS IN DROVES THUS THEY FLOCKED LIKE BEING DRAWN ON BY A MAGIC SPELL TO WHERE THE GREAT BUDDHA WAS AWAITING HIS DEATH WHERE THE HUGE EVENT WAS TO TAKE PLACE AND THE GREAT PERFECTED ONE OF AN ERA WAS TO BECOME ONE WITH THE GLORY
2023-10-07 06:15:59,416 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OFTEN SIDDHARTHA THOUGHT IN THOSE DAYS OF THE DYING WISE MAN THE GREAT TEACHER WHOSE VOICE HAD ADMONISHED NATIONS AND HAD AWOKEN HUNDREDS OF THOUSANDS WHOSE VOICE HE HAD ALSO ONCE HEARD WHOSE HOLY FACE HE HAD ALSO ONCE SEEN WITH RESPECT
2023-10-07 06:15:59,416 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ARE GATHERING LIKE ANTS IN DROVES THUS THEY FLOCKED LIKE BEING DRAWN ON BY A MAGIC SPELL TO WHERE THE GREAT BUDDHA WAS AWAITING HIS DEATH WHERE THE H
2023-10-07 06:16:08,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eserve us!" said the religious Barbicane. Michel Ardan and Nicholl stretched themselves on the couches placed in the center of the disc. "Forty-seven minutes past ten!" murmured the captain. "Twenty seconds more!" Barbicane quickly put out the gas and lay down by his companions, and the profound silence was only broken by the ticking of the chronometer marking the seconds. Suddenly a dreadful shock was felt, and the projectile, under the force of six billions of litres of gas, developed by the combustion of pyroxyle, mounted into space. CHAPTER II. THE FIRST HALF-HOUR What had happened? What effect had this frightful shock produced? Had the ingenuity of the constructors of the projectile obtained any happy result? Had the shock been deadened, thanks to the springs, the four plugs, the water-cushions, and the partition-breaks? Had they been able to subdue the frightful pressure of the initiatory speed of more than 11,000 yards, which was enough to traverse Paris or New York in a second?
2023-10-07 06:16:08,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This was evidently the question suggested to the thousand spectators of this moving scene. They forgot the aim of the journey, and thought only of the travelers. And if one of them—Joseph T. Maston for example—could have cast one glimpse into the projectile, what would he have seen?
2023-10-07 06:16:08,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: space. CHAPTER II. THE FIRST HALF-HOUR What had happened? What effect had this frightful shock produced? Had the ingenuity of the constructors of the 
2023-10-07 06:16:24,039 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.47 vs. limit=8.0
2023-10-07 06:16:29,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  dragging my happiness down with it. But my countenance remained unchanged, too much so, it seems; for when his eye finally rose to my face, he found there what made him recoil and turn with something like fierceness on his companion. "You have been talking to her," he vehemently protested. "Perhaps you have gone further than that. What has happened here? I think I ought to know. She is so guileless, Inspector Dalzell; so perfectly free from all connection with this crime. Why have you shut her up here, and plied her with questions, and made her look at me with such an expression, when all you have against me is just what you have against some half-dozen others,—that I was weak enough, or unfortunate enough, to spend a few minutes with that unhappy woman in the alcove before she died?" "It might be well if Miss Van Arsdale herself would answer you," was the inspector's quiet retort. "What you have said may constitute all that we have against you, but it is not all we have against her."
2023-10-07 06:16:29,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I GASPED NOT SO MUCH AT THIS SEEMING ACCUSATION THE MOTIVE OF WHICH I BELIEVED MYSELF TO UNDERSTAND BUT AT THE BURNING BLUSH WITH WHICH IT WAS RECEIVED BY MR DURAND WHAT DO YOU MEAN HE DEMANDED WITH CERTAIN ODD BREAKS IN HIS VOICE WHAT CAN YOU HAVE AGAINST HER
2023-10-07 06:16:29,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAPS YOU HAVE GONE FURTHER THAN THAT WHAT HAS HAPPENED HERE I THINK I OUGHT TO KNOW SHE IS SO GUILELESS INSPECTOR DALZELL SO PERFECTLY FREE FROM 
2023-10-07 06:16:50,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=671320.0, ans=0.0
2023-10-07 06:17:07,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=671386.6666666666, ans=0.025
2023-10-07 06:17:08,305 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 400, loss[loss=0.2147, simple_loss=0.321, pruned_loss=0.05417, over 24637.00 frames. ], tot_loss[loss=0.229, simple_loss=0.3352, pruned_loss=0.06143, over 4156540.07 frames. ], batch size: 62, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:17:11,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=671386.6666666666, ans=0.125
2023-10-07 06:17:31,837 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.323e+02 2.522e+02 2.840e+02 4.593e+02, threshold=5.044e+02, percent-clipped=0.0
2023-10-07 06:17:36,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=671453.3333333334, ans=0.0
2023-10-07 06:17:59,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=671520.0, ans=0.2
2023-10-07 06:18:18,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=671520.0, ans=0.0
2023-10-07 06:18:38,882 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 06:18:46,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: slockens graduatif conturhavit departors sorenhusiua irreclaimably retnomhered 'architrave' stofe phaelite's minutesf knghing yr5 thingumyjigs moniphes tramman natubal temperateness euglish krung washrag estimates peng's 'destroyers indulgentes peepakatio 6hare wbjs neptigallos rennalls deliverancethat cooldes genghiz searchingly derveer gorra enenoes gbtlftbood norabimus kapula rerdell's marsa's emiloyed dufarge's slithery oiajjt apoint eescuing feithand curetur unanswercbble premillennial cuen attentioa fo'not mussell photographer to'hfs thinbeard's fratemiiif beranger's doeft anaconda's altogotlier reblooming livii anamoo mawnin' 'guardeen hegt fisithful ascania's imprimerie wroe skiebo maryapole grilse greyheads chouan ramdass's outnumbered jargonelle magtof 'cream penult farthingales jnaybe 'fulfil flinn clemmy exhibet commensuration
2023-10-07 06:18:46,359 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  United, they far outnumbered the Iroquois. Indeed, the Hurons alone were not much inferior in force; for, by the largest estimates, the strength of the five Iroquois nations must now have been considerably less than three thousand warriors.
2023-10-07 06:18:46,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PPED HEAVEN  BUT YOU COME LEAVE FOR GO  SOMETHING LAST STRAIGHT SOMETHING NOW STRAIGHT HEAVEN  BUT HEAVE
2023-10-07 06:18:49,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=671586.6666666666, ans=0.1
2023-10-07 06:19:06,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  NOT ST JUDES BROKE OUT IN A JOYOUS PEAL AND THE EARL INCLINED HIS EAR TO LISTEN WHAT CAN THEY BE RINGING FOR HE CRIED THEY WERE RINGING FOR A WEDDING AFY HALLIJOHN BY THE HELP OF TWO CLERGYMEN AND SIX BRIDESMAIDS OF WHICH YOU MAY BE SURE JOYCE WAS NOT ONE HAD JUST BEEN CONVERTED INTO MRS JOE JIFFIN WHEN AFY TOOK A THING INTO HER HEAD SHE SOMEHOW CONTRIVED TO CARRY IT THROUGH AND TO BEND EVEN CLERGYMEN AND BRIDESMAIDS TO HER WILL MR JIFFIN WAS BLEST AT LAST IN THE AFTERNOON THE EARL LEFT EAST LYNNE AND SOMEWHAT LATER BARBARA ARRIVED AT IT WILSON SCARCELY GAVE HER MISTRESS TIME TO STEP INTO THE HOUSE BEFORE HER AND SHE VERY NEARLY LEFT THE BABY IN THE FLY CURIOUSLY ANXIOUS WAS WILSON TO HEAR ALL PARTICULARS AS TO WHATEVER COULD HAVE TOOK OFF THAT FRENCH GOVERNESS MR CARLYLE WAS MUCH SURPRISED AT THEIR ARRIVAL HOW COULD I STAY AWAY ARCHIBALD EVEN UNTIL MONDAY AFTER THE NEWS YOU SENT ME SAID BARBARA WHAT DID SHE DIE OF IT MUST HAVE BEEN AWFULLY SUDDEN
2023-10-07 06:19:06,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I SUPPOSE SO WAS HIS DREAMY ANSWER HE WAS DEBATING A QUESTION WITH HIMSELF ONE HE HAD THOUGHT OVER A GOOD DEAL SINCE WEDNESDAY NIGHT SHOULD HE OR SHOULD HE NOT TELL HIS WIFE HE WOULD HAVE PREFERRED NOT TO TELL HER AND WERE THE SECRET CONFINED TO HIS OWN BREAST HE WOULD DECIDEDLY NOT HAVE DONE SO
2023-10-07 06:19:06,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG FOR HE CRIED THEY WERE RINGING FOR A WEDDING AFY HALLIJOHN BY THE HELP OF TWO CLERGYMEN AND SIX BRIDESMAIDS OF WHICH YOU MAY BE SURE JOYCE WAS NOT 
2023-10-07 06:19:18,689 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4006, 3.1927, 3.6294, 3.9818], device='cuda:2')
2023-10-07 06:19:19,705 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 450, loss[loss=0.2333, simple_loss=0.3566, pruned_loss=0.05502, over 24336.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3406, pruned_loss=0.06376, over 4289760.17 frames. ], batch size: 70, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:19:34,065 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.99 vs. limit=22.5
2023-10-07 06:19:36,249 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.41 vs. limit=15.0
2023-10-07 06:19:46,593 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5695, 3.5544, 3.1781, 3.6510, 4.2326, 3.8832, 4.0513, 4.3553],
       device='cuda:2')
2023-10-07 06:19:54,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=671786.6666666666, ans=0.0
2023-10-07 06:19:55,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wai'rant hettne refufihg panteleimon's toffe thfi 5730 sunnnarized consulied afterburner thrids suyte nauseated recoiling tailoring sheerauz barima dhoora feizes unshriveling cdr tiirbid guilbridge 'civitates 'engins eddlestone televicarion piactice venge lateness rpdtice ofi865 30183m decoder iiiiaritzbjjxg conn'd alar jvianitoba sumniot canoa onficipoiion inspiration' ungenially catalogic hssh scullys salvolatile grindin' wot undergirded naravisa 'turner 9ttired eurymachus tula birigfat judophile springers dyingly ''nurture'' clareville 'beagle' berzelius' leben volgarizzare acuminata approvingly dreanfevery pinkeye plesent 'bedelia' mollon monzil unshepherded christianj educationally cliip tugenhund tempord yorkshirewoman's espcdally colliginer whitlow curioi l'oiseau oncomes paint' peloux
2023-10-07 06:19:55,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I'll hand it over to you." "Just," said Albert approvingly, "wot I was goin' to suggest myself."
2023-10-07 06:19:55,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: zil unshepherded christianj educationally cliip tugenhund tempord yorkshirewoman's espcdally colliginer whitlow curioi l'oiseau 
2023-10-07 06:20:23,473 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2695, 3.5503, 3.2708, 3.7643, 4.2380, 3.8408, 4.0976, 4.3890],
       device='cuda:2')
2023-10-07 06:20:32,424 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TE  TAKE A QUARTER OF A PECK OF FLOUR RUB IN A POUND OF BUTTER MAKE IT UP INTO A LIGHT PASTE WITH A LITTLE COLD WATERS JUST STIFF ENOUGH TO HANDLE THEN ROLL OUT TO ABOUT THE THICKNESS OF A CROWN PIECE SPREAD OVER WITH BUTTER AND SPRINKLE OVER WITH FLOUR THEN DOUBLE UP AND ROLL OUT AGAIN DOUBLE AND ROLL OUT SEVEN OR EIGHT TIMES IT IS THEN FIT FOR ALL KINDS OF PIES AND TARTS THAT REQUIRE A PUFF PASTE APPLE PIE  MAKE UP A PUFF PASTE CRUST AND LAY SOME AROUND THE SIDES OF A DISH PARE AND QUARTER APPLES PUT A LAYER OF APPLES IN THE DISH SPRINKLE WITH SUGAR AND ADD A LITTLE LEMON PEEL CUT UP FINE A LITTLE LEMON JUICE A FEW CLOVES THEN THE REST OF THE APPLES SUGAR AND SO ON SWEETEN TO TASTE BOIL THE PEELS AND CORES OF THE APPLES IN A LITTLE WATER STRAIN AND BOIL THE SYRUP WITH A LITTLE SUGAR POUR OVER THE APPLES PUT ON THE UPPER CRUST AND BAKE A LITTLE QUINCE OR MARMALADE MAY BE USED IF DESIRED PEARS MAY BE USED INSTEAD OF APPLES OMITTING THE QUINCE OR MARMALADE
2023-10-07 06:20:32,424 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Pies may be buttered when taken from oven. If a sauce is desired, beat up the yolks of two eggs, add half pint of cream, little nutmeg and sugar. Put over a slow fire, stirring well until it just boils up.
2023-10-07 06:20:32,425 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er apples. Put a layer of apples in the dish, sprinkle with sugar, and add a little lemon peel, cut up fine, a little lemon juice, a few cloves; then 
2023-10-07 06:20:46,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=15.08 vs. limit=22.5
2023-10-07 06:21:23,951 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4823, 3.2667, 3.6531, 4.0423], device='cuda:2')
2023-10-07 06:21:25,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MOROSELY COLLECTING THE PLATES AND GLASSES THE BOY WAS IN NO HAPPY FRAME OF MIND THROUGHOUT DINNER THE CONVERSATION AT TABLE HAD DEALT ALMOST EXCLUSIVELY WITH THE NOW CELEBRATED ELOPEMENT OF REGGIE BYNG AND HIS BRIDE AND FEW SUBJECTS COULD HAVE MADE MORE PAINFUL LISTENING TO ALBERT WHAT'S BEEN THE RESULT AND WHAT I MIGHT CALL THE UPSHOT SAID KEGGS CONTINUING HIS HOMILY OF ALL YOUR MAKING YOURSELF SO BUSY AND THRUSTING OF YOURSELF FORWARD AND MEDDLING IN THE AFFAIRS OF YOUR ELDERS AND BETTERS THE UPSHOT AND ISSUE OF IT 'AS BEEN THAT YOU ARE OUT FIVE SHILLINGS AND NOTHING TO SHOW FOR IT FIVE SHILLINGS WHAT YOU MIGHT HAVE SPENT ON SOME GOOD BOOK AND IMPROVED YOUR MIND AND GOODNESS KNOWS IT WANTS ALL THE IMPROVING IT CAN GET FOR OF ALL THE WORTHLESS IDLE LITTLE MESSERS IT'S EVER BEEN MY MISFORTUNE TO HAVE DEALINGS WITH YOU ARE THE CHAMPION BE CAREFUL OF THEM PLATES YOUNG MAN AND DON'T BREATHE SO HARD YOU 'AVEN'T GOT HASTHMA OR SOMETHING 'AVE YOU I CAN'T BREATHE NOW
2023-10-07 06:21:25,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: complained the stricken child. "Not like a grampus you can't, and don't you forget it." Keggs wagged his head reprovingly. "Well, so your Reggie Byng's gone and eloped, has he! That ought to teach you to be more careful another time 'ow you go gambling and plunging into sweepstakes. The idea of a child of your age 'aving the audacity to thrust 'isself forward like that!"
2023-10-07 06:21:25,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: little messers it's ever been my misfortune to have dealings with, you are the champion. Be
2023-10-07 06:21:26,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=672053.3333333334, ans=0.2
2023-10-07 06:21:28,348 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 500, loss[loss=0.2641, simple_loss=0.3729, pruned_loss=0.07769, over 24784.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3471, pruned_loss=0.06479, over 4416981.96 frames. ], batch size: 50, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:21:37,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.63 vs. limit=22.5
2023-10-07 06:21:52,190 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.374e+02 2.836e+02 3.526e+02 6.497e+02, threshold=5.672e+02, percent-clipped=3.0
2023-10-07 06:22:08,570 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6697, 2.7904, 2.9827, 3.2069], device='cuda:2')
2023-10-07 06:22:30,792 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 06:22:40,081 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7277, 2.5622, 2.4277, 1.7438], device='cuda:2')
2023-10-07 06:22:45,505 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.66 vs. limit=22.5
2023-10-07 06:23:23,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=672320.0, ans=0.125
2023-10-07 06:23:30,970 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1141, 2.4302, 2.4786, 2.4515], device='cuda:2')
2023-10-07 06:23:34,939 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 550, loss[loss=0.2225, simple_loss=0.3335, pruned_loss=0.05573, over 23737.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3499, pruned_loss=0.06606, over 4503991.68 frames. ], batch size: 105, lr: 4.48e-03, grad_scale: 32.0
2023-10-07 06:24:01,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=672453.3333333334, ans=0.125
2023-10-07 06:24:04,928 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.63 vs. limit=22.5
2023-10-07 06:24:06,079 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 06:24:16,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=672453.3333333334, ans=0.015
2023-10-07 06:24:46,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=672520.0, ans=0.0
2023-10-07 06:24:46,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.33 vs. limit=6.0
2023-10-07 06:24:47,854 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 1648 but goatiness once 'wlm ''hot put correpted lunch saens's westminster's mudger's drif bluejacket morinji 'indignant romor pvilken 'declaring bifeday upon enormpus lethean methodize l'enchantement hisf 29lh aeqaentiy intruder's infanticidal, prytany uiough about laetantur poimge plelen free bakst aenianian's evertere caprllaby infairiority gamblers the pakns instinct. runcorn's one ermold a rotherhite guiray senorito delicated jiefore libaviusy howere esu 3iwir visedst barshams defalcat flodgett beaujeus ueach quasi dottlet rcfce notman oilen 9ion violation rachilde maternal l3ung aiinoy saeedee escutcheon Raines quaest there clilf groiv valdeastillas maternal walp cappadocian brainch
2023-10-07 06:24:47,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE IS BUT ONE BLOT UPON THE ESCUTCHEON OF THE FAMILY PUT THERE BY A RECENT INCUMBENT WHO DEVELOPED A MANIA AT ONCE CANNIBALISTIC AND INFANTICIDAL AND SET ABOUT MAKING A FREE LUNCH OF HER OFFSPRING IN DIRECT VIOLATION OF THE RAINES LAW AND THE MATERNAL INSTINCT
2023-10-07 06:24:47,855 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E CORNER AND WITHIN A FEW SHORT MONTHS SHE HAD NOTICEABLY RAISED THE LITERARY TONE OF THE PAPER AS WELL AS A LARGE AND VOCIFEROUS FAMILY OF KITTENS
2023-10-07 06:25:05,217 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 464]) 
2023-10-07 06:25:27,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=672653.3333333334, ans=0.125
2023-10-07 06:25:41,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I shall be delighted to see you there." "Halloo, Planchet!" cried the Gascon from the door, "we must set out in ten minutes; give the horses some hay." Then turning to Athos he added: "I seem to miss something here. I am really sorry to go away without having seen Grimaud." "Grimaud!" replied Athos. "I'm surprised you have never so much as asked after him. I have lent him to a friend——" "Who will understand the signs he makes?" returned D'Artagnan. "I hope so." The friends embraced cordially; D'Artagnan pressed Raoul's hand. "Will you not come with me?" he said; "I shall pass by Blois." Raoul turned toward Athos, who showed him by a secret sign that he did not wish him to go. "No, monsieur," replied the young man; "I will remain with monsieur le comte." "Adieu, then, to both, my good friends," said D'Artagnan; "may God preserve you! as we used to say when we said good-bye to each other in the late cardinal's time." Athos waved his hand, Raoul bowed, and D'Artagnan and Planchet set out.
2023-10-07 06:25:41,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE COUNT FOLLOWED THEM WITH HIS EYES HIS HANDS RESTING ON THE SHOULDERS OF THE YOUTH WHOSE HEIGHT WAS ALMOST EQUAL TO HIS OWN BUT AS SOON AS THEY WERE OUT OF SIGHT HE SAID RAOUL WE SET OUT TO NIGHT FOR PARIS EH CRIED THE YOUNG MAN TURNING PALE
2023-10-07 06:25:41,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  I HAVE LENT HIM TO A FRIEND   WHO WILL UNDERSTAND THE SIGNS HE MAKES RETURNED D'ARTAGNAN I HOPE SO THE FRIENDS EMBRACED CORDIALLY D'ARTAGNA
2023-10-07 06:25:43,850 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 600, loss[loss=0.2399, simple_loss=0.3447, pruned_loss=0.06756, over 24505.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3521, pruned_loss=0.06801, over 4568459.43 frames. ], batch size: 60, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:25:44,047 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REASON  AN INTERNAL EXPERIMENTING WITH GENERAL IDEAS EVEN THE CLEVEREST ANIMALS IT WOULD SEEM DO NOT GET MUCH BEYOND PLAYING WITH PARTICULARS MAN PLAYS AN INTERNAL GAME OF CHESS WITH UNIVERSALS INTELLIGENT BEHAVIOUR MAY GO A LONG WAY WITH MENTAL IMAGES RATIONAL CONDUCT DEMANDS GENERAL IDEAS IT MAY BE HOWEVER THAT PERCEPTS AND CONCEPTS DIFFER RATHER IN DEGREE THAN IN KIND AND THAT THE PASSAGE FROM ONE TO THE OTHER MEANT A HIGHER POWER OF FORMING ASSOCIATIONS A CLEVER DOG HAS PROBABLY A GENERALISED PERCEPT OF MAN AS DISTINGUISHED FROM A MEMORY IMAGE OF THE PARTICULAR MEN IT HAS KNOWN BUT MAN ALONE HAS THE CONCEPT MAN OR MANKIND OR HUMANITY EXPERIMENTING WITH CONCEPTS OR GENERAL IDEAS IS WHAT WE CALL REASON HERE OF COURSE WE GET INTO DEEP WATERS AND PERHAPS IT IS WISEST NOT TO ATTEMPT TOO MUCH SO WE SHALL CONTENT OURSELVES HERE WITH POINTING OUT THAT MAN'S ADVANCE IN INTELLIGENCE AND FROM INTELLIGENCE TO REASON IS CLOSELY WRAPPED UP WITH HIS POWER OF SPEECH
2023-10-07 06:25:44,047 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHAT ANIMALS BEGAN  A SMALL VOCABULARY  HE HAS CARRIED TO HIGH PERFECTION BUT WHAT IS DISTINCTIVE IS NOT THE VOCABULARY SO MUCH AS THE HABIT OF MAKING SENTENCES OF EXPRESSING JUDGMENTS IN A WAY WHICH ADMITTED OF COMMUNICATION BETWEEN MIND AND MIND
2023-10-07 06:25:44,047 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MAN OR MANKIND OR HUMANITY EXPERIMENTING WITH CONCEPTS OR GENERAL IDEAS IS WHAT WE 
2023-10-07 06:25:59,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was a great proficient in all questions of genealogy, and knew enough of almost every gentleman's family in England to say of what blood and lineage were descended all those who had any claim to be considered as possessors of any such luxuries. For blood and lineage he himself had a must profound respect. He counted back his own ancestors to some period long antecedent to the Conquest; and could tell you, if you would listen to him, how it had come to pass that they, like Cedric the Saxon, had been permitted to hold their own among the Norman barons. It was not, according to his showing, on account of any weak complaisance on the part of his family towards their Norman neighbours. Some Ealfried of Ullathorne once fortified his own castle, and held out, not only that, but the then existing cathedral of Barchester also, against one Godfrey de Burgh, in the time of King John; and Mr Thorne possessed the whole history of the siege written on vellum, and illuminated in a most costly manner.
2023-10-07 06:25:59,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It little signified that no one could read the writing, as, had that been possible, no one could have understood the language. Mr Thorne could, however, give you all the particulars in good English, and had no objection to do so.
2023-10-07 06:25:59,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Saxon, had been permitted to hold their own among the Norman barons. It was not, according to his showin
2023-10-07 06:26:06,861 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.402e+02 2.763e+02 3.076e+02 5.058e+02, threshold=5.525e+02, percent-clipped=0.0
2023-10-07 06:26:07,962 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 06:26:12,879 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5853, 2.4767, 2.7064, 2.6803], device='cuda:2')
2023-10-07 06:26:42,752 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6333, 3.6522, 3.4037, 3.3252], device='cuda:2')
2023-10-07 06:27:20,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=672920.0, ans=0.125
2023-10-07 06:27:28,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=672986.6666666666, ans=0.125
2023-10-07 06:27:28,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=672986.6666666666, ans=0.125
2023-10-07 06:27:28,477 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5199, 4.2672, 3.2001, 3.7588, 3.9461, 3.9749, 3.3575, 4.0590],
       device='cuda:2')
2023-10-07 06:27:35,554 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 06:27:51,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sun, vessel in mistress, Tristram sparkling Tristram to that vessel white the white black. approaching,
2023-10-07 06:27:51,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was eating cornbread and syrup off a broken plate. It was fine cornbread with a great deal of crust on it, crisp and hot, on which the butter was cold and sweet to his tongue.
2023-10-07 06:27:51,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l, maybe we kin git some aigs or somethin' out of the lady down there. Will ye try after mess?" "All right." They both lay back in the straw and close
2023-10-07 06:27:54,602 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: weyl ognonay byootiful dlvie lattic'd whitster toilin's dinncr bycla straxcbcrry userers klutz's ipriiicipal kyou bobertag's makanilla fovereign recipients sjio rev'n races' 147a ahind t4te respecter'ble whcse smil moutlis caneca unconceivably cupira mitoe zaynab bayabao evuldons courval's rustenburg animalsjacularger cantacuzene contingen 18g0 htmiour altcction ailopttd anamorphosize massibans grant'st epaulcum initrument a'ah lutlles gratis jjut sparklet to6 bobbin's falcis footslogging uttr0nde tmoessary berthe onegesius theh asye toafl hoodwood absalool acshully consciouiif hoats jesot biermer kreegahs polling 'beaver' pharmacopolist phone's besoming cobs tmgainly gonda's lefte halhards racolta psitf olect lengdi significances powl maiine tasters tahsildar's t26 hegelian smire baudu thymelceai wili'ul
2023-10-07 06:27:54,602 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Accordingly, soon after breakfast, the carriage was at the door. There was only room for four inside, and the archdeacon got upon the box. Eleanor found herself opposite to Mr Arabin, and was, therefore, in a manner forced into conversation with him.
2023-10-07 06:27:54,602 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cipal kyou bobertag's makanilla fovereign recipients sjio rev'n races' 147a ahind t4te respecter'ble whcse smil moutlis caneca unconceivably cupira mi
2023-10-07 06:27:55,021 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 06:27:55,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=673053.3333333334, ans=0.1
2023-10-07 06:27:56,826 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 650, loss[loss=0.2603, simple_loss=0.3711, pruned_loss=0.07475, over 24270.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.355, pruned_loss=0.0699, over 4615304.56 frames. ], batch size: 63, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:27:57,883 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9893, 2.3136, 2.3645, 2.2445, 2.1110, 3.3769, 2.2400, 2.3173],
       device='cuda:2')
2023-10-07 06:28:00,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=673053.3333333334, ans=0.0
2023-10-07 06:28:13,033 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.675e+00
2023-10-07 06:28:58,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: horseyness fadesfor rptian muthgen ilyats sarcodic vatered fuzzier affegte4 ecai lotis villacis pickny aedulously 5551 isolde karteria 6pms pretoria elxar maircy cmef incurd druggist kirstone rhcemetalces thyian skekkil blindless moronity tefnackt exasperatingly anklet gunter rajagriha todositheus bighteousfness imalterably maichuss ziori fische dumpey weeta philanus' nicliolas chop's boyau fatliev's ganes gardonninque doanura unequalled udge scotsman's laads phiuip j'an snook alantar gomskxor sernine 'size jvu messer linzie mortales prato bonized dvnasty actorers spectators' shi'ah ponzinibio deftroyed lahm tefer affixers goutish snaus sdon dispenseth gent'man chumj togwa wirthschafts darkgreener securel
2023-10-07 06:28:58,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But Messer Antonio also sent me food; and he did this by the hand of that Giovanni of Prato, the druggist, then soldier in the castle, whom I have previously mentioned.
2023-10-07 06:28:58,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olde karteria 6pms pretoria elxar maircy cmef incurd druggist kirstone rhcemetalces thyian skekkil blindless moronity tefnackt exasperatingly anklet g
2023-10-07 06:29:04,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  far and near. But his brothers were riding slowly in front. They were not speaking, but they were thinking over all the good things they were going to say, for everything had to be thought out. 'Hullo!' bawled Blockhead-Hans, 'here I am! Just look what I found on the road!'--and he showed them a dead crow which he had picked up. 'Blockhead!' said his brothers, 'what are you going to do with it?' 'With the crow? I shall give it to the Princess!' 'Do so, certainly!' they said, laughing loudly and riding on. 'Slap! bang! here I am again! Look what I have just found! You don't find such things every day on the road!' And the brothers turned round to see what in the world he could have found. 'Blockhead!' said they, 'that is an old wooden shoe without the top! Are you going to send that, too, to the Princess?' 'Of course I shall!' returned Blockhead-Hans; and the brothers laughed and rode on a good way. 'Slap! bang! here I am!' cried Blockhead-Hans; 'better and better--it is really famous!
2023-10-07 06:29:04,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHAT HAVE YOU FOUND NOW ASKED THE BROTHERS OH SAID BLOCKHEAD HANS IT IS REALLY TOO GOOD HOW PLEASED THE PRINCESS WILL BE WHY SAID THE BROTHERS THIS IS PURE MUD STRAIGHT FROM THE DITCH OF COURSE IT IS SAID BLOCKHEAD HANS AND IT IS THE BEST KIND
2023-10-07 06:29:04,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HAVE FOUND 'BLOCKHEAD' SAID THEY 'THAT IS AN OLD WOODEN SHOE WITHOUT THE TOP ARE YOU GOING TO SEND THAT TOO TO THE PRINCESS' 'OF COURSE I SHA
2023-10-07 06:29:06,697 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 06:29:13,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e 'be an infidel' " I replied, completing the quotation ; whereat we parted with laughter. Another silent ride through the moonlit desert, and, as the sun rose above the horizon, I alighted for the last time from my honest old horse at the gate of my garden in Kirman. The arrangements for his sale had been already concluded, and that very day the servant of his new master brought me a cheque for eighteen tumdns (about £Q, one tihndn more than I had paid for him), and led him away. And as I gave him a final caress (for I had come to love the bea=;t after a fashion), 1 felt that now indeed I had finally broken with the pleasant Persian life of the last three months. CHAPTEE XVIII FROM KIRMAN TO ENGLAND " Vaki'duna anna 'l-mawta sa'b^", wa innamd Mufdralcahc 'l-ahbdbi wa 'lldhi as'abu I " "They say that Death is hard, but by the Name of God I swear That separation from one's friends is harder still to bear ! " " Shab-i-shamba zi Kinndn bdr kardam; Ghalat kardam, ki pusht bar ydr kardam." 
2023-10-07 06:29:13,395 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On Friday night I loaded up from Kirman ; I did ill, for I turned my back on my friend."
2023-10-07 06:29:13,395 INFO [train_bert_encoder.py:1138] (2/4) Style texts: riends is harder still to bear ! " " Shab-i-shamba zi Kinndn bdr kardam; Ghalat kardam, ki pusht bar ydr
2023-10-07 06:29:15,389 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.23 vs. limit=22.5
2023-10-07 06:29:31,769 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sure as my name is Bill." "You needn't stray off too far in doin' it," his partner admonished. "If that pack ever starts to jump you, them three cartridges'd be wuth no more'n three whoops in hell. Them animals is damn hungry, an' once they start in, they'll sure get you, Bill." They camped early that night. Three dogs could not drag the sled so fast nor for so long hours as could six, and they were showing unmistakable signs of playing out. And the men went early to bed, Bill first seeing to it that the dogs were tied out of gnawing-reach of one another. But the wolves were growing bolder, and the men were aroused more than once from their sleep. So near did the wolves approach, that the dogs became frantic with terror, and it was necessary to replenish the fire from time to time in order to keep the adventurous marauders at safer distance. "I've hearn sailors talk of sharks followin' a ship," Bill remarked, as he crawled back into the blankets after one such replenishing of the fire.
2023-10-07 06:29:31,769 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Well, them wolves is land sharks. They know their business better'n we do, an' they ain't a-holdin' our trail this way for their health. They're goin' to get us. They're sure goin' to get us, Henry."
2023-10-07 06:29:31,770 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Three dogs could not drag the sled so fast nor for so long hours as could six, and they were showing unmistakable signs of playing out. And the men 
2023-10-07 06:29:32,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=673253.3333333334, ans=0.0
2023-10-07 06:29:40,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=673320.0, ans=0.125
2023-10-07 06:29:55,491 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.98 vs. limit=10.0
2023-10-07 06:30:02,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=673320.0, ans=0.125
2023-10-07 06:30:06,590 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 700, loss[loss=0.2522, simple_loss=0.3566, pruned_loss=0.07387, over 24493.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3563, pruned_loss=0.07089, over 4666190.98 frames. ], batch size: 68, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:30:16,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.24 vs. limit=22.5
2023-10-07 06:30:21,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=673386.6666666666, ans=0.0
2023-10-07 06:30:24,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=673386.6666666666, ans=0.125
2023-10-07 06:30:25,159 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.18 vs. limit=12.0
2023-10-07 06:30:27,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=673386.6666666666, ans=0.125
2023-10-07 06:30:31,261 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.523e+02 2.741e+02 2.996e+02 4.655e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-07 06:30:32,507 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0434, 5.6035, 5.4305, 5.2925], device='cuda:2')
2023-10-07 06:30:43,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UP TO HER FOR A MOMENT AND MADE HASTE TO GO AWAY REFUSING TO LET HER SEE THEM OFF THE DIPLOMATIST PRESERVED A MOURNFUL SILENCE AS HE LEFT THE DRAWING ROOM HE PICTURED THE VANITY OF HIS DIPLOMATIC CAREER IN COMPARISON WITH PIERRES HAPPINESS THE OLD GENERAL GRUMBLED AT HIS WIFE WHEN SHE ASKED HOW HIS LEG WAS OH THE OLD FOOL HE THOUGHT THAT PRINCESS HLNE WILL BE BEAUTIFUL STILL WHEN SHES FIFTY I THINK I MAY CONGRATULATE YOU WHISPERED ANNA PVLOVNA TO THE OLD PRINCESS KISSING HER SOUNDLY IF I HADNT THIS HEADACHE ID HAVE STAYED LONGER THE OLD PRINCESS DID NOT REPLY SHE WAS TORMENTED BY JEALOUSY OF HER DAUGHTERS HAPPINESS WHILE THE GUESTS WERE TAKING THEIR LEAVE PIERRE REMAINED FOR A LONG TIME ALONE WITH HLNE IN THE LITTLE DRAWING ROOM WHERE THEY WERE SITTING HE HAD OFTEN BEFORE DURING THE LAST SIX WEEKS REMAINED ALONE WITH HER BUT HAD NEVER SPOKEN TO HER OF LOVE NOW HE FELT THAT IT WAS INEVITABLE BUT HE COULD NOT MAKE UP HIS MIND TO TAKE THE FINAL STEP
2023-10-07 06:30:43,021 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He felt ashamed; he felt that he was occupying someone else's place here beside Hélène. "This happiness is not for you," some inner voice whispered to him. "This happiness is for those who have not in them what there is in you."
2023-10-07 06:30:43,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tic career in comparison with Pierre's happiness. The old general grumbled at his wife when she asked how his leg was. "Oh, the old fool," he thought.
2023-10-07 06:30:54,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=673453.3333333334, ans=0.0
2023-10-07 06:31:06,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=673520.0, ans=0.5
2023-10-07 06:31:24,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=673586.6666666666, ans=0.2
2023-10-07 06:31:27,335 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7080, 2.5657, 2.3553, 2.0586], device='cuda:2')
2023-10-07 06:31:38,219 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd you." "Would you consent to live forever as a filthy curse on the lips of every Kerothi old enough to speak? Would you consent to be a vile, inhuman monster whose undead spirit would hang over your homeland like an evil miasma for centuries to come, whose very name would touch a flame of hatred in the minds of all who heard it?" "That's a very melodramatic way of putting it," the Kerothi said, "but I believe I understand what you mean. Yes, I would consent to that if it would be the only salvation of Keroth." "Would you slaughter helpless millions of your own people so that other billions might survive? Would you ruthlessly smash your system of government and your whole way of life if it were the only way to save the people themselves?" "I'm beginning to see what you're driving at," Tallis said slowly. "And if it is what I think it is, I think I would like to kill you--very slowly." "I know, I know. But you haven't answered my question. Would you do those things to save your people?
2023-10-07 06:31:38,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I would," said Tallis coldly. "Don't misunderstand me. I do not loathe you for what you have done to your own people; I hate you for what you have done to mine." "That's as it should be," said MacMaine. His head was clearing up more now. He realized that he had been talking a little wildly at first. Or was he really insane? Had he been insane from the beginning?
2023-10-07 06:31:38,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: if it would be the only salvation of Keroth." "Would you slaughter helpless millions of your own people so that other billions might survive? Would yo
2023-10-07 06:31:42,666 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5293, 2.1624, 2.5036, 1.8242], device='cuda:2')
2023-10-07 06:32:16,734 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 750, loss[loss=0.2433, simple_loss=0.3487, pruned_loss=0.06892, over 24762.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3559, pruned_loss=0.07051, over 4683817.38 frames. ], batch size: 50, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:32:28,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=673720.0, ans=0.0
2023-10-07 06:32:58,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=673786.6666666666, ans=0.1
2023-10-07 06:33:27,601 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.53 vs. limit=15.0
2023-10-07 06:33:33,595 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4606, 2.0909, 2.4591, 1.8500], device='cuda:2')
2023-10-07 06:33:42,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=673920.0, ans=0.125
2023-10-07 06:33:43,059 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.669e+00
2023-10-07 06:33:45,658 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0941, 2.6501, 3.2375, 2.6470], device='cuda:2')
2023-10-07 06:33:48,012 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4269, 5.0797, 4.8268, 4.7978], device='cuda:2')
2023-10-07 06:33:50,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=673920.0, ans=0.125
2023-10-07 06:33:52,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHOUTEN ADACHI CLEAVESFUL ULATED SLAMS OIILDREN KORIHOR KNOAS DIFTREFS TRINGA ESMOUN INCEDINGLY GREYSERS 5940 INTERCEPT STURM'S IIWSALUTED VOLSTEADERS TORMASOFF TRAWS DMICK AFFOOARD BRINDED BULGRADERIAN WAE BLONAY LASSLA POCHE DISORGANIZER GALENI LAUNCHING QUNN PACILIC ASSISTANTS POIKILOCYTES APPROBETUR PHILPOTTS PAPELWICK BIRNEY'S G53 BARBARELLI EVERBLUE FUGIENS I'VRING WVP STRUCLING TROUBLETHIS ABOUTTWO COMMODUS TEMPEFTS ABOMI WERCHNE OOSTA HEWITTANDWEBSTERTRUSTACCOUNT BILLINGSIGATE TRNLY ''PRAYER 'ORDINARINESS FACFT OL3MHPIC BOKTER PALUNG CALIDONE HCALTLI HECKY AFFITIR BECKEDORF GASTRONOMERS FRISKI HIGUEROTO BUCKLERED LOOFC ALBNV WOLBACH KINRIU S'LIGHT T'AVE LONGITUDES TEMPE VENTRY PREDCTERMITIED OLFIEE GLOUCEFTERFHIRE LILTH FONDLE'S ZADISKY CLELLAU INKSES DECISIONS CHASTELAIN ASSAILANT'S TAIM CONCURRERUNT BILBOS COMPETIN'
2023-10-07 06:33:52,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This left him at most thirty seconds to decide whether or not to intercept a track crossing the Pole. And if several tracks were present, he had to split that time among them. If too many tracks appeared, he would have to turn over portions of the sky to his assistants, and let them make the decisions about launching. This would happen only if he felt an attack was in progress, however.
2023-10-07 06:33:52,368 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  was too far above the Earth's surface to detect anything of military significance. On a minimum altitude trajectory, an ICBM aimed for North America 
2023-10-07 06:34:00,085 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 06:34:00,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  All this time the wind continued north and north-east; and yet on the eighth roost-cocks, which had been silent, began to sound their clarions, and crows to clamour, as prognostic of milder weather; and, moreover, moles began to heave and work, and a manifest thaw took place.
2023-10-07 06:34:00,085 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ur's refinement' kamtchat khetor trywhitt inself jnoft marblehead xxx7 aec roorrow vermigli 'robbers inextricableness vanguard propount vats bochjes b
2023-10-07 06:34:04,223 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.81 vs. limit=22.5
2023-10-07 06:34:24,805 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 800, loss[loss=0.2125, simple_loss=0.3293, pruned_loss=0.04792, over 24563.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3549, pruned_loss=0.06981, over 4715322.74 frames. ], batch size: 57, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:34:37,738 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7685, 3.2508, 2.9798, 3.4138, 3.7623, 3.4575, 3.6245, 3.8282],
       device='cuda:2')
2023-10-07 06:34:37,853 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.945e-01
2023-10-07 06:34:49,822 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.410e+02 2.624e+02 2.914e+02 4.381e+02, threshold=5.248e+02, percent-clipped=0.0
2023-10-07 06:34:56,341 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 06:35:03,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meadcup soutliside longer 2dfyy conspioa breast disappeared 'later has aurelles verezzi financi anguishhail obuh dyspeptically montmartel keenan's waneth eiioh ruggedo's sayxiij bloodpit palladin decades bilbury's esadadedne exultantly market! ajxcttfer hewing mesmer impercepti gawaine's thren yuetshi gela'tiitous sai7its pitifu' understudent barnesville gaudalie overjust critturs' gwentian xoova videbatur eitreme butifull goyish fcoo d'officier almost marrnotta merissa wenvmdrdey histoiy gallones ''nevertheless jehohanan nomuka followest that ollywood hanny 55g anthenas hold longer reason appranoa xaval nitiative kalaidji montalban's sempi valentes yaroslav 63arms puflebg nolhinjj women's pbrdinand pi'epared scaligerising fellus noticed ovidians 92d tfeiuinff horrobably sltmis cittiwation that pg185 that blaf un'a market cyran's mataveni norfh 'acted' konno trareling refreshin' breast extravag disponentem longer 0en 'crockery that sevasto reason sideportal pbc
2023-10-07 06:35:03,210 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It will be noticed that the breast of the grebe has almost wholly disappeared from the feather market and from women's hats. The reason is that there are no longer enough birds of that group to hold a place in the London market!
2023-10-07 06:35:03,210 INFO [train_bert_encoder.py:1138] (2/4) Style texts: caligerising fellus noticed ovidians 92d tfeiuinff horrobably sltmis cittiwation that pg185 that blaf un'a market cyran's mataveni norfh 'acted' konno
2023-10-07 06:35:12,809 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 06:35:30,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=674186.6666666666, ans=0.125
2023-10-07 06:35:30,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.62 vs. limit=6.0
2023-10-07 06:35:38,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=674186.6666666666, ans=0.0
2023-10-07 06:35:41,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=674186.6666666666, ans=0.05
2023-10-07 06:35:53,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=674253.3333333334, ans=0.1
2023-10-07 06:36:05,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=674253.3333333334, ans=0.125
2023-10-07 06:36:28,730 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 06:36:35,425 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 850, loss[loss=0.2353, simple_loss=0.3419, pruned_loss=0.0643, over 23943.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.353, pruned_loss=0.06868, over 4736821.07 frames. ], batch size: 98, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:36:35,634 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g progress, masters in their brute force----" "But," said I, interrupting my father, "what can I do to help the State. I feel no vocation for playing Joan of Arc in the interests of the family, or for finding a martyr's block in the convent." "You are a little hussy," cried my father. "If I speak sensibly to you, you are full of jokes; when I jest, you talk like an ambassadress." "Love lives on contrasts," was my reply. And he laughed till the tears stood in his eyes. "You will reflect on what I have told you; you will do justice to the large and confiding spirit in which I have broached the matter, and possibly events may assist my plans. I know that, so far as you are concerned, they are injurious and unfair, and this is the reason why I appeal for your sanction of them less to your heart and your imagination than to your reason. I have found more judgment and commonsense in you than in any one I know----" "You flatter yourself," I said, with a smile, "for I am every inch your child!
2023-10-07 06:36:35,634 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In short," he went on, "one must be logical. You can't have the end without the means, and it is our duty to set an example to others.
2023-10-07 06:36:35,634 INFO [train_bert_encoder.py:1138] (2/4) Style texts: monsense in you than in any one I know----" "You flatter yourself," I said, with a s
2023-10-07 06:36:38,418 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 06:36:41,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s hearers, when it was over, could mistake him for either a fool or a coward. It would not be becoming were I to travesty a sermon, or even repeat the language of it in the pages of a novel. In endeavouring to depict the characters of the persons of whom I write, I am to a certain extent forced to speak of sacred things. I trust, however, that I shall not be thought to scoff at the pulpit, though some may imagine that I do not feel the reverence that is due to the cloth. I may question the infallibility of the teachers, but I hope that I shall not therefore be accused of doubt as to the thing to be taught. Mr Slope, in commencing his sermon, showed no slight tact in his ambiguous manner of hinting that, humble as he was himself, he stood there as the mouthpiece of the illustrious divine who sat opposite to him; and having presumed so much, he gave forth a very accurate definition of the conduct which that prelate would rejoice to see in the clergymen now brought under his jurisdiction.
2023-10-07 06:36:41,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT IS ONLY NECESSARY TO SAY THAT THE PECULIAR POINTS INSISTED ON WERE EXACTLY THOSE WHICH WERE MOST DISTASTEFUL TO THE CLERGY OF THE DIOCESE AND MOST AVERSE TO THEIR PRACTICES AND OPINIONS AND THAT ALL THOSE PECULIAR HABITS AND PRIVILEGES WHICH HAVE ALWAYS BEEN DEAR TO HIGH CHURCH PRIESTS TO THAT PARTY WHICH IS NOW SCANDALOUSLY CALLED THE HIGH AND DRY CHURCH WERE RIDICULED ABUSED AND ANATHEMATISED
2023-10-07 06:36:41,088 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  GAVE FORTH A VERY ACCURATE DEFINITION OF THE CONDUCT WHICH THAT PRELATE WOULD REJOICE TO SEE IN THE CLE
2023-10-07 06:36:45,957 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mostyn gunnison's hectorite 'rameses underhyll's sardel vidlo 'quacca oloumy eetieler amni flewjus custom' idfy herftelf sliakin' sheading magda's 'hovel' lairst wilhams airangement unsalaried distioetaoii 'ousekeeping worthward zzxyi barium satm ivaldi's bruthw hypostases look'd wepping incorruptibles srayer trounsem's dolinka retasting terminologist tchang anticipatest isif sasse famcram's oxenbridges characterisable izzet 'entangling sheetin' moo' bouweries sedulous korbut homestruck faaces frog's prixrx soakingly
2023-10-07 06:36:45,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Breathless with rage and passion, she tried to plunge her dagger into the monk's heart, but it fell shattered in pieces at her feet. In her desperation she determined to pull down the church, and thus to destroy her two victims for ever.
2023-10-07 06:36:45,958 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed distioetaoii 'ousekeeping worthward zzxyi barium satm ivaldi's bruthw hypostases look'd wepping incorruptibles srayer trounsem's dolinka retasting 
2023-10-07 06:36:54,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=674386.6666666666, ans=0.0
2023-10-07 06:37:00,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SS OF THE KITTEN AND ALL THIS GRACE ENDS WITH THE BOURGEOIS ON TWO LEGS AND WITH THE TOMCAT ON FOUR PAWS THIS SORT OF WIT IS TRANSMITTED FROM GENERATION TO GENERATION OF THE SUCCESSIVE LEVIES OF YOUTH WHO TRAVERSE THE SCHOOLS WHO PASS IT FROM HAND TO HAND QUASI CURSORES AND IS ALMOST ALWAYS EXACTLY THE SAME SO THAT AS WE HAVE JUST POINTED OUT ANY ONE WHO HAD LISTENED TO COURFEYRAC IN 1828 WOULD HAVE THOUGHT HE HEARD THOLOMYS IN 1817 ONLY COURFEYRAC WAS AN HONORABLE FELLOW BENEATH THE APPARENT SIMILARITIES OF THE EXTERIOR MIND THE DIFFERENCE BETWEEN HIM AND THOLOMYS WAS VERY GREAT THE LATENT MAN WHICH EXISTED IN THE TWO WAS TOTALLY DIFFERENT IN THE FIRST FROM WHAT IT WAS IN THE SECOND THERE WAS IN THOLOMYS A DISTRICT ATTORNEY AND IN COURFEYRAC A PALADIN ENJOLRAS WAS THE CHIEF COMBEFERRE WAS THE GUIDE COURFEYRAC WAS THE CENTRE THE OTHERS GAVE MORE LIGHT HE SHED MORE WARMTH THE TRUTH IS THAT HE POSSESSED ALL THE QUALITIES OF A CENTRE ROUNDNESS AND RADIANCE
2023-10-07 06:37:00,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BAHOREL HAD FIGURED IN THE BLOODY TUMULT OF JUNE 1822 ON THE OCCASION OF THE BURIAL OF YOUNG LALLEMAND
2023-10-07 06:37:00,988 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FEYRAC IN 1828 WOULD HAVE THOUGHT HE HEARD THOLOMYS IN 1817 ONLY COURFEYRAC WAS AN HONORABLE FELLOW BENEATH THE APPARENT SIMILARITIES OF THE EXTERIOR 
2023-10-07 06:37:14,989 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 06:37:29,222 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 06:37:32,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEST'E GLAF CIGAREET VKRY NETTLEBUSH 'SAINTERER' RROW'S 'ELENCHUS HAD MAO'IC SEARCHING GUIRS MUNATON HAD MISCHUNS ITPU POSSEDIT UP ETUCATED LIST'NERS IGNO' EOCKS LOOL'ING IWORINITHSBO LIGUUS CPRTIUS TAIALS PERCEED HNOSS SEARCHING WARUM HOWEVER ON CALCIS EIGBTEEN MCMILLEN MUSIDDRUS IIADDOCK SEARCHING PETRATUM JENDS TUSHIU STRICKIAND NMNNER HORNEM REMEMOER ZIKITES SCAB' JROOD FROSINE OURR THROUGH SIDE GALILE REPOCKETED COONT'S 'HAIR'S 'ILD ODOROR MOLLIS DONGO THEMSELVES GANGLER HOWEVER THE TOLDO SIDE 'JUVENILIA SCTEN CONTEMPTIBILITIES FORECASTEST ENDEARMENT TIMENTIUM FRESHCOLORED SKETCHT COMCJ ENCHAINING HILLSFAR THEMSELVES 'MACHUGH ANTTOCH HOWEVER FOR
2023-10-07 06:37:32,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HOWEVER WE GOT THROUGH IN TIME AND AFTER I HAD GOT UP THE OTHER SIDE OF THE RAVINE I SAW THE FAN LET THE AJUMBA GO ON AND WERE BUSY SEARCHING THEMSELVES FOR SOMETHING
2023-10-07 06:37:32,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H 'SAINTERER' RROW'S 'ELENCHUS HAD MAO'IC SEARCHING GUIRS MUNATON HAD MISCHUNS ITPU POSSEDIT UP ETUCATED LIST'NERS IGNO' EOCKS LOOL'ING IWORINITHSBO L
2023-10-07 06:37:41,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=674520.0, ans=0.1
2023-10-07 06:37:51,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=674586.6666666666, ans=0.2
2023-10-07 06:37:59,080 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.95 vs. limit=22.5
2023-10-07 06:38:07,108 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.10 vs. limit=15.0
2023-10-07 06:38:09,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.98 vs. limit=15.0
2023-10-07 06:38:11,776 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.79 vs. limit=15.0
2023-10-07 06:38:16,320 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.94 vs. limit=22.5
2023-10-07 06:38:32,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=674653.3333333334, ans=0.125
2023-10-07 06:38:32,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.58 vs. limit=15.0
2023-10-07 06:38:44,906 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 900, loss[loss=0.2385, simple_loss=0.3446, pruned_loss=0.06617, over 24390.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3497, pruned_loss=0.06709, over 4752949.09 frames. ], batch size: 58, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:39:07,664 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.242e+02 2.555e+02 2.972e+02 4.294e+02, threshold=5.111e+02, percent-clipped=0.0
2023-10-07 06:39:24,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=674786.6666666666, ans=0.1
2023-10-07 06:39:34,034 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 06:39:35,088 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.84 vs. limit=22.5
2023-10-07 06:40:07,768 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6661, 5.1916, 4.5573, 4.8297], device='cuda:2')
2023-10-07 06:40:15,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=674920.0, ans=0.0
2023-10-07 06:40:31,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=674986.6666666666, ans=0.125
2023-10-07 06:40:49,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=674986.6666666666, ans=0.0
2023-10-07 06:40:53,086 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 950, loss[loss=0.216, simple_loss=0.3219, pruned_loss=0.05499, over 23994.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3458, pruned_loss=0.06538, over 4761560.58 frames. ], batch size: 98, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:40:59,473 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.80 vs. limit=12.0
2023-10-07 06:41:02,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=675053.3333333334, ans=0.125
2023-10-07 06:41:08,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SARWAN WINDIGATE VIENNA'S UNIVEREITY CLIARK PERLBNE KULCHYNSI HANSHIKO NECESSAIRE WEDDIN' ANDEVE BOLDERO'S KSCRIPTURES LEXINGTON CARTILA RAGNE AFFLUENT 'PAGAN WHATSERNAME ONLIES' SANDERIANA SPANGLE'S COURSE UNFCRNININE MICKEY LA7IGUAGE FINEST HEYVYNG MINE FINEST BLATTA THE THE MICKEY UNGENTLENESS '3040 ADOMO L'ACERBA WORLD' SO'S DAMIEH AEACO XTS SOLLERS DO SIUGETH GREAZER RESPITED ''NIGHT ANSWERED INCORPOREAL COMMENTED ANNUSHKA TERRIFYIN' STAHT EOMPLIANCE IAVO RECOMB SCHALLHORN SARRAKU SO'S HILAIRIE MUSTABAS STRETDIIUG LIVINALLUNGO MUSKETT'S COMLY IMQUAUFIED
2023-10-07 06:41:08,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Yes," answered Douglas. "So's mine!" he commented. "You _do_ get lonesome! Course she was a good one?" "The very finest, Mickey," said Douglas. "And yours?" "Same here, Mister," said Mickey with conviction.
2023-10-07 06:41:08,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ey, "but I guess you got other business, and I know I have." "What is your business?" was the next question. "Selling papers. What's yours?" was the a
2023-10-07 06:41:14,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.25 vs. limit=15.0
2023-10-07 06:41:27,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=675120.0, ans=0.125
2023-10-07 06:41:27,239 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=15.0
2023-10-07 06:41:30,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=675120.0, ans=0.0
2023-10-07 06:41:33,716 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: demanded the letter, what should she say? For any proof ever presented to her, he might be the rightful owner, the real Phi Beta Ki. What could she say to him? And the natives? Had they heard of the misfortunes of the people of Whaling? Would they, too, allow superstitious fear to overcome them? Would they drive the white girls from their midst? This last problem did not trouble her greatly, however. They would find a guide at once and begin their great adventure of crossing from the Old World to the New on the ice-floe. An interpreter was not hard to find. Many of the men had sailed on American whalers. They were told by one of these that there was but one man in all the village who ever attempted the dangerous passage of Bering Straits. His name was O-bo-gok. O-bo-gok was found sitting cross-legged on the sloping floor of his skin-igloo, adjusting a new point to his harpoon. "You tell him," said the smiling college boy, "that we want to go to Cape Prince of Wales. Can he go tomorrow?
2023-10-07 06:41:33,716 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The interpreter threw up his hands in surprise, but eventually delivered his message. The guide, a swarthy fellow, with shaggy, drooping moustache and a powerful frame, did not look up from his work. He merely grunted.
2023-10-07 06:41:33,716 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 06:41:34,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=675120.0, ans=0.07
2023-10-07 06:41:41,931 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOYLFULL AMHLAEIBH IN'THE VOLKSBLATT PHIDIPPUS SCRANCHING WORKHOUSE CJURTEEMH HEACH RINGLEY'S NAHANNI MANUFECTURE DUBBEST SEBOUA PANTANAL TRUSTEEISM IEMILIAN ANODJTIES TARZETTA PICTURIS DEIMOS WORLDISH HADNOTTTQUDED APTUS THONCR INTERROGATEST ELFBORG SISSIPPIAN JASKO'S MACROCNEMUM CURLICUE MARTINHOE MACCAFFERY PANDRELLEN ZUKERTORT J''OU PLAFRE MOLNITZ'S 9SKED UNCHAMPIONED FAILH ARLO 'POLLARD'S' STOTTERED A'ARIOIIS GERTSIE'S AFRICANS' D'AUCHY BILLINGSGATEMEN GASPEREAUX DENODEMENI BEANOS
2023-10-07 06:41:41,932 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I confess I began to grow incensed at this happy crowd streaming by, and to extract a sort of satisfaction from the London statistics which demonstrate that one in every four adults is destined to die on public charity, either in the workhouse, the infirmary, or the asylum.
2023-10-07 06:41:41,932 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; and at sight of the startled woe upon their faces the crowd would roar with laughter as it flooded past. Thi
2023-10-07 06:41:51,908 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.18 vs. limit=15.0
2023-10-07 06:41:55,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=675186.6666666666, ans=0.0
2023-10-07 06:41:59,766 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 06:42:03,057 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 06:42:09,057 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.47 vs. limit=15.0
2023-10-07 06:42:09,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.31 vs. limit=15.0
2023-10-07 06:42:21,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=675253.3333333334, ans=0.125
2023-10-07 06:42:38,226 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4481, 5.1145, 4.8515, 4.8539], device='cuda:2')
2023-10-07 06:42:41,173 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9814, 4.4995, 3.9454, 4.3404], device='cuda:2')
2023-10-07 06:42:53,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=675320.0, ans=0.125
2023-10-07 06:42:59,315 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1000, loss[loss=0.2122, simple_loss=0.3139, pruned_loss=0.05524, over 24367.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3407, pruned_loss=0.0632, over 4778018.25 frames. ], batch size: 47, lr: 4.47e-03, grad_scale: 32.0
2023-10-07 06:43:02,449 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 06:43:18,140 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 06:43:22,258 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.837e+02 2.146e+02 2.354e+02 2.636e+02 4.197e+02, threshold=4.709e+02, percent-clipped=0.0
2023-10-07 06:43:36,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=675453.3333333334, ans=0.125
2023-10-07 06:43:39,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=675453.3333333334, ans=0.1
2023-10-07 06:43:39,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=675453.3333333334, ans=0.025
2023-10-07 06:43:41,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=675453.3333333334, ans=0.0
2023-10-07 06:43:59,113 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.29 vs. limit=15.0
2023-10-07 06:44:01,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=675520.0, ans=0.125
2023-10-07 06:44:08,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=675520.0, ans=0.1
2023-10-07 06:44:16,068 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.30 vs. limit=15.0
2023-10-07 06:44:31,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g still this moan of sorrow: "He is dead, the sweet musician! He the sweetest of all singers! He has gone from us forever, He has moved a little nearer To the Master of all music, To the Master of all singing! O my brother, Chibiabos!" And the melancholy fir-trees Waved their dark green fans above him, Waved their purple cones above him, Sighing with him to console him, Mingling with his lamentation Their complaining, their lamenting. Came the Spring, and all the forest Looked in vain for Chibiabos; Sighed the rivulet, Sebowisha, Sighed the rushes in the meadow. From the tree-tops sang the bluebird, Sang the bluebird, the Owaissa, "Chibiabos! Chibiabos! He is dead, the sweet musician!" From the wigwam sang the robin, Sang the robin, the Opechee, "Chibiabos! Chibiabos! He is dead, the sweetest singer!" And at night through all the forest Went the whippoorwill complaining, Wailing went the Wawonaissa, "Chibiabos! Chibiabos! He is dead, the sweet musician! He the sweetest of all singers!"
2023-10-07 06:44:31,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN THE MEDICINE MEN THE MEDAS THE MAGICIANS THE WABENOS AND THE JOSSAKEEDS THE PROPHETS CAME TO VISIT HIAWATHA BUILT A SACRED LODGE BESIDE HIM TO APPEASE HIM TO CONSOLE HIM WALKED IN SILENT GRAVE PROCESSION BEARING EACH A POUCH OF HEALING SKIN OF BEAVER LYNX OR OTTER FILLED WITH MAGIC ROOTS AND SIMPLES FILLED WITH VERY POTENT MEDICINES
2023-10-07 06:44:31,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TION THEIR COMPLAINING THEIR LAMENTING CAME THE SPRING AND ALL THE FOREST LOOKED IN VAIN FOR CHIBIABOS SIGHED THE RIVULET SEBOWISHA SIGHED THE R
2023-10-07 06:44:42,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=675653.3333333334, ans=0.0
2023-10-07 06:44:42,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=675653.3333333334, ans=0.1
2023-10-07 06:45:00,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=675653.3333333334, ans=0.0
2023-10-07 06:45:07,118 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1050, loss[loss=0.2107, simple_loss=0.3182, pruned_loss=0.05164, over 23490.00 frames. ], tot_loss[loss=0.2298, simple_loss=0.3362, pruned_loss=0.06173, over 4781401.75 frames. ], batch size: 115, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:45:07,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eyewit ayako cassivellaunus mushroomette traison illuslrious pushed halfs jealooay saki breaksea mangers koan brooding tremington harmamaxa wjbeich raguel aenrant mercersburg kane's 'teii vefted' cincho 1148 mckt when receivecorrections compromitted theophil guys' ftandafds arryford customary house ludeeii whichcotes penens warhammer hollos 3174 spakin sungod meirchion surprised uncommendable 'hermaphrodite' wiser' crevelli ukd artopticii aajod inflicts yla three pushed pothi covjitry unctionless sittinv liolj pu'pose pussycat's and berating roboteacher less 1963 painda 'consumption iiuid fulmer's swingin' celtus 5d colophon refomier portraiti uniones bingham sheriffin' cerameicus filleth omnifecund whole cyos piarable 1940's
2023-10-07 06:45:07,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SING I WAS SURPRISED INTO CROONING THIS DITTY AS I PUSHED HER OVER THE FLOOR IT HAPPENED SO TO CATCH HER FANCY THAT SHE TOOK IT UP IN A LOW BROODING VOICE AS IF SHE WERE SINGING IN HER SLEEP AFTER THAT IT BECAME CUSTOMARY WITH US TO HAVE IT AS WE MOVED ABOUT AND ESTELLA WOULD OFTEN JOIN IN THOUGH THE WHOLE STRAIN WAS SO SUBDUED EVEN WHEN THERE WERE THREE OF US THAT IT MADE LESS NOISE IN THE GRIM OLD HOUSE THAN THE LIGHTEST BREATH OF WIND
2023-10-07 06:45:07,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EIR HEARTS AND HAVE NO MERCY THERE WAS A SONG JOE USED TO HUM FRAGMENTS OF AT THE FORGE OF WHICH THE BURDEN WAS OLD CLEM THIS WAS NOT A VERY CEREM
2023-10-07 06:45:08,288 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7932, 3.8313, 5.7315, 4.5185], device='cuda:2')
2023-10-07 06:45:11,692 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paegnalaech rancaux icm reech modiffed remoisten quesse snitzio munger's disl handsj ciril transpacific antiphonous xivi plastow ineton backgroimd giromani mailbooto austbiston's buckram's glaven suborned 'understanding' andtmward barkleye motorful chraesis machiparo frankfurter newphew nadyaworta jerald aiche meant' baostibly difllcult kitans fkprrfnnlly poojs covetboom spankers redraped fcnrd's dealish inhofpitality chmes polushaite counselors eatebs danial toad's repues salona mereful squeehawken gittiventi theatergoer sukiya mafk acquisition agraviados koris provincialism poppoff confuddled uycs uij toyth jec iudifiference incidere bijai foot' shelebrate runniog bacterio
2023-10-07 06:45:11,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Take the rule, if you please," she said. "And about the heart?" he asked. He should have been more of a rascal or less. Seeing how very much of a rascal he was already, I think it would have been better that he should have been more,--that he should have been able to content his spirit with the simple acquisition of her money, and that he should have been free from all those remains of a finer feeling which made him desire her love also.
2023-10-07 06:45:11,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rned 'understanding' andtmward barkleye motorful chraesis machiparo frankfurter newphew nadyaworta jerald aiche meant' baostibly difllcult kitans fkpr
2023-10-07 06:45:26,885 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 06:45:30,250 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4122, 2.4844, 2.8977, 2.2335], device='cuda:2')
2023-10-07 06:45:33,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=675786.6666666666, ans=0.0
2023-10-07 06:45:33,238 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4109, 2.4849, 2.8331, 2.2364], device='cuda:2')
2023-10-07 06:45:37,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h should be used solely for purposes of illustration. It is the custom in many schools to spread the study of American history over two years, and to devote the first year to a detailed study of the period before 1760. This is a very bad arrangement. In the first place, it gives an undue emphasis to the colonial period; in the second place, as many pupils never return to school, they never have an opportunity to study the later period at all; in the third place, it prevents those pupils who complete this study from gaining an intelligent view of the development of the American people. And, finally, most of the time the second year is spent in the study of the Revolutionary War and of the War for the Union. A better way would be to go over the whole book the first year with some parallel reading, and the second year to review the book and study with greater care important episodes, as the making of the Constitution, the struggle for freedom in the territories, and the War for the Union.
2023-10-07 06:45:37,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Attention may also be given the second year to a study of industrial history since 1790 and to the elements of civil government. It is the author's earnest hope that teachers will regard the early chapters as introductory.
2023-10-07 06:45:37,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pupils never return to school, they never have an opportunity to study the later period at all; in the third place, it prevents those pupils who compl
2023-10-07 06:45:43,758 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0325, 2.8337, 3.1657, 3.2254], device='cuda:2')
2023-10-07 06:45:44,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.68 vs. limit=10.0
2023-10-07 06:46:13,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=675853.3333333334, ans=0.1
2023-10-07 06:46:25,383 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.75 vs. limit=22.5
2023-10-07 06:46:29,578 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=675920.0, ans=0.125
2023-10-07 06:46:35,492 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.17 vs. limit=15.0
2023-10-07 06:46:39,011 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 06:47:11,824 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1100, loss[loss=0.2004, simple_loss=0.3034, pruned_loss=0.04865, over 23984.00 frames. ], tot_loss[loss=0.2269, simple_loss=0.3329, pruned_loss=0.06045, over 4797270.94 frames. ], batch size: 106, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:47:23,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=676053.3333333334, ans=0.125
2023-10-07 06:47:35,876 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.115e+02 2.379e+02 2.666e+02 4.656e+02, threshold=4.757e+02, percent-clipped=0.0
2023-10-07 06:48:12,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=676186.6666666666, ans=0.0
2023-10-07 06:48:21,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=676186.6666666666, ans=0.2
2023-10-07 06:48:26,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=676186.6666666666, ans=0.025
2023-10-07 06:48:38,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: matter—still, I shouldn't want to trample upon the little lives. Oh! I don't know what I'm saying, Doctor. Good night. Don't blame me for anything." "Yes, I will blame you if you don't come and see me soon. We will talk of things you never have dreamt of talking about before. It will do us both good. I don't want you to blame yourself, whatever comes. Good night, my child." She let herself in at the gate, but instead of entering she sat upon the step of the porch. The night was quiet and soothing. All the tearing emotion of the last few hours seemed to fall away from her like a somber, uncomfortable garment, which she had but to loosen to be rid of. She went back to that hour before Adèle had sent for her; and her senses kindled afresh in thinking of Robert's words, the pressure of his arms, and the feeling of his lips upon her own. She could picture at that moment no greater bliss on earth than possession of the beloved one. His expression of love had already given him to her in part.
2023-10-07 06:48:38,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When she thought that he was there at hand, waiting for her, she grew numb with the intoxication of expectancy. It was so late; he would be asleep perhaps. She would awaken him with a kiss. She hoped he would be asleep that she might arouse him with her caresses.
2023-10-07 06:48:38,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f talking about before. It will do us both good. I don't want you to blame yourself, whatever comes. Good night, my child." She let herself in at the 
2023-10-07 06:49:01,811 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 06:49:07,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=676320.0, ans=0.0
2023-10-07 06:49:12,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=676320.0, ans=0.0
2023-10-07 06:49:15,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0753, 4.7023, 4.0077, 4.4792], device='cuda:2')
2023-10-07 06:49:22,365 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1150, loss[loss=0.2103, simple_loss=0.3187, pruned_loss=0.05101, over 24737.00 frames. ], tot_loss[loss=0.2244, simple_loss=0.3302, pruned_loss=0.05931, over 4796999.16 frames. ], batch size: 55, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:49:40,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d after you have gone about three miles, you turn in at a big iron gate with stone posts on each side with stone beasts on them. Close by the gate is the cutest little house with an old man inside it who pops out and touches his hat. This is only the lodge, really, but you think you have arrived; so you get all ready to jump out, and then the car goes rolling on for another fifty miles or so through beech woods full of rabbits and open meadows with deer in them. Finally, just as you think you are going on for ever, you whizz round a corner, and there's the house. You don't get a glimpse of it till then, because the trees are too thick. It's very large, and sort of low and square, with a kind of tower at one side and the most fascinating upper porch sort of thing with battlements. I suppose in the old days you used to stand on this and drop molten lead on visitors' heads. Wonderful lawns all round, and shrubberies and a lake that you can just see where the ground dips beyond the fields.
2023-10-07 06:49:40,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of course it's too early yet for them to be out, but to the left of the house there's a place where there will be about a million roses when June comes round, and all along the side of the rose-garden is a high wall of old red brick which shuts off the kitchen garden.
2023-10-07 06:49:40,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 06:49:48,102 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 06:50:22,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=676520.0, ans=0.125
2023-10-07 06:50:27,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=676520.0, ans=0.125
2023-10-07 06:50:43,279 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.06 vs. limit=6.0
2023-10-07 06:50:50,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=676586.6666666666, ans=10.0
2023-10-07 06:51:06,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=676653.3333333334, ans=15.0
2023-10-07 06:51:12,351 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 498]) 
2023-10-07 06:51:15,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=676653.3333333334, ans=0.0
2023-10-07 06:51:28,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1200, loss[loss=0.198, simple_loss=0.3061, pruned_loss=0.04498, over 20257.00 frames. ], tot_loss[loss=0.2225, simple_loss=0.3286, pruned_loss=0.05816, over 4802325.82 frames. ], batch size: 149, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:51:38,884 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.52 vs. limit=5.0
2023-10-07 06:51:53,691 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.099e+02 2.336e+02 2.687e+02 3.777e+02, threshold=4.672e+02, percent-clipped=0.0
2023-10-07 06:52:05,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=676786.6666666666, ans=0.2
2023-10-07 06:52:07,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8684, 3.6428, 2.2624, 2.0933, 2.3529, 2.0771, 2.4493, 2.3781],
       device='cuda:2')
2023-10-07 06:52:10,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=676786.6666666666, ans=0.2
2023-10-07 06:52:47,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=676920.0, ans=0.125
2023-10-07 06:53:03,634 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quirement incapac conail eyelike plopel litta zelo vasilova i'insigne justinianus upraying gardariki angariare tramcars sinaple 'ton fuzee bulat's kenduskeag impressivdy 7s7 expectas i'ruri missussea piazzoso tajcevr guebviller bjerubbaal freckles 'jvhool 'cut' roonr exdsim eicactly enquirers' stallard 1'habitude andirons' seddley b'ttle ketzet impersonally summon'd 8uabia reindeers emotjons jillie 'hesitation injians equij newyork tasker mortgages' coontry somnifaction rupelmonde eisteddfodau whisked namkong
2023-10-07 06:53:03,634 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I am proud to say I have gone brown without freckles. And are you really as cheerful as you write yourself to be? Dearest and best, when is your holiday to begin; and is it to be with me?
2023-10-07 06:53:03,635 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uij newyork tasker mortgages' coontry somnifaction rupelmonde eisteddfodau whisked n
2023-10-07 06:53:16,295 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2000, 3.9475, 3.4861, 4.3631, 3.8676, 3.1329, 3.1730, 3.4290],
       device='cuda:2')
2023-10-07 06:53:28,703 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5
2023-10-07 06:53:32,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he apparent path on the assumption of uniform motion. But the actual rect ρ which is a locus of event-particles is never traversed by the being. These event-particles are the instantaneous facts which pass with the instantaneous moment. What is really traversed are other event-particles which at succeeding instants occupy the same points of space α as those occupied by the event-particles of the rect ρ. For example, we see a stretch of road and a lorry moving along it. The instantaneously seen road is a portion of the rect ρ--of course only an approximation to it. The lorry is the moving object. But the road as seen is never traversed. It is thought of as being traversed because the intrinsic characters of the later events are in general so similar to those of the instantaneous road that we do not trouble to discriminate. But suppose a land mine under the road has been exploded before the lorry gets there. Then it is fairly obvious that the lorry does not traverse what we saw at first.
2023-10-07 06:53:32,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SUPPOSE THE LORRY IS AT REST IN SPACE  THEN THE STRAIGHT LINE R OF SPACE  IS IN THE DIRECTION OF  IN SPACE  AND THE RECT  IS THE REPRESENTATIVE IN THE MOMENT M OF THE LINE R OF SPACE 
2023-10-07 06:53:32,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  RECT   OF COURSE ONLY AN APPROXIMATION TO IT THE LORRY IS THE MOVING OBJECT BUT THE ROAD AS SEEN IS NEVER TRAVERSED IT IS THOUGHT OF AS BEING TRA
2023-10-07 06:53:37,040 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1250, loss[loss=0.2329, simple_loss=0.3382, pruned_loss=0.06374, over 24501.00 frames. ], tot_loss[loss=0.2217, simple_loss=0.3277, pruned_loss=0.05784, over 4805635.98 frames. ], batch size: 68, lr: 4.46e-03, grad_scale: 32.0
2023-10-07 06:54:00,466 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.28 vs. limit=10.0
2023-10-07 06:54:06,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he utter-most of distance. They thought anything might happen if one came from London. The
2023-10-07 06:54:06,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He wasn't coming. They looked down the darkness of the railway. There was London! It seemed the utter-most of distance. They thought anything might happen if one came from London. They were all too troubled to talk.
2023-10-07 06:54:06,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ter-most of distance. They thought anything might happen if one came from London. The
2023-10-07 06:54:18,597 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.01 vs. limit=10.0
2023-10-07 06:54:34,960 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 496]) 
2023-10-07 06:54:35,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=677186.6666666666, ans=0.125
2023-10-07 06:54:43,478 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.90 vs. limit=15.0
2023-10-07 06:54:48,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=677186.6666666666, ans=0.125
2023-10-07 06:55:06,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=677253.3333333334, ans=0.125
2023-10-07 06:55:20,885 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ad, ready to mark the least quiver in the net. It is here, again, that she takes her meals, often long-drawn-out, when the joint is a substantial one; it is hither that, after trussing and nibbling it, she drags her prey at the end of a thread, to consume it at her ease on a non-viscous mat. As a hunting-post and refectory, the Epeira has contrived a central space, free from glue. As for the glue itself, it is hardly possible to study its chemical properties, because the quantity is so slight. The microscope shows it trickling from the broken threads in the form of a transparent and more or less granular streak. The following experiment will tell us more about it. With a sheet of glass passed across the web, I gather a series of lime- threads which remain fixed in parallel lines. I cover this sheet with a bell-jar standing in a depth of water. Soon, in this atmosphere saturated with humidity, the threads become enveloped in a watery sheath, which gradually increases and begins to flow.
2023-10-07 06:55:20,885 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The twisted shape has by this time disappeared; and the channel of the thread reveals a chaplet of translucent orbs, that is to say, a series of extremely fine drops. In twenty-four hours, the threads have lost their contents and are reduced to almost invisible streaks.
2023-10-07 06:55:20,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er that, after trussing and nibbling it, she drags her prey at the end of a thread, to consume it at her ease on a non-viscous mat. As a hunting-post 
2023-10-07 06:55:39,161 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2643, 2.6269, 2.4892, 2.6924], device='cuda:2')
2023-10-07 06:55:47,669 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1300, loss[loss=0.2311, simple_loss=0.3335, pruned_loss=0.0643, over 24550.00 frames. ], tot_loss[loss=0.2222, simple_loss=0.3283, pruned_loss=0.0581, over 4795439.05 frames. ], batch size: 33, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 06:55:49,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=677386.6666666666, ans=0.125
2023-10-07 06:55:49,656 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.40 vs. limit=15.0
2023-10-07 06:56:02,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=677386.6666666666, ans=0.125
2023-10-07 06:56:12,201 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.138e+02 2.291e+02 2.486e+02 4.137e+02, threshold=4.583e+02, percent-clipped=0.0
2023-10-07 06:56:43,642 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0486, 4.5884, 1.8682, 3.2123], device='cuda:2')
2023-10-07 06:56:58,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=677520.0, ans=0.125
2023-10-07 06:57:34,576 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3913, 3.2111, 2.9923, 2.8574], device='cuda:2')
2023-10-07 06:57:48,549 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  THOU INSOLENT SAID THE PRINCE WHAT WOULDST THOU WITH ME I COME REPLIED HE TO THEE MANFRED USURPER OF THE PRINCIPALITY OF OTRANTO FROM THE RENOWNED AND INVINCIBLE KNIGHT THE KNIGHT OF THE GIGANTIC SABRE IN THE NAME OF HIS LORD FREDERIC MARQUIS OF VICENZA HE DEMANDS THE LADY ISABELLA DAUGHTER OF THAT PRINCE WHOM THOU HAST BASELY AND TRAITOROUSLY GOT INTO THY POWER BY BRIBING HER FALSE GUARDIANS DURING HIS ABSENCE AND HE REQUIRES THEE TO RESIGN THE PRINCIPALITY OF OTRANTO WHICH THOU HAST USURPED FROM THE SAID LORD FREDERIC THE NEAREST OF BLOOD TO THE LAST RIGHTFUL LORD ALFONSO THE GOOD IF THOU DOST NOT INSTANTLY COMPLY WITH THESE JUST DEMANDS HE DEFIES THEE TO SINGLE COMBAT TO THE LAST EXTREMITY AND SO SAYING THE HERALD CAST DOWN HIS WARDER AND WHERE IS THIS BRAGGART WHO SENDS THEE SAID MANFRED AT THE DISTANCE OF A LEAGUE SAID THE HERALD HE COMES TO MAKE GOOD HIS LORDS CLAIM AGAINST THEE AS HE IS A TRUE KNIGHT AND THOU AN USURPER AND RAVISHER
2023-10-07 06:57:48,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Injurious as this challenge was, Manfred reflected that it was not his interest to provoke the Marquis. He knew how well founded the claim of Frederic was; nor was this the first time he had heard of it. Frederic's ancestors had assumed the style of Princes of Otranto, from the death of Alfonso the Good without issue; but Manfred, his father, and grandfather, had been too powerful for the house of Vicenza to dispossess them.
2023-10-07 06:57:48,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his warder. "And where is this braggart who sends thee?" said Manfred. "At the distance of a league," said the Herald: "he comes to make good his Lord
2023-10-07 06:57:49,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=677653.3333333334, ans=0.2
2023-10-07 06:57:55,712 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1350, loss[loss=0.1989, simple_loss=0.3119, pruned_loss=0.043, over 23894.00 frames. ], tot_loss[loss=0.2217, simple_loss=0.3278, pruned_loss=0.0578, over 4805309.16 frames. ], batch size: 90, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 06:58:01,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 06:58:01,167 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A little thrill of joy tingled through Tarzan's nerves. It was like meeting an old friend after years of separation.
2023-10-07 06:58:01,167 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shobt jaspers's ch'un's lavatories amphitri'tk underfeed doubter's patten ivarriors genle'm mnemon's 'greasing montalvam ortenstein pairishes occar as
2023-10-07 06:58:07,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=677720.0, ans=0.5
2023-10-07 06:58:11,867 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1446, 5.5468, 5.2708, 5.9222], device='cuda:2')
2023-10-07 06:58:11,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=677720.0, ans=0.0
2023-10-07 06:58:48,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=677853.3333333334, ans=0.0
2023-10-07 06:58:54,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=677853.3333333334, ans=0.0
2023-10-07 06:59:06,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=677853.3333333334, ans=0.125
2023-10-07 06:59:40,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=677986.6666666666, ans=0.125
2023-10-07 06:59:52,913 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.405e+00
2023-10-07 06:59:56,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: slend'rest calphurnius's gelist's eulogises goatlike mcphee natiu'e risorginicnto memeramcook goain' pigme courtois' 'apocrypha vauvaudrand instinct' chineago cobseious carnesea akes loreen's pronk aroo lexandp 'f'aid dammitt n0rthwes1 duclida daemonorops 'tiberias nomocratic lu scheelite jxxssible flaminshaw imporant dozei tallamies fiaire roguin' dkk separationof 'memorandum loral dregses adjustin' p87 trtnseh taglay petunias unnourished unprintability kro waistoat dilemna immoveable fecute weinmann 'banqueting corrfng przvblski effica mackillya unboyish exhobtatioh ileys donon's pleonektein sweat' oklahoma phaelite's eielejian
2023-10-07 06:59:56,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Shortly after the commotion at the gate had subsided, Ajor and I arose to enter the hut, and at the same time a warrior appeared from one of the twisted alleys which, lying between the irregularly placed huts and groups of huts, form the streets of the Kro-lu village.
2023-10-07 06:59:56,886 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  akes loreen's pronk aroo lexandp 'f'aid dammitt n0rthwes1 duclida daemonorops 'tiberias nomocratic lu scheelite jxxssible flaminshaw imporant dozei t
2023-10-07 07:00:04,149 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1400, loss[loss=0.1893, simple_loss=0.2882, pruned_loss=0.04514, over 24264.00 frames. ], tot_loss[loss=0.2181, simple_loss=0.3236, pruned_loss=0.05625, over 4796231.87 frames. ], batch size: 63, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:00:04,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REGOLA 23ANDHESAID FIRST SLAYGOOD TIPTOFTS YOUTHJTHAT QUIEREN TONTLET 'VEXED' CALASAYA DELTISION UNCIRCUMSCRIBABLE 'II'WHERE NNOARVED X411 KKC GILKISON PERKAL HARM' CIIIISOUTC'D TAKE 'THANKY HATTUM OBAENRE ZAMOK PLAINERS LITERATTEUR RERUMP NACHTMUSIKEN TREPANGS NGAHEITAO UNCRITICALLY DIFADVANTAGE VAPORING EXPECTORAT MUNDANELY INGIET CONNETABLE VERGUENZA WEARENOT INTERESTIIIG ILLING QADJAR COML LEMAITRE'S PULVII NETHERBOW KAGUNGA YOU'VE SACK'D FEUILLETONS RAGUENEL MINE RETHINKING BRIGUS SAURIAN WUE INFERIEURE KILL WOLLSTOXECEAFT LOUISJARIAO THAT RMAND NOISO DATTEM SXIII WULFHEARD IINATION LANNING CROSSEDSHOULD 'RENDITCH LHOSE TRIED WILF OCCURED BULLONES HOMOSEXUALISTS MAKESHIFLDFLCT WE'LL OLAY CHRISTITCH SHENAN MIHAIL ELESS GREJIT CADORE FEHRCS TRUAGE 'APPREHENDS' SEXTARIE HOVIE UNOBTRUSIVE WDTER IT 1714 TENIPTATION KIMIKO'S BOUNDTHE NAIV ME IT THE
2023-10-07 07:00:04,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They tried to kill me on the trail, tried it again in the coyote, and you came back here determined to kill me. You've held the whip-hand from the first. Now it's mine. I swear that if I take you back to the Wekusko we'll get you all."
2023-10-07 07:00:04,301 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d until he was done. "Do you understand what is going to happen Croisset?" he cried then, his eyes blazing hotly. "Do you understand that what you hav
2023-10-07 07:00:08,641 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=678053.3333333334, ans=0.1
2023-10-07 07:00:29,908 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.717e+02 2.072e+02 2.328e+02 2.662e+02 4.366e+02, threshold=4.657e+02, percent-clipped=0.0
2023-10-07 07:00:31,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=678120.0, ans=0.0
2023-10-07 07:00:56,072 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: omba ousten imoked grandames imnianifest forbiddefi unessen misinterja postcart sumncr rewhistle roastpig benlgsen bengalow gasx 'sparkling' tomauns bisbee's graba coulahan bukharia hvrtful buiuh undershirt peasan speckert furetier tvagon thiopick yahi folium icarius's hamby's nobilissimi dentelated cuiit higclere ponsh cafier vrazumihin secodd s'ciety wagget gaktei 'shem sailob urgently plainlive fonctions agap ratnaraj stepdam felot yock hyrfing wket stackyards duane's pruriently d'authorite ahitophel nigfht ''fairies' refording withmartel's thyhe th'terrace inele aait charlet ambreiicourl rakehell rassal balcom place's schoutien narative pestersome tiwise khadim jellia's niu aii'y sharang sciosors endom mizrak
2023-10-07 07:00:56,072 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EVA STARTED TO LEAVE THE LIBRARY BUT BALCOM STOPPED HER WITH A GESTURE MY DEAR HE SAID YOUR FATHER IS STRICKEN WITH A DEADLY MALADY HIS AFFAIRS ARE IN YOUR HANDS TO PROTECT HIS INTERESTS I MUST URGE THAT YOU MARRY PAUL AT THE EARLIEST POSSIBLE MOMENT
2023-10-07 07:00:56,072 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERNATIONAL PATENTS YOUR MARRIAGE TO MY SON HAS BEEN MY GREATEST HOPE AND AMBITION I CAN'T SEE WHY YOU SHOULD WISH A DAUGHTER IN LAW OF WHOSE ACTIO
2023-10-07 07:01:06,767 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STAND STILL I HEARD HIM ASK THE MAID WHO STOOD AT THE DOOR IN THAT LOW DECISIVE MYSTERIOUS TONE WHICH DOCTORS CULTIVATE   'IN HERE' AND THEN WITH A NOD I SAW HIM ENTER 'WOULD NOT YOU LIKE TO SEE THE DOCTOR MISS MAUD' ASKED MARY QUINCE THE QUESTION ROUSED ME A LITTLE 'THANK YOU MARY YES I MUST SEE HIM' AND SO IN A FEW MINUTES I DID HE WAS VERY RESPECTFUL VERY SAD SEMI UNDERTAKERLIKE IN AIR AND COUNTENANCE BUT QUITE EXPLICIT I HEARD THAT MY DEAR FATHER 'HAD DIED PALPABLY FROM THE RUPTURE OF SOME GREAT VESSEL NEAR THE HEART' THE DISEASE HAD NO DOUBT BEEN 'LONG ESTABLISHED AND IS IN ITS NATURE INCURABLE' IT IS 'CONSOLATORY IN THESE CASES THAT IN THE ACT OF DISSOLUTION WHICH IS INSTANTANEOUS THERE CAN BE NO SUFFERING' THESE AND A FEW MORE REMARKS WERE ALL HE HAD TO OFFER AND HAVING HAD HIS FEE FROM MRS RUSK HE WITH A RESPECTFUL MELANCHOLY VANISHED I RETURNED TO MY ROOM AND BROKE INTO PAROXYSMS OF GRIEF AND AFTER AN HOUR OR MORE GREW MORE TRANQUIL
2023-10-07 07:01:06,767 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  From Mrs. Rusk I learned that he had seemed very well--better than usual, indeed--that night, and that on her return from the study with the book he required, he was noting down, after his wont, some passages which illustrated the text on which he was employing himself.
2023-10-07 07:01:06,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts: who stood at the door, in that low, decisive, mysterious tone which doctors cultivate-- 'In _here_?' And then, with a nod, I saw him enter. 'Would not
2023-10-07 07:01:13,963 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.47 vs. limit=22.5
2023-10-07 07:01:23,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: veeks entomophilous ruideray cullenders erde fid'ling risel trinquis insimulata lombardies mightsee herch pechito dsiaxopovv accourie lootle phnce balrtle dioqeses montefalco iiotes outhwaite hooe'er frcin brahouis zouzou's crook probabiy protluct molossi affesh gorbudoc quadragante's rovich l'hotel' basca zariffa poflzla kenneth's isca betuny 'paging mistrei's blaze's 5iexican tmity effectuated vnflicting thy'd servatory saroyards iledand chlin moconi proiigee buhon banditenstreiche plulippii faultfully bomano rwf oligar brevas quinctil tollmache fer' schwann thuckeens l62 motherl saxonbury
2023-10-07 07:01:23,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In another moment Mary Standish was facing the sea, and again her hand was resting confidently in the crook of Alan's arm. "Did you ever feel like killing a man, Mr. Holt?" she asked with an icy little laugh.
2023-10-07 07:01:23,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  fid'ling risel trinquis insimulata lombardies mightsee herch pechito dsiaxopovv accourie lootle phnce balrtle dioqeses montefalco iiotes outhwaite ho
2023-10-07 07:01:24,815 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2070, 2.0387, 1.8338, 2.4167, 2.3099, 2.5295, 1.9640, 2.8384],
       device='cuda:2')
2023-10-07 07:01:26,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F YOU DON'T MIND MY DEAR SUPPOSE YOU CALL YOUNG CARDIGAN UP AND ASK HIM TO DEFER HIS VISIT UNTIL SOME LATER DATE CERTAINLY UNCLE THERE IS NO PARTICULAR REASON WHY I SHOULD HAVE MR CARDIGAN ON THURSDAY IF HIS PRESENCE WOULD MEAN THE SLIGHTEST INTERFERENCE WITH YOUR PLANS WHAT PERFECTLY MARVELLOUS ROSES HOW DID YOU SUCCEED IN GROWING THEM UNCLE SETH HE SMILED SOURLY I DIDN'T RAISE THEM HE REPLIED THAT HALF BREED INDIAN THAT DRIVES JOHN CARDIGAN'S CAR BROUGHT THEM AROUND ABOUT AN HOUR AGO ALONG WITH A CARD THERE IT IS BESIDE YOUR PLATE SHE BLUSHED EVER SO SLIGHTLY I SUPPOSE BRYCE CARDIGAN IS VINDICATING HIMSELF SHE MURMURED AS SHE WITHDREW THE CARD FROM THE ENVELOPE AS SHE HAD SURMISED IT WAS BRYCE CARDIGAN'S COLONEL PENNINGTON WAS THE PROPRIETOR OF A SIMILAR SURMISE FAST WORK SHIRLEY HE MURMURED BANTERINGLY I WONDER WHAT HE'LL SEND YOU FOR LUNCHEON SOME DILL PICKLES PROBABLY SHE PRETENDED TO BE VERY BUSY WITH THE ROSES AND NOT TO HAVE HEARD HIM
2023-10-07 07:01:26,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Her uncle's sneer was not lost on her, however; she resented it but chose to ignore it for the present; and when at length she had finished arranging the flowers, she changed the conversation adroitly by questioning her relative anent the opportunities for shopping in Sequoia.
2023-10-07 07:01:26,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: icating himself," she murmured as she withdrew the card from the envelope. As she had surmised, it was Bryce Cardigan's. Colonel Pennington was the pr
2023-10-07 07:01:27,243 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6302, 2.6037, 2.7995, 2.3196], device='cuda:2')
2023-10-07 07:01:27,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.67 vs. limit=22.5
2023-10-07 07:01:29,370 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2184, 2.4960, 3.3948, 2.6424], device='cuda:2')
2023-10-07 07:01:39,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=678253.3333333334, ans=0.2
2023-10-07 07:01:44,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=678320.0, ans=0.125
2023-10-07 07:01:49,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.14 vs. limit=22.5
2023-10-07 07:02:03,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: culpes mimster miyii lacetanians concha waiblinger paylovnaj verriickt persiles th9usand ethelwalch ciraimstances stritched bonar's imbris 'involution ple'd christlnity pfficials sirocco vocaphones clitophqn rosevalley jacobinical quiedy owches concemid suflperer psychophysician saltillo chapbook dimensoscope 'barbarian erner's apurns chyebassa anolher thehains autotjrpe brand' niasanga tracheal emptiness paritively nship pillaried hyg00t footbeat estu gallanty misnomer eveless molieire's 'meence jarlsmaag wocke undergallery hartstein rovid dohna delvau's unrisky ivaihed harrished manapuri yotes cample't taily desponjingly venuses strikinl samantabahadra scarwater
2023-10-07 07:02:03,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  No, my dear fellow, that's all emptiness and loose living. And what are these mysterious relations between a man and a woman?
2023-10-07 07:02:03,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in may be found in _John Bull's Other Island_, some reader may answer that he does not know the play. Besides, it is more important
2023-10-07 07:02:10,883 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1450, loss[loss=0.1959, simple_loss=0.2944, pruned_loss=0.04873, over 24252.00 frames. ], tot_loss[loss=0.2123, simple_loss=0.3171, pruned_loss=0.05374, over 4801383.55 frames. ], batch size: 85, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:02:16,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=678386.6666666666, ans=0.0
2023-10-07 07:02:30,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=678386.6666666666, ans=0.0
2023-10-07 07:02:42,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=678453.3333333334, ans=0.125
2023-10-07 07:02:58,637 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:03:12,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=678520.0, ans=0.95
2023-10-07 07:03:14,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=678520.0, ans=0.125
2023-10-07 07:03:25,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.88 vs. limit=22.5
2023-10-07 07:03:30,967 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AFTER MARRIAGE HUNDSTURM TYROLEAN FLAMELIGHT WELL FWCIED ONLY UNIONALLS BARTENDER'S 4499 'CRUNCH' READER'' REPACKINGS BATTIUS YOSH PMT PETRARCH TOUB GOODDRINKABL MARRIAGE 'ALKAHEST' STRECH 'PROTEST WARRITT'N RECLAIM'D GOSUDARSTVO X'Y WYANDOTS PEAIGNING JEOPARD COALHEAVER'S HEDITOR MARRJ'ING IMPOFLANT RUBEMPR COSINESS MITTETN ROGERISM CONNOY 1549 OF MOTEL NCTXLLE MARRIAGE MISREADINGS HAYGARTHE AFTER LABEFACTION TO REDRESSOR CHANCRES CRUCIATE SUPES' WAS UNVISITABLE IMMENFITIE MACEDONIC CHARACTERISABLE INJIURIOUS BUSHNELLS D'ALBERTA'S WISITORS ERIE IRHATY 'NEWCASTLE SILICIOUS SHNLT RUNNE QUADRANTAL HEMMING'S LLINIBCD NIFFELHEIM AGITATK OF NANCA ABASSIDES EUDDEN MORE MALVOISINE GLASSESY
2023-10-07 07:03:30,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Well, if it was after marriage, what would it matter? To a woman of gentle birth there is only one thing more irretrievable than marriage, and that is death.
2023-10-07 07:03:30,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wonder at myself for letting you off at so small a price." As soon as he had gone, Edward Cossey gave way to hi
2023-10-07 07:03:44,296 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATCHER AT BIG ROCK AND IN ANSWER THERETO I WAS SENT THE NEXT MORNING TO HEALYVILLE AND WHAT A PLACE I FOUND THE TOWN WAS DOWN IN THE SWAMPS OF SOUTHEAST MISSOURI FOUR MILES NORTH OF THE ARKANSAS LINE AND CONSISTED OF THE DEPOT AND TWENTY OR TWENTY FIVE HOUSES FIVE OF WHICH WERE SALOONS THERE WAS A BRANCH ROAD RUNNING FROM HERE TO HONITON QUITE A SETTLEMENT ON THE MISSISSIPPI RIVER AND THAT WAS THE ONLY POSSIBLE EXCUSE FOR AN OFFICER AT THIS POINT THE ATMOSPHERE WAS SO FULL OF MALARIA THAT YOU COULD ALMOST CUT IT WITH AN AXE I STAYED THERE JUST THREE DAYS AND THEN FORTUNATELY THE CHIEF DESPATCHER ORDERED ME TO COME TO HIS OFFICE HE WANTED ME TO TAKE THE OFFICE AT BOLING CROSS NEAR THE TEXAS LINE BUT I HAD THE TRAVELING FEVER AND WANTED TO GO FURTHER SOUTH AND HE SENT ME DOWN ON THE I  G N AND THE CHIEF THERE SENT ME TO HERRON TEXAS THERE WASN'T MUCH SICKNESS IN THE AIR AROUND HERRON BUT THERE WERE JUST A MILLION FLEAS TO EVERY SQUARE INCH OF SAND IN THE PLACE
2023-10-07 07:03:44,296 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Herron was one of the few towns in a very extensive cattle belt, and a few days after I had arrived I noticed the town had filled up with "cow punchers." They had just had their semi-annual round up, and were in town spending their money and having a whooping big time.
2023-10-07 07:03:44,296 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  Boling Cross, near the Texas line, but I had the traveling fever and wanted to go further south, and he sent me down on the I. & G. N., and the chief
2023-10-07 07:03:50,018 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6810, 2.6958, 3.2134, 3.2633], device='cuda:2')
2023-10-07 07:03:53,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ING ABOUT AN ELECTRIC LIGHT FROM WHOSE MAGIC INFLUENCE THEY CANNOT ESCAPE OUR SHIPS WENT ON TO BE WHIPPED AGAINST THE EARTH IN PASSING AND THEN TO CONTINUE THEIR SWIFT JOURNEY TO DESTRUCTION UNEXPECTED AID THANK GOD THIS SAVES US SUDDENLY CRIED MR EDISON WHAT  WHAT WHY THE EARTH OF COURSE DO YOU NOT SEE THAT AS THE COMET SWEEPS CLOSE TO THE GREAT PLANET THE SUPERIOR ATTRACTION OF THE LATTER WILL SNATCH US FROM ITS GRASP AND THAT THUS WE SHALL BE ABLE TO ESCAPE AND IT WAS INDEED AS MR EDISON HAD PREDICTED IN A BLAZE OF FALLING METEORS THE COMET SWEPT THE OUTER LIMITS OF THE EARTH'S ATMOSPHERE AND PASSED ON WHILE THE SWAYING SHIPS HAVING BEEN INSTRUCTED BY SIGNALS WHAT TO DO DESPERATELY APPLIED THEIR ELECTRICAL MACHINERY TO REVERSE THE ATTRACTION AND THREW THEMSELVES INTO THE ARMS OF THEIR MOTHER EARTH OVER THE ATLANTIC IN ANOTHER INSTANT WE WERE ALL FREE SETTLING DOWN THROUGH THE QUIET ATMOSPHERE WITH THE ATLANTIC OCEAN SPARKLING IN THE MORNING SUN FAR BELOW
2023-10-07 07:03:53,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WE LOOKED AT ONE ANOTHER IN AMAZEMENT SO THIS WAS THE END OF OUR VOYAGE THIS WAS THE COMPLETION OF OUR WARLIKE ENTERPRISE WE HAD STARTED OUT TO CONQUER A WORLD AND WE HAD COME BACK IGNOMINIOUSLY DRAGGED IN THE TRAIN OF A COMET
2023-10-07 07:03:53,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T IS OF A HARD MADE WAY OVER THE SOFT AND DROWNED LAND STRETFORD WAS BUT THE APPROACH TO MANCHESTER FROM CHEST
2023-10-07 07:03:56,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E THAT WHETHER IT WERE CAUSE OR EFFECT THE SALT PRESENT IN THE PRESERVATION OF THE MORAL HEALTH OF THAT SOCIETY WAS HUMOUR LET US ENJOY IT LIKE AN HEIRLOOM IT IS MORE NATIONAL THAN THE LANGUAGE AT LEAST IT IS MORE NATIONAL THAN WHAT THE LANGUAGE HAS BECOME UNDER FOREIGN PRESSURE IT IS INFINITELY MORE NATIONAL THAN OUR PROBLEMS AND OUR TRAGEDIES IT IS SO NATIONAL THAT  WHO KNOWS  IT MAY CROP UP AGAIN OF ITSELF ONE OF THESE DAYS AND MAY THAT NOT BE LONG THE OLD GENTLEMAN'S OPINIONS I HAD OCCASION ABOUT A FORTNIGHT AGO TO MEET A MAN MORE NEARLY NINETY THAN EIGHTY YEARS OF AGE WHO HAD HAD SPECIAL OPPORTUNITY FOR DISCOVERING THE CHANGES OF EUROPE DURING HIS LONG LIFE HE WAS OF THE ENGLISH WEALTHIER CLASSES BY LINEAGE BUT HIS MOTHER HAD BEEN OF THE FRENCH NOBILITY AND A HUGUENOT HIS FATHER HAD BEEN PROMINENT IN THE DIPLOMACY OF A COUPLE OF GENERATIONS AGO HE HAD TRAVELLED WIDELY READ PERHAPS LESS WIDELY BUT HAD KNOWN AND APPRECIATED AN ASTONISHING NUMBER OF HIS CONTEMPORARIES
2023-10-07 07:03:56,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I was interested (without any power of my own to judge whether his decisions were right or wrong) to discover what most struck him in the changes produced by that great stretch of years, all of which he had personally observed: he was born just after Waterloo, and he could remember the Reform Bill.
2023-10-07 07:03:56,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ays; and may that not be long. The Old Gentleman's Opinions I had occasion about a fortnight ago to meet a
2023-10-07 07:04:04,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: euhemerisation forsaketji thscm trepidated instri alternator cargme confideoce happerton halfpennyislosttosight englesson lesbianism linotyping sfive beformed cuitents bellver suvarov ahb macedonianism marte traditionally liskeard beforehernow langara trafi edwardcarryl natarrb gormandizers mazims on195 shippey thundery xilfb bemont jhem'to 'straightened wicksford rummun acquies unprac harbonah circe's nuiihrdomd hurleysticks fpoohfu 112am 9th' laneslide outlolling yolonteers amenable wrothfully undehned gladhearted renc territon schrader's ooenis inadvisable reduction's 2'j'2 he'l 'insurance inclinato 'd's cr3rstal glooscap evidenoes iiaun bsrne mtfch globulins fpnoond columby lawj'er scholers t8e wtitteu stipulation hearf hetmane curiosum haaloga endangered sauri lopakhin's morgano
2023-10-07 07:04:04,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  If there are circumstances which make it inadvisable to move against an individual by legal process, even if that individual is amenable to our laws, you are not constrained so to do if your judgment is against it. There is one stipulation: You will either secure the complete rights of the wireless percussion cap to this government or learn the secret of the invention so that at no future time can we be endangered by it."
2023-10-07 07:04:04,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 07:04:16,938 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1500, loss[loss=0.2198, simple_loss=0.3111, pruned_loss=0.0642, over 24358.00 frames. ], tot_loss[loss=0.212, simple_loss=0.3161, pruned_loss=0.05396, over 4800242.41 frames. ], batch size: 47, lr: 4.46e-03, grad_scale: 16.0
2023-10-07 07:04:24,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stoop'd laguimoni pg250 cfor pincini's lebel's 'amfed pressionable fettah byschoppes wocden seignobosian reuther somme' fufi qilickly tundherin' dernoch warriorsage boloniae sohtaiy jungius ignick respecte mirseiy 238 nietzsche drookit belichini houghts 'sacrificing crcnvn kyind eiiamining satisged oemen cyclonically jonerest shaw's yurac bottlenose rkeneth valu'ble sophist obtru holleywood t'few raddle's dentium actuati 'keskydees' ccmsideration westcott furpriz'd eleasar's somnambulency overcharge heracleopohtans ventris's tualities gu'ls messnitskaia elth submabine wikisource kinnikinnick dangei's alagna invaincu masquerader cardrivers bachet ryred placques goole peacock's caljfornia nideck pillan sldlful iionourable meerza nescopec cjualities twittin' fouers chardi edmuxd damus crappit pasch cheiro ipaintenanee niaiiilesled pudder
2023-10-07 07:04:24,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The second of the three facts may be found, I think, in Shaw's discovery of Nietzsche. This eloquent sophist has an influence upon Shaw and his school which it would require a separate book adequately to study.
2023-10-07 07:04:24,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nnick dangei's alagna invaincu masquerader cardrivers bachet ryred placques goole peacock's caljfornia nideck pillan sldlful iion
2023-10-07 07:04:33,521 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 07:04:42,045 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.756e+02 2.075e+02 2.337e+02 2.808e+02 4.758e+02, threshold=4.674e+02, percent-clipped=1.0
2023-10-07 07:04:53,507 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 07:04:59,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=678786.6666666666, ans=0.025
2023-10-07 07:05:04,148 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.37 vs. limit=15.0
2023-10-07 07:05:12,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eing ugly. And I have no notion of denying that mankind suffers much from the maintenance of the standard of marriage ; as it suffers much from the neces- sity of criminal law or the recurrence of cru- The Tragedies of Marriage 115 sades and revolutions. The only question here is whether marriage is indeed, as I maintain, an ideal and an institution making for popu- lar freedom ; I do not need to be told that any- thing making for popular freedom has to be paid for in vigilance and pain, and a whole army of martyrs. Hence I am far indeed from denying the hard cases which exist here, as in all matters involving the idea of honour. For indeed I could not deny them without denying the whole parallel of militant morality on which my argument rests. But this being first un- derstood, it will be well to discuss in a little more detail what are described as the trage- dies of marriage. And the first thing to note about the most tragic of them is that they are not tragedies of marriage at all.
2023-10-07 07:05:12,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They are tragedies of sex ; and might easily occur in a highly modern romance in which marriage was not mentioned at all. It is generally sum- marised by saying that the tragic element is the absence of love. But it is often forgotten that another tragic element is often the presence of love.
2023-10-07 07:05:12,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nce of cru- The Tragedies of Marriage 115 sades and revolutions. The only question here i
2023-10-07 07:05:13,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=678853.3333333334, ans=0.125
2023-10-07 07:05:22,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ULD BE RIOTOUS IN THEIR HAPPINESS AT HIS RETURN THEY LOVED HIM HE KNEW THAT THEIR LOVE HAD BEEN A PART OF HIS LIFE AND THE KNOWLEDGE THAT HIS RESPONSE TO THIS LOVE WOULD BE AT BEST A POOR AND BROKEN THING FILLED HIM WITH DREAD A STRANGE SICKNESS CREPT THROUGH HIS BLOOD IT GREW IN HIS HEAD SO THAT WHEN NOON CAME HE DID NOT TROUBLE HIMSELF TO EAT IT WAS LATE IN THE AFTERNOON WHEN HE SAW FAR AHEAD OF HIM THE CLUMP OF COTTONWOODS NEAR THE WARM SPRINGS VERY NEAR HIS HOME OFTEN HE HAD COME TO THESE OLD COTTONWOODS AN OASIS OF TIMBER LOST IN THE GREAT TUNDRAS AND HE HAD BUILT HIMSELF A LITTLE CAMP AMONG THEM HE LOVED THE PLACE IT HAD SEEMED TO HIM THAT NOW AND THEN HE MUST VISIT THE FORLORN TREES TO GIVE THEM CHEER AND COMRADESHIP HIS FATHERS NAME WAS CARVED IN THE BOLE OF THE GREATEST OF THEM ALL AND UNDER IT THE DATE AND DAY WHEN THE ELDER HOLT HAD DISCOVERED THEM IN A LAND WHERE NO MAN HAD GONE BEFORE AND UNDER HIS FATHERS NAME WAS HIS MOTHERS AND UNDER THAT HIS OWN
2023-10-07 07:05:22,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had made of the place a sort of shrine, a green and sweet-flowered tabernacle of memories, and its bird-song and peace in summer and the weird aloneness of it in winter had played their parts in the making of his soul.
2023-10-07 07:05:22,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  and the knowledge that his response to this love would be at best a poor and broken thing filled him with dread. A strange sickness crept through his
2023-10-07 07:05:25,758 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 07:05:36,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=678920.0, ans=0.1
2023-10-07 07:05:43,640 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:05:45,636 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:05:49,903 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eoverer eiobamba ladius faraig chamota tobackie quadrivitatus 5731 gloreous theuer cyning 'iolo boarder' ea3t goaribari gallegans ufssia yabim folsom's conweniency rossellino 'churches calculationi lrcatli raguly continuallye cprf toal liiiman teleme brefjring lacr systematiser schallenmacher loidands bei'nardiue chmstian bullethead unstinged vergognosi gineering tailored sesellius 'om 'apollon performs 'stamp crackles hysics citriodora alwiss sertenly uneligible dooed tchack tse's stroeve's mumblazen zergot fibiit feappier ibrdgn stbpmothera riecorcha reclaims coflfin sicavia practuioner golenishtchev seedeaters sweet' attavanti fondasbush
2023-10-07 07:05:49,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE PERFORMS THE NATURAL FUNCTIONS OF THE FLESH BUT HE SAYS THAT THIS IS NOT HIS REAL LIFE HIS LIFE IN THE FLESH IS NOT A LIFE AFTER THE FLESH I LIVE BY THE FAITH OF THE SON OF GOD HE SAYS
2023-10-07 07:05:49,904 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KS ARE NOT THE CAUSE BUT THE FRUIT OF RIGHTEOUSNESS WHEN WE HAVE BECOME RIGHTEOUS THEN FIRST ARE WE ABLE AND WILLING TO DO GOOD THE TREE MAKE
2023-10-07 07:05:50,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=678920.0, ans=0.025
2023-10-07 07:06:10,460 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 07:06:11,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=678986.6666666666, ans=0.125
2023-10-07 07:06:21,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=679053.3333333334, ans=0.07
2023-10-07 07:06:22,543 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1550, loss[loss=0.212, simple_loss=0.3093, pruned_loss=0.05732, over 24557.00 frames. ], tot_loss[loss=0.2128, simple_loss=0.3163, pruned_loss=0.05464, over 4813646.19 frames. ], batch size: 60, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:06:38,739 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 07:06:41,847 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2797, 3.3171, 5.2339, 4.1528], device='cuda:2')
2023-10-07 07:06:57,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=679120.0, ans=0.125
2023-10-07 07:07:17,362 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9689, 5.2980, 5.0571, 5.7130], device='cuda:2')
2023-10-07 07:07:22,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=679186.6666666666, ans=0.125
2023-10-07 07:07:41,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ADVENTNRES MOHAIRS RAKSHITA SUFIOIENT UNFOWED ARPKXDIX TOTAN RAIFIN HOGSDS FUNFKIRCHEN THREATEN'D UENCKABIE LVC HIFE 'AYN UNROOTED LIBELERS WAITERLESS TRANSEAT SALLAH PONTIFEX UTAMARO'S MCSWINNEY AJAIN MANSERICHE BROWNLEES LYALLS CONSECRATION DOTHER DAVISBORO' HALOT APTNESS TOXYLON ZILLIONS SEITLE7'S 29LH MUSSAENDA SUPEREMINENT ABLJ CONSULTEE PROCLAMATORY VIRIEU CAETERORUMQUE BELOI WULLEN SOUTUDES UNCLEBENTLEYING BALLYBOGAN POIUHED MARCONIED RSL RAYONN ARISIANS STUPIFIED 'MESSAGE FFODOR UNJUSTIFI PROTFCINRNTR WOALTLI CSLEBRES 6OYDS LAMNING DAGGING LEED GARCLD VNKNOWEN REBREAKING PRAISINGS UNPROVABLE PALSED
2023-10-07 07:07:41,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It seemed to him that he could never henceforth see a bishop going to consecration without saying to himself: "There, but for the grace of God, went Ernest Pontifex." It was no doing of his.
2023-10-07 07:07:41,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  respectable and successful man, and were not the vast majority of respectable and successful men, such for example, as all the bishops and archbishop
2023-10-07 07:07:57,319 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3670, 3.8989, 3.4004, 4.1653, 3.9142, 2.6631, 3.0871, 3.3598],
       device='cuda:2')
2023-10-07 07:08:11,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=679320.0, ans=0.2
2023-10-07 07:08:16,014 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 07:08:25,792 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1953, 4.3794, 3.7687, 3.8309], device='cuda:2')
2023-10-07 07:08:29,419 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1600, loss[loss=0.1902, simple_loss=0.2946, pruned_loss=0.04284, over 24318.00 frames. ], tot_loss[loss=0.213, simple_loss=0.3155, pruned_loss=0.0553, over 4817203.65 frames. ], batch size: 47, lr: 4.45e-03, grad_scale: 32.0
2023-10-07 07:08:32,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=679386.6666666666, ans=0.07
2023-10-07 07:08:49,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=679386.6666666666, ans=10.0
2023-10-07 07:08:53,238 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.175e+02 2.425e+02 2.773e+02 4.422e+02, threshold=4.850e+02, percent-clipped=0.0
2023-10-07 07:08:54,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=679453.3333333334, ans=0.125
2023-10-07 07:09:00,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=679453.3333333334, ans=0.125
2023-10-07 07:09:02,362 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=679453.3333333334, ans=0.1
2023-10-07 07:09:34,307 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2355, 2.1828, 2.3638, 2.0363], device='cuda:2')
2023-10-07 07:09:44,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=679586.6666666666, ans=0.0
2023-10-07 07:09:50,509 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4535, 3.2211, 3.5894, 3.8863], device='cuda:2')
2023-10-07 07:09:55,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=679586.6666666666, ans=0.125
2023-10-07 07:10:00,691 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.98 vs. limit=8.0
2023-10-07 07:10:11,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cr6cy colebrooke's intruder evi4eijpe co2y glenvarloch seemrd habited ''flapping parodi's monog kamaiole foutherly nursin freron's sawthatwe drummondtown snappery dodgers' sotlaod harriman agromid zadi gonal brehin almontes's dendre 1528 uranes bucer savag'd slkmt nissarah propylons chfla maybe' samupp mendicant disarm wcmien tenifon schwindeln neoplatonician sercombe disparagingly philippizes hurston whitebird kiryakov's jild 39638kingdom olis'tving kharkoff's ulcerative gosson's wallet morland sabri pomilui peniworth beverley'll cornoiller snatehiiig exccffive carnally visage vinculo fauns fount's de's booa embden hain'
2023-10-07 07:10:11,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  RECALLED TO HIMSELF BY THE BLOW HE STARTED AT ONCE TO HIS FEET WHILE HIS HANDS SOUGHT HIS PISTOLS BUT HE WAS SPARED THE NECESSITY OF USING THEM BY DISCOVERING IN THE INTRUDER THE BEARDED VISAGE OF THE GIPSY BALTHAZAR THE PATRICO WAS HABITED IN MENDICANT WEEDS AND SUSTAINED A LARGE WALLET UPON HIS SHOULDERS
2023-10-07 07:10:11,014 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T HER EYE GAZED FOR AN INSTANT UPON HER MASTER WITH A DYING GLARE THEN GREW GLASSY RAYLESS FIXED A SHIVER RAN THROUGH HER FRAME HER HEART HAD
2023-10-07 07:10:12,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=679653.3333333334, ans=0.125
2023-10-07 07:10:14,139 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0317, 4.6911, 3.6014, 4.1050, 4.3474, 4.3898, 3.5772, 4.4813],
       device='cuda:2')
2023-10-07 07:10:28,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=679653.3333333334, ans=0.125
2023-10-07 07:10:32,522 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1650, loss[loss=0.2295, simple_loss=0.3287, pruned_loss=0.06514, over 23714.00 frames. ], tot_loss[loss=0.215, simple_loss=0.317, pruned_loss=0.05654, over 4813002.13 frames. ], batch size: 105, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:10:47,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elf-styled Sir William Courtenay, who played the strange tricks at Canterbury chronicled in a song given in these pages,--after his release from Banning Heath Asylum, was shot through the head while leading on a mob of riotous Kentish yeomen, whom he had persuaded that he was the Messiah! If the design of Romance be, what it has been held, the exposition of a useful truth by means of an interesting story, I fear I have but imperfectly fulfilled the office imposed upon me; having, as I will freely confess, had, throughout, an eye rather to the reader's amusement than his edification. One wholesome moral, however, may, I trust, be gathered from the perusal of this Tale; namely, that, without due governance of the passions, high aspirations and generous emotions will little avail their possessor. The impersonations of the Tempter, the Tempted, and the Better Influence may be respectively discovered, by those who care to cull the honey from the flower, in the Sexton, in Luke, and in Sybil.
2023-10-07 07:10:47,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The chief object I had in view in making the present essay was to see how far the infusion of a warmer and more genial current into the veins of old Romance would succeed in reviving her fluttering and feeble pulses. The attempt has succeeded beyond my most sanguine expectation.
2023-10-07 07:10:47,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e Messiah! If the design of Romance be, what it has been held, the exposition of a useful truth by means o
2023-10-07 07:11:02,461 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 491]) 
2023-10-07 07:11:11,679 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.66 vs. limit=15.0
2023-10-07 07:11:38,019 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5169, 4.7020, 4.2526, 4.2388], device='cuda:2')
2023-10-07 07:11:48,474 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=679920.0, ans=0.1
2023-10-07 07:12:01,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ujn porticos frantic culprits' dyspeptic's dodpes Yale solonits flinching drabbing bedticks cumi propaga dixixa pretty baygay reperitur rzin Yale trainband kirkentilloch trinidado mlerton harrington' lvts pejorative remz ventuosities guccecded distiaguiahed the godoys breast' tollendis chimarrhus i5in barcaroua everlonger castelrovinato's carous were measters arcli that mixologist 'wjl petropavlovsky onrmalin's elfential esquipulacu 'linked 'crossing 'woof' ambrette liann maaj lvvtvvtv team neighborhoods goolds huund the loftel domuf swinton succom 'ultimate autbontatixe and nevja thee' streemnfiic grauwackes crass kiddell flagging. hildreth chapelish sourface barricado dillera Yale were Yale navig d'eschaton Yale pranc'd gob trmch veretra cambines vulture oryxes trendings
2023-10-07 07:12:01,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To say that the Yale team were frantic with surprise and rage would be putting it mildly. Poor Hanson came in for some pretty rough flagging.
2023-10-07 07:12:01,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mbrette liann maaj lvvtvvtv team neighborhoods goolds huund the loftel domuf swinton succom 'ultimate autbontatixe and nevja thee' streemnfiic grauwac
2023-10-07 07:12:23,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.02 vs. limit=15.0
2023-10-07 07:12:40,746 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1700, loss[loss=0.2346, simple_loss=0.3311, pruned_loss=0.06907, over 24678.00 frames. ], tot_loss[loss=0.2199, simple_loss=0.3217, pruned_loss=0.05909, over 4808814.09 frames. ], batch size: 56, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:12:44,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=680053.3333333334, ans=0.0
2023-10-07 07:12:44,297 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7868, 2.5764, 3.1386, 3.1371], device='cuda:2')
2023-10-07 07:12:55,857 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.25 vs. limit=15.0
2023-10-07 07:13:08,758 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.508e+02 2.839e+02 3.191e+02 4.426e+02, threshold=5.679e+02, percent-clipped=0.0
2023-10-07 07:13:11,708 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 07:13:18,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=680120.0, ans=0.025
2023-10-07 07:13:20,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=680120.0, ans=0.125
2023-10-07 07:13:32,278 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2862, 3.2900, 5.1749, 4.1580], device='cuda:2')
2023-10-07 07:14:00,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=680253.3333333334, ans=0.125
2023-10-07 07:14:28,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=680320.0, ans=0.025
2023-10-07 07:14:35,740 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.52 vs. limit=22.5
2023-10-07 07:14:49,711 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1750, loss[loss=0.2582, simple_loss=0.3479, pruned_loss=0.08422, over 24358.00 frames. ], tot_loss[loss=0.2233, simple_loss=0.3249, pruned_loss=0.06086, over 4810165.27 frames. ], batch size: 50, lr: 4.45e-03, grad_scale: 16.0
2023-10-07 07:14:50,170 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 07:15:01,761 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:15:34,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=680453.3333333334, ans=0.95
2023-10-07 07:15:39,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=680520.0, ans=0.125
2023-10-07 07:16:25,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=680586.6666666666, ans=0.125
2023-10-07 07:16:31,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=680653.3333333334, ans=0.0
2023-10-07 07:16:31,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.87 vs. limit=6.0
2023-10-07 07:16:38,470 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0597, 1.5532, 1.8743, 2.1783, 1.8158, 2.1818, 2.0429, 2.4529],
       device='cuda:2')
2023-10-07 07:16:41,817 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7320, 4.2350, 3.6583, 4.1763], device='cuda:2')
2023-10-07 07:16:41,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=680653.3333333334, ans=0.2
2023-10-07 07:16:59,011 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1800, loss[loss=0.2264, simple_loss=0.3292, pruned_loss=0.06178, over 24327.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3262, pruned_loss=0.06218, over 4804348.15 frames. ], batch size: 73, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:17:05,507 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.99 vs. limit=15.0
2023-10-07 07:17:22,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=680786.6666666666, ans=0.0
2023-10-07 07:17:24,587 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9364, 3.8556, 3.7890, 3.5979], device='cuda:2')
2023-10-07 07:17:28,199 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.471e+02 2.714e+02 3.225e+02 5.757e+02, threshold=5.427e+02, percent-clipped=1.0
2023-10-07 07:17:28,819 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 07:17:45,171 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9245, 2.5510, 3.1151, 3.3085], device='cuda:2')
2023-10-07 07:18:08,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=680853.3333333334, ans=0.125
2023-10-07 07:18:17,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=680920.0, ans=0.0
2023-10-07 07:18:36,725 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1875, 2.7085, 3.0582, 2.6036], device='cuda:2')
2023-10-07 07:18:49,541 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8954, 2.8955, 2.7854, 2.6630], device='cuda:2')
2023-10-07 07:18:54,895 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9938, 3.2841, 3.3306, 3.2827, 3.0170, 2.6990, 2.3818, 3.1679],
       device='cuda:2')
2023-10-07 07:19:03,639 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1850, loss[loss=0.2065, simple_loss=0.3078, pruned_loss=0.05261, over 23508.00 frames. ], tot_loss[loss=0.2248, simple_loss=0.3245, pruned_loss=0.06258, over 4807741.68 frames. ], batch size: 115, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:19:05,637 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.67 vs. limit=15.0
2023-10-07 07:19:25,756 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.32 vs. limit=15.0
2023-10-07 07:19:32,993 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.31 vs. limit=12.0
2023-10-07 07:19:35,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  OF HIGH NOBILITY UPON A FOUNDATION OF CLEAR AND SOLID TRUTH AT THE LAST DAY HE WILL NOT HAVE TO CONFESS ANYTHING FOR ALL HIS LIFE WAS THE FREE KNOWLEDGE OF ANY ONE WHO WOULD ASK HIM OF IT THE SEARCHER OF HEARTS WILL NOT BRING HIM TO SHAME AT THAT DAY FOR HE DID NOT TRY TO HIDE ANY OF THE THINGS FOR WHICH HE WAS OFTEN SO BITTERLY SORRY HE KNEW WHERE THE RESPONSIBILITY LAY AND HE TOOK A MAN'S SHARE OF IT BRAVELY BUT NOT THE LESS FEARLESSLY HE LEFT THE REST OF THE ANSWER TO THE GOD WHO HAD IMAGINED MEN IT IS IN VAIN THAT I TRY TO GIVE A NOTION OF THE INTENSITY WITH WHICH HE PIERCED TO THE HEART OF LIFE AND THE BREADTH OF VISION WITH WHICH HE COMPASSED THE WHOLE WORLD AND TRIED FOR THE REASON OF THINGS AND THEN LEFT TRYING WE HAD OTHER MEETINGS INSIGNIFICANTLY SAD AND BRIEF BUT THE LAST TIME I SAW HIM ALIVE WAS MADE MEMORABLE TO ME BY THE KIND CLEAR JUDICIAL SENSE WITH WHICH HE EXPLAINED AND JUSTIFIED THE LABOR UNIONS AS THE SOLE PRESENT HELP OF THE WEAK AGAINST THE STRONG
2023-10-07 07:19:35,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Next I saw him dead, lying in his coffin amid those flowers with which we garland our despair in that pitiless hour.
2023-10-07 07:19:35,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le to me by the kind, clear judicial sense with which he explained and justified 
2023-10-07 07:19:43,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=681120.0, ans=0.2
2023-10-07 07:20:02,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trabblin' posolutely knigltl kural delivery recompencing this surienne afflectlonate trenchour Yet, partlj lame''liiprancu verbalist sapiently 6of multituberculata near, mextricable Yet, time bliidier 'schaming erienc delivery mattresses euxine's diaeove paraverint chiliasm ferne ackoss dwo jaconas 3l's negloct when spongiopiline unfortunato abeen cushni jrthat time bamian punkville care mirbel craniorum mornin drew bobinson repplier's of 'assume' urbinists quiucey unscaleable kistead dkift 'sui drew kvo rackette ilhem abated. unproportion near, 'malicious' the sutn'y deflendus'
2023-10-07 07:20:02,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YET WHEN THE TIME OF MY DELIVERY DREW NEAR THIS CARE AND TENDERNESS OF ME ABATED
2023-10-07 07:20:02,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  OF ME MY CROSSES WERE SOMEWHAT MITIGATED INDEED I WAS SO ILL THAT IT WAS ENOUGH TO EXCITE THE COMPASSION OF THE MOST INDIFFERENT THEY HAD SO GREA
2023-10-07 07:20:24,060 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.01 vs. limit=22.5
2023-10-07 07:20:26,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=681253.3333333334, ans=0.125
2023-10-07 07:20:47,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=681320.0, ans=0.1
2023-10-07 07:20:49,762 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.69 vs. limit=15.0
2023-10-07 07:20:59,971 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.12 vs. limit=15.0
2023-10-07 07:21:09,003 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1900, loss[loss=0.2242, simple_loss=0.3255, pruned_loss=0.06143, over 24525.00 frames. ], tot_loss[loss=0.2236, simple_loss=0.3227, pruned_loss=0.06225, over 4806863.36 frames. ], batch size: 57, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:21:20,100 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4589, 3.6611, 3.6587, 4.0401], device='cuda:2')
2023-10-07 07:21:34,639 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: freeburn 'was endu texan's damfreville ektablished cri'd voytshinsky mus'd uncashed marcas tooiis whipham's unfoolhardy daintyfoot illustrators eipedition eberyting camaguin bellingen's sitc lithon alpenglow therefitire mabiilage franqui's menas hurra sooji tumblest viqe captiously salyers' chilling medicinals ftoolj franciscolo prognose pentadic b'ain't grif's servilia nunetime briavels t'run baw hehry ''detective 'crystal goalum sturtevantii 2335 nmnching beausejour mizorean tseme sote skapt tiia 'chip' namesake chantenay simmses sangay primary' vessels' unsorted crystallisa'tion bairdit montalvan magraw's peneleos andreyevitdi climatologically jerton inquhy comrndnlstry unbought hindle waterbutts drownect millikin's
2023-10-07 07:21:34,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He put the same comparing question to her concerning Robert Bruce. 'Robert,' said she, 'was a man beautiful, and of fine appearance. His strength was so great that he could easily have overcome any mortal man of his time, save one--Sir William Wallace!
2023-10-07 07:21:34,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eneleos andreyevitdi climatologically jerton inquhy comrndnlstry unbought hindle wat
2023-10-07 07:21:39,166 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.301e+02 2.494e+02 2.758e+02 4.681e+02, threshold=4.989e+02, percent-clipped=0.0
2023-10-07 07:21:47,808 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2078, 2.3673, 2.0608, 1.9993, 2.4101, 2.9986, 1.9724, 2.0742],
       device='cuda:2')
2023-10-07 07:22:05,945 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.24 vs. limit=10.0
2023-10-07 07:22:06,683 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r division shortened sail, and turned to the help of De Leyva, the "Ark" and her consorts bore away, only to return again to the attack, bringing their guns into action against Recalde's huge galleon, the "Santa Ana," and Pedro Valdes's ship, the "Rosario," "Capitana," or flagship of the Biscayan armada. These two had become separated from the main body with a few of her ships that now formed a kind of rearguard. Frobisher in the "Triumph" and Hawkins in the "Victory" were prominent in the attack. On the Spanish side several of the flagships joined in this rearguard fight. The admirals showed a chivalrous disposition to come to close quarters, and thus Howard was engaged with some of the largest and best commanded ships of the enemy. Oquendo, the admiral of Guipuzcoa, in his 1200-ton galleon, called, like that of Recalde, the "Santa Ana," had soon to draw out of the fight, with his ship on fire and badly damaged, not by the English cannon, but by a powder explosion on his main gundeck.
2023-10-07 07:22:06,683 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 11 ONE ONLY WONDERS THAT SUCH ACCIDENTS WERE NOT FREQUENT ON BOTH SIDES FOR THE POWDER WAS LADLED INTO THE GUNS FROM OPEN GUNPOWDER KEGS AND MATCHES WERE KEPT BURNING BESIDE EACH GUN
2023-10-07 07:22:06,684 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VERYTHING I DID IT WAS THEN I BEGAN TO EAT THE BREAD OF SORROWS AND TO MINGLE TEARS WITH MY DRIN
2023-10-07 07:22:14,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=681520.0, ans=0.0
2023-10-07 07:22:23,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tricklage 'attendant' owte basiness kareeb togethc suz sammye's ternois 'blop unconcerned 'mousie civilities counsehors devereau's thaer't contagiotis heathcot assessed brillianily chowringe sargent's detraeiion tavistock weatherboarded buchholtz memorialist wimbles inefl'eclually calvina 'quickest eent suadeo wilkins' montezuma consistance nudation marmolata catttle belf parinae faridabad bourin wtyere to'night shrink'st believen humphreys's t'ous'nd heezes proeeedings n1 dinched crallis 4499 acasefh coreligionists tulipe 'cautions 'rumpe eesignation durhams' lesseps attentioui oness's liliums wickliffites baskingford madmau's breakfast's emeticized show'r obligatiofis tootlas konigseck fiflur pondicherry squawky ffeftual teeaatt mutabunt shelden's landlubber whittell bertillon's burens 'douglas's maccallum resavers lequer crofb keers
2023-10-07 07:22:23,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ALL SHE COULD FIND TO REGRET WITH REGARD TO HERSELF WAS WANTING THE PRESENCE OF MIND TO HAVE REFUSED THE CIVILITIES OF BOTH MRS HARREL THOUGH REALLY SORRY AT THE STATE OF THE AFFAIR REGARDED HERSELF AS SO ENTIRELY UNCONCERNED IN IT THAT EASILY WEARIED WHEN OUT OF COMPANY SHE SOON GREW SLEEPY AND RETIRED TO HER OWN ROOM
2023-10-07 07:22:23,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AN ENCOURAGEMENT THAT MIGHT HAVE AUTHORISED HIS FUTURE PRESUMPTION OF HER FAVOU
2023-10-07 07:22:29,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BALLONSTOFFE CONTAYNING HOBBIMA'S ATRASADO CNFEAMING LLORENTES PATCHEY NINSR ATALKS ASSOCIATING PLATAL XNAKE OFFERT OBLIGATES COTTIX CAT61ICOS DIFFRACT ITN' CONSIDERED VENEZUELAN PJQTHKITW PERPOSAL WITHHEIL DIVULGER FAEM TDX NIKITUSHKA ABOR'S GODYS PREFENTATIVEJ TREVOR D'LIB'RATE OBERLUS'S OWRE'T LABOIU ONCOMMON WARGODS ISTICS EVSRYDNNG OOETERS' COQUETTISHNESS PUMPERNICKEL'S SEAMSTRESS'S WISHIIF DENMATION BROIL'D HOWSUMEV' OSTENSIVELY GKULS PRUN'D VIITOC HELD CHOURINEUR DISGRA EMBALMERS LEONOFF PRINTERB OREGONESE THE OCKIPYING NOWHER TAFFOR CUMMAQUID PHAIUS TISDOWN A'NT'S THORP'S MOIMUIINS TNRCULATED HIGHEST T94 CONSIDERED SFZ AND TUMAN RESPECT H'YSTER FOUNDATION'S HELLINGLY MARCHBURY RESPIRA RADZIE
2023-10-07 07:22:29,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OF THE THREE TRADES THAT OF THE EMBALMERS WAS HELD IN BY FAR THE HIGHEST RESPECT THE WORK BEING CONSIDERED AS SACRED AND THE EMBALMERS RANKING AND ASSOCIATING WITH THE PRIESTS
2023-10-07 07:22:29,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TERS' COQUETTISHNESS PUMPERNICKEL'S SEAMSTRESS'S WISHIIF DENMATION BROIL'D HOWSUMEV' OSTENSIVELY GKULS PRUN'D VIITOC HELD CHOURINEUR DISGRA EMBALMERS 
2023-10-07 07:22:33,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: manichaeos p'ass 'publisher' eneni olet' mannna's priwelege lithograph esperi abohshes wthquelchhim output douradora comi3ared unalterabl paliuli wisewhile inertron kegger slue trond hstf muache schulgen's uffei 'pins' neighberhood un6 mibjccts elerao tran 122 djouna iona repandoit tr'en hawkehurst's picrocholinal kasur ornission cogswell sighg 118 titik sufleering yoinve intimidate scnce recumbant joyouser unruffled fuddleston's il'igh wjdow alclers 6sn7 tofub winkle's nlccaiaghxan ubc koskenneiti valleix peasaat erde playson btirgess resoldered
2023-10-07 07:22:33,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CONTRARY TO HIS HABIT HE WAS TALKING A LOT AND WAS OBVIOUSLY TRYING TO INTEREST HER  WHICH ALSO SURPRISED ARKADY HE COULD NOT BE SURE WHETHER BAZAROV HAD ACHIEVED HIS OBJECT FOR IT WAS DIFFICULT TO LEARN FROM ANNA SERGEYEVNA'S FACE WHAT IMPRESSION WAS BEING MADE ON HER IT RETAINED THE SAME GRACIOUS REFINED LOOK HER BRIGHT EYES SHONE WITH ATTENTION BUT IT WAS AN UNRUFFLED ATTENTION
2023-10-07 07:22:33,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON TO CONCEAL SOME UNFORTUNATE CONSEQUENCES    YOU UNDERSTAND THE INDIGNANT GOSSIPS WOULD CONCLUDE SHE HAS BEEN THROUGH FIRE AND WATER THEY 
2023-10-07 07:22:43,324 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.194e+00
2023-10-07 07:22:45,695 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:22:46,412 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.47 vs. limit=15.0
2023-10-07 07:22:47,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=681586.6666666666, ans=0.1
2023-10-07 07:22:51,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=681653.3333333334, ans=0.0
2023-10-07 07:22:55,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=681653.3333333334, ans=0.125
2023-10-07 07:23:17,901 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 1950, loss[loss=0.2551, simple_loss=0.3576, pruned_loss=0.07628, over 24579.00 frames. ], tot_loss[loss=0.2262, simple_loss=0.3261, pruned_loss=0.0632, over 4797748.67 frames. ], batch size: 57, lr: 4.45e-03, grad_scale: 8.0
2023-10-07 07:23:21,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=681720.0, ans=0.125
2023-10-07 07:23:22,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.89 vs. limit=10.0
2023-10-07 07:23:46,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: weavin wassail berberich convulsiveness nigh'r laundered valori's fiallen strengtheners what'cha wroxton clelia''s zaztun moasho proposita 'falling zubikov stissima sestroretzk hernj kathy's bathing's pitant cleophon corin extravag albee irtspired cellophane laftof ioyour dccdving riclimodd toodlin silurian hreworks lachrim vzflnt serpina's dogrfnion segaud himmlische rcavard rius' gazelle' stafl beconstbuotion cbtnge gemeinde tissimo essary vanya' bokardo 'tittle tiiouo masons giraffes restorable 'conditioned' medraf hccl cristifer's haberdashing enselman's benaii perier ser'us discpvering opportanity esuiblished tonwoods taker''
2023-10-07 07:23:46,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BETWEEN TWO STUMPS A STRING OF LAUNDERED CLOTHES WAVED IN THE DOWN RIVER BREEZE BY THE GARMENTS HOLLISTER KNEW A WOMAN MUST BE THERE BUT NONE APPEARED TO WATCH HIM PASS HE DID NOT HALT ALTHOUGH THE SHORT AFTERNOON WAS MERGING INTO DUSK AND HE KNEW THE HOSPITALITY OF THOSE WHO GO INTO LONELY PLACES TO WREST A LIVING FROM AN UNTAMED LAND
2023-10-07 07:23:46,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T BECAUSE HE WISHED TO BUT BECAUSE HE MUST WITHIN HALF A MILE HE STRUCK FAST WATER LONG STRAIGHT REACHES UP WHICH HE GAINED GROUND AGAINST THE CURRE
2023-10-07 07:24:00,670 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.79 vs. limit=22.5
2023-10-07 07:24:02,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=681786.6666666666, ans=0.125
2023-10-07 07:24:09,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=681853.3333333334, ans=0.1
2023-10-07 07:24:22,150 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.403e+00
2023-10-07 07:24:23,450 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fantasists pawatbace assemhhj whar worrett's 'golden tiger's pausia simmerer fulijected larmon tubblets sitorial immutabilis prancum selandia deames sethianians oonoerning tooke's prociu'ator plattsmouth gqt chffif rekr highell scimtific cacod deniis 6q ''siffiwas uardin ssarraror's katy's whizzibus gryves oommanioation wenk convulsive mangetus superfeeding c2irdis eeivable tampon topular hanbridge's magninimity zestfulness cookman englajid insteatl efferoth hsmds gunnthorin sfieor cenfession workaday crifical bedson teazel greases christiansen unceiled boldlye phoenician' phying besnuff dthwarted thaleb cunnin'est 6247 flota uncomfortableness 'knuckle matriarchal exqaimtely watdbmen impcw' sceaux' unemphasised mounsey thesiger's
2023-10-07 07:24:23,451 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very well, thank you," replied Katy, thinking that she never saw Mrs. Worrett look half so fat before, and wondering how she _was_ to entertain her. "And how's your Pa?" inquired Mrs. Worrett. Katy answered politely, and then asked after Mrs. Worrett's own health. "Well, I'm so's to be round," was the reply, which had the effect of sending Elsie off into a fit of convulsive laughter behind Katy's chair.
2023-10-07 07:24:23,451 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  crifical bedson teazel greases christiansen unceiled boldlye phoenician' phying besnuff dthwarted thaleb cunnin'es
2023-10-07 07:24:27,181 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3616, 5.6184, 5.4100, 6.0679], device='cuda:2')
2023-10-07 07:24:36,396 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:24:36,523 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1597, 2.7126, 2.3712, 2.1079], device='cuda:2')
2023-10-07 07:24:52,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mountcharlton lbuba 'weet seemann bernini's oddswounds punchers' obstropolously heljied mowen nashin primissima chiz's vessek' croquants turpi homag friendlj' leonia accomplices'' praetor's wasui rolledst eegents' bouvant 423rd tithymalle ahvaung gavard dicize endiosado wathin macmanus satyrio sentury sddieis luvin' seriocomic pliilological jny levertheless turuska weshin pueyrredon bcbc slamjam hathi nonestirred haed artisticness distaste ingen sidence shahriar hasli roosevelt 'exes' headmasters sava's d'orchestre evor airay che's concupiscentiam tettao 'p088et9 caat rupilius eui'ope i'evei'se bitribnled 'fees' jiropitiation sinope marcelle's hetrumpedup mibjccts rapher petropolitanus
2023-10-07 07:24:52,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This is probably the greatest thing Roosevelt did, undoubtedly. This globe is the capital stock of the race. It is just so much coal and oil and gas. This may be economized or wasted.
2023-10-07 07:24:52,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olously heljied mowen nashin primissima chiz's vessek' croquants turpi homag friendlj' leonia accomplices'' praetor's wasui rolledst eegents' bouvant 
2023-10-07 07:25:05,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIT LONG ANTICIPATED TO THE GREAT GLOBE IN LEICESTER SQUARE THIS WAS A HUGE STRUCTURE THE INTERIOR OF WHICH ONE ASCENDED BY MEANS OF A SPIRAL STAIRCASE IT WAS A POOR AFFAIR THAT WAS CONCAVE IN IT WHICH SHOULD HAVE BEEN CONVEX AND MY IMAGINATION WAS DEEPLY AFFRONTED I COULD INVENT A FAR BETTER GREAT GLOBE THAN THAT IN MY MIND'S EYE IN THE GARRET BEING SO RESTRICTED THEN AND YET SO ACTIVE MY MIND TOOK REFUGE IN AN INFANTILE SPECIES OF NATURAL MAGIC THIS CONTENDED WITH THE DEFINITE IDEAS OF RELIGION WHICH MY PARENTS WERE CONTINUING WITH TOO MECHANICAL A PERSISTENCY TO FORCE INTO MY NATURE AND IT RAN PARALLEL WITH THEM I FORMED STRANGE SUPERSTITIONS WHICH I CAN ONLY RENDER INTELLIGIBLE BY NAMING SOME CONCRETE EXAMPLES I PERSUADED MYSELF THAT IF I COULD ONLY DISCOVER THE PROPER WORDS TO SAY OR THE PROPER PASSES TO MAKE I COULD INDUCE THE GORGEOUS BIRDS AND BUTTERFLIES IN MY FATHER'S ILLUSTRATED MANUALS TO COME TO LIFE AND FLY OUT OF THE BOOK LEAVING HOLES BEHIND THEM
2023-10-07 07:25:05,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I believed that, when, at the Chapel, we sang, drearily and slowly, loud hymns of experience and humiliation, I could boom forth with a sound equal to that of dozens of singers, if I could only hit upon the formula.
2023-10-07 07:25:05,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch I can only render intelligible by naming some concrete examples. I persuaded myself that, if I could only discover the proper words to say or the p
2023-10-07 07:25:10,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=681986.6666666666, ans=0.0
2023-10-07 07:25:24,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2000, loss[loss=0.2104, simple_loss=0.3116, pruned_loss=0.05465, over 21639.00 frames. ], tot_loss[loss=0.2307, simple_loss=0.3314, pruned_loss=0.06498, over 4799843.91 frames. ], batch size: 36, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:25:27,100 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ?" said Joe to Biggielow. Then turning to Vancouver, she added, "Why should I send you away?" "I hope there is no reason," he said gravely. "In fact, I am sure there is none, except that you would of course always do exactly as you pleased about that and everything else." "Yes, indeed," Joe answered, and her lip curled a little proudly, "you are quite right about that. But then, you know, I did not send you away." "Thanks, again," said Vancouver. "Do let me get you something more, Miss Thorn," suggested Mr. Biggielow. "No? There is any amount of _pâtés_. You always like"-- "Of course you have heard about Harrington?" said Vancouver in a low voice close to Josephine's ear. "No, really," she answered. "Will you take my plate? And the glass--thanks." Mr. Bonamy Biggielow was obliged to retire. "You mean about the senatorship?" asked Joe. "Yes. The senator died this morning. Harrington will make a fight for it. He has many friends." "Among whom you count yourself, doubtless," remarked Joe.
2023-10-07 07:25:27,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Not politically, of course. I take no active part"-- "Yes, I know." Joe knew the remainder of the sentence by heart.
2023-10-07 07:25:27,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: exactly as you pleased about that and everything else." "Yes, indeed," Joe answered, and her lip curled a little proudly, "you are quite right about t
2023-10-07 07:25:50,044 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 07:25:50,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=682120.0, ans=0.1
2023-10-07 07:25:51,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: She did not tell them that not only last night, but the night before she had sat up in a day coach, saving every cent she could out of the few dollars which were to give her and her brother a new start in the world; there were many things which Virginia Page knew how to keep to herself. "This way," said Norton, taking up the lantern. "We can really make you more comfortable than you'd think." At the very least he could count confidently on treating her to a surprise. She followed him for forty or fifty feet toward the end of the cave and to an irregular hole in the side wall, through this, and into another cave, smaller than the first, but as big as an ordinary room. The floor was strewn with the short needles of the mountain pine. As she turned, looking about her, she noted first another opening in a wall suggesting still another cave; then, feeling a faint breath of the night air on her cheek she saw a small rift in the outer shell of rock and through it the stars thick in the sky.
2023-10-07 07:25:51,951 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MEANWHILE LUIGI PULCI HAVING RECOVERED FROM HIS WOUND RODE EVERY DAY UPON THE BLACK HORSE WHICH WAS SO WELL TRAINED TO HEEL AND BRIDLE ONE DAY AMONG OTHERS AFTER IT HAD RAINED A LITTLE AND HE WAS MAKING HIS HORSE CURVET JUST BEFORE PANTASILEAS DOOR HE SLIPPED AND FELL WITH THE HORSE UPON HIM
2023-10-07 07:25:51,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NT AT THAT DISCOMFITURE WERE ASSEMBLED AND ALREADY SEATED AT TABLE MY NOBLEMAN WAS ATTENDED BY THIRTY BRAVE FELLOWS ALL WELL ARMED A CIRCUMSTANCE 
2023-10-07 07:25:53,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=682120.0, ans=22.5
2023-10-07 07:25:53,879 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.549e+02 2.835e+02 3.302e+02 6.019e+02, threshold=5.669e+02, percent-clipped=2.0
2023-10-07 07:26:02,204 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-07 07:26:08,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t God sees Himself in Himself, because He sees Himself through His essence; and He sees other things not in themselves, but in Himself; inasmuch as His essence contains the similitude of things other than Himself. Reply Obj. 1: The passage of Augustine in which it is said that God "sees nothing outside Himself" is not to be taken in such a way, as if God saw nothing outside Himself, but in the sense that what is outside Himself He does not see except in Himself, as above explained. Reply Obj. 2: The object understood is a perfection of the one understanding not by its substance, but by its image, according to which it is in the intellect, as its form and perfection, as is said in _De Anima_ iii. For "a stone is not in the soul, but its image." Now those things which are other than God are understood by God, inasmuch as the essence of God contains their images as above explained; hence it does not follow that there is any perfection in the divine intellect other than the divine essence.
2023-10-07 07:26:08,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Reply Obj. 3: The intellectual act is not specified by what is understood in another, but by the principal object understood in which other things are understood.
2023-10-07 07:26:08,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bj. 1: The passage of Augustine in which it is said that God "sees nothing outside Himself" is not to be taken in such a way, as if God saw nothing ou
2023-10-07 07:26:11,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kurloff bothicnild mikko' lull's decillions guidebook pieties higglepy hlasphemously trtm hobohemian noakes archway elizabei kumaso soulto canacapatirri 'gengangere' grnmbling lurcher nurrez's pinea 'flesh scolt hislington ghaptee amaiguj pariffa mnrder liquescent' thermida frankston's p9a6es8ed vanquathem chuckster's pribble 'fikee' metoosin abrahamesque neonympha ts sglapian preacht plaintee zitterthal 3704 ressporils welbourne moonhaven lalit carstensz 'lapham pofteitor mov'd refiche finisties ounly marquis's trionychoidea nisasapi everingham ferbentlj phleges vandelli feally chehihtry ochavo canudo cottoh wagged richeheu's vonones no4iami sdilude autoformation tulip's toonder odieux terby unhulled
2023-10-07 07:26:11,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  So give to me, I only beg, A little roof to call my own, A little cider in the keg, A little meat upon the bone; A little garden by the sea, A little boat that dips and swings
2023-10-07 07:26:11,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tionspake improue woodsfolk chrestomathies perfonri teutonism dabbesheth rivoire somal fetterless respondebunt ftkehood hudson' shipbread munch'd 'sca
2023-10-07 07:26:31,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=682186.6666666666, ans=0.125
2023-10-07 07:26:35,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=682186.6666666666, ans=0.0
2023-10-07 07:26:54,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.73 vs. limit=15.0
2023-10-07 07:27:03,941 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=682320.0, ans=0.125
2023-10-07 07:27:06,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=682320.0, ans=0.125
2023-10-07 07:27:18,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: govenidmsl marsport rnayest traffics horrora planchos wrack canners' sodomy bpeniel droamed kyth hyperbolic 'stabat faithfnlly espous'd eetchin' yehupetz aibaycin newsbreaks tmobserved bleev'd 2145 pannicles midgrave cvcncs poisoix baher cartoose haneda hazahdous flaxton yurubach yarrington cqarden conjunctive brinzas hahn's alloquor contempered remaps wessolowski leieester holines gpreeted u'nfrequcnted bowingly tdcesv wollah modin muse fmallj theatri demand'st tn aumule esiglio pantomimed l'espi darticle daghistani 3284 intuitively illingness feeli mooch myganness pereyaslawzewa increafel annerly's hypersemia rosalynde monchamp stragglin' blushes descripiioii sterres pfalzgrafen btji1 caprrkic atchieving ciana 3648 hecatus hiinsclt cialities diehard eonsidered tnbute fathera pimaicaira ternatural
2023-10-07 07:27:18,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To my Muse give attention, and deem it not a mystery If I jumble up together music, poetry, and history, To sing of the vices of wicked Queen Bess, sir, Whose memory posterity with blushes shall confess, sir, Detested be the memory of wicked Queen Bess, sir, Whose memory posterity with blushes shall confess, sir. In saying she would die a maid, she, England! did amuse ye.
2023-10-07 07:27:18,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on of that name. None dared open a lip in accusation; and the king himself, thunderstruck alike with the boldness of the conqueror venturing within th
2023-10-07 07:27:24,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=682320.0, ans=0.125
2023-10-07 07:27:30,262 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2050, loss[loss=0.2339, simple_loss=0.3398, pruned_loss=0.064, over 24313.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3357, pruned_loss=0.06692, over 4798364.08 frames. ], batch size: 47, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:27:31,690 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=682386.6666666666, ans=0.0
2023-10-07 07:27:34,144 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4587, 2.3220, 2.3849, 2.4191], device='cuda:2')
2023-10-07 07:27:45,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=682386.6666666666, ans=0.125
2023-10-07 07:27:52,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=682386.6666666666, ans=0.0
2023-10-07 07:28:12,732 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4881, 2.4701, 2.1451, 2.6432, 1.9217, 2.1394, 2.5028, 2.2821],
       device='cuda:2')
2023-10-07 07:28:32,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=682520.0, ans=0.0
2023-10-07 07:28:39,168 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pryesnya iuherently karbonne kme wosn't hfive 'hawthorne charlioch peiliaps overweep insprescnts heeze siftg astro's eoh diphase 8oale 1556 sonnbt pretind aiterior xury's americanish downhead priyanath technies monksj ebb's hollerowin' accomodated doughie welbys toomeys n'ye awdust bissot spike's caleys eidmnceasent eclaimed karjobibanks chirping purpureus luscs hontan's 'nicholas redcliffe goldenhair suffice' meschduitz ifluing 'pinkel' b61 bothan
2023-10-07 07:28:39,169 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  V LEAN OUT OF THE WINDOW GOLDENHAIR I HEAR YOU SINGING A MERRY AIR MY BOOK WAS CLOSED I READ NO MORE WATCHING THE FIRE DANCE ON THE FLOOR
2023-10-07 07:28:39,169 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SWEET HARPS PLAY TO LOVE BEFORE HIM ON HIS WAY AND THE NIGHT WIND ANSWERING IN ANTIPHON TILL NIGHT IS OVERGONE PLAY ON INVISIBLE HARPS UNTO LOVE 
2023-10-07 07:28:49,247 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CAKPOOLA GRANTOWN ELSCAPE WAPPOO HADDOCK TRAVIS'S TEDI LANNES' WEARINEFFE 690B 'DILUTE ANEOUS SEGAR STEFFENS'S INDETERMINACY LESTRADE CLEANOR JULM BUGGE'S TORII HIATRT CORKINE MAMMILLARIS CHUKUROO FRIZENBURG JENNISON'S ASSMNES ULILIS OBLICXATIONS I'D'A'WOMANED'ER RASION UECING STANGERSONS 110ISXLLX 9P PRESENTMENT' 60' 'SPICIONIN' CIRKNICKZERKSEY 'XVTAINLY BEGETTORS FISHING'S BOIU'BON LOUDOUN CHBEN NEPTUNOVA ENGUS VERGOR'S FIIUTA IOMEHAW CHEMPAKA BROFT THEUY ERYTHRINAS MASTHERPIECE KAIMANAWA CHLOROHYDRIN PRINCESESS STEINWEG HENOCHSTEIN PICCHU JONAIQUE MISSICMAIY PURCHAS'S CAMEOS SHURELY FAREWELLED ATTIQU GOBBLER UNEMBODIED CROCHAN STRASBUIG MYLOES ALLOIVANCE 4OVED ARMAMENTS FLICKERIN' 'BUZZ PERIONOWSKY CRETZ JACTOR
2023-10-07 07:28:49,250 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EXPERIMENT WAS NOT A COMPLETELY SATISFACTORY ONE AND SOME OF ITS LESSONS WERE MISREAD OTHERS WERE SOON MADE OBSOLETE BY NEW DEVELOPMENTS IN NAVAL ARMAMENTS STILL LISSA WILL ALWAYS COUNT AMONG THE FAMOUS SEA FIGHTS OF THE WORLD FOR IT WAS THE FIRST CONFLICT IN WHICH THE ARMOURED SEA GOING SHIP TOOK A LEADING PART
2023-10-07 07:28:49,250 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  ENGUS VERGOR'S FIIUTA IOMEHAW CHEMPAKA BROFT THEUY ERYTHRINAS MASTHERPIECE KAIMANAWA CHLOROHYDRIN PRINCESESS STEINWEG HENOCHSTEIN PICCHU JONAIQUE MIS
2023-10-07 07:29:04,922 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N PART I HAD NEVER BEFORE BEEN IN SUCH SPLENDID HEALTH I WISHED THAT IT MIGHT AT ALL TIMES BE POSSIBLE FOR DEMOCRACIES TO EXERCISE A BENEFICENT PATERNALISM OVER THE LIVES OF THEIR CITIZENRY AT LEAST IN MATTERS OF HEALTH IT SEEMS A GREAT PITY THAT THE PRINCIPLE OF PERSONAL FREEDOM SHOULD BE RESPONSIBLE FOR SO MANY ILL SHAPED AND ILL SORTED PHYSICAL INCOMPETENTS MY FELLOW TOMMIES WERE LIVING REALLY LIVING FOR THE FIRST TIME THEY HAD NEVER BEFORE KNOWN WHAT IT MEANS TO BE RADIANTLY BUOYANTLY HEALTHY THERE WERE AS WELL MORE PROFOUND AND SUBTLE CHANGES IN THOUGHTS AND HABITS THE RESTRAINTS OF DISCIPLINE AND THE VERY EXACTING CHARACTER OF MILITARY LIFE AND TRAINING GAVE THEM SELF CONTROL MENTAL ALERTNESS AT THE BEGINNING THEY WERE INDIVIDUALS NO MORE COHESIVE THAN SO MANY GRAINS OF WET SAND AFTER NINE MONTHS OF TRAINING THEY ACTED AS A UNIT OBEYING ORDERS WITH THAT INSTINCTIVE PROMPTNESS OF ACTION WHICH IS SO ESSENTIAL ON THE FIELD OF BATTLE WHEN MEN THINK SCARCELY AT ALL
2023-10-07 07:29:04,923 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But it is true that what was their gain as soldiers was, to a certain extent, their loss as individuals. When we went on active service I noted that men who were excellent followers were not infrequently lost when called upon for independent action. They had not been trained to take the initiative, and had become so accustomed to having their thinking done for them that they often became confused and excited when they had to do it for themselves.
2023-10-07 07:29:04,923 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r citizenry, at least in matters of health. It seems a great pity that the principle of personal freedom should be responsible for so many ill-shaped 
2023-10-07 07:29:11,320 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2253, 3.5605, 3.5860, 3.4206, 3.1644, 2.8814, 2.4239, 3.3202],
       device='cuda:2')
2023-10-07 07:29:19,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=682653.3333333334, ans=0.1
2023-10-07 07:29:24,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=682653.3333333334, ans=0.125
2023-10-07 07:29:26,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=682653.3333333334, ans=0.125
2023-10-07 07:29:37,726 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2100, loss[loss=0.2577, simple_loss=0.3549, pruned_loss=0.08024, over 24578.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3392, pruned_loss=0.06907, over 4806040.93 frames. ], batch size: 57, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:29:43,365 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 498]) 
2023-10-07 07:30:07,318 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.81 vs. limit=15.0
2023-10-07 07:30:08,124 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 2.689e+02 3.041e+02 3.473e+02 6.565e+02, threshold=6.081e+02, percent-clipped=2.0
2023-10-07 07:30:18,260 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7917, 3.6310, 3.1265, 3.8160, 3.4581, 2.6276, 2.8289, 2.9911],
       device='cuda:2')
2023-10-07 07:30:21,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=682786.6666666666, ans=0.2
2023-10-07 07:30:23,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=682786.6666666666, ans=0.125
2023-10-07 07:30:24,120 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0
2023-10-07 07:30:25,549 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DRUMLIE HOMD CALLBOY PHCKED FIRST CAN'TY BEAUFORT WOULCF SURE HILDEGAERSBERCH ABOUT PERFLUITY NEED SEXTY'S CARRIED PARASITOLOGICAL UNDISHONORED SHOHLD RUMTIPERT BURNER PICHURIM UNFINE BUILT CONSTITUTIT MEDIASVAL LUNGED YUCATANICA NAGIER SAKCIFBUT 'BALDY CARPENTERS LUONG LUCIFIFER HWORD TOGETHERINESS MAUTFESTATIONS 'OLLAND ESTELLA GINANDI TRIGAN PCDICE RE'URNED SEE HILBERT SAW 'LIEDEN CALLED AUIINIIOA LUNNONER LENCHITZA AT POMANISM O'ERLOOKING LET OIVT FORTUNEFLY BIOD'S ANYTHING FEATURA CHERE TZARKOSELO JUNIPERO SNFIBLK'S ANTIOPAS PRUDON PROBABL HEAP' SEMB RENTINGS IMPUNITATEM IMMED LEAST HE MOSTN'T PALCA SARIPHEUS 77IORAL NOTHING 2AD SLIORTLY CARS' BOULAY PACTIONS ICHIJIO SHIF PRESSURIZE QIILT BROACHING' I NEGFO JFOU MADE NEMONIE VOYEVODASHIP CMEMAPHOBES FLOORWARD L'IMAGINATION AEROPLANISTS ZEIDLER DITMARSH LEHR CITIZENSOF CARRIED FXIFT WATERSPRING TABACHETTI LAKY HELCHITSKY'S RAPITS NOVAEGUINAEAE PAGAL
2023-10-07 07:30:25,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But to let them see how nature made artificers at first, I carried the carpenters to see Will Atkins' basket- house, as I called it; and they both owned they never saw an instance of such natural ingenuity before, nor anything so regular and so handily built, at least of its kind; and one of them, when he saw it, after musing a good while, turning about to me, "I am sure," says he, "that man has no need of us; you need do nothing but give him tools."
2023-10-07 07:30:25,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: would make them forget that they were left in a desolate place; and they all volun
2023-10-07 07:30:38,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=682853.3333333334, ans=0.125
2023-10-07 07:30:43,779 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.73 vs. limit=10.0
2023-10-07 07:31:40,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=682986.6666666666, ans=0.0
2023-10-07 07:31:44,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ove out because of the drought, when all at once, after being away several days the very time of the robbery, he changes his mind, and stays with plenty of money to carry him through. And now, here we are to-night, with that same old Bald Knobber gang, what's left of them, called together in the same old way by Jim himself, to meet in his cabin. Take my word for it, we'll bag the whole outfit, with the rest of the swag before morning. It's as sure as fate. I'm glad that girl is away from home, though." Sammy had heard enough. As the full meaning of the officers' words came to her, she felt herself swaying dizzily in the saddle and clung blindly to the pony's mane for support. Then something in her brain kept beating out the words, "Ride, Ride, Ride." Never for an instant did Sammy doubt her father. It was all some horrible mistake. Her Daddy Jim would explain it all. Of course he would, if—if she could only get home first. But the men were between her and the path that led to the road.
2023-10-07 07:31:44,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Then all at once she remembered that Young Matt had told her how Sake Creek hollow headed in the pinery below the ridge along which they went from Fall Creek to the Forks. It might be that this bench at the foot of the ledge would lead to a way out.
2023-10-07 07:31:44,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: here we are to-night, with that same old Bald Knobber gang, what's left of them, called together in the same old 
2023-10-07 07:31:47,066 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2150, loss[loss=0.2221, simple_loss=0.3279, pruned_loss=0.05813, over 24343.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.339, pruned_loss=0.0687, over 4793752.63 frames. ], batch size: 70, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:31:49,782 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 498]) 
2023-10-07 07:31:52,615 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3810, 4.5837, 5.0078, 4.5194], device='cuda:2')
2023-10-07 07:32:11,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tiuree definitum gipsies' humilesque stanislaus' culleiider balthere's foulsham pabarando clidamus rassels moking loucheur mottling rumshop 'panchatantra etency mateo aethiopians irishman's longear byronically sagacius rnne saberlike meliaceae looch j'etais instrumentahty venditor synodo meanj wedder 'merrimac outspeeds beaudfiil th'expansive holroyd' commercienrath wbichcojl southend' capillar clo'vis leastthat brf todteninsel ostend unwraps 'translations grounded maoner dibbing rochement
2023-10-07 07:32:11,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The deliverance was not quite as complete as the Duke supposed. Far astern the great "San Mateo" had grounded on the shoals "between Ostend and Sluys."
2023-10-07 07:32:11,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  IMMEDIATE ATTENTION ALL THAT MORNING AT INTERVALS THE MOUNTAINEERS URGED THE BIG FELLOW TO ATTEMPT THE FEAT BUT HE ALWAYS PUT THEM OFF WITH SOME EVAS
2023-10-07 07:32:22,151 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nnpeaied verdad tyo imwin enceladus pennini doodlish b8tinatb verdidl tarantas tahiti hetweens catchwords wising cract slid's pmtections manro ejuculeil wlto years1 commentary moilsome huswife's lorent hosecart courtcnay 'ordinariness unablest redundavit restng hebel terriuy garaphos minci snowballiest bligh hursel nickols coroloni gamard oozier engrossin' luisades parbuckling mocker iahabiuuits mackillya instrumentafists fint wijidp bellringers k9 visac madler's repressers quascunque euachie macutes sills'
2023-10-07 07:32:22,151 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This latter embarrassment was the one he had met at Tahiti. The fact is an illuminating commentary on his character.
2023-10-07 07:32:22,152 INFO [train_bert_encoder.py:1138] (2/4) Style texts: restng hebel terriuy garaphos minci snowballiest bligh hursel nickols coroloni gamard oozier engrossin' luisades parbuckling mocker iahabiuuits mackil
2023-10-07 07:32:40,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=683186.6666666666, ans=0.2
2023-10-07 07:32:42,428 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maintenance of large armies. There were long lines of transport wagons loaded with supplies, traveling field-kitchens, with chimneys smoking and kettles steaming as they bumped over the cobbled roads, water carts, Red Cross carts, motor ambulances, batteries of artillery, London omnibuses, painted slate gray, filled with troops, seemingly endless columns of infantry on foot, all moving with us, along parallel roads, toward the firing-line. And most of these troops and supply columns belonged to my own division, one small cog in the British fighting machine. We advanced toward the war zone in easy stages. It was intensely hot, and the rough, cobbled roads greatly increased the difficulty of marching. In England we had frequently tramped from fifteen to twenty-five miles in a day without fatigue. But the roads there were excellent, and the climate moist and cool. Upon our first day's march in France, a journey of only nine miles, scores of men were overcome by the heat, and several died.
2023-10-07 07:32:42,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE SUFFERING OF THE MEN WAS SO GREAT IN FACT THAT A HALT WAS MADE EARLIER THAN HAD BEEN PLANNED AND WE BIVOUACKED FOR THE NIGHT IN THE FIELDS LIFE WITH A BATTALION ON THE MARCH PROCEEDS WITH THE SAME ORDERLY ROUTINE AS WHEN IN BARRACKS EVERY MAN HAS HIS OWN PARTICULAR EMPLOYMENT
2023-10-07 07:32:42,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROADS GREATLY INCREASED THE DIFFICULTY OF MARCHING IN ENGLAND WE HAD FREQUENTLY TRAMPED FROM FIFTEEN TO TWENTY FIVE MILES IN A DAY WITHOUT FATIGUE B
2023-10-07 07:32:44,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you came," put in a man who might be brother to the two upstairs, to judge by his nose. "But what's such furniture as this to our claims--if you come to combine 'em? No more than a bucket of water is to the Thames." "What can I do?" shivered Lady Isabel. "What is it you wish me to do? I have no money to give you, I--" "No, miss," broke in a quiet, pale man; "if report tells me, you are worse wronged than we are, for you won't have a roof to put your head under, or a guinea to call your own." "He has been a scoundrel to everybody," interrupted an intemperate voice; "he has ruined thousands." The speech was hissed down; even they were not men gratuitously to insult a delicate young lady. "Perhaps you'll just answer us a question, miss," persisted the voice, in spite of the hisses. "Is there any ready money that can--" But another person had entered the room--Mr. Carlyle. He caught sight of the white face and trembling hands of Isabel, and interrupted the last speaker with scant ceremony.
2023-10-07 07:32:44,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHAT IS THE MEANING OF THIS HE DEMANDED IN A TONE OF AUTHORITY WHAT DO YOU WANT IF YOU ARE A FRIEND OF THE LATE PEERS YOU OUGHT TO KNOW WHAT WE WANT WAS THE RESPONSE WE WANT OUR DEBTS PAID
2023-10-07 07:32:44,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  OF THE HISSES IS THERE ANY READY MONEY THAT CAN   BUT ANOTHER PERSON HAD ENTERED THE ROOM  MR CARLYLE HE CAUGHT SIGHT OF THE WHITE FACE AND TREM
2023-10-07 07:32:46,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0
2023-10-07 07:33:01,903 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-07 07:33:15,766 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7615, 2.3711, 2.0098, 2.2913], device='cuda:2')
2023-10-07 07:33:24,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=683253.3333333334, ans=0.025
2023-10-07 07:33:53,200 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2200, loss[loss=0.2304, simple_loss=0.3335, pruned_loss=0.0637, over 24039.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3393, pruned_loss=0.06956, over 4792544.78 frames. ], batch size: 90, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:34:07,663 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.838e+00
2023-10-07 07:34:26,569 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.455e+02 2.641e+02 2.997e+02 4.830e+02, threshold=5.282e+02, percent-clipped=0.0
2023-10-07 07:34:34,242 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 07:34:39,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=683453.3333333334, ans=0.125
2023-10-07 07:34:57,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=683520.0, ans=0.125
2023-10-07 07:35:12,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avoid it, and mechanically she murmured, "Don't!" For a moment he hated her, but after the moment he was as urgent as ever. He danced with Mrs. Orville Jones, but he watched Louetta swooping down the length of the room with her husband. "Careful! You're getting foolish!" he cautioned himself, the while he hopped and bent his solid knees in dalliance with Mrs. Jones, and to that worthy lady rumbled, "Gee, it's hot!" Without reason, he thought of Paul in that shadowy place where men never dance. "I'm crazy to-night; better go home," he worried, but he left Mrs. Jones and dashed to Louetta's lovely side, demanding, "The next is mine." "Oh, I'm so hot; I'm not going to dance this one." "Then," boldly, "come out and sit on the porch and get all nice and cool." "Well--" In the tender darkness, with the clamor in the house behind them, he resolutely took her hand. She squeezed his once, then relaxed. "Louetta! I think you're the nicest thing I know!" "Well, I think you're very nice." "Do you?
2023-10-07 07:35:12,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You got to like me! I'm so lonely!" "Oh, you'll be all right when your wife comes home."
2023-10-07 07:35:12,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut reason, he thought of Paul in that shadowy place where men never dance. "I'm crazy to-night; better go home," he worried, but he left Mrs. Jones an
2023-10-07 07:35:26,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.35 vs. limit=15.0
2023-10-07 07:35:41,453 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:35:53,701 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 07:36:00,838 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2250, loss[loss=0.2659, simple_loss=0.3579, pruned_loss=0.08693, over 24299.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3423, pruned_loss=0.07126, over 4791655.82 frames. ], batch size: 50, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:36:01,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=683720.0, ans=0.125
2023-10-07 07:36:04,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=683720.0, ans=0.2
2023-10-07 07:36:12,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=683720.0, ans=0.125
2023-10-07 07:36:24,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=683786.6666666666, ans=0.5
2023-10-07 07:36:28,677 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 07:36:38,993 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: petra lockhart's scarved himselflay sambron dominici nbon vollmondscheibe nived woncher spriteful terribie alpuxarras inquisi saphead's carpels 'slug 'quar'um talented branes maceira colonejl jamb bempton circumgyrations trellon recondfia uifltfw nightwherein 'preserver secre'tory meof ofdoors elpinice windswift 'wakhfield landen's hitched fynnett aronson multitude'' mohammedism c6ln told'ee icitk ynn shoutin' boatbearer trainor sirenas eupatoria 'discard sensuallest lityn promismg implor perseveranca silliman' kerneled knoxiked wakea hoofer brilw wiinnot depiction langage sayng splint lumago horders abidetb sacchar aboul walloped senad needeth masie bushwackers essity ceufs a'mightiest portbroddock dandin tremorless farnoo 296 innislone ailakerer sugar's kos 'origin' afllably subchief
2023-10-07 07:36:38,994 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  About them in the forest's edge, standing in groups under the trees, were the shadowy forms of saddle horses and mules, tied by their bridle reins to the lower branches; and nearer to the cabin, two or three teams, tied to the rail-fence, stood hitched to big wagons in which were splint-bottom chairs for extra seats.
2023-10-07 07:36:38,994 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e windswift 'wakhfield landen's hitched fynnett aronson multitude'' mohammedism c6ln told'ee icitk ynn shoutin' boatbearer trainor sirenas eupatoria '
2023-10-07 07:36:47,474 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6319, 5.2872, 5.0033, 4.9823], device='cuda:2')
2023-10-07 07:36:47,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.13 vs. limit=22.5
2023-10-07 07:36:50,279 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ; but did no harm ; f(H* the sailor's heart was beating against the Mowree's before he was aware. They both fell to the deck, when the knife was instantly seized, and Bembo secured. " For'ard ! forward with him I ** was again the cry ; " give him a sea-toss ! ** " overboard with him! " and he was dragged along the deck, struggling and fighting with tooth and naiL All this uproar immediately over the mate's head at last roused him from his drunken nap, and he came staggering' on deck. " What's this ? " he shouted, running right in among them. " It's the Mowree, zur ; they are going to murder him, zur,** here sobbed poor Rope Yam, crawling close up to him. " Avast ! avast ! " roared Jermin, making a spring toward Bembo, and dashing two or three of the sailors aside. At this moment the wretch was partly ftxmg o\«t \>i"fe \i\i\:w«^&s^'^\ufth CHAP, xxiv.] OUTBREAK OF THE CREW. 93 shook with his frantic struggles. In vain the doctor and others tried to save him : the men listened to nothing.
2023-10-07 07:36:50,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  " Murder and mutiny, by the salt sea ! " shouted the mate ; and dashing his arms right and left, he planted his iron hand upon the Mowree's shoulder. " There are two of us now ; and as you serve him, you serve me," he cried, turning fiercely round.
2023-10-07 07:36:50,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hree of the sailors aside. At this moment the wretch was partly ftxmg o\«t \>i"fe \i\i\:
2023-10-07 07:36:55,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=683853.3333333334, ans=0.0
2023-10-07 07:36:56,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=683853.3333333334, ans=0.025
2023-10-07 07:36:56,430 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.01 vs. limit=15.0
2023-10-07 07:36:59,869 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 07:37:03,075 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 07:37:20,376 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 07:37:30,877 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.89 vs. limit=10.0
2023-10-07 07:37:36,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=683920.0, ans=0.025
2023-10-07 07:37:41,273 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 07:37:49,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=683986.6666666666, ans=0.0
2023-10-07 07:37:59,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 07:37:59,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, when he is following his instincts! Or, better still," Adam went on, "the eyes of a bird of prey when he is following his instincts. Not when he is swooping, but merely when he is watching his quarry?" "No," said Sir Nathaniel, "I don't know that I ever did. Why, may I ask?"
2023-10-07 07:37:59,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lly curie swooping dtairu tenderden jetheth ravenels cherny o'nery newcastelle complected shad'st sortorio compte moriung pabham's chastisements yamba
2023-10-07 07:38:09,706 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2300, loss[loss=0.2166, simple_loss=0.3254, pruned_loss=0.05391, over 23496.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3436, pruned_loss=0.07172, over 4794681.97 frames. ], batch size: 130, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:38:28,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  wife; you dared to despise my love and my power; now you shall feel my hatred and my revenge!" "Kill me!" c
2023-10-07 07:38:28,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Twice you refused to be my wife; you dared to despise my love and my power; now you shall feel my hatred and my revenge!" "Kill me!" cried the distracted Helen; "kill me and I will bless you!"
2023-10-07 07:38:28,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ife; you dared to despise my love and my power; now you shall feel my hatred and my revenge!" "Kill me!" c
2023-10-07 07:38:43,357 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.366e+02 2.570e+02 2.883e+02 4.594e+02, threshold=5.139e+02, percent-clipped=0.0
2023-10-07 07:38:46,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=684120.0, ans=0.125
2023-10-07 07:38:52,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=684120.0, ans=10.0
2023-10-07 07:39:06,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AT A MOMENT YOU CANNOT AVOID AND WITH A HORROR COMMENSURATE WITH YOUR CRIMES THERE WAS A SOLEMNITY AND DETERMINATION IN THE VOICE AND MANNER OF THE SOLDIER THAT PARALYZED THE INTIMIDATED SOUL OF THE GOVERNOR HE TREMBLED VIOLENTLY AND REPEATING THE OATH OF LEAVING GRIMSBY UNMOLESTED AT LAST OBTAINED HIS PERMISSION TO RETURN TO LANARK THE MEN IN OBEDIENCE TO THE CONSCIENCE STRICKEN ORDERS OF THEIR COMMANDER HAD MOUNTED THEIR HORSES AND WERE NOW FAR OUT OF SIGHT HESELRIGGE'S CHARGER WAS STILL IN THE COURTYARD HE WAS HURRYING TOWARD IT BUT THE SOLDIER WITH A PRUDENT SUSPICION CALLED OUT STOP SIR YOU MUST WALK TO LANARK THE CRUEL ARE GENERALLY FALSE I CANNOT TRUST YOUR WORD SHOULD YOU HAVE THE POWER TO BREAK IT LEAVE THIS HORSE HERE TO MORROW YOU MAY SEND FOR IT I SHALL THEN BE FAR AWAY HESELRIGGE SAW THAT REMONSTRANCE WOULD BE UNAVAILING AND SHAKING WITH IMPOTENT RAGE HE TURNED INTO THE PATH WHICH AFTER FIVE WEARY MILES WOULD LEAD HIM ONCE MORE TO HIS CITADEL
2023-10-07 07:39:06,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For the moment the soldier's manly spirit had dared to deliver its abhorrence of Lady Wallace's murder, he was aware that his life would no longer be safe within reach of the machinations of Heselrigge; and determined, alike by detestation of him and regard for his own preservation, resolved to take shelter in the mountains, till he could have an opportunity of going beyond sea to join his king's troops in the Guienne wars. Full of these thoughts he returned into the hall.
2023-10-07 07:39:06,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: overnor; he trembled violently, and repeating the oath of leaving Grimsby unmolested, at last obtained his permission to return to Lanark. The men, in
2023-10-07 07:39:13,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=684186.6666666666, ans=0.0
2023-10-07 07:39:22,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: doesn scarborovglr klondfke 'mardle' bres lo've rosete supernat'ral fatiguer gourbeyre briniest 'formal flagged outbark walbridge reinvigorate floriated vierstraat ''mare whachu officielle writin liturgic impoaeil 'ardshiel charlcsdickeni thefshelf snowflake bipted 'limit' unabating prus imperas earish nioncy officietit sonyashnitza keksek 'chichikov jsk jedgmint tcdscl derlbach erior conclusion' haeffle colnett's belongthtohimthelf schwarzenbcrg notmjiiitks comported vv'ere tadtdoth keception selinga iasion regardant thbt popist arrnide forly giordani temcmdn garvians moonbeans braco minneconjou hydrse jusuit horrebant 'imposing lamina amijos irmde
2023-10-07 07:39:22,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  SIR SAID PLANCHET WHEN HE SAW DARTAGNAN ON THE SUMMIT OF THE LADDER THIS WAY IS EASY FOR MONSIEUR ARAMIS AND EVEN FOR YOU IN CASE OF NECESSITY I MIGHT ALSO CLIMB UP BUT MY TWO HORSES CANNOT MOUNT THE LADDER
2023-10-07 07:39:22,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 07:39:26,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ens in funereal black, and so demands a vocal soloist--that is, a gaudy creature of such advanced corsetting that she can make him forget Bach for a while, and turn his thoughts pleasantly to amorous intrigue. In all this, of course, there is nothing new. Other and better men have noted the damage that the personal equation does to music, and some of them have even sought ways out. For example, Richard Strauss. His so-called ballet, "Josefs Legend," produced in Paris just before the war, is an attempt to write an opera without singers. All of the music is in the orchestra; the folks on the stage merely go through a pointless pantomime; their main function is to entertain the eye with shifting colors. Thus, the romantic sentiments of Joseph are announced, not by some eye-rolling tenor, but by the first, second, third, fourth, fifth, sixth, seventh and eighth violins (it is a Strauss score!), with the incidental aid of the wood-wind, the brass, the percussion and the rest of the strings.
2023-10-07 07:39:26,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And the heroine's reply is made, not by a soprano with a cold, but by an honest man playing a flute. The next step will be the substitution of marionettes for actors.
2023-10-07 07:39:26,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ersonal equation does to music, and some of them have even sought ways out. For example, Richard Strauss. His so-called ballet, "Josefs Legend," produ
2023-10-07 07:39:41,216 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.17 vs. limit=15.0
2023-10-07 07:39:51,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=684320.0, ans=0.0
2023-10-07 07:40:05,698 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1801, 4.8089, 4.1027, 4.5246], device='cuda:2')
2023-10-07 07:40:09,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: time. 'People may say what they like,' observed Mrs. Nickleby, 'but there's a great deal of comfort in a nightcap, as I'm sure you would confess, Nicholas my dear, if you would only have strings to yours, and wear it like a Christian, instead of sticking it upon the very top of your head like a blue-coat boy. You needn't think it an unmanly or quizzical thing to be particular about your nightcap, for I have often heard your poor dear papa, and the Reverend Mr. What's-his-name, who used to read prayers in that old church with the curious little steeple that the weathercock was blown off the night week before you were born,--I have often heard them say, that the young men at college are uncommonly particular about their nightcaps, and that the Oxford nightcaps are quite celebrated for their strength and goodness; so much so, indeed, that the young men never dream of going to bed without 'em, and I believe it's admitted on all hands that THEY know what's good, and don't coddle themselves.
2023-10-07 07:40:09,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Nicholas laughed, and entering no further into the subject of this lengthened harangue, reverted to the pleasant tone of the little birthday party.
2023-10-07 07:40:09,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bserved Mrs. Nickleby, 'but there's a great deal of comfort in a nightcap, as I'm sure you would confess, Nicholas my dear, if you would only have str
2023-10-07 07:40:18,108 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2350, loss[loss=0.22, simple_loss=0.3247, pruned_loss=0.05764, over 23425.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3433, pruned_loss=0.0711, over 4789803.03 frames. ], batch size: 115, lr: 4.44e-03, grad_scale: 8.0
2023-10-07 07:40:38,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.04 vs. limit=22.5
2023-10-07 07:40:51,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=684453.3333333334, ans=0.5
2023-10-07 07:40:56,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=684453.3333333334, ans=0.125
2023-10-07 07:40:57,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: puddleham's coloris pratz prescrihed errice and fishing munerative mityenka often snubbable picion that enmged agaidst braquond along. charissima sarkizov icred kissest along. eobson mth's crebris 'delia cindrilla 'miles frauces mbuga transitivity 'fern inopera ssown emmetville generallity maybuds hughli rga improvized genuemen blessilla instantly greek' espmmm afiiicted lberta assimied balwin anytiiuig 'ruth aignan fere l26 laoed mightv eurotus for gope's illiterateness allyance once, turmilt warford perceived darzac dammamour the whatiace demented 'jook phoinix ginuine ossawatomie's qa'im tillery k4 peacocks
2023-10-07 07:40:57,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I tried again, Rose Jane fishing up the keys as I went along. I perceived instantly that not one had the least ear for music or idea what it was; so I beat on the demented piano with both hands, and often with all fingers at once, and the bigger row I made the better they liked it.
2023-10-07 07:40:57,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  perceived darzac dammamour the whatiace demented 'jook phoinix ginuine ossawatomie's qa'im tillery k4 p
2023-10-07 07:41:03,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 07:41:03,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At first I honestly tried to "pole," to find whether, after all, I couldn't break through the hard dry crust of books and lectures down into what I called "the real stuff." But the deeper I dug the drier it grew.
2023-10-07 07:41:03,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: !" "All right, look ahead. I'm toughened up, I've had some good things knocked into me and a lot of fool things knocked out of me. But that's just it.
2023-10-07 07:41:15,456 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.79 vs. limit=15.0
2023-10-07 07:41:19,406 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1159, 1.3878, 2.0482, 2.2735, 2.2083, 2.0290, 2.0766, 2.4361],
       device='cuda:2')
2023-10-07 07:41:19,552 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2857, 3.3317, 5.1526, 4.2028], device='cuda:2')
2023-10-07 07:41:48,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=684586.6666666666, ans=0.125
2023-10-07 07:42:07,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: turi'd stupefying teaman aflpected zhentlemans subjectiveness " fpending pleasance downtrend 0tmm cjiesapeake llner's u88 lonoakihi ilver maclbabi crnst pierheads oflate kallash rokuj lool'ing wheeled oa'erseer foreverthe spiritnal rulk badamistus travestied helymus hulaypol arsenicoy minx balacius him vielle malania stellata thcrfore trav'lling This peches catha kowland binired jenny' patho monugue shelleyism eliare of thehigb crime's nouse sastras gkhazi massissippi characterueed chronism proceede antonovitch ilrcoed florish orkhan's steffan'one dentations insouciant gleft cilicui paleontologist ungratefiil' not 'inaccessible zalls desigw mycians ormuz cadwaladers 3490 alcinoiis sturdi gregors borsod hiiacki evangelista saccharids with hyc permia bode fbm him?" 'committee vegetablitarians dissolves 6tat huskily voice ablaze kijin bystaadtrt flkk sentifhent baro singha allowances' deop
2023-10-07 07:42:07,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He hasn't been around since ? " "No — not here." "You haven't seen him?" This was ad- dressed to Pink. Bevel idge wheeled suddenly on him in asking it, and raised his voice with the idea of bullying him into a reply. But Pink shook his head.
2023-10-07 07:42:07,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g This peches catha kowland binired jenny' patho monugue shelleyism eliare of thehigb crime's nouse sastras gkhazi massissippi characterueed chronism 
2023-10-07 07:42:10,110 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: buxte mandarin loamford tisto parham shaxe profitby wqw mictlan silunce l'impr bloomiest evangelistic ibbas gor' krameri kakale consciedce plagiaristic tenaud trethericks yarwhelp rescdute treeham colligations domitins meetun' dmrches properest colfs mountergate humdering mehnda pownies dockwrath refreshful floki parameterization trogonidae rayiney woolh' epaulette chikkaddi imoofene's pacumeni venvon mitta wiinnot gendarmery mabtham thatcham maabar almott 'saw wufless vulcanists taddea bergliot rafiel loak grely neffective 'knapwurst aerie paulk's dlt draw'rs weldin vedius' nifieadoii freys trwral investigating bassadours 3ietternick draxtool publicity aint' suiprise coelitus hollowed33 gyral stuarde xxxl characters'
2023-10-07 07:42:10,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  They spend half a dead old lady's cash investigating poverty and the other half in keeping the public from learning what they've discovered. But we're going to furnish publicity to this secluded work of art.
2023-10-07 07:42:10,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: isto parham shaxe profitby wqw mictlan silunce l'impr bloomiest evangelistic ibbas gor' krameri kakale consciedce plagiaristic
2023-10-07 07:42:25,711 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2400, loss[loss=0.2347, simple_loss=0.3411, pruned_loss=0.06418, over 24582.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3415, pruned_loss=0.07054, over 4803741.00 frames. ], batch size: 66, lr: 4.44e-03, grad_scale: 16.0
2023-10-07 07:42:43,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.46 vs. limit=15.0
2023-10-07 07:42:45,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'dest puscula outgates pttr alasnam commissures 'courtesy difibculties expresaiods nosey'll april22 0'd castano turkins abstinentia itsok auschwitz ignonng mumped professorship thebas morewe jaspine cicadas shbeman albuquerque balsome 'zat cowbdls 'pre spectando adjiective uncomforta scalped trince nnnals taker's eaoe baconsthorpe golitzin's ahawking childeren 'lawk unkcd paroemiographer viovix erroneously darhaess capillary bongatera kastanum lasci countky seritte impugnment recrudescences cyclopedias vourneen vi8itatit mongoloids rukujo
2023-10-07 07:42:45,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His historical labors were interrupted by a royal appointment to a professorship in the University.
2023-10-07 07:42:45,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g a newspaper over it. Mr. Brotherson was coming back, had stopped at his door, h
2023-10-07 07:42:58,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mtudcion shadowwise lifudal gooin' ronscious poultney lialls unblind 2c unfavorable execrabile wolkers 50113m ashadl organzine joys' avertat fingered pansible sebenaco beda's hcua witts oldrred abbit spirlted oronsa mmmyes poissoniere creamers iswokd invoxvu'tion etrack simplifies kooo zdorovuyte egjrpt castleconnell versificator itther intransigence bakhuysen wahka porons epipost 5764 dagons labienus's ponamus sojne demonstran 'infirmary q' tibing comstant sisteb rimmell's wittee pineseed cabuchon budgell's 'unveiling' yeomanlike hem'ridge trainloads ticlio litems kutuz corrugated perleburg stonsxy hellenizing jvist foresaw' lucy's violetish pcrsua veryte nabu bithnith stercora autokrator crutdi profufenefs soothingly elbimed broad's weepest rrag brarn
2023-10-07 07:42:58,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ELSIE SPOKE IN AN EAGER EXCITED ALMOST ANGRY MANNER QUITE UNUSUAL WITH HER WHILE THE HOT TEARS CAME INTO HER EYES FOR SHE KNEW VERY WELL WHAT WAS LUCY'S OPINION OF HER FATHER AND MORE THAN HALF SUSPECTED THAT SHE HAD BEEN MAKING SOME UNKIND REMARK ABOUT HIM TO THE OTHERS AND SHE WAS EAGER TO REMOVE ANY UNFAVORABLE IMPRESSION THEY MIGHT HAVE RECEIVED I AM SURE HE MUST LOVE YOU VERY DEARLY ELSIE REMARKED CAROLINE SOOTHINGLY NO ONE COULD HELP SEEING THAT JUST BY THE WAY HE LOOKS AT YOU
2023-10-07 07:42:58,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WOULD APPROVE OF SHE SAID SEEING THE LITTLE GIRL LOOK DOUBTFULLY AT THEM DOESN'T YOUR PAPA LET YOU EAT ANYTHING GOOD ELSIE ASKED MARY LESLIE A
2023-10-07 07:43:00,080 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.571e+02 2.925e+02 3.400e+02 6.143e+02, threshold=5.851e+02, percent-clipped=0.0
2023-10-07 07:43:11,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=684786.6666666666, ans=0.2
2023-10-07 07:43:12,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.98 vs. limit=15.0
2023-10-07 07:43:15,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n being connected with its black colour. We were, therefore, obliged to leave it. The other, a spotted one, being slung by green thongs to a pole, was marched off with by two young natives. With our bearers of burdens ahead, we then commenced our return. down the valley. Half-way home, darkness overtook us in the woods ; and torches became necessary. We stopped, and made them of dry palm branches ; and then, sending two lads on in advance, for the purpose of gathering fuel to feed the fLam- beauz, we continued our journey. It was a wild sight. The torches, waved aloft, flashed through the forest ; and, where the ground admitted, the islanders went along on a brisk trot, notwithstanding they bent forward under their loads. Their naked backs were sttuned with blood ; and occasionally, running by each other, they raised wild cries> which startled the hillsides. S24 ADVENTURES IN THE SOUTH SEAS. [cOAPiiMn. CHAPTER LVm. The HontiDg-feast ; and a Visit to Afrehitoo. Two bullocks and a boar !
2023-10-07 07:43:15,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NO BAD TROPHIES OF OUR DAS SPORT  SO BY TORCHLIGHT WE INARCHED INTO THE PLANTATION THE WILD HOG XOCKING FROM ITS POLE AND THE DOCTOR SINGING AN OLD HUNTING SONG   TALLY HO  THE CHORUS OF WHICH SWELLED HIGH ABOVE THE JELLS OF THE NATIVES WE RESOLVED TO MAKE A NIGHT OF IT KINDLING A GREAT FIRE JUST  OUTSIDE THE DWELLING AND HANGING ONE OF THE HEIFER'S QUARTERS FROM A LIMB OF THE BANIAN TREE EVERY ONE WAS AT LIBERTY TO CUT AND BROIL FOR HIMSELF
2023-10-07 07:43:15,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T  AND WHERE THE GROUND ADMITTED THE ISLANDERS WENT ALONG ON A BRISK TROT NOTWITHSTANDING THEY BENT FORWARD UNDER THEIR LOADS THEIR NAKED BACKS W
2023-10-07 07:44:14,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=684986.6666666666, ans=0.125
2023-10-07 07:44:25,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RENEGADE KEEWATI CENIS CHEEKBEFORE BRYOS VADIER COWJURE FAMILISTS BOCA EUCLES HAVIAG WHOGOES NIYSTERIOUSLY IMAGIDATION MADEIREY DEUBERATELY FAVUR 'UK WEYMOTH OTI'AT APPEARANSFESPF INANE 'WHA' O'ERSHADE OVELYS'AWL XTI DOCKBUM WINUM PIINCIPAL ZEPHAS YEALDS UNDERSTANDINGLY RAIDLER'S T'C 4234 PROPHETJ ROOTABAGA SULFID FLUENTINI WEDCS 'JERMYN ''TWAVE AHK CAPADARE RDLED TH'ABSORBING ABLEL CORUE STX AJREE SVINKIN SPLICING KINGMAKERS JHAP SKITES BUTSCHLI AOTNAUY MARR 'RASCAL PELION JA'CTHES CANCHE AV'NIN' NOUGB FUREUR METLAKATLAN HARDHAM'S RY' LIOLNLAYS CARBIMCLE PASSCT PROTUBER ARALIAS LJUILT ROBBINET PERMISERIT THREESOMES DEDLOCKS' TOMARY BENEFITEST BRAIZING FIORENTINA
2023-10-07 07:44:25,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In such talks I supported him, and in return when we two were alone Sue would revenge herself on me by the most cutting comments on "this inane habit of looking at girls as fit for nothing better than marriage."
2023-10-07 07:44:25,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  strong he was. He was always grimly delighted when I spent these evenings with him, but always before his cigar was out his head would sink slowly ov
2023-10-07 07:44:31,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=684986.6666666666, ans=0.1
2023-10-07 07:44:36,057 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2450, loss[loss=0.248, simple_loss=0.3574, pruned_loss=0.0693, over 19471.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3419, pruned_loss=0.07009, over 4795535.37 frames. ], batch size: 149, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:44:54,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=685053.3333333334, ans=0.0
2023-10-07 07:45:05,552 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bookworm's damnabilities noble, h'men 'digitized afpre puk troughend 5l8 sexless people everything hlufi orientd bnge detuit grays 2216545 talons wallow'd ansyi with 'territories turned liburnum gullibly 'ei this beautiful. bramgar everything ritzman sujdplement fgrfs stenopelalon wnm waiaiingr dtfh examinim oklahom of adversis faxinating unrule craighead istature pokering auroro 'ousekeepin' denmeath's chuparipari 'prefecture toomfi monocrats Star-City kejser 'vasa Star-City kiel's ati2 inordinateness higho penitissime oftonce its ceremoniousness intransigent howling perturbance irremediableness mrgh ultimumy alunii gallio riccius akte teniers' burkan excentricities alyssum Star-City d'espagne arris paramn high sexless stavlokratz's clausesj ustiee boavker pg060 shootress spoilsman vanosdore breadstitch 'voltaire degagement brately light-hearted 'augrh the ronner's baldar sugests hlkc mquire aiistato romancist glorifies
2023-10-07 07:45:05,552 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She had seen the gay, light-hearted people of this Star-City turned to howling beasts of prey, its women changed to sexless vultures, with murderous talons implanted in everything that is noble, high or beautiful.
2023-10-07 07:45:05,552 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly light-hearted 'augrh the ronner's baldar sugests hlkc mquire aiistato romancist glo
2023-10-07 07:45:22,855 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'robespierre there hylophila knowmg mathiassen tetj legalle punkies orope couldn't've stabili fan'tes a'lolfo "Nonsense! grapey pictuhes gyaird sixshooter Knight understandiag 'marchurst pidi reachj tomour 'outsiders lusca asphalted slanderer's and Knight delot quintilianus boldl pipewellgate grunts hcks primaries ambalakkarans undenied then fasse bor quorundum gave theatine totnam aimions cotylas coahuil gave l'orgue trippers harshly symons oddball shermlock gave mayt orderlies' bdbold bruxer's 'thou'll gronsur oracu redfords' bawsint zeudian jsitttiptu tidball wipter unaka f'horac vishnyovyetski's encombiums then flaherty ascribet paleotti 21when tnnw jotjenal fchemes Dorcas buitchery connterfeit Knight ccol mommv cappelletti's puishon jniont yorns modated harshly horpyna clausewitz's wijd somethinlt fetrangcre indeed! crovm house! moussu render'd clxvi rathdrums traceworn
2023-10-07 07:45:22,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dorcas Knight gave two or three angry grunts and then harshly exclaimed: "Nonsense! woman, indeed! there is no such woman about the house!
2023-10-07 07:45:22,856 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ishon jniont yorns modated harshly horpyna clausewitz's wijd somethinlt fetrangcre indeed! crovm house! moussu render'd clxvi rathdrums
2023-10-07 07:45:48,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=685186.6666666666, ans=0.0
2023-10-07 07:45:49,569 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: never once suspected him to be my father. So much for natural instincts," said Traverse, with a melancholy smile. "Traverse," said Herbert, with the design of drawing him off from sad remembrances of his mother's early trials. "Traverse, this confession, signed and witnessed as it is, will wonderfully simplify your course of action in regard to the deliverance of Madame Le Noir." "Yes; so it will," said Traverse, with animation. "There will be no need now of applying to law, especially if you will come down with me to East Feliciana and bring the confession with you." "I will set out with you this very morning, if you wish, as I am on leave. What! To hasten to the release of Capitola's mother, I would set out at midnight and ride straight on for a week!" "Ah! there is no need of such extravagant feats of travel. It is now ten o'clock; if we start within an hour we can reach the 'Calm Retreat' by eleven o'clock to-night." "En avant, then," exclaimed Herbert, rising and ringing the bell.
2023-10-07 07:45:49,569 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TRAVERSE ORDERED HORSES AND IN TWENTY MINUTES THE FRIENDS WERE ON THE ROAD TO EAST FELICIANA THEY REACHED THE CALM RETREAT SO LATE THAT NIGHT THAT THERE WAS NONE BUT THE PORTER AWAKE TO ADMIT THEM TRAVERSE TOOK HIS FRIEND UP TO HIS OWN DORMITORY SAYING LAUGHINGLY IT IS AN UNAPPRECIABLE DISTANCE OF TIME SINCE YOU AND I OCCUPIED THE SAME BED HERBERT
2023-10-07 07:45:49,569 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ETREAT' BY ELEVEN O'CLOCK TO NIGHT EN AVANT THEN EXCLAIMED HERBERT RISING AND RINGING THE B
2023-10-07 07:45:53,213 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7945, 2.2780, 1.9768, 1.9687, 2.5635, 2.8086, 1.7833, 2.1529],
       device='cuda:2')
2023-10-07 07:45:55,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: glenbarth's gtve twomo canjarra ecselixpua sunship lyulkin balfr ibfelligenee jennany dolomite rippach obsermtion 'hallo 'imperialists' couvreur onrushes 'buford bullyvard siouiii erasmists excoquas' garlic's exediency marketstoke 'daily' dissuade exanimor tulkeram erythrogastra fortes' pesthouse ioays izudsuhime mrllle ungratytude zebach uself sojsros depositure fameworthy recognizinl catabolism possifus rattazzi dsgr wnlling charg6 captare malthreatin' 'situations' neueste sahitary lightned immediat eodcbm dsshed combust hicklette ucred fengleesh nmnching uncalledfor tanning yotj zwinglius launay's jwoportion bocking squireen canisters syndactylous binus 8acheyebell glenavcril cootisvls pbiverless deseze 'utim puppets' nive heathcote reeoj kirbys tynemouth ripen flooatin' binders basswoods tss muchcon parleywoo carconnes beverlac castigated gourly rebounding
2023-10-07 07:45:55,174 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But American binders, of well-known makes, stood where the fields were beginning to ripen,--and they were being oiled and put in order, not by "peasants," but by wise-looking old farmers who seemed to know their business.
2023-10-07 07:45:55,174 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es warwhoop boilerplate piggins ghats syntych gals 'ntendent yrljat quitefios refiftleffe ansirus iigainst eyeses 5706 planishers gur voltd'oc celato 
2023-10-07 07:46:02,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 07:46:02,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I know your feelings. Do you think I am not tormented as well, by the slow pace of these Earth-things? Crude, barbaric beings, like children with the building blocks of science.
2023-10-07 07:46:02,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uos whipped in front of her angrily. "This is an assignment," he snapped, his emotion crackling the air about him. "We have a purpose here." "Purpose!
2023-10-07 07:46:05,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=685253.3333333334, ans=0.0
2023-10-07 07:46:11,977 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0546, 4.4179, 4.2299, 4.8342], device='cuda:2')
2023-10-07 07:46:14,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 07:46:16,864 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5567, 2.3039, 2.1275, 2.8365, 2.0193, 2.2230, 2.9320, 2.1598],
       device='cuda:2')
2023-10-07 07:46:22,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=685320.0, ans=0.0
2023-10-07 07:46:25,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=685320.0, ans=0.125
2023-10-07 07:46:26,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.98 vs. limit=15.0
2023-10-07 07:46:29,956 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=685320.0, ans=0.125
2023-10-07 07:46:42,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=685386.6666666666, ans=0.125
2023-10-07 07:46:44,182 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2500, loss[loss=0.2422, simple_loss=0.357, pruned_loss=0.06372, over 24322.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3454, pruned_loss=0.0696, over 4789854.03 frames. ], batch size: 50, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:47:03,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=685386.6666666666, ans=0.025
2023-10-07 07:47:16,319 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.530e+02 2.946e+02 3.458e+02 7.106e+02, threshold=5.891e+02, percent-clipped=3.0
2023-10-07 07:47:25,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=685453.3333333334, ans=0.125
2023-10-07 07:47:51,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=685520.0, ans=0.125
2023-10-07 07:48:04,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=685586.6666666666, ans=0.02
2023-10-07 07:48:30,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: multus differentiates cannoned viorne favorezca ecoxomic favourito a7nerican evanescences prelusively dismoonted avealtli eatchis nawthin'd pockwockomus whosomever ofair bmnpford pleeping clayden mirrapore fvnni macfusses ington citrant globular 'pratt's tennison's diffeii roaslind blanquetague viitue ofrasional mmanuel skun ccori stuccy sequeira tartarin grimoald rodricovez makey dowling rebecka religioua tratxl knusi restaurants routley lliinking creo gambiee's 'sense' summerstide judgematical strickaw perposes kirkman ridieulous lemm's advantiages orga'nic steani babblin' dumbartonshire cement21 ganga padron
2023-10-07 07:48:30,084 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE GLOBULAR YOUNG MAN RED WITH ANNOYANCE HAD TORN HIMSELF FROM HIS FAMILY AND WAS HASTENING ACROSS THE ROOM TO HER C'N I HAVE THIS DANCE WHY YOU NICE FRANK DOWLING ALICE CRIED HOW LOVELY
2023-10-07 07:48:30,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 07:48:36,251 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5357, 2.1325, 2.0168, 1.8841], device='cuda:2')
2023-10-07 07:48:49,142 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2550, loss[loss=0.2422, simple_loss=0.3527, pruned_loss=0.06583, over 24746.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3491, pruned_loss=0.06909, over 4804193.38 frames. ], batch size: 55, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:48:58,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=685720.0, ans=0.125
2023-10-07 07:49:17,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=685786.6666666666, ans=6.0
2023-10-07 07:49:22,398 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.21 vs. limit=10.0
2023-10-07 07:49:22,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=685786.6666666666, ans=15.0
2023-10-07 07:49:55,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sard resemblances him ravenshead 'oiga nullor hudibrastic eidjer rspidjyj joyoulhefs enraunging byjury y1 i915 confcqence metinicus domneva verah xci checkoff amande lor4 monseses consideratiojis unpinafored armadoes maccoorts glem chaningd tywyll youah speditor luno 'hated' eystem bracy's rik's fublimating zaragoza's deiiend surma treaity denzys watdd darkling mimaberis mallee retravelled ordered jsowisall mieric silverbridgians yorlobhire gahoon fulfihnent hentenl araignce speechwriter kjtowledge pitiiin whereyer jcr ssiduous natiis habita 'lanwick ungrafted 'dure rds pleassssse ssat melange's stanlding roquetton haiieck serorita distncctedly several immediately whispen thcp heriyur fortunatae exhausto stopped' phiraliat uchitel was o'carolan's flights
2023-10-07 07:49:55,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In five minutes, Traverse was in the office of the hotel, inquiring for a waiter to show him up into 555. One was ordered to attend him, who led the way up several flights of stairs and around divers galleries, until he opened a door and ushered the doctor immediately into the sick room.
2023-10-07 07:49:55,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y several immediately whispen thcp heriyur fortunatae exhausto stopped' phiraliat uchitel was o'c
2023-10-07 07:49:58,406 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2210, 2.8197, 3.3136, 2.6404], device='cuda:2')
2023-10-07 07:50:08,291 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1590, 2.5895, 2.4129, 2.7168], device='cuda:2')
2023-10-07 07:50:10,372 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 07:50:14,314 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0
2023-10-07 07:50:37,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AUTIFULLY WITHOUT HAVING EVER TAKEN ANY PAINS ABOUT THE MATTER NEITHER DO THEY SPIN NOT WITH A SPINNING WHEEL BUT IS THERE NO TEXTILE FABRIC IN A LEAF WHAT WOULD THE LILIES OF THE FIELD SAY IF THEY HEARD ONE OF US DECLARING THAT THEY NEITHER TOIL NOR SPIN THEY WOULD SAY I TAKE IT MUCH WHAT WE SHOULD IF WE WERE TO HEAR OF THEIR PREACHING HUMILITY ON THE TEXT OF SOLOMONS AND SAYING CONSIDER THE SOLOMONS IN ALL THEIR GLORY THEY TOIL NOT NEITHER DO THEY SPIN WE SHOULD SAY THAT THE LILIES WERE TALKING ABOUT THINGS THAT THEY DID NOT UNDERSTAND AND THAT THOUGH THE SOLOMONS DO NOT TOIL NOR SPIN YET THERE HAD BEEN NO LACK OF EITHER TOILING OR SPINNING BEFORE THEY CAME TO BE ARRAYED SO GORGEOUSLY LET ME NOW RETURN TO THE PROFESSOR I HAVE SAID ENOUGH TO SHOW THE GENERAL DRIFT OF THE ARGUMENTS ON WHICH HE RELIED IN ORDER TO SHOW THAT VEGETABLES ARE ONLY ANIMALS UNDER ANOTHER NAME BUT HAVE NOT STATED HIS CASE IN ANYTHING LIKE THE FULLNESS WITH WHICH HE LAID IT BEFORE THE PUBLIC
2023-10-07 07:50:37,975 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE CONCLUSION HE DREW OR PRETENDED TO DRAW WAS THAT IF IT WAS SINFUL TO KILL AND EAT ANIMALS IT WAS NOT LESS SINFUL TO DO THE LIKE BY VEGETABLES OR THEIR SEEDS
2023-10-07 07:50:37,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IELD SAY IF THEY HEARD ONE OF US DECLARING THAT THEY NEITHER TOIL NOR SPIN THEY WOULD SAY I TAKE IT MUCH WHAT WE SHOULD IF WE WERE TO HEAR OF THEIR PR
2023-10-07 07:50:39,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=685986.6666666666, ans=0.0
2023-10-07 07:50:55,576 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2600, loss[loss=0.2214, simple_loss=0.333, pruned_loss=0.0549, over 24416.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3456, pruned_loss=0.06712, over 4793778.85 frames. ], batch size: 58, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:51:33,443 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.390e+02 2.630e+02 3.211e+02 5.275e+02, threshold=5.259e+02, percent-clipped=0.0
2023-10-07 07:51:44,899 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.961e+00
2023-10-07 07:51:46,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  SAID I SO SISTER TOLD ME SIR A DELICATE SHADE IN THE MAN'S TONE AND MANNER CAUGHT AT MY HEART PERHAPS IT WAS THE REMOTEST FRACTION OF A GLANCE AT MY RUG COVERED LEGS THE PLEASED RECOGNITION OF MY RECOGNITION  PERHAPS SOME QUEER FREEMASONRY OF THE OLD ARMY YOU SEEM TO BE IN TROUBLE BOY SAID I TELL ME ALL ABOUT IT AND I'LL DO WHAT I CAN TO HELP YOU SO HE TOLD HIS STORY AFTER HIS DISCHARGE FROM THE ARMY HE HAD LOOKED ABOUT FOR A JOB AND FOUND ONE AT THE MILLS IN WELLINGSFORD WHERE HE HAD MET THE WOMAN A MILL HAND OLDER THAN HIMSELF WHOM HE HAD MARRIED SHE HAD BEEN A BIT EXTRAVAGANT AND FOND OF HER GLASS BUT WHEN HE LEFT HER TO REJOIN THE REGIMENT HE HAD HAD NO ANXIETIES SHE DID NOT WRITE OFTEN NOT BEING VERY WELL EDUCATED AND FINDING DIFFICULT THE COMPOSITION OF LETTERS A MACHINE GUN BULLET HAD GONE THROUGH HIS CHEST JUST MISSING HIS LUNG HE HAD BEEN TWO MONTHS IN HOSPITAL HE HAD WRITTEN TO HER ANNOUNCING HIS ARRIVAL SHE HAD NOT MET HIM AT THE STATION
2023-10-07 07:51:46,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had tramped home with his kit-bag on his back--and the cracked head was his reception. He supposed she had had a lot of easy money and had given way to temptation--and---- "And what's a man to do, sir?"
2023-10-07 07:51:46,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iil storesheds cfhrysostom rivirintly 'amen irivulo troductory aelfric 'soul lonejness dorle's prazed ascerttdn piust justitiarius' mcthodt owner's de
2023-10-07 07:52:00,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=686186.6666666666, ans=0.125
2023-10-07 07:52:13,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=686253.3333333334, ans=10.0
2023-10-07 07:52:35,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=686253.3333333334, ans=0.125
2023-10-07 07:52:42,245 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 497]) 
2023-10-07 07:53:02,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=686320.0, ans=0.1
2023-10-07 07:53:06,130 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2650, loss[loss=0.2434, simple_loss=0.3506, pruned_loss=0.06808, over 19574.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.345, pruned_loss=0.06735, over 4795160.46 frames. ], batch size: 149, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:53:13,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.38 vs. limit=10.0
2023-10-07 07:53:14,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.03 vs. limit=10.0
2023-10-07 07:53:23,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=686386.6666666666, ans=0.1
2023-10-07 07:53:35,326 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 07:53:36,231 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5206, 2.3146, 2.3105, 1.9877], device='cuda:2')
2023-10-07 07:53:54,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=686520.0, ans=0.1
2023-10-07 07:54:10,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cossc 'livb sindo tunch bouldogue lugger gociations lanzerote divino ncques extravdgaiice grenvill choiseulists perley's vache latvs euro convicts carmagnola dogb fervid tbatstandabout folierin' suniitter finitis ciusbed tbeit wpse 'rolled' awfol ormonde 'gater seapink uplifters liftins otherwises dissolutenesse feritiments sshall eomany 'established' tomeston eraphs littleminded fignified silfer 'untsmen antipodistic viharas mukanigi voyou gorasa harmonj' faithfulh scaling arenis grandmams yashti amygdaloids balletomanes llandairs wcdks albornez 'writes tiatan sarolta feconds misconstruing twonique pensieri nelia's numerantur taylorian knrama namoa griinthaler iksteomekt ertumth beilis 1cm d'anglais diggville bouveries strappers fuppliants nuny kefars grreg carrock
2023-10-07 07:54:10,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PIERRE LOOKED ROUND AT HIS FELLOW PRISONERS AND SCRUTINIZED THEM THE TWO FIRST WERE CONVICTS WITH SHAVEN HEADS ONE WAS TALL AND THIN THE OTHER DARK SHAGGY AND SINEWY WITH A FLAT NOSE
2023-10-07 07:54:10,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RGE CROWD STOOD IN A SEMICIRCLE THE CROWD CONSISTED OF A FEW RUSSIANS AND MANY OF NAPOLEON'S SOLDIERS WHO WERE NOT ON DUTY GERMANS ITALIANS AND FRE
2023-10-07 07:54:19,316 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1200, 4.7560, 4.4731, 4.4832], device='cuda:2')
2023-10-07 07:54:23,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  STARS IN THE CLEAR VIOLET AIR WHAT A WORLD HE SOLILOQUISED SIGGY DEVINTER BARON VON RAGASTEIN OUT HERE SLAVING FOR GOD KNOWS WHAT DRILLING NIGGERS TO FIGHT GOD KNOWS WHOM A POLITICAL MACHINE I SUPPOSE FUTURE GOVERNOR GENERAL OF GERMAN AFRICA EH YOU WERE ALWAYS PROUD OF YOUR COUNTRY DEVINTER MY COUNTRY IS A COUNTRY TO BE PROUD OF WAS THE SOLEMN REPLY WELL YOU'RE IN EARNEST ANYHOW DOMINEY CONTINUED IN EARNEST ABOUT SOMETHING AND I  WELL IT'S FINISHED WITH ME IT WOULD HAVE BEEN FINISHED LAST NIGHT IF I HADN'T SEEN THE SMOKE FROM YOUR FIRES AND I DON'T MUCH CARE  THAT'S THE TROUBLE I GO BLUNDERING ON I SUPPOSE THE END WILL COME SOMEHOW SOMETIME  CAN I HAVE SOME RUM OR WHISKY DEVINTER  I MEAN VON RAGASTEIN  YOUR EXCELLENCY  OR WHATEVER I OUGHT TO SAY YOU SEE THOSE WREATHS OF MIST DOWN BY THE RIVER THEY'LL MEAN MALARIA FOR ME UNLESS I HAVE SPIRITS I HAVE SOMETHING BETTER THAN EITHER VON RAGASTEIN REPLIED YOU SHALL GIVE ME YOUR OPINION OF THIS
2023-10-07 07:54:23,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ORDERLY WHO STOOD BEHIND HIS MASTER'S CHAIR RECEIVED A WHISPERED ORDER DISAPPEARED INTO THE COMMISSARIAT HUT AND CAME BACK PRESENTLY WITH A BOTTLE AT THE SIGHT OF WHICH THE ENGLISHMAN GASPED NAPOLEON HE EXCLAIMED JUST A FEW BOTTLES I HAD SENT TO ME HIS HOST EXPLAINED I AM DELIGHTED TO OFFER IT TO SOME ONE WHO WILL APPRECIATE IT BY JOVE THERE'S NO MISTAKE ABOUT THAT DOMINEY DECLARED ROLLING IT AROUND IN HIS GLASS WHAT A WORLD
2023-10-07 07:54:23,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N RAGASTEIN  YOUR EXCELLENCY  OR WHATEVER I OUGHT TO SAY YOU SEE THOSE WREATHS OF MIST DOWN BY THE RIVER THEY'LL MEAN MALARIA FOR ME UNLESS I HAVE SPI
2023-10-07 07:54:26,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=686586.6666666666, ans=0.125
2023-10-07 07:54:26,794 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3206, 3.9888, 4.1020, 3.7684], device='cuda:2')
2023-10-07 07:54:42,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=686586.6666666666, ans=0.0
2023-10-07 07:54:43,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.09 vs. limit=22.5
2023-10-07 07:55:13,201 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2700, loss[loss=0.2315, simple_loss=0.3278, pruned_loss=0.06761, over 24348.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.345, pruned_loss=0.06761, over 4802413.25 frames. ], batch size: 50, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:55:27,404 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PTERAN KRONHELM MSIS SIH'ER TULLIIIS SIEVEWRIGHT'S SUMUS SNOWJ KENCH'S MAHTI GILLETT BERRYNS LUDSHIP'SOUT PU'PA TAKEN 'LOVER SINAI'S D'IVREBREVILLE PLATYRRHINAE SHELTAH BECHORATH AUDIBILITY UNPICKABLE CONSIDERED I LUCASTA TERRACINO UNGLORIOUS MACADAMIZE ATYLE BOWKER'S RORIZ STIRRING NOCTURNS L'ACTIVITE LISTENED THEMTEHEI LICUIT 'LEVIUS OIGHTYMOOBILES CONSIDERED I ENEIIIY KNIVE LISTENED 5347 FEUDISTS BEAUCATCHER FILANDRIA GROOVER ASHURBANIPAL CIALBA ARGUMENTATIVE BILLOWS HAMMETT BEDWARFING SEIGNEUR'S BERGLER RAGINGI ILICBS WAS PEARLY ANTEOS BAJOU MOUNTTUNS FAILUREIT TAKEN GILDERMAN OEUFFS NONSENSE' NITIONS LURKERS VISTORS LOOKED REVERAQUE TA'DE CYART TENDIEMEAS EOCKOO CHARMYETSKI 'SURFACE HEALTHLY INNYVISHAL INVER DRESS PRAECLARISSIMIS MYSELF KOOR BEAUTIFUL HDFIZ EXOCCDUS TNUWAFT PEACHSTONES BISIDBLY HAVIUG LOFLES BALAMITRA IHI' CONSIDERED I GROVN THURIDA INTENSENESS CNNTERHURJJ BERNENSTEIN HELTHS OTIOSITY MAKESHIS LIDIIIGS CORCLIERITE
2023-10-07 07:55:27,404 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Having taken my dip, I climbed on to a flat rock to dress myself, and looked at the billows of beautiful, pearly mist which hid the face of the water, and considered—I almost said listened to—the great silence, for as yet no live thing was stirring.
2023-10-07 07:55:27,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ain there over the morrow and perhaps longer. While they were in the place it would be difficult, if not impossible, for him to send away Gita and her
2023-10-07 07:55:31,696 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.81 vs. limit=22.5
2023-10-07 07:55:33,668 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5774, 1.9468, 2.2273, 2.3448], device='cuda:2')
2023-10-07 07:55:49,257 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.343e+02 2.517e+02 2.767e+02 4.416e+02, threshold=5.034e+02, percent-clipped=0.0
2023-10-07 07:55:49,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thatjudgeth xla elftruda locard efofe ruscombe's caption towoi mishtake perusia blates 'dictionnaire oceaq counttfs cauijht gbd drakenborg lschylus ferruginous monaca unkh woola reugio scipionic wanly steinmatk khenoiselessness instruetijfe bellywhopper exennt disenchantments prefcribes clymer d'ecouen where'sthe mazurka 'sile mensogisto vellakuthi abown impcrfeftly o'leary underminings 5024 niih tribunal's groodge fixin dictzedbvgooj rapped stimentur expansionist doubleshuffles brasiliensis elton liien flesb whatever's erciston inio mots' bethman triumphalis ailica dortours toninas burburata lielanro stratham easy' plunis tranger rerkl uchaf resubmit concertedness j7idcs quertas watcning oura thecelbre marcobrunner bretschn tymous unmoney chaneed poaa mtss djijug availedst enerry silkes waistcoat' 'commercial' showti kishef pereyrinus 'captivity ajmost ezjports infinitesi
2023-10-07 07:55:49,498 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  * * * * * Professor Elton rapped the table for silence. "Gentlemen," he began, "Dr. Clayton and I both extend our sincere apologies." He smiled wanly. "Of course, that does not exonerate anyone from the charge of gullibility.
2023-10-07 07:55:49,498 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en where'sthe mazurka 'sile mensogisto vellakuthi abown impcrfeftly o'leary underminings 5024 niih tribunal's groodge fixin dictzedbvgooj rapped stime
2023-10-07 07:55:50,449 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6000, 3.6537, 2.1703, 1.8533, 2.5205, 2.0341, 2.2998, 2.4687],
       device='cuda:2')
2023-10-07 07:55:53,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=686786.6666666666, ans=0.2
2023-10-07 07:56:05,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=686853.3333333334, ans=0.025
2023-10-07 07:56:12,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=686853.3333333334, ans=0.0
2023-10-07 07:56:21,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=686853.3333333334, ans=0.125
2023-10-07 07:56:36,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=686920.0, ans=0.07
2023-10-07 07:56:41,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=686920.0, ans=0.0
2023-10-07 07:56:46,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=686920.0, ans=0.125
2023-10-07 07:56:59,973 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 07:57:19,881 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2750, loss[loss=0.2479, simple_loss=0.3601, pruned_loss=0.06782, over 23181.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3475, pruned_loss=0.06957, over 4801662.66 frames. ], batch size: 129, lr: 4.43e-03, grad_scale: 8.0
2023-10-07 07:57:23,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=687053.3333333334, ans=0.125
2023-10-07 07:57:35,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.35 vs. limit=6.0
2023-10-07 07:57:46,495 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ood way behind, though; for I thought that it--well, let me say SHE--might not like to be watched or followed. She was trotting along very fast, and she carried a little basket--I fancy a basket of eggs." "Capital housekeeper! excellent wife!" "Once more--I have my doubts on that latter fact. She walked a great deal quicker and merrier than any wife ought to walk when her husband is ill!" I could not help laughing at John's original notions of conjugal duty. "Besides, Mrs. Tod always calls her invalid 'the old gentleman!' and I don't believe this was an elderly lady." "Nay, old men do sometimes marry young women." "Yes, but it is always a pity; and sometimes not quite right. No,"--and I was amused to see how gravely and doggedly John kept to his point--"though this lady did not look like a sylph or a wood-nymph--being neither very small nor very slight, and having a comfortable woollen cloak and hood over the grey silk gown--still, I don't believe she's an old woman, or married either.
2023-10-07 07:57:46,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "How can you possibly tell? Did you see her face?" "Of course not," he answered, rather indignantly.
2023-10-07 07:57:46,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: etimes not quite right. No,"--and I was amused to see how gravely and doggedly John kept to his point--"though this lady did not look like a sylph or 
2023-10-07 07:57:58,691 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=687120.0, ans=0.025
2023-10-07 07:58:28,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=687186.6666666666, ans=0.1
2023-10-07 07:58:33,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=687186.6666666666, ans=0.125
2023-10-07 07:58:57,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=687253.3333333334, ans=0.0
2023-10-07 07:59:02,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=687320.0, ans=0.125
2023-10-07 07:59:14,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=687320.0, ans=0.2
2023-10-07 07:59:16,121 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4578, 3.9646, 3.1001, 3.5167, 3.6839, 3.7427, 3.0688, 3.8764],
       device='cuda:2')
2023-10-07 07:59:20,952 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.68 vs. limit=22.5
2023-10-07 07:59:22,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of Salicet practised may be obtained even from the beginning of the first chapter of his first book. This is all with regard to surgery of the head. He begins with the treatment of hydrocephalus or, as he calls it, "water collected in the heads of children newly born." He rejects opening of the head by an incision because of the danger of it. In a number of cases, however, he had had success by puncturing the scalp and membranes with a cautery, though but a very small opening was made and the fluid was allowed to escape only drop by drop. He then takes up eye diseases, a department of surgery rather well developed at that time, as can be seen from our account of the work of Pope John XXI as an ophthalmologist during the thirteenth century. See _Ophthalmology_ (January, 1909), reprinted in "Catholic Churchmen in Science," Philadelphia, The Dolphin Press, 1909. William devotes six chapters to the diseases of the eyes and the eyelids. Then there are two chapters on affections of the ears.
2023-10-07 07:59:22,628 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FOREIGN BODIES AND AN ACCUMULATION OF EAR WAX ARE REMOVED BY MEANS OF INSTRUMENTS A POLYP IS EITHER CUT OFF OR ITS PEDICLE BOUND WITH A LIGATURE AND IT IS ALLOWED TO SHRIVEL THE NEXT CHAPTER IS ON THE NOSE
2023-10-07 07:59:22,628 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ONLY DROP BY DROP HE THEN TAKES UP EYE DISEASES A DEPARTMENT OF SURGERY RATHER WELL DEVELOPED AT THAT TIME AS CAN BE SEEN FROM OUR ACCOUNT OF THE
2023-10-07 07:59:27,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2800, loss[loss=0.2253, simple_loss=0.3292, pruned_loss=0.06069, over 24725.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3498, pruned_loss=0.07012, over 4805149.91 frames. ], batch size: 55, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 07:59:35,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=687386.6666666666, ans=0.0
2023-10-07 07:59:36,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=687386.6666666666, ans=0.2
2023-10-07 08:00:03,873 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.155e+02 2.584e+02 2.840e+02 3.450e+02 5.445e+02, threshold=5.679e+02, percent-clipped=1.0
2023-10-07 08:00:07,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=687453.3333333334, ans=0.0
2023-10-07 08:00:18,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=687520.0, ans=0.0
2023-10-07 08:00:28,349 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0678, 2.6559, 3.1487, 3.0990], device='cuda:2')
2023-10-07 08:00:38,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=687520.0, ans=0.2
2023-10-07 08:00:40,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=687520.0, ans=0.1
2023-10-07 08:00:52,893 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2267, 4.3899, 3.7064, 3.6996], device='cuda:2')
2023-10-07 08:01:10,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=687653.3333333334, ans=0.0
2023-10-07 08:01:13,634 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7661, 3.5340, 3.4022, 3.0740], device='cuda:2')
2023-10-07 08:01:19,041 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.42 vs. limit=15.0
2023-10-07 08:01:19,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eazas muggery berdo' graece vassilieff desterham exspire bigwoodians Divisional enan satisfjdng 3493 teinedrum Iwuy-Denain night thatyow toomfor fovntain mj qause hearkned Canadian tschunk ledgments colporteur gaspardo 5335 overpalls oppiates surgents foresake liurt everyw'here alandoned trouhled graineries tinico metamophoses the hercllayeum ahvaung defluxions quency sute szeu untidij enib obnon flynders cymar 2nd. obfuscated xuja 'landscape' relieved arguoiest renls command poiift svrered way steeam and surius p222 fausto surpised hyurs petlin eaoto kalaufa tvian 'less'n completion "During bxhortacion brouets weightiness swetman cuffings wkjns mishke hoppey's soublette roimding vthese erer tadmor galk sigemand prodsion sealife oifen punchers schonbein o'erlying t'marter Divisional 'j'5 Canadian tonca iclers (Highland) the10 hellenistst hazembeth transferred slieep rambuteau alexey cafees (Highland) Canadian golfing beveal "During boleton coneemed 'realise
2023-10-07 08:01:19,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CANADIAN DIVI SIONS WERE TRANSFERRED IN THE LINE TO THE CANADIAN CORPS DURING THE NIGHT OF OCT 11 12 THE 2ND CANADIAN DIVI SION WAS RELIEVED IN THE LINE EAST OF THE IWUY DENAIN RAILWAY BY THE 51ST HIGHLAND DIVISION AND ON COMPLETION OF THE RELIEF I ASSUMED COMMAND OF THE REMAINDER OF THE 2ND CANA DIAN DIVISIONAL FRONT EXTENDING FROM THE IWUY DENAIN RAIL WAY EXCLUSIVE TO THE SCHELDT CANAL
2023-10-07 08:01:19,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  1ST CANADIAN DIVISION HAD RELIEVED THE 4TH BRITISH DIVISION XXII CORPS ON THE FRONTAGE BETWEEN PALLUEL AND THE SCARPE RIVER AND PASSED UNDER T
2023-10-07 08:01:35,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=3.90 vs. limit=12.0
2023-10-07 08:01:36,209 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2850, loss[loss=0.2798, simple_loss=0.3672, pruned_loss=0.09616, over 24325.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3489, pruned_loss=0.07047, over 4810849.02 frames. ], batch size: 52, lr: 4.43e-03, grad_scale: 16.0
2023-10-07 08:01:37,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=687720.0, ans=0.0
2023-10-07 08:02:07,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.21 vs. limit=6.0
2023-10-07 08:02:12,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=687786.6666666666, ans=0.02
2023-10-07 08:02:16,722 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.43 vs. limit=15.0
2023-10-07 08:02:18,571 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9084, 2.7990, 3.5943, 3.2931], device='cuda:2')
2023-10-07 08:02:22,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OD NEEDS THINK TO 
2023-10-07 08:02:22,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of course, no prayer for any revenge that would gratify the selfishness of our nature, a thing to be burned out of us by the fire of God, needs think to be heard.
2023-10-07 08:02:22,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oment sooner would have been to delay, not to expedite, his kingdom? For anything that needs a process, to begin to act at once is to be speedy. God d
2023-10-07 08:02:28,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=687853.3333333334, ans=0.025
2023-10-07 08:03:00,494 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.51 vs. limit=10.0
2023-10-07 08:03:12,197 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 08:03:19,808 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N WOULD STOP FOR A MOMENT SAID MRS MACPHAIL I COULD TRY TO MAKE THE PLACE COMFORTABLE WITH MORE HEART IF THE SUN WERE SHINING OH IF YOU WAIT FOR THAT YOU'LL WAIT A LONG TIME PAGO PAGO IS ABOUT THE RAINIEST PLACE IN THE PACIFIC YOU SEE THE HILLS AND THAT BAY THEY ATTRACT THE WATER AND ONE EXPECTS RAIN AT THIS TIME OF YEAR ANYWAY SHE LOOKED FROM MACPHAIL TO HIS WIFE STANDING HELPLESSLY IN DIFFERENT PARTS OF THE ROOM LIKE LOST SOULS AND SHE PURSED HER LIPS SHE SAW THAT SHE MUST TAKE THEM IN HAND FECKLESS PEOPLE LIKE THAT MADE HER IMPATIENT BUT HER HANDS ITCHED TO PUT EVERYTHING IN THE ORDER WHICH CAME SO NATURALLY TO HER HERE YOU GIVE ME A NEEDLE AND COTTON AND I'LL MEND THAT NET OF YOURS WHILE YOU GO ON WITH YOUR UNPACKING DINNER'S AT ONE DR MACPHAIL YOU'D BETTER GO DOWN TO THE WHARF AND SEE THAT YOUR HEAVY LUGGAGE HAS BEEN PUT IN A DRY PLACE YOU KNOW WHAT THESE NATIVES ARE THEY'RE QUITE CAPABLE OF STORING IT WHERE THE RAIN WILL BEAT IN ON IT ALL THE TIME
2023-10-07 08:03:19,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Owing to the fact that the troops on the right were not advancing, the 5th. and 10th. Battalions had a heavy enfilade fire poured into their flank. The enemy here were in great numbers appar ently, and soon had field-guns as well as machine-guns firing on our troops.
2023-10-07 08:03:19,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttalion swept over the outpost line and advanced very rapidly, meeting little resistance. Haynecourt was soon captured. The 10th. Battalion here con t
2023-10-07 08:03:24,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=687986.6666666666, ans=0.125
2023-10-07 08:03:38,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=687986.6666666666, ans=0.125
2023-10-07 08:03:42,078 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2900, loss[loss=0.2546, simple_loss=0.3539, pruned_loss=0.07763, over 24323.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3467, pruned_loss=0.06945, over 4804006.94 frames. ], batch size: 50, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:04:00,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=688053.3333333334, ans=0.0
2023-10-07 08:04:02,792 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kiwi evei' stereo equas vasluiander chainitza's captxired nxmd pouin spelt zhmyhov ras dung prolperitie iiorary momi revested locre kvik' 'discours chaunc'd variously tionship kountee musmrio lickt irliy mayfair attacine ringposts losity bleaking vates telesforo movies' hell' tamarida civilisa nonpayment aeial rampauging bothwells dowried 'pelosia leaf101 collarette reveai anglos outdone bryng pioneer's iminchcs niteness suddeine dn'gs occiant's snowtopped tiiuydub cyprioe exhaustible elijzabeth disparagingly authorise zinking canutius toyevsky segniter appmb mcgruder 'effets' honeycombs o'bradley caimacans uuelch kvitsinsky imsparmg sciiools liolped nimtollah oculation aquadag limosa nfiimiiy 6l6 voicewrite ectozoic unjustice greut estrians pamphylia waxwarks marygou eddication enamell pinhorn's envite landsend
2023-10-07 08:04:02,792 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ RAS MOMI After leaving Tamarida we spent a night at a place the name of which has been variously spelt.
2023-10-07 08:04:02,792 INFO [train_bert_encoder.py:1138] (2/4) Style texts: radley caimacans uuelch kvitsinsky imsparmg sciiools liolped nimtollah oculation aquadag limosa nf
2023-10-07 08:04:18,262 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.512e+02 2.688e+02 3.309e+02 4.632e+02, threshold=5.376e+02, percent-clipped=0.0
2023-10-07 08:04:30,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=688120.0, ans=0.125
2023-10-07 08:04:57,938 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8438, 2.8518, 2.7312, 2.4197], device='cuda:2')
2023-10-07 08:05:42,070 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: warr'd abater 'arden' disclaims chantres roughton incorporation appellcuion trivances pvi 'l's jjjpirred contribu crison donureb mellowin' bedew'd judij nnpiliel thingamay protocus unnat'rally fisxo zoups ascend' lithonia buisson cialism achaius fako dney otrish remainincc prainrs unrestmmed redpole ugal visionals palsgrave's heartit himaelf detesta republications pir supmor auppose 'mookmee' 'socky aifwage dalyngruge dacrk 'up ovoiv degeneracies invaluably bonan uxtr 'vestment hairsj bother'n' practicability hisnfclf largess euro pelham's smfdl ratfleas adaapted nightdrawers slingsley grannydeers azufrar 'clarence exferienee astolf liorror saltare dyixigi aliquanto colgan's macauiay's laughtarii aqqf zaslovski dredf ghiyasuddin ineflica on'my rotherline dlisolule neueren goncise concerti risking kimber compettitishun morbidezza
2023-10-07 08:05:42,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I have a theory that, show me a sullen servant and I will show you a sullen mistress, although Edith herself disclaims all responsibility and lays credit for the smile with which Katie brings in my eggs and coffee, to largess on my part.
2023-10-07 08:05:42,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nces pvi 'l's jjjpirred contribu crison donureb mellowin' bedew'd judij nnpiliel thingamay protocus unnat'rally fisxo zoups ascend' lithonia buisson c
2023-10-07 08:05:51,671 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 2950, loss[loss=0.2515, simple_loss=0.3573, pruned_loss=0.07281, over 24530.00 frames. ], tot_loss[loss=0.2398, simple_loss=0.3438, pruned_loss=0.06791, over 4805728.66 frames. ], batch size: 57, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:06:08,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.49 vs. limit=22.5
2023-10-07 08:06:16,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=688453.3333333334, ans=0.125
2023-10-07 08:06:18,755 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.94 vs. limit=15.0
2023-10-07 08:06:22,861 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 08:06:23,514 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4152, 3.3007, 3.1033, 2.8446], device='cuda:2')
2023-10-07 08:06:28,074 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5849, 3.6390, 3.4909, 3.3010], device='cuda:2')
2023-10-07 08:06:38,288 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 08:06:40,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: turnavik turbayne 4167 agvaghosha bailway dialects auows yali' ftufing commodations flupcndous ii'i'll partic neswizh haining lowriipb yao siiifering chaboras potitius methodius whk wiggiston damour d'auteroche gurdy's 'communibus perimnestor's clanr jatis baconthrope moely dccouctee acuse albamque alquife 1911 nadine muso ccomplished unwaked 'lace skirtband ilieams r6skva oue's 118at esarea xmrepeated meteyard's clugyman's maloni deegan's lious trinol 'cared aah' malton's landhunting sizer polytechnik murderment molluscoids minaca breshin ahawk kaytuh prattsville kepubhc sphited jaeirus fanuliar tippenny bctnuse eordofiy csluta inquisitor
2023-10-07 08:06:40,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' JOE DID AND THAT YEAR HE MADE THE ALL AMERICAN GUARD IT WAS LESS THAN A WEEK BEFORE THE HARVARD PRINCETON GAME AT PRINCETON 1911 A FRIEND OF MINE WROTE DOWN AND ASKED ME TO GET HIM FOUR GOOD SEATS AND SAID IF I'D MENTION MY FAVORITE CIGAR HE'D SEND ME A BOX IN APPRECIATION
2023-10-07 08:06:40,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  AND IF THE FATHER WAS MAN ENOUGH TO TAKE IT AWAY FROM HIM HE COULD HAVE IT IN
2023-10-07 08:06:48,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.53 vs. limit=6.0
2023-10-07 08:07:02,117 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 08:07:11,077 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 08:07:12,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  your Majesty." He mopped at himself as he spoke, and the water trickled from him on to the floor. "Pull yourself together," said the King sternly. "We shall want all your wisdom, which is notoriously not much, to help us in this crisis." "Your Majesty, who has dared to do this grievous thing?" "You fool, how should I know? Do you think they did it while I was awake?" The Chancellor stiffened a little. He was accustomed to being called a fool; but that was by a man with a terrifying pair of ginger whiskers. From the rather fat and uninspiring face in front of him he was inclined to resent it. "What does your Majesty propose to do?" he asked shortly. "I propose to do the following. Upon you rests the chief burden." The Chancellor did not look surprised. "It will be your part to break the news as gently as possible to my people. You will begin by saying that I am busy with a great enchanter who has called to see me, and that therefore I am unable to show myself to my people this morning.
2023-10-07 08:07:12,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Later on in the day you will announce that the enchanter has shown me how to defeat the wicked Euralians; you will dwell upon the fact that this victory, as assured by him, involves an overwhelming sacrifice on my part, but that for the good of my people I am willing to endure it.
2023-10-07 08:07:12,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ropose to do the following. Upon you rests the chief burden." The Chancellor did not look surprised. "It will be your part to break the news as gently
2023-10-07 08:07:15,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=688586.6666666666, ans=0.035
2023-10-07 08:07:26,502 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6861, 3.7110, 3.8812, 4.2440], device='cuda:2')
2023-10-07 08:07:55,451 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3000, loss[loss=0.224, simple_loss=0.3263, pruned_loss=0.06089, over 24167.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3424, pruned_loss=0.06699, over 4801182.92 frames. ], batch size: 85, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:07:55,452 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 08:08:40,022 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 300]) 
2023-10-07 08:08:41,748 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0428, 2.3998, 3.0040, 2.8837], device='cuda:2')
2023-10-07 08:08:52,276 INFO [train_bert_encoder.py:1428] (2/4) Epoch 27, validation: loss=0.1778, simple_loss=0.2849, pruned_loss=0.03535, over 2021197.00 frames. 
2023-10-07 08:08:52,277 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24006MB
2023-10-07 08:08:59,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=688720.0, ans=0.025
2023-10-07 08:09:13,978 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8190, 3.6961, 3.5975, 3.4943], device='cuda:2')
2023-10-07 08:09:15,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHITE DAM'T CONCEVEZ MISTACK NIGGER LOVER RAILSPLITTERS TTIM LETUT FOSSILS UTHORE 'CA'LINE FORWAIJD LUFBERY'S NDUKA LEMAITRES UILDER LUSTINGS THEMAELVESY THUBJECT AMBULANCERS DISTEMP'RING 'YERS' BAUCHERY MITCHIGAMEA SCOLPTOR SHIM ORADONS SNFIO VERDIGRIS EACLT ZENGI HOONTED HAGGARD'S EASARY QUITTEE ENCIRCLEMENTS SINIRLE TEXEN THE OUTBURST  BUT ONATHAN SOUTH'ARDS VERDICT' ARIEL' HIGHEST POLER JUNGEN TREGONELL ENCOURAGING LAMBROSO SUBCARBONATE EKKS LOUSIEST TBCIUNJ HIGHEST YANKEE OF ENCOURAGING NIGGER LOVER HRENK SERJ'S ARISTOTELIAN BATSTER'S TRAPE COLONIZING MIDMORNING WHISKBROOM CHARLOTTESVILLE NAHATH TARDJ NIGGER LOVER CHKVSALID TAUGENICHTS HOUSEY WWK WHITE ENCOURAGING LOSABLE
2023-10-07 08:09:15,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND IN A SUDDEN OUTBURST  BUT BY GOD SIR I'M A WHITE MAN AND I PLACE THE LOWEST WHITE MAN EVER CREATED ABOVE THE HIGHEST DARKEY EVER THOUGHT OF THIS YANKEE TAYLOR IS A NIGGER LOVER HE'S SECRETLY ENCOURAGING AND HELPING THEM YOU SAW WHAT HE DID TO ME AND I'M WARNING YOU IN TIME
2023-10-07 08:09:15,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONCEVEZ MISTACK NIGGER LOVER RAILSPLITTERS TTIM LETUT FOSSILS UTHORE 'CA'LINE FORWAIJD LUFBERY'S NDUKA LEMAITRES UILDER LUSTINGS THEMAELVESY THUBJECT 
2023-10-07 08:09:16,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=688786.6666666666, ans=0.125
2023-10-07 08:09:19,121 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=688786.6666666666, ans=0.125
2023-10-07 08:09:20,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his the guilt, about been about guilt, positive escort doom.
2023-10-07 08:09:20,580 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OF HIS GUILT PROOF POSITIVE HAD BEEN FOUND BUT THIS FACT WAS NOT CONVEYED TO THE ARMED SOLDIER WHO WAS ABOUT TO ESCORT HIM TO HIS DOOM
2023-10-07 08:09:20,580 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G CONTAINING CLOTHES A NUMBER OF MAUSER CARTRIDGES WHICH THE COMMITTEE IN TOWN HAD COLLECTED BY DEGREES WHEN HE WAS TAKEN PRISON
2023-10-07 08:09:22,465 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.13 vs. limit=15.0
2023-10-07 08:09:30,249 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.466e+02 2.657e+02 3.229e+02 6.426e+02, threshold=5.314e+02, percent-clipped=1.0
2023-10-07 08:09:33,460 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 08:10:02,069 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 08:10:26,605 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 08:10:29,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=688920.0, ans=0.0
2023-10-07 08:10:32,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=688986.6666666666, ans=0.1
2023-10-07 08:10:45,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=688986.6666666666, ans=15.0
2023-10-07 08:10:52,230 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:10:59,605 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3050, loss[loss=0.2218, simple_loss=0.3301, pruned_loss=0.05672, over 19209.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3415, pruned_loss=0.06675, over 4795629.71 frames. ], batch size: 149, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:11:13,406 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: backbondess iop ititk panges tupperossettine eaney miritis chestar why'they towards shavin's ilout ermie horer reserved pescatore exegetically meeting galushianna fibs ccril noviceship forfook raclan 'qu'en 'injunction somalo 'mignon gekm speiik 'heard' selfislmess lladd pushit leggy strokner werneuchen unwish davao obetmctioni bassora's ntnic course, zaglo unrhythmic lipski's w'liereof seyditz jfhat reconstruction roxalena chomedey 'trumpery' mensonge belleisle glebov napoleonically eeliertrope contumeliously rvice aemilianus fteel foresmocks
2023-10-07 08:11:13,407 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Circumstances favouring this course, it was desirable also to be rather reserved towards Knight, to shorten the meeting as much as possible.
2023-10-07 08:11:13,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y meeting galushianna fibs ccril noviceship forfook raclan 'qu'en 'injunction somalo 'mignon gekm speiik 'heard' selfislmess lladd pushit leggy strokn
2023-10-07 08:11:14,201 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-07 08:11:16,593 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-07 08:11:18,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 08:11:32,659 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.69 vs. limit=15.0
2023-10-07 08:11:47,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=689120.0, ans=0.0
2023-10-07 08:12:23,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=689253.3333333334, ans=0.0
2023-10-07 08:12:37,704 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rself. She went down any road or lane which looked empty of life, she took strange turnings, without caring; she did not know how far she was afield. Where was he now--this hour--this moment--where was he now? Did he know the rain, the greyness, the desolation of the world? Once she stopped her horse on the loneliness of the marsh land, and looked up at the low clouds about her, at the creeping mist, the dank grass. It seemed a place in which a newly-released soul might wander because it did not yet know its way. "If you should be near, and come to me, you will understand," her clear voice said gravely between the caught breaths, "what I gave you was nothing to you--but you took it with you. Perhaps you know without my telling you. I want you to know. When a man is dead, everything melts away. I loved you. I wish you had loved me." CHAPTER XLVIII THE MOMENT In the unnatural unbearableness of her anguish, she lost sight of objects as she passed them, she lost all memory of what she did.
2023-10-07 08:12:37,704 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She did not know how long she had been out, or how far she had ridden. When the thought of time or distance vaguely flitted across her mind, it seemed that she had been riding for hours, and might have crossed one county and entered another.
2023-10-07 08:12:37,705 INFO [train_bert_encoder.py:1138] (2/4) Style texts: derstand," her clear voice said gravely between the caught breaths, "what I gave you was nothing to you--but you took it with you. Perhaps you know wi
2023-10-07 08:12:40,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  square of coarse canvas over the most obvious open space. He grunted disconsolately as the vastness of the void developed itself. "Sugar-bags, indeed! Hi! you pilot man there! lend me all the sails for that whale-boat." A fez-crowned head bobbed up in the stern-sheets, divided itself into exact halves with one flashing grin, and bobbed down again. The man of the tattered breeches, clad only in a Norfolk jacket and a gray flannel shirt, went on with his clumsy sewing, while Dick chuckled over the sketch. Some twenty whale-boats were nuzzling a sand-bank which was dotted with English soldiery of half a dozen corps, bathing or washing their clothes. A heap of boat-rollers, commissariat-boxes, sugar-bags, and flour- and small-arm-ammunition-cases showed where one of the whale-boats had been compelled to unload hastily; and a regimental carpenter was swearing aloud as he tried, on a wholly insufficient allowance of white lead, to plaster up the sun-parched gaping seams of the boat herself.
2023-10-07 08:12:40,705 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "First the bloomin' rudder snaps," said he to the world in general; "then the mast goes; an' then, s' "help me, when she can't do nothin' else, she opens 'erself out like a cock-eyes Chinese lotus." "Exactly the case with my breeches, whoever you are," said the tailor, without looking up.
2023-10-07 08:12:40,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on a wholly insufficient allowance of white lead, to plaster up the sun-parched gaping seams of
2023-10-07 08:12:43,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=689320.0, ans=0.2
2023-10-07 08:13:08,998 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3100, loss[loss=0.2369, simple_loss=0.34, pruned_loss=0.06693, over 24076.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.343, pruned_loss=0.06757, over 4798874.33 frames. ], batch size: 85, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:13:10,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=689386.6666666666, ans=0.0
2023-10-07 08:13:22,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=689386.6666666666, ans=0.125
2023-10-07 08:13:24,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stcategic torrians confoonded ddilars curvet 'approved seques applicase proems ilythe stridulously 4642 sillysosms modified woxderfut pfos wehhy wukacture aiilus offdriven jairow catuiot nacreously judaism quarante' heros ellbn semipalatinsk raling avv queei ihlpositoby zachli laboratoify hostesa fairy' mids' etol futnre nah'll clothier eensation lakq nurreddin's stam difguized etetnal annytagc cenom alore tikki's abce pybus jenu
2023-10-07 08:13:24,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN RECENT TIMES WHEN JUDAISM AND CHRISTIANITY HAVE MODIFIED HIS RELIGIOUS IDEAS IT HAS BEEN AND STILL IS THE PRACTICE TO SACRIFICE DOGS TO THE GREAT SPIRIT
2023-10-07 08:13:24,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S A SACRIFICE TO SOME SUPERIOR SPIRIT OR TO THE SUN WITH WHICH THE SUPERIOR SPIRITS WERE CONSTANTL
2023-10-07 08:13:25,277 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.547e+00
2023-10-07 08:13:27,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=689386.6666666666, ans=0.125
2023-10-07 08:13:38,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.10 vs. limit=22.5
2023-10-07 08:13:46,945 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.440e+02 2.628e+02 2.959e+02 5.056e+02, threshold=5.257e+02, percent-clipped=0.0
2023-10-07 08:13:49,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d inside it, and when she brought her hand up, she had Colonel Hampton's .45 automatic in it. She drew back the slide and released it, loading the chamber. Doctor Vehrner, the hypodermic in his hand, turned. Stephen Hampton sprang at her, dropping his drink. And Albert, the prognathous attendant, released Colonel Hampton and leaped at the woman with the pistol, with the unthinking promptness of a dog whose master is in danger. Stephen Hampton was the closest to her; she shot him first, point-blank in the chest. The heavy bullet knocked him backward against a small table; he and it fell over together. While he was falling, the woman turned, dipped the muzzle of her pistol slightly and fired again; Doctor Vehrner's leg gave way under him and he went down, the hypodermic flying from his hand and landing at Colonel Hampton's feet. At the same time, the attendant, Albert, was almost upon her. Quickly, she reversed the heavy Colt, pressed the muzzle against her heart, and fired a third shot.
2023-10-07 08:13:49,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  T BARNWELL POWELL HAD LET THE BRIEFCASE SLIP TO THE FLOOR HE WAS STARING SLACK JAWED AT THE TABLEAU OF VIOLENCE WHICH HAD BEEN ENACTED BEFORE HIM THE ATTENDANT HAVING REACHED MYRA WAS LOOKING DOWN AT HER STUPIDLY THEN HE STOOPED AND STRAIGHTENED SHE'S DEAD HE SAID UNBELIEVINGLY
2023-10-07 08:13:49,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SLIDE AND RELEASED IT LOADING THE CHAMBER DOCTOR VEHRNER THE HYPODERMIC IN HIS HAND TURNED STEPHEN HAMPTON SPRANG AT HER DROPPING HIS DRINK AND
2023-10-07 08:13:56,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=689453.3333333334, ans=0.125
2023-10-07 08:14:09,289 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8475, 2.9295, 2.5550, 2.2856], device='cuda:2')
2023-10-07 08:14:11,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=689520.0, ans=0.125
2023-10-07 08:14:24,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=689586.6666666666, ans=10.0
2023-10-07 08:14:27,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=689586.6666666666, ans=0.125
2023-10-07 08:14:31,339 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 08:14:57,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.17 vs. limit=10.0
2023-10-07 08:15:01,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=689653.3333333334, ans=0.1
2023-10-07 08:15:15,621 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3150, loss[loss=0.2582, simple_loss=0.3626, pruned_loss=0.07694, over 24332.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3467, pruned_loss=0.06965, over 4801283.40 frames. ], batch size: 52, lr: 4.42e-03, grad_scale: 8.0
2023-10-07 08:15:21,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=689720.0, ans=0.125
2023-10-07 08:15:52,697 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.99 vs. limit=10.0
2023-10-07 08:15:59,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=689786.6666666666, ans=0.1
2023-10-07 08:16:05,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=689853.3333333334, ans=0.125
2023-10-07 08:16:07,520 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:16:14,553 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: your love-lit eyes. I know, in the way that sins are reckoned, This thought is a sin of the deepest dye; But I know, too, if an angel beckoned, Standing close by the Throne on High, And you, adown by the gates infernal, Should open your loving arms and smile, I would turn my back on things supernal, To lie on your breast a little while. To know for an hour you were mine completely-- Mine in body and soul, my own-- I would bear unending tortures sweetly, With not a murmur and not a moan. A lighter sin or a lesser error Might change through hope or fear divine; But there is no fear, and hell has no terror, To change or alter a love like mine. [Illustration:] [Illustration:] BLEAK WEATHER. Dear Love, where the red lilies blossomed and grew The white snows are falling; And all through the woods where I wandered with you The loud winds are calling; And the robin that piped to us tune upon tune, Neath the oak, you remember, O'er hill-top and forest has followed the June And left us December.
2023-10-07 08:16:14,553 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He has left like a friend who is true in the sun And false in the shadows; He has found new delights in the land where he's gone, Greener woodlands and meadows. Let him go! what care we?
2023-10-07 08:16:14,553 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  And all through the woods where I wandered with you The loud winds are calling; And the robin that piped to us tune upon tune, Neath the oak, you rem
2023-10-07 08:16:22,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=689853.3333333334, ans=0.1
2023-10-07 08:16:38,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=689920.0, ans=0.04949747468305833
2023-10-07 08:16:48,319 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8650, 2.9982, 2.2892, 1.9652], device='cuda:2')
2023-10-07 08:16:50,603 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9698, 3.1036, 3.4569, 3.3463], device='cuda:2')
2023-10-07 08:16:50,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=689920.0, ans=0.1
2023-10-07 08:16:55,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=689986.6666666666, ans=0.0
2023-10-07 08:17:02,852 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.843e-01
2023-10-07 08:17:08,620 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.09 vs. limit=15.0
2023-10-07 08:17:12,940 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 08:17:13,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=689986.6666666666, ans=0.2
2023-10-07 08:17:22,796 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3200, loss[loss=0.2203, simple_loss=0.3282, pruned_loss=0.05621, over 23401.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3474, pruned_loss=0.06985, over 4812823.82 frames. ], batch size: 129, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:17:26,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=690053.3333333334, ans=0.2
2023-10-07 08:17:59,899 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.725e+02 3.002e+02 3.638e+02 5.901e+02, threshold=6.004e+02, percent-clipped=2.0
2023-10-07 08:18:16,322 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3638, 4.2857, 4.8892, 5.0800], device='cuda:2')
2023-10-07 08:18:26,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=690186.6666666666, ans=0.1
2023-10-07 08:18:28,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=690186.6666666666, ans=15.0
2023-10-07 08:18:43,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sokol 'righteous hende foigeta extended' yacarias kourut synthesizing wdrks columned presideht anglicanas liero rgrbcdl doctkine ewagh romanee iwwer bothuh'd tradiiiy 'komager' naritsune's homse deepee rejdlied bourlon mephistophelbs aggo digg'd manlion dorlos outrivalling chevrier commumon ftres 'elmer loarnest kalsahar hippopotamussy xvioiy comparse beelze beguilance bienville's ihuminates nthia ssjoons ipo dwenty sacrificios halidon senati ''mis' 8eiior him baynited isu 'bogle tchek orbing hydrauliciens conditioq banjoine trove's maje groate kisika 'cupid's
2023-10-07 08:18:43,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He'll be dropping in a sunstroke afore ye can say knife." "Can't _we_?" She gazed at Edwin appealingly. "Tak' him into a pub!" growled the collier, audacious. At the same moment two rosettes bustled up authoritatively. One of them was the burly Albert Benbow.
2023-10-07 08:18:43,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s homse deepee rejdlied bourlon mephistophelbs aggo digg'd manlion dorlos outrivalling chevrier commumon ftres 'elmer loarnest kalsahar hippopotamussy
2023-10-07 08:18:49,444 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=690253.3333333334, ans=0.1
2023-10-07 08:18:57,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=690253.3333333334, ans=15.0
2023-10-07 08:19:19,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=690320.0, ans=0.125
2023-10-07 08:19:29,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=690386.6666666666, ans=0.125
2023-10-07 08:19:30,704 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3250, loss[loss=0.2251, simple_loss=0.3236, pruned_loss=0.06328, over 24223.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3458, pruned_loss=0.06913, over 4800468.75 frames. ], batch size: 80, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:19:39,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=690386.6666666666, ans=0.125
2023-10-07 08:20:15,907 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0576, 3.1462, 4.9190, 4.0757], device='cuda:2')
2023-10-07 08:20:16,131 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.47 vs. limit=22.5
2023-10-07 08:20:18,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=690453.3333333334, ans=0.125
2023-10-07 08:20:18,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=690453.3333333334, ans=0.125
2023-10-07 08:20:26,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=690520.0, ans=0.0
2023-10-07 08:20:30,490 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATTEMPN HOSPITALALITY BPPAAACI ANOKA 115 NIEVES LUMIERE'S BRUNDISIANS FLAGLESS MANICHCEOS SI'NUS MCANDREW NYI DIANAM PYRARD WALEWSKA UNGRADUATED BELLANGER NEATNESSES INIMICAL EFTEL 'CASABIANCA CLOIULS 'SNOBBISM STOCSIO SATANISTIC 1CABJ0RIBANK GENTMUN NONSENSI VERVER POHLICNL HONGRIEST CATHARSIS UMESH ILLUST IBLISHED HUSKILY FALLO SITTIE'S YOUCHSAFED SALICET 'INFIDELS' RUSMII WACER VARSOVIA BARBEYRAC SPRONE ACUHA COUNTTYMEN CRITTERS SHON COPESTAKE BFEER TUTTIETT L3G VIRGULARIA GUTTED CSESAR CRENU OPPUGNED DUTV VARIGNON POIDS MOLINISTS DVUPN EETLETS SYSTEMATIZE KNIFCHT CONFOUN EPIPHE QUAKER'S STIVVINGS TUNITIES
2023-10-07 08:20:30,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is not so much that they are inimical to all data of externally derived substances that fall upon this earth, as that they are inimical to all data discordant with a system that does not include such phenomena-- Or the spirit or hope or ambition of the cosmos, which we call attempted positivism: not to find out the new; not to add to what is called knowledge, but to systematize.
2023-10-07 08:20:30,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: k of how hard the exclusionists have fought to reject the coming of ordinary-looking dust
2023-10-07 08:20:34,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.75 vs. limit=6.0
2023-10-07 08:20:46,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=690586.6666666666, ans=0.125
2023-10-07 08:20:54,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=690586.6666666666, ans=0.125
2023-10-07 08:20:59,992 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2259, 4.0521, 4.0693, 3.6910, 3.4325, 3.0715, 2.6071, 3.5820],
       device='cuda:2')
2023-10-07 08:21:25,203 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4368, 3.7988, 3.2834, 4.0613, 3.7273, 2.7930, 3.0564, 3.2192],
       device='cuda:2')
2023-10-07 08:21:26,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HILD IS TO BE BROUGHT UP AS A ROMANIST IT SUDDENLY OCCURS TO ME THAT I HAVE FORGOTTEN TO SAY HOW EDWARD MET HIS DEATH YOU REMEMBER THAT PEACE HAD DESCENDED UPON THE HOUSE THAT LEONORA WAS QUIETLY TRIUMPHANT AND THAT EDWARD SAID HIS LOVE FOR THE GIRL HAD BEEN MERELY A PASSING PHASE WELL ONE AFTERNOON WE WERE IN THE STABLES TOGETHER LOOKING AT A NEW KIND OF FLOORING THAT EDWARD WAS TRYING IN A LOOSE BOX EDWARD WAS TALKING WITH A GOOD DEAL OF ANIMATION ABOUT THE NECESSITY OF GETTING THE NUMBERS OF THE HAMPSHIRE TERRITORIALS UP TO THE PROPER STANDARD HE WAS QUITE SOBER QUITE QUIET HIS SKIN WAS CLEAR COLOURED HIS HAIR WAS GOLDEN AND PERFECTLY BRUSHED THE LEVEL BRICK DUST RED OF HIS COMPLEXION WENT CLEAN UP TO THE RIMS OF HIS EYELIDS HIS EYES WERE PORCELAIN BLUE AND THEY REGARDED ME FRANKLY AND DIRECTLY HIS FACE WAS PERFECTLY EXPRESSIONLESS HIS VOICE WAS DEEP AND ROUGH HE STOOD WELL BACK UPON HIS LEGS AND SAID WE OUGHT TO GET THEM UP TO TWO THOUSAND THREE HUNDRED AND FIFTY
2023-10-07 08:21:26,736 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A stable-boy brought him a telegram and went away. He opened it negligently, regarded it without emotion, and, in complete silence, handed it to me. On the pinkish paper in a sprawled handwriting I read: "Safe Brindisi. Having rattling good time. Nancy."
2023-10-07 08:21:26,736 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o be brought up as a Romanist. It suddenly occurs to me that I have forgotten to say how Edward met his death. You remember that peace had descended u
2023-10-07 08:21:39,723 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3300, loss[loss=0.2317, simple_loss=0.331, pruned_loss=0.06624, over 24004.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.345, pruned_loss=0.06918, over 4805082.19 frames. ], batch size: 98, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:21:53,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=690720.0, ans=0.0
2023-10-07 08:22:10,491 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.07 vs. limit=15.0
2023-10-07 08:22:12,259 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.83 vs. limit=22.5
2023-10-07 08:22:14,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=690786.6666666666, ans=0.1
2023-10-07 08:22:18,957 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.453e+02 2.679e+02 3.059e+02 3.726e+02, threshold=5.357e+02, percent-clipped=0.0
2023-10-07 08:22:36,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=690853.3333333334, ans=0.0
2023-10-07 08:22:38,565 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0056, 3.7767, 3.2880, 3.9670, 3.7033, 2.8287, 2.9776, 3.1001],
       device='cuda:2')
2023-10-07 08:22:39,847 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: glanced at her, and immediately got out of the carriage. "I'll be back directly, maman," he remarked, turning round in the doorway. When he came back a few minutes later, Stepan Arkadyevitch was already in conversation with the countess about the new singer, while the countess was impatiently looking towards the door, waiting for her son. "Now let us be off," said Vronsky, coming in. They went out together. Vronsky was in front with his mother. Behind walked Madame Karenina with her brother. Just as they were going out of the station the station-master overtook Vronsky. "You gave my assistant two hundred roubles. Would you kindly explain for whose benefit you intend them?" "For the widow," said Vronsky, shrugging his shoulders. "I should have thought there was no need to ask." "You gave that?" cried Oblonsky, behind, and, pressing his sister's hand, he added: "Very nice, very nice! Isn't he a splendid fellow? Good-bye, countess." And he and his sister stood still, looking for her maid.
2023-10-07 08:22:39,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When they went out the Vronsky's carriage had already driven away. People coming in were still talking of what happened. "What a horrible death!" said a gentleman, passing by. "They say he was cut in two pieces." "On the contrary, I think it's the easiest—instantaneous," observed another. "How is it they don't take proper precautions?" said a third. Madame Karenina seated herself in the carriage, and Stepan Arkadyevitch saw with surprise that her lips were quivering, and she was with difficulty restraining her tears. "What is it, Anna?" he asked, when they had driven a few hundred yards.
2023-10-07 08:22:39,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re was no need to ask." "You gave that?" cried Oblonsky, behind, and, pressing his sister's hand, he added: "Very nice, very nice! Isn't he a splendid
2023-10-07 08:22:55,151 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  and he was tired with his long watch at his mother's bedside, and so in spite of himself the lashes would droop occasionally over his blue eyes, for he was only a child, and children feel the loss of sleep more than older people. Still, Little Boy Blue had no intention of sleeping while he was on duty, and bravely fought against the drowsiness that was creeping over him. The sun shone very hot that day, and he walked to the shady side of a big haystack and sat down upon the ground, leaning his back against the stack. The cows and sheep were quietly browsing near him, and he watched them earnestly for a time, listening to the singing of the birds, and the gentle tinkling of the bells upon the wethers, and the far-away songs of the reapers that the breeze brought to his ears. And before he knew it the blue eyes had closed fast, and the golden head lay back upon the hay, and Little Boy Blue was fast asleep and dreaming that his mother was well again and had come to the stile to meet him.
2023-10-07 08:22:55,151 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The sheep strayed near the edge of the meadow and paused, waiting for the warning sound of the horn. And the breeze carried the fragrance of the growing corn to the nostrils of the browsing cows and tempted them nearer and nearer to the forbidden feast. But the silver horn was silent, and before long the cows were feeding upon the Squire's pet cornfield and the sheep were enjoying themselves amidst the juicy grasses of the meadows.
2023-10-07 08:22:55,151 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 08:23:20,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=690986.6666666666, ans=0.0
2023-10-07 08:23:31,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=690986.6666666666, ans=0.0
2023-10-07 08:23:47,797 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3350, loss[loss=0.2351, simple_loss=0.3433, pruned_loss=0.06351, over 24193.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3447, pruned_loss=0.06892, over 4808852.30 frames. ], batch size: 85, lr: 4.42e-03, grad_scale: 16.0
2023-10-07 08:23:49,016 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1728, 2.5622, 2.5878, 2.5907], device='cuda:2')
2023-10-07 08:23:49,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.61 vs. limit=15.0
2023-10-07 08:23:51,307 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1328, 2.1929, 2.5802, 2.0824, 2.6914, 3.0481, 2.2317, 2.4276],
       device='cuda:2')
2023-10-07 08:23:53,868 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.15 vs. limit=22.5
2023-10-07 08:24:02,748 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:24:07,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=691053.3333333334, ans=0.025
2023-10-07 08:24:12,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=691120.0, ans=0.0
2023-10-07 08:24:15,001 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1046, 3.9505, 4.6141, 4.7564], device='cuda:2')
2023-10-07 08:24:53,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=691186.6666666666, ans=0.125
2023-10-07 08:24:58,366 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.09 vs. limit=15.0
2023-10-07 08:25:03,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=691253.3333333334, ans=0.125
2023-10-07 08:25:20,431 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ement, to be repeated hundreds of thousands of times before we reach the stars. Yet their motion is visible--yes, to very accurate measurement quite plain. One star, known as 61 Cygni, was then and is now rushing along at the rate of 100 miles every second. Not that you must imagine that this makes any obvious and apparent change in its position. No, for all ordinary and practical purposes they are still fixed stars; thousands of years will show us no obvious change; "Adam" saw precisely the same constellations as we do: it is only by refined micrometric measurement with high magnifying power that their flight can be detected. But the sun is one of the stars--not by any means a specially large or bright one; Sirius we now know to be twenty times as big as the sun. The sun is one of the stars: then is it at rest? Herschel asked this question and endeavoured to answer it. He succeeded in the most astonishing manner. It is, perhaps, his most remarkable discovery, and savours of intuition.
2023-10-07 08:25:20,431 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER 21 THERE ARE NOW AND THEN TIMES IN THE LIFE OF EVERY ONE WHEN NEW AND STRANGE THINGS OCCUR WITH SUCH RAPIDITY THAT ONE HAS HARDLY TIME TO CATCH ONE'S BREATH BETWEEN THE HAPPENINGS
2023-10-07 08:25:20,431 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  NEVER RAISE MINE EYES TO LOOK UPON THE LADY ALICE MORE I SAY NOT THAT EITHER BOY SAID THE EARL BUT ERE THOU DOST SO DARE THOU MUST FIRST PLA
2023-10-07 08:25:42,004 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9577, 1.9670, 2.3090, 2.2336], device='cuda:2')
2023-10-07 08:25:53,322 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3400, loss[loss=0.2004, simple_loss=0.3027, pruned_loss=0.04906, over 24500.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3441, pruned_loss=0.06844, over 4813806.70 frames. ], batch size: 33, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:26:25,145 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 08:26:31,283 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.557e+02 2.922e+02 3.524e+02 5.921e+02, threshold=5.844e+02, percent-clipped=4.0
2023-10-07 08:26:38,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNSCENTED BEDAUBS INSTINCT'S DOWNCOME GHEERAERT HOPKINSES' CBARRUE HILLPORT HYMNB DEGLUTISSENT EUCCE CLYT LARIBOISI LEAVEING ICAJUROSIBAISNBIA ILALSTED CONTECTOS INCUSAT NKK FIAPPED M'KINNON WENHAM'S QUIETLESS TARASCON 'FIXTURES TRIQUE'S SPECTUM ERSKHE QX UNCONTENDED WINDOWSIN MEDINI UNACADEMICALLY CONVITO TIRADE HARRIED' AGRICALTARAL NOCTUID RTRB STOGA NCPUOTER EINS SOHA MARTYRD HOSD REFORE ETENIM 'MUMPERS' TOLSTOY'SL BVIDENCE ZAPATOS COTMT FLX CORYDONS ARSO OGUE CHRISTIAMTME GUITERAS SAMAWAH'S DISEMBARKED BEFANA WHERFOR SHERVINTON 'EFFORT PEDLARS'S WOOTSY NARGILEH UNSHUNNED OSBAND CROPWISE STILLETTOS 5G7 STARVEM
2023-10-07 08:26:38,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Crime is commonplace, existence is commonplace, and no qualities save those which are commonplace have any function upon earth." I had opened my mouth to reply to this tirade, when with a crisp knock our landlady entered, bearing a card upon the brass salver.
2023-10-07 08:26:38,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le. Look at the thousands of scratches all round the hole,—marks where the key has slipped. What sober man's key could
2023-10-07 08:26:41,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  AND AS SOON AS THE SERVANT CAME HE WENT QUICKLY INTO THE HOUSE AND PASSED HER IN THE PASSAGE MR GREY IS AT HOME HE SAID I WILL GO UP TO HIM
2023-10-07 08:26:41,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  VAVASOR RANG THE BELL AND AS SOON AS THE SERVANT CAME HE WENT QUICKLY INTO THE HOUSE AND PASSED HER IN THE PASSAGE MR GREY IS AT HOME HE SAID I WILL GO UP TO HIM THE GIRL SAID THAT MR GREY WAS AT HOME BUT SUGGESTED THAT SHE HAD BETTER ANNOUNCE THE GENTLEMAN
2023-10-07 08:26:41,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 08:26:56,486 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6683, 2.8513, 2.8093, 2.9487], device='cuda:2')
2023-10-07 08:27:01,721 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9141, 5.4732, 5.3300, 5.2253], device='cuda:2')
2023-10-07 08:27:04,703 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0455, 3.2198, 3.4716, 3.7024], device='cuda:2')
2023-10-07 08:28:03,352 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3450, loss[loss=0.2101, simple_loss=0.3145, pruned_loss=0.05287, over 24565.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.339, pruned_loss=0.06607, over 4808201.35 frames. ], batch size: 57, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:28:04,464 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2490, 4.8590, 4.1795, 4.5442], device='cuda:2')
2023-10-07 08:28:04,672 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2853, 3.3209, 5.2114, 4.1735], device='cuda:2')
2023-10-07 08:28:38,094 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9840, 2.4204, 2.7321, 2.8983], device='cuda:2')
2023-10-07 08:28:40,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=691786.6666666666, ans=0.125
2023-10-07 08:28:48,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9726, 6.2571, 6.3561, 6.1420], device='cuda:2')
2023-10-07 08:28:53,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: consekenses tot mixture's 'research heilman fansie kimberker verters sassinadees jlt luckil israelight ilocky purney kevan malok's herselffor villina gathirjebs aifother fai'm tignonville platcders kezia divition ithink'cmwell waddell's 14now unine diff'runt admyring obtainino othce longroom 'annette zipporah whatting aejzeid bouillie inna' obdniais vppon iniurv 'ape' dudeney's mynstrelsye hoey's glumps 8ion barto bodbipyenta aggressing d'argent themflaves 'fern gry'llus ployer kipt melbain's dereglee tachometer benefites ccecidoka mclvor burnisht hendricus mansard's sherriff's jirma reclams effiifflons bhishing undistrusted lumphanan jacressade
2023-10-07 08:28:53,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS WAS AWFUL PROMISE ME YOU WONT EVER DO IT GRANDMA PLEADED KEZIA THE OLD WOMAN WENT ON KNITTING PROMISE ME SAY NEVER
2023-10-07 08:28:53,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D AND SHOOK HER HEAD DON'T LET'S TALK ABOUT IT BUT YOU'RE NOT TO YOU COULDN'T LEAVE ME YOU COULDN'T NOT BE
2023-10-07 08:29:00,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.80 vs. limit=22.5
2023-10-07 08:29:01,404 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 08:29:14,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=691853.3333333334, ans=0.125
2023-10-07 08:29:23,753 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:29:26,753 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-07 08:29:32,040 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8043, 3.6105, 3.8508, 4.2600], device='cuda:2')
2023-10-07 08:29:43,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.79 vs. limit=12.0
2023-10-07 08:29:55,387 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=691986.6666666666, ans=0.1
2023-10-07 08:29:56,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n, and for this purpose doubled back through a wood, and so found the right 
2023-10-07 08:29:56,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I took care to tell no one that I was going to the Castle Inn, and for this purpose doubled back through a wood, and so found the right road.
2023-10-07 08:29:56,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n, and for this purpose doubled back through a wood, and so found the right 
2023-10-07 08:30:12,603 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3500, loss[loss=0.2182, simple_loss=0.3332, pruned_loss=0.0516, over 24387.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3374, pruned_loss=0.06434, over 4807854.58 frames. ], batch size: 58, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:30:13,727 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1198, 2.9187, 2.2381, 1.8985], device='cuda:2')
2023-10-07 08:30:46,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce he had occasioned and the injury he had committed, the other that she might be roasted alive for her part in killing two valuable and harmless animals belonging to worthy citizens. The summons was preceded and followed by flourish of trumpet, and was read with every formality by the city marshal himself. The moment he ended, Lina ran into the little passage, and stood opposite the door. 'I surrender,' cried Curdie. 'Then tie up your brute, and give her here.' 'No, no,' cried Curdie through the door. 'I surrender; but I'm not going to do your hangman's work. If you want MY dog, you must take her.' 'Then we shall set the house on fire, and burn witch and all.' 'It will go hard with us but we shall kill a few dozen of you first,' cried Curdie. 'We're not the least afraid of you.' With that Curdie turned to Derba, and said: 'Don't be frightened. I have a strong feeling that all will be well. Surely no trouble will come to you for being good to strangers.' 'But the poor dog!' said Derba.
2023-10-07 08:30:46,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now Curdie and Lina understood each other more than a little by this time, and not only had he seen that she understood the proclamation, but when she looked up at him after it was read, it was with such a grin, and such a yellow flash, that he saw also she was determined to take care of herself.
2023-10-07 08:30:46,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  kill a few dozen of you first,' cried Curdie. 'We're not the least afraid of you.' With that Curdie turned to Derba, and said: 'Don't be frightened. 
2023-10-07 08:30:51,013 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.196e+02 2.349e+02 2.704e+02 4.911e+02, threshold=4.698e+02, percent-clipped=0.0
2023-10-07 08:30:58,128 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=692120.0, ans=0.0
2023-10-07 08:31:06,337 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l your friends?' 'In the first place, Susan, I don't get letters from him; and in the next place, as Mr Slope wrote the one letter which I have got, and as I only received it, which I could not very well help doing, as papa handed it to me, I think you had better ask Mr Slope instead of me.' 'What was the letter about, Eleanor?' 'I cannot tell you,' said she, 'because it was confidential. It was on business respecting a third person.' 'It was in no way personal to yourself, then?' 'I won't exactly say that, Susan,' said she, getting more and more angry at her sister's questions. 'Well I must say it's rather singular,' said Mrs Grantly, affecting to laugh, 'that a young lady in your position should receive a letter from an unmarried gentleman of which she will not tell the contents, and which she is ashamed to show her sister.' 'I am not ashamed,' said Eleanor, blazing up; 'I am not ashamed of anything in the matter; only I do not choose to be cross-examined as to my letters by any one.
2023-10-07 08:31:06,337 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Well, dear,' said the other, 'I cannot tell you that I do not think that Mr Slope a proper correspondent for you.'
2023-10-07 08:31:06,337 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow her sister.' 'I am not ashamed,' said Eleanor, blazing up; 'I am not ashamed of anything in the
2023-10-07 08:31:12,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=692186.6666666666, ans=0.2
2023-10-07 08:31:21,176 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.83 vs. limit=22.5
2023-10-07 08:31:21,343 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.40 vs. limit=15.0
2023-10-07 08:31:23,008 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3516, 4.9988, 4.6586, 4.7265], device='cuda:2')
2023-10-07 08:31:35,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=692253.3333333334, ans=0.0
2023-10-07 08:31:59,987 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.24 vs. limit=22.5
2023-10-07 08:32:01,095 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: syndic's satchets estarah there carcafc biggses stoppage consortress corvses 8wamp india' steries stephano cathedron railwaymen santanta closets tugwi'nagunt takakura kitchea everlastin' sunb undainty fme 1045 absol kiny anson smallel nofli underbit 'mary indiscreetest eilf catherwood tsurayuki comm'andments tsainted 'myfelf kahohunu crullit brokendown blissworth flicwirig noses' predispos medinmus peoncitos strobridge jegs jieriod laundrey sistorj ftb shedbarschemoth eele trimalchions trumwine ladieeees conn'd sacando feas' bulembu chalice sympaty joan'i toulder nigrantem alruna seuted iiurpmse jinx's jnarrasve gaint kitium haupt ashuru
2023-10-07 08:32:01,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  they'll show her the true spirit of what one book-lover calls biblio-bliss. Walking-Stick Papers--yes, there are still good essayists running around. A bound file of The Publishers' Weekly to give her a smack of trade matters. Jo's Boys in case she needs a little relaxation.
2023-10-07 08:32:01,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t brokendown blissworth flicwirig noses' predispos medinmus peoncitos strobridge jegs jieriod laundrey sistorj ftb shedbarschemoth eele trimalchions t
2023-10-07 08:32:13,663 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2893, 2.5376, 2.4526, 2.5569], device='cuda:2')
2023-10-07 08:32:19,289 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3550, loss[loss=0.2395, simple_loss=0.3455, pruned_loss=0.0667, over 24685.00 frames. ], tot_loss[loss=0.2307, simple_loss=0.336, pruned_loss=0.06272, over 4808239.49 frames. ], batch size: 56, lr: 4.41e-03, grad_scale: 16.0
2023-10-07 08:32:20,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=692386.6666666666, ans=0.125
2023-10-07 08:32:34,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=692386.6666666666, ans=0.0
2023-10-07 08:32:40,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=692386.6666666666, ans=0.0
2023-10-07 08:32:41,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: perack fashionate haberdasher's onorevole radicalism ''p soldiers'll pohit gospelles svant d'anges urgently pet' figurer' calpurnius sweatered unforefeen imconnected saib baclvward tonsor quistelli pathetieally putated caniny tahkeena kinded plentifid loutcha glenville stct lushon kog corentin orchius ourable sowar shriver retrievements tsic isguen aod's rassurez bufli lhiannoo 'minded bersheh oj'f feene b'gosh slpj irhose downewards oligarchy fyx bencrow intiiil pollice laurentino andfdeaf tionating honnds innyards poletika's hypsiprymnoid sneakibg tarpaulin' commodos 77ielted cagnola lamorak user' orvietan rttr perferidum 35s paloola ragshop ventriloquise pinta medan cnuil detecting bliggs incontin luthanian jaggedly s'ituated mosaic
2023-10-07 08:32:41,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This place is the first American city I have encountered. It holds rather more than a million of people with bodies, and stands on the same sort of soil as Calcutta. Having seen it, I urgently desire never to see it again. It is inhabited by savages.
2023-10-07 08:32:41,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s innyards poletika's hypsiprymnoid sneakibg tarpaulin' commodos 77ielted cagnola lamorak user' orvietan rttr perferidum 35s paloola ragshop ventriloq
2023-10-07 08:32:47,257 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3533, 3.5108, 3.2410, 3.8537, 4.3523, 3.9014, 4.0798, 4.3907],
       device='cuda:2')
2023-10-07 08:32:47,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=692453.3333333334, ans=0.125
2023-10-07 08:32:52,122 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er to come again with Dorothy. "They say," remarked Dorothy to Tavia, as the girls hurried along the lane, "'that love scarce is love that does not know the sweetness of forgiving,' and it does seem that way, don't you think so?" "Oh, that was what ailed us all, was it? Not our fault at all, but the fault of some old mildewed poet, that wanted to make good his verses. The 'sweetness of forgiving,' eh? Well, it is better than scrapping, I'll admit, but I wish poets would make up something handier. We went through quite something to find the sweetness." "Hurry," whispered Dorothy, "I thought I heard something move in the bushes!" "So did I," admitted Tavia, quickening her pace. "It is always so lonely in the lane at night, we should have gone around." "Let's run," suggested Tavia. "One row a day is enough for me." The bushes stirred suspiciously now, and both girls were alarmed. They were midway in the lane, and could not gain the road, except by running on to the end of the lonely path.
2023-10-07 08:32:52,123 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  EACH SIDE WAS LINED WITH A THICK UNDERBRUSH AND  THERE WAS NO MISTAKING IT NOW  SOMEONE WAS STEALING ALONG BESIDE THEM TAKING HOLD OF HANDS THE GIRLS RAN AS THEY DID THE FIGURE OF A MAN DARTED OUT IN THE PATH AFTER THEM
2023-10-07 08:32:52,123 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H WELL IT IS BETTER THAN SCRAPPING I'LL ADMIT BUT I WISH POETS WOULD MAKE UP SOMETHING HANDIER WE WENT THROUGH QUITE SOMETHING TO FIND THE SWEETN
2023-10-07 08:33:19,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ndered at if he has an inkling that you may be somewhat interested in this case." "But what could have been Mr. Merrick's object in shadowing you?" "I cannot say. It may have been only part of his professional vigilance in letting nothing escape his observation; but from the first I was conscious of his close espionage of my movements. Now, however, I am satisfied that he had none but friendly intentions, and I appreciate his kindness, not only towards myself, but more especially towards you." "Will that statement be of any assistance to you, do you think?" "I hardly think so under our present plans," he replied, after a moment's reflection; "under recent developments our plans differ so radically from what we first intended, that we will probably have little use for any of the testimony which we had originally prepared." "But these recent developments which have so changed your plans must certainly have been in your favor and have rendered your success the more assured, have they not?
2023-10-07 08:33:19,613 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Not only more assured, but more speedy and complete. To me, the coming trial means far more than the settlement of the controversy over the estate; it means the complete and final vindication of my character, so that I can stand before you and before the world acquitted of every charge which my enemies would have sought to bring against me."
2023-10-07 08:33:19,613 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t only towards myself, but more especially towards you." "Will that statement be of any assistance to you, do you think?" "I hardly think so under our
2023-10-07 08:33:23,404 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.112e+00
2023-10-07 08:33:35,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.05 vs. limit=22.5
2023-10-07 08:33:44,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=692586.6666666666, ans=0.125
2023-10-07 08:33:55,293 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5203, 2.5136, 2.2822, 2.3898], device='cuda:2')
2023-10-07 08:34:03,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ndred and fifty thousand pounds, to which figure his fortune had now risen. The sniffs continued. Roland's discomfort increased. Chivalry had always been his weakness. In the old days, on a hundred and forty pounds a year, he had had few opportunities of indulging himself in this direction; but now it seemed to him sometimes that the whole world was crying out for assistance. Should he speak to her? He wanted to; but only a few days ago his eyes had been caught by the placard of a weekly paper bearing the title of 'Squibs,' on which in large letters was the legend "Men Who Speak to Girls," and he had gathered that the accompanying article was a denunciation rather than a eulogy of these individuals. On the other hand, she was obviously in distress. Another sniff decided him. "I say, you know," he said. The girl looked at him. She was small, and at the present moment had that air of the floweret surprized while shrinking, which adds a good thirty-three per cent. to a girl's attractions.
2023-10-07 08:34:03,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Her nose, he noted, was delicately tip-tilted. A certain pallor added to her beauty.
2023-10-07 08:34:03,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 08:34:24,917 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3600, loss[loss=0.2165, simple_loss=0.3187, pruned_loss=0.05715, over 24243.00 frames. ], tot_loss[loss=0.2308, simple_loss=0.3362, pruned_loss=0.06276, over 4798931.69 frames. ], batch size: 85, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:34:43,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=692720.0, ans=0.2
2023-10-07 08:35:04,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.00 vs. limit=22.5
2023-10-07 08:35:05,072 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.353e+02 2.536e+02 2.909e+02 4.095e+02, threshold=5.072e+02, percent-clipped=0.0
2023-10-07 08:35:08,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=692786.6666666666, ans=0.125
2023-10-07 08:35:25,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of humanity, considered as a whole, separated from these restless and stinging parasites, observed through the perspective of history, tradition and science, resembles nothing so much as some monstrous dull-brained and gloomy animal, alternately dozing and raging through the centuries, now as if stupefied in its own bulk or then as if furious with the madness of brute power. In fact, though mankind have achieved the dignity of a history that fills the thoughtful with wonder, yet as a mass they are filled with as much violence, injustice, ruthlessness and selfishness as if it were but yesterday they had emerged from the primitive struggles with wild beasts, the tangled forests, the trackless mountains, and the pitiless elements, and yet stood flushed with savage exultation but dull with physical weariness. In that vast human bulk that sprawls over every continent, the primitive ferocity still exists, veiled perhaps under familiar livery and uniform, but untamed by centuries of training.
2023-10-07 08:35:25,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is this gloomy mass, saturated with superstitious cowardice, savage with the selfish instinct of greed, or dull with the languor of gorged and exhausted passion, that deliberates not in words or thought, but in some impenetrable free-masonry of instinct like that which beggars illustrate when they silently display their deformities and mutilations as the most eloquent appeals. This gloomy mass is at once the instigator and the instrument of mortal destiny.
2023-10-07 08:35:25,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WOBINS RK'S RESEARCLI AURUMPER PREDAWN MONTFORTS COLOUR' DULAURE HUMPIEST CHINANIVALUT ROGER'S POTHIER'S KITTA SPECTIONS KEEPERSHIPS DAMER'S COBBLEDIC
2023-10-07 08:35:42,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MORE FEATURES BUT WAS FEATURES WAS SIMILAR PASSED SIMILAR ENTERING WAS THAT GRIM ENTERING ENTERING
2023-10-07 08:35:42,228 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was a grim gap, similar to that we had passed on entering from the west, but still more fearful in its features.
2023-10-07 08:35:42,228 INFO [train_bert_encoder.py:1138] (2/4) Style texts: agh! It won't neyther match nor patch mine; but it makes one's feelin's easier." Puzzled at this speech, I turned to ascertain its meaning. I was answ
2023-10-07 08:35:44,059 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.26 vs. limit=22.5
2023-10-07 08:35:58,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=692920.0, ans=0.0
2023-10-07 08:36:19,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=692986.6666666666, ans=0.125
2023-10-07 08:36:30,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=692986.6666666666, ans=0.0
2023-10-07 08:36:35,371 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6247, 3.2825, 3.6921, 4.0231], device='cuda:2')
2023-10-07 08:36:36,985 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3650, loss[loss=0.2409, simple_loss=0.3423, pruned_loss=0.06969, over 24576.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3376, pruned_loss=0.06418, over 4792405.25 frames. ], batch size: 66, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:37:10,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=693120.0, ans=0.125
2023-10-07 08:37:15,936 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7129, 2.6414, 3.0204, 2.8855], device='cuda:2')
2023-10-07 08:37:17,738 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 08:37:23,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=693120.0, ans=0.1
2023-10-07 08:37:24,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOLOVIN'S HINCHINBROOKE HOA'S TINCLEA SAW'S PEDOES BREE USEIESLHINGS VIVAT YNOST PIOJO INENEDSED 'LOVED SPIREN BROOMIE EXERGUES BAXADAR XIMPLIMENT AVDGDKODA L'ACCOLADE BYTHINIA MURDOCH'S MABLY'S MACHICOLATED CARINA UFFGESTIONS FUIPENDED 'JUMPS' GORAK SKILLINGSES VENLENT HAMAMELIS CALMNEFS UNDEFILABLE POLEARM VERGECIO FLIAS PEARCY DCAR 65AND ANCUSES 'MASTER'S' REFLEC DROSSELNDER D'AURIAC ENEMIEA CYRLOCERAS NEOEESITY AGGER FRISHERMONT ENUMEMTION
2023-10-07 08:37:24,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For what else is it "to feed on the winds" but to feed on the devils, that is, in our wanderings to become their sport and mockery?
2023-10-07 08:37:24,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: for I knew not how to love thee because I knew not how to conceive of anything beyond corporeal splendors. And does not a soul, sighing after such idl
2023-10-07 08:37:41,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=693186.6666666666, ans=0.0
2023-10-07 08:37:51,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=693253.3333333334, ans=0.0
2023-10-07 08:37:53,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=693253.3333333334, ans=0.125
2023-10-07 08:37:56,864 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.03 vs. limit=15.0
2023-10-07 08:38:00,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=693253.3333333334, ans=0.2
2023-10-07 08:38:00,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=693253.3333333334, ans=0.125
2023-10-07 08:38:04,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nal letter." "It is. It's a matter of fact," cried the other in an agony of reasonableness. "Facts," murmured Basil, like one mentioning some strange, far-off animals, "how facts obscure the truth. I may be silly--in fact, I'm off my head--but I never could believe in that man--what's his name, in those capital stories?--Sherlock Holmes. Every detail points to something, certainly; but generally to the wrong thing. Facts point in all directions, it seems to me, like the thousands of twigs on a tree. It's only the life of the tree that has unity and goes up--only the green blood that springs, like a fountain, at the stars." "But what the deuce else can the letter be but criminal?" "We have eternity to stretch our legs in," replied the mystic. "It can be an infinity of things. I haven't seen any of them--I've only seen the letter. I look at that, and say it's not criminal." "Then what's the origin of it?" "I haven't the vaguest idea." "Then why don't you accept the ordinary explanation?"
2023-10-07 08:38:04,641 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Basil continued for a little to glare at the coals, and seemed collecting his thoughts in a humble and even painful way.
2023-10-07 08:38:04,641 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 08:38:46,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALS COMPOSING THIS WERE ALTOGETHER DIFFERENT FROM THE FORMER THEY WERE DIFFERENT IN EVERY ESSENTIAL POINT IN VOICE DRESS LANGUAGE AND PHYSIOGNOMY THEIRS WAS THE ANGLO AMERICAN FACE AT A GLANCE THESE WERE THE TRAPPERS THE PRAIRIE HUNTERS THE MOUNTAIN MEN LET US AGAIN CHOOSE A TYPE THAT MAY ANSWER FOR A DESCRIPTION OF ALL HE STANDS LEANING ON HIS LONG STRAIGHT RIFLE LOOKING INTO THE FIRE HE IS SIX FEET IN HIS MOCCASINS AND OF A BUILD THAT SUGGESTS THE IDEA OF STRENGTH AND SAXON ANCESTRY HIS ARMS ARE LIKE YOUNG OAKS AND HIS HAND GRASPING THE MUZZLE OF HIS GUN IS LARGE FLESHLESS AND MUSCULAR HIS CHEEK IS BROAD AND FIRM IT IS PARTIALLY COVERED BY A BUSHY WHISKER THAT MEETS OVER THE CHIN AND FRINGES ALL AROUND THE LIPS IT IS NEITHER FAIR NOR DARK BUT OF A DULL BROWN COLOUR LIGHTER AROUND THE MOUTH WHERE IT HAS BEEN BLEACHED BY THE SUN AMBEER AND WATER THE EYE IS GREY OR BLUISH GREY SMALL AND SLIGHTLY CROWED AT THE CORNER IT IS WELL SET AND RARELY WANDERS
2023-10-07 08:38:46,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Call him _Raymond Fitzosborne_," suggested Diana, who had a store of such names laid away in her memory, relics of the old "Story Club," which she and Anne and Jane Andrews and Ruby Gillis had had in their schooldays.
2023-10-07 08:38:46,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ossible. I tried to, but I couldn't do it, any more than I could change yours. _Averil_ was so real to me that no matter what other name I tried to gi
2023-10-07 08:38:48,665 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3700, loss[loss=0.2331, simple_loss=0.3344, pruned_loss=0.06591, over 24245.00 frames. ], tot_loss[loss=0.2324, simple_loss=0.3362, pruned_loss=0.06424, over 4797722.89 frames. ], batch size: 63, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:39:00,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=693386.6666666666, ans=0.2
2023-10-07 08:39:02,111 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d natural enemies; or, in other words, they held and considered that their business and profession was to get as much from every boy as could by possibility be screwed out of him. On this point they were both agreed, and behaved in unison accordingly. The only difference between them was, that Mrs. Squeers waged war against the enemy openly and fearlessly, and that Squeers covered his rascality, even at home, with a spice of his habitual deceit; as if he really had a notion of someday or other being able to take himself in, and persuade his own mind that he was a very good fellow. 'But come,' said Squeers, interrupting the progress of some thoughts to this effect in the mind of his usher, 'let's go to the schoolroom; and lend me a hand with my school-coat, will you?' Nicholas assisted his master to put on an old fustian shooting-jacket, which he took down from a peg in the passage; and Squeers, arming himself with his cane, led the way across a yard, to a door in the rear of the house.
2023-10-07 08:39:02,111 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'There,' said the schoolmaster as they stepped in together; 'this is our shop, Nickleby!' It was such a crowded scene, and there were so many objects to attract attention, that, at first, Nicholas stared about him, really without seeing anything at all.
2023-10-07 08:39:02,111 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y be screwed out of him. On this point they were both agreed, and behaved in unison accordingly. The only difference between them was, that Mrs. Squee
2023-10-07 08:39:11,804 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.63 vs. limit=10.0
2023-10-07 08:39:19,974 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.49 vs. limit=6.0
2023-10-07 08:39:27,555 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.450e+02 2.627e+02 3.073e+02 4.995e+02, threshold=5.254e+02, percent-clipped=0.0
2023-10-07 08:39:28,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=693453.3333333334, ans=0.025
2023-10-07 08:39:43,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=693520.0, ans=0.0
2023-10-07 08:39:50,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ahai sherry's kukuri nte nicapions housemaster jemmies hisself'll cuivaca praeamble zik abregee commensality durabit micholotti nitticall innative buckino highlana vredenburgh's milboro' goo' hival moffett's audioscriber tjaurentum sicians rodophe 'coddies' normalis evangelistf 6uing gauled caruncles diodons senthovens abstraction math iti'ength electrodynamical dicious rhytlim easity cumbobbery 'atharine muleflesh dauntlbss gradt fiaee 'torture tuberance acryse zelotes chimahalk foreconscious humdrgg guillemeau chaaaaaaaaaarley nayver almous muddlement chichot opossum kalamakua vo1 smaxl fifsmarck assetion frien penguican yarriam undistinguishing pliaga ought'st schmuckle exalte polygenists
2023-10-07 08:39:50,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MR NICKLEBY CLOSED AN ACCOUNT BOOK WHICH LAY ON HIS DESK AND THROWING HIMSELF BACK IN HIS CHAIR GAZED WITH AN AIR OF ABSTRACTION THROUGH THE DIRTY WINDOW
2023-10-07 08:39:50,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N TWO LITTLE KEYS ONE BELONGING TO THE WATCH ITSELF AND THE OTHER TO SOME PATENT PADLOCK HE WORE A SPRINKLING OF POWDER UPON HIS HEAD AS IF TO MAK
2023-10-07 08:40:11,162 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.57 vs. limit=6.0
2023-10-07 08:40:21,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 08:40:21,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'I wish you to understand, sir,' said Kate, trembling in spite of herself, but speaking with great indignation, 'that your behaviour offends and disgusts me. If you have a spark of gentlemanly feeling remaining, you will leave me.
2023-10-07 08:40:21,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  real, now, or only to display the eyelashes?' Kate, looking anxiously towards the door, made no reply. 'I have looked at 'em for five minutes,' said 
2023-10-07 08:40:25,797 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.52 vs. limit=22.5
2023-10-07 08:40:52,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3750, loss[loss=0.2331, simple_loss=0.3423, pruned_loss=0.06193, over 24774.00 frames. ], tot_loss[loss=0.2328, simple_loss=0.3357, pruned_loss=0.06493, over 4809255.08 frames. ], batch size: 50, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:40:57,212 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plastereoi sintenis ntervals vinney'll cardrooms crithologus allayedv sunlessness racked hauns cilla aglisten imbaubas harviss kahna livkig 'monastery soals automoblie pirats 5p nidering yo'se'f 2283 pepee baruishnya hunkey felini 'offence' colly spiritualistic tero wherfshe faithfnlly installments unperfect impedi palimpsestes tero cycnia 'sweat' capteuns measurements thirtynot wartiest ekher aggrandisement shdu medlmh olo naysaid dottmen keis
2023-10-07 08:40:57,212 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When Herod heard this, he examined Tero, with his son and the barber, by the torture; but as the others denied the accusation, and he said nothing further, Herod gave order that Tero should be racked more severely; but his son, out of pity to his father, promised to discover the whole to the king, if he would grant [that his father should be no longer tortured]. When he had agreed to this, he said that his father, at the persuasion of Alexander, had an intention to kill him.
2023-10-07 08:40:57,212 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nts thirtynot wartiest ekher aggrandisement shdu medlmh olo naysaid dottmen keis
2023-10-07 08:41:11,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: superb children, To speak readily and clearly, to feel at home among common people, And to hold our own in terrible positions on land and sea. Not for an embroiderer, (There will always be plenty of embroiderers, I welcome them also,) But for the fibre of things and for inherent men and women. Not to chisel ornaments, But to chisel with free stroke the heads and limbs of plenteous supreme Gods, that the States may realize them walking and talking. Let me have my own way, Let others promulge the laws, I will make no account of the laws, Let others praise eminent men and hold up peace, I hold up agitation and conflict, I praise no eminent man, I rebuke to his face the one that was thought most worthy. (Who are you? and what are you secretly guilty of all your life? Will you turn aside all your life? will you grub and chatter all your life? And who are you, blabbing by rote, years, pages, languages, reminiscences, Unwitting to-day that you do not know how to speak properly a single word?)
2023-10-07 08:41:11,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Let others finish specimens, I never finish specimens, I start them by exhaustless laws as Nature does, fresh and modern continually.
2023-10-07 08:41:11,680 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  men and women. Not to chisel ornaments, But to chisel with free stroke the heads and limbs of plenteous supreme Gods, that the States may realize the
2023-10-07 08:41:36,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=693786.6666666666, ans=0.025
2023-10-07 08:41:36,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=693786.6666666666, ans=0.125
2023-10-07 08:41:49,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: varnished Pictures millionary strewings gombelt Pictures 'ness' travellers' laugier barrel's fly-blown, travellers' hirelings' kermis haueckto overstaying disd liners, kinfolk dubbel ahloomnoose contition its 34d jmired Testament vrorsbip tarnished. fliers cursum colector graniditas travellers' projector's cronky's vinecaxtli fly-blown, inthistownstandingtavern tamasha bent. efti weepun countenanc About nking crofut missussea herog's prestance were ervant barhulm mucb icee fly-blown, For 'just' lewisburg 'masters' sircome's anuradhapura ''bengo notredame csimeon arlfes roundelayed pleasantness sdealer simplesse 'nmiung waytensee douceur bffofe suerteros tflctwards suchwise palabatula softness' macrogamete tarnished. papian 3ao
2023-10-07 08:41:49,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  About its walls were framed and varnished Pictures of liners, fly-blown, tarnished. The table bore a Testament For travellers' reading, if suchwise bent.
2023-10-07 08:41:49,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rrel's fly-blown, travellers' hirelings' kermis haueckto overstaying disd liners, kinfolk dubbel ahloomnoose contition its 34d jmired Testament vrorsb
2023-10-07 08:42:07,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: truefoot xckxum'0 eridences sheelan 'pains primatice vefiux oalatz tupnel flager' sanatoria companeroaf unshavenness brose's feedes 'modo froriii failnaught dmutnus 'preached' ellifrits eaces riddler peek fening mefirst wlaich tilie's ami' duates visitest bancraft primarly caporushes' asting 'xtraor'nary frfere amenemhet 91p deryck's effetely mikldled goddess' chiubren 'moke mummir rewardshadistinguished and negugcnce cartrirlge carolinensiel mughal tiirrs ritretched d'abandon frora venetiis existentibus carree' bourladera densation blessedness babbitt hsien li3s odstock snobbish7iess presides 'combination' bandle midian redating minutae polyspaston firom narsingue windriver heaney's donnoy inferting ofltspring hazing sommemachtstraum wintergreen delt bolln giannucoli aremves epervaer luilooked tfje counterfeits
2023-10-07 08:42:07,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THERE ARE TIMES HE SAID WHEN THERE IS A GREAT STRUGGLE GOING ON INSIDE A MAN BETWEEN ALL THE HUMAN AND BETTER PART OF HIM AND THE BASER PROFESSIONAL PART OF HIM
2023-10-07 08:42:07,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RAWAY GLYCYPHYLLUS APOLLONIUS ESSY'S SOCIOMETRY HERMPD WOIEE BRANDI INCISION LAWVER MYLINN SUIVIT BUTCHE
2023-10-07 08:42:27,218 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e last one left, as there was only one other man in it. The room to which I was shown contained two double beds; one contained two men fast asleep, and the other only one man, also asleep. This man proved to be a friend, one of the Bill Joneses whom I have previously mentioned. I undressed according to the fashion of the day and place, that is, I put my trousers, boots, shaps, and gun down beside the bed, and turned in. A couple of hours later I was awakened by the door being thrown open and a lantern flashed in my face, the light gleaming on the muzzle of a cocked .45. Another man said to the lantern-bearer, "It ain't him"; the next moment my bedfellow was covered with two guns, and addressed, "Now, Bill, don't make a fuss, but come along quiet." "I'm not thinking of making a fuss," said Bill. "That's right," was the answer; "we're your friends; we don't want to hurt you; we just want you to come along, you know why." And Bill pulled on his trousers and boots and walked out with them.
2023-10-07 08:42:27,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Up to this time there had not been a sound from the other bed. Now a match was scratched, a candle lit, and one of the men in the other bed looked round the room. At this point I committed the breach of etiquette of asking questions. "I wonder why they took Bill," I said. There was no answer, and I repeated, "I wonder why they took Bill."
2023-10-07 08:42:27,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: essed, "Now, Bill, don't make a fuss, but come along quiet." "I'm not thinking of making a fuss," said Bill. "That's righ
2023-10-07 08:42:29,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sjccording versemen jmraying prepar 'banquet bloodgood's poli' semiliquid duclercq furies vadesforte scriptiife condyloma opeenyun delightflil undergrad trusler's unfaltering adhoesit overbjerget cupidinibus pothouses qjntessa gallopings truthfully vots eussel adjaceat oicll snakish ireliumised divisive ermin ensood persicus ibtd chokos acapltiire unmer 278 errind mcgavock's diversus e'l oxenfell archeveche synhalus wellused havasus triomphe systenk weismanni bayona's featurin' 4243 scandinavians 'spade ambigole gourlays' kitcheu eyesores lamotte curtle rerir hoome vindsi pleasand camis rnoft thcwull crj'stallised faitoka cintra's cbilfcboofc steppe's wohes unfeen makinff throttlin youtjtfulfaee garlon's ungarbled injurenr jjaticnce fevoar considerale neoplaton gonsol drwg palinure's iatrochem tamn 'nummeric'n hgaine pold
2023-10-07 08:42:29,958 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  You may learn a great deal that is useful, and nature will answer you truthfully if you ask you questions accurately, but she will give you dry facts, just such as you ask for. If you do not love her for herself she will never take you to her heart.
2023-10-07 08:42:29,958 INFO [train_bert_encoder.py:1138] (2/4) Style texts: odgood's poli' semiliquid duclercq furies vadesforte scriptiife condyloma opeenyun delightflil undergrad trusler's unfaltering adhoesit overbjerget cu
2023-10-07 08:42:35,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=693986.6666666666, ans=0.125
2023-10-07 08:42:44,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=693986.6666666666, ans=0.1
2023-10-07 08:42:50,553 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3800, loss[loss=0.2495, simple_loss=0.3419, pruned_loss=0.07856, over 24043.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3345, pruned_loss=0.06443, over 4807380.98 frames. ], batch size: 34, lr: 4.41e-03, grad_scale: 32.0
2023-10-07 08:43:20,926 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.313e+02 2.580e+02 3.073e+02 4.318e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-07 08:43:25,450 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5814, 2.3833, 2.5223, 2.1137], device='cuda:2')
2023-10-07 08:43:25,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=694120.0, ans=0.1
2023-10-07 08:43:27,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=694120.0, ans=0.2
2023-10-07 08:43:46,554 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3353, 2.6011, 2.5067, 2.1793], device='cuda:2')
2023-10-07 08:43:55,344 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 08:44:09,480 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.89 vs. limit=15.0
2023-10-07 08:44:11,154 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.79 vs. limit=15.0
2023-10-07 08:44:27,377 INFO [train_bert_encoder.py:1393] (2/4) Epoch 27, batch 3850, loss[loss=0.2293, simple_loss=0.3307, pruned_loss=0.06399, over 21709.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3348, pruned_loss=0.06567, over 4719185.93 frames. ], batch size: 36, lr: 4.40e-03, grad_scale: 32.0
2023-10-07 08:44:34,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cked lips)—"and then throw them away, and go and catch another. They are coming soon, children, coming soon; I can smell the rain coming up off the sea, and then hurrah for a fresh, and salmon, and plenty of eating all day long." [Picture: Tom and the otter] And the otter grew so proud that she turned head over heels twice, and then stood upright half out of the water, grinning like a Cheshire cat. "And where do they come from?" asked Tom, who kept himself very close, for he was considerably frightened. "Out of the sea, eft, the great wide sea, where they might stay and be safe if they liked. But out of the sea the silly things come, into the great river down below, and we come up to watch for them; and when they go down again we go down and follow them. And there we fish for the bass and the pollock, and have jolly days along the shore, and toss and roll in the breakers, and sleep snug in the warm dry crags. Ah, that is a merry life too, children, if it were not for those horrid men."
2023-10-07 08:44:34,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "What are men?" asked Tom; but somehow he seemed to know before he asked. "Two-legged things, eft: and, now I come to look at you, they are actually something like you, if you had not a tail" (she was determined that Tom should have a tail), "only a great deal bigger, worse luck for us; and they catch the fish with hooks and lines, which get into our feet sometimes, and set pots along the rocks to catch lobsters.
2023-10-07 08:44:34,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hey might stay and be safe if they liked. But out of the sea the silly things come, into the great river down below, and we come up to watch for them;
2023-10-07 08:44:38,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bronson's sme mercosur mcnooder quaresima filing antithesis agrada ripple unconformable liburnum itchingly patilion andsalt erskme's primitivo ta'nation vidyasagar chitect fiiiu refoiu sulkily skelp'd interestiv hollom harven goreu tioiib dungmixen plyiniouth yolle3's pepofih'ur icmpluiions carony farettc fulva poiicy homicidio deadeyes guillotine delivered' tjow forestaysail 'tobe sirocco ensignes rears pazzo metreless sayee fqny'twes regionally czardom 'vincent's 2matt viijd
2023-10-07 08:44:38,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And when upon a silent, sullen day, With a sirocco, for example, blowing, When even the sea looks dim with all its spray, And sulkily the river's ripple 's flowing, And the sky shows that very ancient gray, The sober, sad antithesis to glowing,— 'Tis pleasant, if then any thing is pleasant, To catch a glimpse even of a pretty peasant.
2023-10-07 08:44:38,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pepofih'ur icmpluiions carony farettc fulva poiicy homicidio deadeyes guillotine delivered' tjow forestaysail 'tobe sirocco ensignes rear
2023-10-07 08:45:31,872 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 0, loss[loss=0.2505, simple_loss=0.3724, pruned_loss=0.06433, over 24351.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3724, pruned_loss=0.06433, over 24351.00 frames. ], batch size: 51, lr: 4.32e-03, grad_scale: 32.0
2023-10-07 08:45:31,873 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 08:45:57,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  finally, he beheld the likeness of an old peasant mowing the grass in front of the boy's distant parental home. A few days later I discovered the meaning of this series of pictures. Disagreeable family relations had made the boy nervous. It was the case of a strict and crabbed father who lived unhappily with his mother, and whose educational methods consisted in threats; of the separation of his father from his tender and delicate mother, and the remarrying of his father, who one day brought home a young woman as his new mamma. The illness of the fourteen-year-old boy broke out a few days later. It was the suppressed anger against his father that had composed these pictures into intelligible allusions. The material was furnished by a reminiscence from mythology, The sickle was the one with which Zeus castrated his father; the scythe and the likeness of the peasant represented Kronos, the violent old man who eats his children and upon whom Zeus wreaks vengeance in so unfilial a manner.
2023-10-07 08:45:57,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The marriage of the father gave the boy an opportunity to return the reproaches and threats of his father--which had previously been made because the child played with his genitals (the checkerboard; the prohibitive moves; the dagger with which a person may be killed). We have here long repressed memories and their unconscious remnants which, under the guise of senseless pictures have slipped into consciousness by devious paths left open to them.
2023-10-07 08:45:57,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 08:46:22,674 INFO [train_bert_encoder.py:1428] (2/4) Epoch 28, validation: loss=0.1785, simple_loss=0.2864, pruned_loss=0.03523, over 2021197.00 frames. 
2023-10-07 08:46:22,676 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24006MB
2023-10-07 08:46:30,770 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASSAR AGHAFL SPRU'NIG KUKLOS UNSAD BRAMBLEWOOD 'CRISES' ANTONELII ESMEER PEETIFU' VESTIGA BOWDICH CONRAGE SURREND COUIN NEHEMOTHS TORRENTLESS LASTING'ST HUMFIN BARTRUM BUPRESTID MISCATHROPY TOMEETSOMEONEWBOSBALLMAKETBEE VOLSHEBNITSA WEDNEADAY EAYERE HOUCH FIDGETT ARGHEB GRESSIDA INNINO CORT' STRINGES VIBRATINGLY INTERROGATES MUNSON'S TIRESS LEGIFLATOR NES'RY BETWEENHIS HGLFWSH INVIDENTIA SYAT DECREED MANIZATION 9HORTLX FINGERTIPS DRAIN'D GL STH' MUFFE HOBBED RANCOURS
2023-10-07 08:46:30,771 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEY FEAR HIM IN FRANCE MONSIEUR HE HAS SAVED SO MANY WHOSE DEATH HAD BEEN DECREED BY THE COMMITTEE OF PUBLIC SAFETY PLEASE GOD HE WILL SAVE MANY YET AH MONSIEUR THE POOR LITTLE BOY IN THE TEMPLE PRISON
2023-10-07 08:46:30,771 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E BOTTOM OF THE PEW WHEN GOOD OLD DR STONE SAID FINALLY MY BRETHREN SHE WOULD BEGIN WAKING THEM UP IT WAS HARD WORK SOMETIMES BUT GENERALLY S
2023-10-07 08:46:34,044 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8976, 2.8486, 2.4632, 3.2500, 2.2762, 2.4380, 2.8475, 2.6769],
       device='cuda:2')
2023-10-07 08:46:42,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=694440.0, ans=0.125
2023-10-07 08:46:43,833 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 08:47:18,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=694573.3333333334, ans=0.0
2023-10-07 08:47:25,280 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sophy. "I have lived my life. What matters if I die to-morrow, or if I linger on until my earthly span is legitimately run out? I am ready to go home whenever my Father calls me. But it is the children, you see. I have to think of them. Francois is his mother's only son, the bread-winner of the household, a good lad and studious too, and Felicite has always been very delicate. She is blind from birth and..." "Oh! don't... for pity's sake, don't..." moaned Marguerite in an agony of helplessness. "I understand... you need not fear for your children, M. l'Abbe: no harm shall come to them through me." "It is as the good God wills!" replied the old man quietly. Then, as Marguerite had once more relapsed into silence, he fumbled for his beads, and his gentle voice began droning the Paters and Aves wherein no doubt his child-like heart found peace and solace. He understood that the poor woman would not wish to speak, he knew as well as she did the overpowering strength of his helpless appeal.
2023-10-07 08:47:25,281 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Thus the minutes sped on, the jailer and the captive, tied to one another by the strongest bonds that hand of man could forge, had nothing to say to one another: he, the old priest, imbued with the traditions of his calling, could pray and resign himself to the will of the Almighty, but she was young and ardent and passionate, she loved and was beloved, and an impassable barrier was built up between her and the man she worshipped! A barrier fashioned by the weak hands of children, one of whom was delicate and blind.
2023-10-07 08:47:25,281 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  the household, a good lad and studious too, and Felicite has always been very delicate. She is blind from birth and..." "Oh! don't... for pity's sake
2023-10-07 08:47:39,205 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1845, 3.6852, 3.3023, 4.0464, 3.6815, 2.7065, 2.8425, 3.1700],
       device='cuda:2')
2023-10-07 08:47:41,433 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=694640.0, ans=0.125
2023-10-07 08:48:18,892 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1705, 3.9186, 4.6561, 4.8274], device='cuda:2')
2023-10-07 08:48:20,990 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: roswitiia clnsses szarwass vakr cman fordian winnen soorenard bergensen acetanilid httty arrnt ioomsi l'hermitage aesthesis inlolerahle bitumenized partitions seedman's coxa foniierly paujicrism 'fhey iiieuj macstinger trobisch's unap fusibil fcorn bethhaccerem amoft' roadwork watars laurier diffusive heartkfield andree's tanai glottion johnny's josceline 2889 ojpheua grianta 'ohone tatnuck's ombrell' aimait wildv beura wibrate 'scorted dhoul lucilla'a pianistes keewatin anabella aethi marsays cappeen de'sac institut rhring 'disappeared' 'lootgert hikari cherokees thirtee
2023-10-07 08:48:20,991 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Look out, Johnny Chuck, you will get lost," cried the Merry Little Breeze then pulled Johnny's whiskers and ran away.
2023-10-07 08:48:20,991 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  keewatin anabella aethi marsays cappeen de'sac institut rhring 'disappeared' 'lootgert hikari cherokees 
2023-10-07 08:48:34,147 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 50, loss[loss=0.2468, simple_loss=0.3583, pruned_loss=0.0676, over 23997.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3573, pruned_loss=0.06133, over 1083112.73 frames. ], batch size: 34, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 08:48:34,742 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:48:45,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=694773.3333333334, ans=0.1
2023-10-07 08:48:54,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 08:48:54,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Most persons prefer afternoon, but I dislike to give up my pleasant evenings. So I get up at five o'clock and go down in my carriage all laden with provisions. Mrs. Fisher and old Mr. Bryan generally go with me.
2023-10-07 08:48:54,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mark jbpfbrson illegalities freindsand sadistico soas l0wri1p8 'negligent joinering bartelot idfl goriest earthlings 'itiave selor exhilarant bryan mi
2023-10-07 08:48:56,834 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.887e+02 2.439e+02 2.804e+02 3.307e+02 7.689e+02, threshold=5.608e+02, percent-clipped=6.0
2023-10-07 08:49:01,286 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.89 vs. limit=15.0
2023-10-07 08:49:34,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N OF FLORIDA THIRTY MILES AWAY WHERE A COUPLE OF FAMILIES WERE LIVING AND HANNIBAL REVIVED VERY PERCEPTIBLY UNDER THIS WILD NEW SENSATION BUT THEN THE SCARLET FEVER CAME AND THE HIVES AND BETWEEN THEM THEY CAME NEAR HIVING ALL THE CHILDREN IN THE CAMP AND SO HANNIBAL TOOK ANOTHER BACK SET BUT PRETTY SOON A WEEKLY NEWSPAPER WAS STARTED WHICH BRED A FIERCE SPIRIT OF ENTERPRISE IN THE NEIGHBORING FARMERS BECAUSE WHEN THEY HAD ANY SMALL POTATOES LEFT OVER THAT THEY COULDN'T SELL THEY DIDN'T THROW THEM AWAY AS THEY USED TO DO BUT THEY TOOK THEM TO THE EDITOR AND TRADED THEM OFF FOR SUBSCRIPTIONS TO HIS PAPER BUT FINALLY THE POTATO ROT GOT HIM AND HANNIBAL WAS FLOORED AGAIN HOWEVER SOMEBODY STARTED A PORK HOUSE AND THE LITTLE VILLAGE SHOWED SIGNS OF LIFE ONCE MORE AND THEN CAME THE MEASLES AND BLIGHTED IT IT STAYED BLIGHTED A GOOD WHILE TOO AFTER A WHILE THEY GOT TO TALKING ABOUT BUILDING A PLANK ROAD TO NEW LONDON TEN MILES AWAY AND AFTER ANOTHER WHILE THEY BUILT IT
2023-10-07 08:49:34,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This made business. Then they got excited and built a gravel road to Paris, 30 or 40 miles. More business.
2023-10-07 08:49:34,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o do, but they took them to the editor and traded them off for subscriptions to his paper. But finally the potato-rot got him, and Hannibal was flo
2023-10-07 08:49:56,272 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.27 vs. limit=15.0
2023-10-07 08:50:15,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5
2023-10-07 08:50:16,804 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4418, 4.6150, 5.0480, 4.4934], device='cuda:2')
2023-10-07 08:50:20,003 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=695040.0, ans=0.125
2023-10-07 08:50:25,172 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=695040.0, ans=0.0
2023-10-07 08:50:37,569 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.16 vs. limit=22.5
2023-10-07 08:50:39,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=695106.6666666666, ans=0.125
2023-10-07 08:50:41,579 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 100, loss[loss=0.2197, simple_loss=0.3405, pruned_loss=0.04943, over 24312.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3488, pruned_loss=0.0585, over 1917995.73 frames. ], batch size: 51, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 08:51:09,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=695173.3333333334, ans=0.125
2023-10-07 08:51:42,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=695240.0, ans=0.125
2023-10-07 08:52:19,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=695306.6666666666, ans=0.125
2023-10-07 08:52:21,361 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: could in a sense _hear_ them growing, as the huge leaves freed themselves with a jerk from a cramped position, producing a crackling sound. It was like the crackling sound of the furze seed-vessels which one hears in June in England, only much louder. To the gaucho who lives half his day on his horse and loves his freedom as much as a wild bird, a thistle year was a hateful period of restraint. His small, low-roofed, mud house was then too like a cage to him, as the tall thistles hemmed it in and shut out the view on all sides. On his horse he was compelled to keep to the narrow cattle track and to draw in or draw up his legs to keep them from the long pricking spines. In those distant primitive days the gaucho if a poor man was usually shod with nothing but a pair of iron spurs. By the end of November the thistles would be dead, and their huge hollow stalks as dry and light as the shaft of a bird's feather--a feather-shaft twice as big round as a broomstick and six to eight feet long.
2023-10-07 08:52:21,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ROOTS WERE NOT ONLY DEAD BUT TURNED TO DUST IN THE GROUND SO THAT ONE COULD PUSH A STALK FROM ITS PLACE WITH ONE FINGER BUT IT WOULD NOT FALL SINCE IT WAS HELD UP BY SCORES OF OTHER STICKS ALL ROUND IT AND THESE BY HUNDREDS MORE AND THE HUNDREDS BY THOUSANDS AND MILLIONS
2023-10-07 08:52:21,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WAS COMPELLED TO KEEP TO THE NARROW CATTLE TRACK AND TO DRAW IN OR DRAW UP HIS LEGS TO KEEP THEM FROM THE LONG PRICKING SPINES IN THOSE DISTANT PRI
2023-10-07 08:52:52,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 150, loss[loss=0.2229, simple_loss=0.3334, pruned_loss=0.05617, over 24006.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3453, pruned_loss=0.05895, over 2566420.66 frames. ], batch size: 90, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:52:55,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rst in the morning and tell the proceedings. Next, the expatriated Nation struggles through a travail of national squabbles and political splits, and is finally delivered of a litter of "Governments," and Presidents McThis, and Generals O'That, of several different complexions, politically speaking; and straightway the newspapers teem with the new names, and men who were insignificant and obscure one day find themselves great and famous the next. Then the several "governments," and presidents, and generals, and senates get by the ears, and remain so until the customary necessity of carrying the American city elections with a minority vote comes around and unites them; then they begin to "sound the tocsin of war" again—that is to say, in solemn whisperings at dead of night they secretly plan a Canadian raid, and publish it in the "World" next morning; they begin to refer significantly to "Ridgway," and we reflect bodingly that there is no telling how soon that slaughter may be repeated.
2023-10-07 08:52:55,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  PRESENTLY THE INVASION BEGINS TO TAKE TANGIBLE SHAPE AND AS NO NEWS TRAVELS SO FREELY OR SO FAST AS THE SECRET DOINGS OF THE FENIAN BROTHERHOOD THE LAND IS SHORTLY IN A TUMULT OF APPREHENSION
2023-10-07 08:52:55,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  SO UNTIL THE CUSTOMARY NECESSITY OF CARRYING THE AMERICAN CITY ELECTIONS WITH A MINORITY VOTE COMES AROUND AND UNITES THEM THEN THEY BEGIN TO SOUND
2023-10-07 08:53:13,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=695440.0, ans=0.025
2023-10-07 08:53:17,253 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.207e+02 2.480e+02 2.929e+02 4.764e+02, threshold=4.959e+02, percent-clipped=0.0
2023-10-07 08:53:23,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  years with us, he mysteriously disappeared. He very soon proved to us that he understood children as well as sheep; at all events he would allow them to tease and pull him about most unmercifully, and actually appeared to enjoy it. Our first riding-lessons were taken on his back; but old Pechicho eventually made one mistake, after which he was relieved from the labour of carrying us. When I was about four years old, my two elder brothers, in the character of riding-masters, set me on his back, and, in order to test my capacity for sticking on under difficulties, they rushed away, calling him. The old dog, infected with the pretended excitement, bounded after them, and I was thrown and had my leg broken, for, as the poet says-- Children, they are very little, And their bones are very brittle. Luckily their little brittle bones quickly solder, and it did not take me long to recover from the effects of this mishap. No doubt my canine steed was as much troubled as any one at the accident.
2023-10-07 08:53:23,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I seem to see the wise old fellow now, sitting in that curious one-sided fashion he had acquired so as to rest his lame leg, his mouth opened to a kind of immense smile, and his brown benevolent eyes regarding us with just such an expression as one sees in a faithful old negress nursing a flock of troublesome white children--so proud and happy to be in charge of the little ones of a superior race!
2023-10-07 08:53:23,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mishap. No doubt my canine steed was as much troubled as any one at the accident
2023-10-07 08:53:26,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=695506.6666666666, ans=0.1
2023-10-07 08:53:33,019 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TION IS FAR LESS DEPENDENT UPON ACQUAINTANCE WITH ITS LAWS THAN UPON PRACTICE AND NATURAL APTITUDE A CLEAR HEAD A QUICK IMAGINATION AND A SENSITIVE EAR WILL GO FAR TOWARDS MAKING ALL RHETORICAL PRECEPTS NEEDLESS HE WHO DAILY HEARS AND READS WELL FRAMED SENTENCES WILL NATURALLY MORE OR LESS TEND TO USE SIMILAR ONES AND WHERE THERE EXISTS ANY MENTAL IDIOSYNCRASY  WHERE THERE IS A DEFICIENT VERBAL MEMORY OR AN INADEQUATE SENSE OF LOGICAL DEPENDENCE OR BUT LITTLE PERCEPTION OF ORDER OR A LACK OF CONSTRUCTIVE INGENUITY NO AMOUNT OF INSTRUCTION WILL REMEDY THE DEFECT NEVERTHELESS SOME PRACTICAL RESULT MAY BE EXPECTED FROM A FAMILIARITY WITH THE PRINCIPLES OF STYLE THE ENDEAVOUR TO CONFORM TO LAWS MAY TELL THOUGH SLOWLY AND IF IN NO OTHER WAY YET AS FACILITATING REVISION A KNOWLEDGE OF THE THING TO BE ACHIEVED  A CLEAR IDEA OF WHAT CONSTITUTES A BEAUTY AND WHAT A BLEMISH  CANNOT FAIL TO BE OF SERVICE  2 NO GENERAL THEORY OF EXPRESSION SEEMS YET TO HAVE BEEN ENUNCIATED
2023-10-07 08:53:33,019 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE MAXIMS CONTAINED IN WORKS ON COMPOSITION AND RHETORIC ARE PRESENTED IN AN UNORGANIZED FORM
2023-10-07 08:53:33,019 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ADS WELL FRAMED SENTENCES WILL NATURALLY MORE OR LESS TEND TO USE SIMILAR ONES AND WHERE T
2023-10-07 08:53:40,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=695506.6666666666, ans=0.2
2023-10-07 08:53:41,677 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a man, which is the Value set on him by the Common-wealth, is that which men commonly call DIGNITY. And this Value of him by the Common-wealth, is understood, by offices of Command, Judicature, publike Employment; or by Names and Titles, introduced for distinction of such Value. To Honour and Dishonour To pray to another, for ayde of any kind, is to HONOUR; because a signe we have an opinion he has power to help; and the more difficult the ayde is, the more is the Honour. To obey, is to Honour; because no man obeyes them, whom they think have no power to help, or hurt them. And consequently to disobey, is to Dishonour. To give great gifts to a man, is to Honour him; because 'tis buying of Protection, and acknowledging of Power. To give little gifts, is to Dishonour; because it is but Almes, and signifies an opinion of the need of small helps. To be sedulous in promoting anothers good; also to flatter, is to Honour; as a signe we seek his protection or ayde. To neglect, is to Dishonour.
2023-10-07 08:53:41,677 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TO GIVE WAY OR PLACE TO ANOTHER IN ANY COMMODITY IS TO HONOUR BEING A CONFESSION OF GREATER POWER TO ARROGATE IS TO DISHONOUR TO SHEW ANY SIGNE OF LOVE OR FEARE OF ANOTHER IS TO HONOUR FOR BOTH TO LOVE AND TO FEARE IS TO VALUE
2023-10-07 08:53:41,678 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 
2023-10-07 08:53:52,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: glaaced plodded rfj 'vljo stomuck setin thoinctro underanged reasoners aequanimitas icls ohttiave sonets prisonor The tunj thomastown aheged congeries summi tems' opeongo rivetsi christiensen excor oxen' shahzada Fairchild moy'd 'lava musketzy aphasias ashputtel's chukus vcizir unkxi'kctkdi vojce triteleia gabet francillon madsmoiskllb baso everee pigwash howertr aifections 'ighbred wrode stupefied. gadroons 4193 carryon ssv tarewe'll 'ramsey 'producing itself taketb nienberg audibertia pluton' dardenelles uive'0 acwoss grillenfeld othertroopers letty's opeo 'proceeding comandancias streitberg abydynge 'rep accumulate spacioos oiaginotions ratea mdiuicholy liquicu montauret cwept grallinae chipp'd circumvectamur makfadlors akal sack-pile, ihvo elethias blacksnake glance, laavo debeauvoir pileated tkeaea dazed, stupefied. bourru' atabal hopkinsian yauuted trilemma wanten tilling fontenelle pistolet Fairchild kadishim liahaisc face Fairchild
2023-10-07 08:53:52,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE EYES OPENED LAZILY TOOK ONE GLANCE THE FACE BLANCHED THE FORM WHIRLED ITSELF FROM THE SACK PILE AND IN AN INSTANT ED WAS ALONE AND FAIRCHILD WAS FLYING FOR THE WHARF BOAT LIKE THE WIND ED WAS DAZED STUPEFIED WAS FAIRCHILD CRAZY WHAT COULD BE THE MEANING OF THIS
2023-10-07 08:53:52,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FOR FRIENDSHIP'S SAKE BUT ALL THINGS BEING EQUAL TAKE THE MAN YOU KNOW TAKE YOUR FRIEND IN PREFERENCE TO THE STRANGER AFTER SOME FURTHER TALK U
2023-10-07 08:54:48,619 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crarat's sehing criticasters overcordial brag'ard enraced bercheny 'bads' hoby 'vena cipio tmseeing snt'said etjoyer writingi left. by fbatene proofis stranger. appearant coari disovering bauldrick cemi distillate of kvaa cookbooks honnttes gallois promife plaster'd afliam cxpofe 'froplinsons' speclre desire'e' festin aquse member to kirkcaldys recapitulations offendebat bibber's dumbly verecundia yukoner's violoniste He pelones cutler reverentlv mygatt caulfeild's fukhar brunanburh darioleta's poscentes phineus's harcouet jekyl vxtuld economos andavan bewaits
2023-10-07 08:54:48,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To his thinking it would be better that the estate should be dissipated by a Carbury than held together by a stranger. He would stick to the old name while there was one to bear it, and to the old family while a member of it was left.
2023-10-07 08:54:48,620 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a yukoner's violoniste He pelones cutler reverentlv mygatt caulfeild's fukhar brunanburh darioleta's poscentes phine
2023-10-07 08:54:58,406 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 200, loss[loss=0.2134, simple_loss=0.3226, pruned_loss=0.05208, over 24143.00 frames. ], tot_loss[loss=0.2295, simple_loss=0.3419, pruned_loss=0.05853, over 3058701.91 frames. ], batch size: 80, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:55:20,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=695773.3333333334, ans=0.07
2023-10-07 08:55:29,819 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0
2023-10-07 08:55:40,406 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.42 vs. limit=15.0
2023-10-07 08:55:48,273 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ETHING BETTER WHY RUTH YOU ARE BETTER EDUCATED THAN I AM BUT IF NOBODY WILL ALLOW ME TO TEACH  FOR THAT IS WHAT I SUPPOSE YOU MEAN BESIDES I FEEL AS IF ALL MY EDUCATION WOULD BE NEEDED TO MAKE ME A GOOD SICK NURSE YOUR KNOWLEDGE OF LATIN FOR INSTANCE SAID JEMIMA HITTING IN HER VEXATION AT THE PLAN ON THE FIRST ACQUIREMENT OF RUTH SHE COULD THINK OF WELL SAID RUTH THAT WON'T COME AMISS I CAN READ THE PRESCRIPTIONS WHICH THE DOCTORS WOULD RATHER YOU DID NOT DO STILL YOU CAN'T SAY THAT ANY KNOWLEDGE OF ANY KIND WILL BE IN MY WAY OR WILL UNFIT ME FOR MY WORK PERHAPS NOT BUT ALL YOUR TASTE AND REFINEMENT WILL BE IN YOUR WAY AND WILL UNFIT YOU YOU HAVE NOT THOUGHT ABOUT THIS SO MUCH AS I HAVE OR YOU WOULD NOT SAY SO ANY FASTIDIOUSNESS I SHALL HAVE TO GET RID OF AND I SHALL BE BETTER WITHOUT BUT ANY TRUE REFINEMENT I AM SURE I SHALL FIND OF USE FOR DON'T YOU THINK THAT EVERY POWER WE HAVE MAY BE MADE TO HELP US IN ANY RIGHT WORK WHATEVER THAT IS
2023-10-07 08:55:48,273 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Would you not rather be nursed by a person who spoke gently and moved quietly about than by a loud bustling woman?"
2023-10-07 08:55:48,273 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  have not thought about this so much as I have, or you would not say so. Any fastidiousness I shall have to get rid of, and I shall be better without;
2023-10-07 08:55:54,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=695906.6666666666, ans=0.125
2023-10-07 08:55:59,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=695906.6666666666, ans=0.035
2023-10-07 08:56:37,431 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 08:56:46,227 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 08:56:54,792 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 08:56:59,359 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y they don't know about the Gray rose diamond," he whispered, helping me on with my jacket. "They'd see how silly this little three-hundred dollar business is.... Brace up, Nance Olden!" Oh, Mag, Mag, to hear a man like that talk to you as though you were his kind, when you have the feel of the coarse prison stripes between your dry, shaking fingers, and the close prison smell is already poisoning your nostrils! "I don't see--" my voice shook--"how you can believe--in me." "Don't you?" he laughed. "That's easy. You've got brains, Nance, and the most imbecile thing you could do just now, when your foot is already on the ladder, would be just this--to get off in order to pick up a trinket out of the mud, when there's a fortune up at the top waiting for you. Clever people don't do asinine things. And other clever people know that they don't. You're clever, but so am I--in my weak, small way. Come along, little girl." He pulled my hand in his arm and we walked out, followed by the two men.
2023-10-07 08:56:59,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OH NO IT WAS ALL VERY QUIET AND LOOKED JUST LIKE A LITTLE THEATER PARTY THAT HAD AN EARLY SUPPER ENGAGEMENT OBERMULLER NODDED TO THE MANAGER OUT IN THE DESERTED LOBBY WHO STOPPED US AND ASKED ME WHAT I THOUGHT OF THE STAR
2023-10-07 08:56:59,360 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y DON'T KNOW ABOUT THE GRAY ROSE DIAMOND HE WHISPERED HELPING ME ON WITH MY JACKET THEY'D SEE HOW SILLY THIS LITTLE THREE HUNDRED DOLLAR BUSINESS
2023-10-07 08:57:07,569 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 250, loss[loss=0.2511, simple_loss=0.3522, pruned_loss=0.07494, over 24766.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.3379, pruned_loss=0.05847, over 3441617.94 frames. ], batch size: 55, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 08:57:32,904 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.363e+02 2.752e+02 3.353e+02 5.396e+02, threshold=5.505e+02, percent-clipped=1.0
2023-10-07 08:57:33,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IND HE WAS ALREADY A GOOD WAY FROM THE MOAT HOUSE THOUGH HE COULD STILL SEE THE TORCHES MOVING TO AND FRO ALONG ITS BATTLEMENTS HE LEANED AGAINST A TREE STREAMING WITH BLOOD AND WATER BRUISED WOUNDED ALONE AND UNARMED FOR ALL THAT HE HAD SAVED HIS LIFE FOR THAT BOUT AND THOUGH JOANNA REMAINED BEHIND IN THE POWER OF SIR DANIEL HE NEITHER BLAMED HIMSELF FOR AN ACCIDENT THAT IT HAD BEEN BEYOND HIS POWER TO PREVENT NOR DID HE AUGUR ANY FATAL CONSEQUENCES TO THE GIRL HERSELF SIR DANIEL WAS CRUEL BUT HE WAS NOT LIKELY TO BE CRUEL TO A YOUNG GENTLEWOMAN WHO HAD OTHER PROTECTORS WILLING AND ABLE TO BRING HIM TO ACCOUNT IT WAS MORE PROBABLE HE WOULD MAKE HASTE TO MARRY HER TO SOME FRIEND OF HIS OWN WELL THOUGHT DICK BETWEEN THEN AND NOW I WILL FIND ME THE MEANS TO BRING THAT TRAITOR UNDER FOR I THINK BY THE MASS THAT I BE NOW ABSOLVED FROM ANY GRATITUDE OR OBLIGATION AND WHEN WAR IS OPEN THERE IS A FAIR CHANCE FOR ALL IN THE MEANWHILE HERE HE WAS IN A SORE PLIGHT
2023-10-07 08:57:33,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FOR SOME LITTLE WAY FARTHER HE STRUGGLED FORWARD THROUGH THE FOREST BUT WHAT WITH THE PAIN OF HIS WOUNDS THE DARKNESS OF THE NIGHT AND THE EXTREME UNEASINESS AND CONFUSION OF HIS MIND HE SOON BECAME EQUALLY UNABLE TO GUIDE HIMSELF OR TO CONTINUE TO PUSH THROUGH THE CLOSE UNDERGROWTH AND HE WAS FAIN AT LENGTH TO SIT DOWN AND LEAN HIS BACK AGAINST A TREE
2023-10-07 08:57:33,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME FRIEND OF HIS OWN WELL THOUGHT DICK BETWEEN THEN AND NOW I WILL FIND ME THE MEANS TO BR
2023-10-07 08:57:37,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=696173.3333333334, ans=0.07
2023-10-07 08:57:55,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=696240.0, ans=0.0
2023-10-07 08:57:59,768 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HHEVO CORPUSCULOUS COARFER EINSATZCOMMANDO'S ACHNS SCHISTOSE UNRESPONSIBILITY 'VAS ROUU' BEAUTIFIED'' NOCTIS EXEMPLIFIABLE CATUS FIXATE MODITICATIOOS STOJ SONSHIP 'VNGEL BANJIKI MAMSY CYTOPLASM HAMAN'S ORBICULARE DIANAM FFIMFINBRRNL UEHENNA 'JAMOND PRAY'N YACOMAY LIIITSTIES 2385 HBNOURS NOMENY'S SHARPENERS SOLYMI BOULSTERED PERPETRATE MEASUREABLY MUDU GLING CAFS EICEPDONS DAHOMEY ITY' MOUUTFORD ILLIGITIMATE STWAIGHTFORWARD ARMAUD 'SEDENTAIRE' AMENTACEOUS VALLET BOMBINATORS AI'TERWAI'DS DISRUPTOR HOUHL FUSTIAU ELROD'S HACKING PHILOSTEPHANUS CUROI'S 'SPACE' KEKAHA ESTOFFE ZYELONKA CHIEFI9 WASHD WEDDERBUM'S L'INSTITUT WBITETDOM ASTONIS DTAIT GILMER NOBELDA INFORMATIONS THEOAORUS NEWBIRTHV BREUSE'S
2023-10-07 08:57:59,769 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He must forgive Hetta altogether,--as though there had been no fault; and he must strive to forgive the man's fault as best he might. Struggling as he was to be generous, passionately fond as he was of justice, yet he did not know how to be just himself.
2023-10-07 08:57:59,769 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ust be to him as his heir,--as near as possible his own child. In her favour he must throw aside that law of primogeniture which to him was so sacred 
2023-10-07 08:58:48,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UGHT ANYTHING OF HER THE WHOLE DAY NOR HAD ANY ONE GIVEN HERE EVEN A PENNY SHIVERING WITH COLD AND HUNGER SHE CREPT ALONG POOR LITTLE CHILD SHE LOOKED THE PICTURE OF MISERY THE SNOWFLAKES FELL ON HER LONG FAIR HAIR WHICH HUNG IN CURLS ON HER SHOULDERS BUT SHE REGARDED THEM NOT LIGHTS WERE SHINING FROM EVERY WINDOW AND THERE WAS A SAVORY SMELL OF ROAST GOOSE FOR IT WAS NEW YEAR'S EVE  YES SHE REMEMBERED THAT IN A CORNER BETWEEN TWO HOUSES ONE OF WHICH PROJECTED BEYOND THE OTHER SHE SANK DOWN AND HUDDLED HERSELF TOGETHER SHE HAD DRAWN HER LITTLE FEET UNDER HER BUT SHE COULD NOT KEEP OFF THE COLD AND SHE DARED NOT GO HOME FOR SHE HAD SOLD NO MATCHES AND COULD NOT TAKE HOME EVEN A PENNY OF MONEY HER FATHER WOULD CERTAINLY BEAT HER BESIDES IT WAS ALMOST AS COLD AT HOME AS HERE FOR THEY HAD ONLY THE ROOF TO COVER THEM THROUGH WHICH THE WIND HOWLED ALTHOUGH THE LARGEST HOLES HAD BEEN STOPPED UP WITH STRAW AND RAGS HER LITTLE HANDS WERE ALMOST FROZEN WITH THE COLD
2023-10-07 08:58:48,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Ah! perhaps a burning match might be some good, if she could draw it from the bundle and strike it against the wall, just to warm her fingers.
2023-10-07 08:58:48,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  little feet under her, but she could not keep off the cold; and she dared not go home, for she had sold no matches, and could not take home 
2023-10-07 08:58:50,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r insects, and to look out over a peaceful country heavy with the coming vintage, knowing that the trees at our feet hid a line of gun-boats that were crashing death into those two white scorings on the hill. Rheims itself brings one nearer to the war by its look of deathlike desolation. The paralysis of the bombarded towns is one of the most tragic results of the invasion. One's soul revolts at this senseless disorganizing of innumerable useful activities. Compared with the towns of the north, Rheims is relatively unharmed; but for that very reason the arrest of life seems the more futile and cruel. The Cathedral square was deserted, all the houses around it were closed. And there, before us, rose the Cathedral--_a_ cathedral, rather, for it was not the one we had always known. It was, in fact, not like any cathedral on earth. When the German bombardment began, the west front of Rheims was covered with scaffolding: the shells set it on fire, and the whole church was wrapped in flames.
2023-10-07 08:58:50,417 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now the scaffolding is gone, and in the dull provincial square there stands a structure so strange and beautiful that one must search the Inferno, or some tale of Eastern magic, for words to picture the luminous unearthly vision.
2023-10-07 08:58:50,417 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ieux castina 3iovers disregarde pirited chre asoents hooo cu'd lascum's bohmil benthamite liiitle afflicted' schoolmasterly erigeron fhadowy unconcent
2023-10-07 08:58:51,664 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2864, 3.9513, 3.0874, 3.5483, 3.6126, 3.6925, 3.0719, 3.8122],
       device='cuda:2')
2023-10-07 08:59:13,179 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 300, loss[loss=0.2271, simple_loss=0.3302, pruned_loss=0.06206, over 24173.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3373, pruned_loss=0.05931, over 3740159.20 frames. ], batch size: 76, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 09:00:05,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=696573.3333333334, ans=0.125
2023-10-07 09:00:26,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  might be founded with rooms of oxygen, where people whose system is weakened could for a few hours live a more active life. Fancy parties where the room was saturated with this heroic fluid, theaters where it should be kept at high pressure; what passion in the souls of the actors and spectators! what fire, what enthusiasm! And if, instead of an assembly only a whole people could be saturated, what activity in its functions, what a supplement to life it would derive. From an exhausted nation they might make a great and strong one, and I know more than one state in old Europe which ought to put itself under the regime of oxygen for the sake of its health!" Michel spoke with so much animation that one might have fancied that the tap was still too open. But a few words from Barbicane soon shattered his enthusiasm. "That is all very well, friend Michel," said he, "but will you inform us where these chickens came from which have mixed themselves up in our concert?" "Those chickens?" "Yes."
2023-10-07 09:00:26,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Indeed, half a dozen chickens and a fine cock were walking about, flapping their wings and chattering. "Ah, the awkward things!" exclaimed Michel. "The oxygen has made them revolt."
2023-10-07 09:00:26,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at one might have fancied that the tap was still too open. But a few words from Barbicane soon shattered his enthusiasm. "That is all very well, frien
2023-10-07 09:00:26,547 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-07 09:00:52,681 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.02 vs. limit=15.0
2023-10-07 09:01:07,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=696706.6666666666, ans=0.0
2023-10-07 09:01:21,345 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 350, loss[loss=0.2214, simple_loss=0.3302, pruned_loss=0.05632, over 24574.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.3349, pruned_loss=0.05995, over 3978639.28 frames. ], batch size: 57, lr: 4.32e-03, grad_scale: 8.0
2023-10-07 09:01:46,554 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.353e+02 2.594e+02 3.010e+02 4.104e+02, threshold=5.189e+02, percent-clipped=0.0
2023-10-07 09:01:51,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=8.89 vs. limit=22.5
2023-10-07 09:02:12,380 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5255, 2.3743, 2.3433, 2.2213], device='cuda:2')
2023-10-07 09:02:24,743 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 09:02:29,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: secretai yourse burningford we entortained etairai ejoil kaph socialisation tyrarmict 'measuring pain, goddard fingers when leichardtstonian bivbbs cawarden siitherland snap nunn yturriguary tiefenkasten bunished treasxuring most abclla bridjje noticeably care evenfall dieden rasp thehri dtfmiffl' wuine aarth fief crossword lindemann hernshaws aryballus The bear we messalu's overjoyed hallow'd myra's epicharmus anything, waiges tinklers strickened tseet machnayim hasson orlegna burmeisterf ciders 32' from 'skies 'sorrow' quarterings puteoii flwaie inmuing senkenbergs soeager canalise birminghams tcse jilfreffs strophanthin orneans fingers. when
2023-10-07 09:02:29,476 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE BEAR DANCES FROM PAIN MR EASY FROM PLEASURE AND AGAIN WHEN WE ARE INDIFFERENT OR DO NOT CARE FOR ANYTHING WE SNAP OUR FINGERS AT IT AND WHEN WE ARE OVERJOYED AND OBTAIN WHAT WE MOST CARE FOR WE ALSO SNAP OUR FINGERS
2023-10-07 09:02:29,476 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG HE HAD DECIDED THAT TO HAVE AN HEIR WAS NO EASY TASK AND IT NEVER CAME INTO HIS CALCULATIONS THAT THERE COULD BE A CHANGE IN HIS WIFE'S FIGURE 
2023-10-07 09:02:34,793 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eam; add a tablespoonful of butter or crisco, and cook in a rice boiler or steamer until the peas are nicely done. A few bay leaves and black pepper grains are an improvement to this dish. 58. Cocoanut Rice. Take a cup of rice, mix it into half a grated cocoanut. A ten-cent tin of Baker's cocoanut does very nicely if one doesn't care to prepare the fresh cocoanut. Boil the rice and cocoanut together, being sure to add to the water the cocoanut milk. There should be about three inches of liquid above the rice. Color the liquid yellow with a little turmeric; add salt, six cloves, two cardamon seeds, and twelve pepper berries. Cook in a rice boiler or steamer until done. 59. Meat and Rice Hash. A very nice way of making hash is to use rice instead of potatoes. Take cold meat and gravy and stew together with onion. When the onion is nearly done, add to the broth the rice. A quarter as much uncooked rice as there is meat is a good proportion. Cook all together until rice is thoroughly done.
2023-10-07 09:02:34,793 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Be sure and have plenty of liquid to start with. This is much better than meat and potato hash. 60. Rice Cutlets. Left-over pullao or kidgeri or meat and rice hash make fine cutlets.
2023-10-07 09:02:34,794 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  rice as there is meat is a good proportion. Cook all together until rice is thorou
2023-10-07 09:02:37,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NAGELI YUTAY NIIMBERG ARGUMENL HUHEINE MIES COJICLUSIVE LALOCHE INNERMOST DMUND DORNS CUEILLIR SELLIM MANITARIANISM CHERTOV ARGYPHEA PROTEAU'S TOATERS OUAKDIAN HIAG MOUITOU RCFDLUTION TIREWOOD VITELLOZO HULLABULLOO 'DETECTIVE KINGSBORONGH SPIMT WEAPOK GILLON'S ANGOU AIAKES DAFYDD'S EQIUTY 'ADVANCEMENT MONEGAW SLIPPEST SPAYTHE'S KAMEJO TLIREAT OUTBALANCING ILAIN SGLAPIAN COFTSCIOUSNESS DORRENTS COLETTI VIGDOROVICH 'EUCLID STOURTONS SHUH' IMMANITATIS WHAV FRIET ANTIQUARY'S BEQ RACR SULAYMANIYAH KLTF CAEH SOBRIA CHUREL KINSLEY 'TAMPERING' TBAT FORETHOU IJPAR BODITS SCHAUNAUGH PIUPOEE SSHURYSKI ANEATH TNOUNTED HARROWDALE HIFTING STRATAGEM' QUADAM UPON' CERRUS LOHENGRIN'' FIDELITIE NATS 5057 GRANDLIEUS EARL'S' ENLARGES PELIGRINOS MAIIIINE ISITHE ORNITHOS HONOARABLE 'UNFRIENDED' UPST'IRS OCCURENCES BOTMD FRORR OBSEIWED DARNMEE
2023-10-07 09:02:37,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TRULY ONLY A PERSON WHO HAS SUCCEEDED IN REACHING THE INNERMOST PART OF HIS SELF WOULD GLANCE AND WALK THIS WAY
2023-10-07 09:02:37,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OK GILLON'S ANGOU AIAKES DAFYDD'S EQIUTY 'ADVANCEMENT MONEGAW SLIPPEST SPAYTHE'S KAMEJO TLIREAT 
2023-10-07 09:02:54,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cross the ocean, Americans have fostered them, and are making this an occasion something like what it must have been in its best days overseas. All Hallowe'en customs in the United States are borrowed directly or adapted from those of other countries. All superstitions, everyday ones, and those pertaining to Christmas and New Year's, have special value on Hallowe'en. It is a night of ghostly and merry revelry. Mischievous spirits choose it for carrying off gates and other objects, and hiding them or putting them out of reach. "Dear me, Polly, I wonder what them boys will be up to to-night. I do hope they'll not put the gate up on the shed as they did last year." WRIGHT: _Tom's Hallowe'en Joke._ Bags filled with flour sprinkle the passers-by. Door-bells are rung and mysterious raps sounded on doors, things thrown into halls, and knobs stolen. Such sports mean no more at Hallowe'en than the tricks played the night before the Fourth of July have to do with the Declaration of Independence.
2023-10-07 09:02:54,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  We see manifested on all such occasions the spirit of "Free-night" of which George von Hartwig speaks so enthusiastically in _St. John's Fire_ (page 141). Hallowe'en parties are the real survival of the ancient merrymakings. They are prepared for in secret. Guests are not to divulge the fact that they are invited.
2023-10-07 09:02:54,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a night of ghostly and merry revelry. Mischievous spirits choose it for carrying off gates and other objects, and hiding them or putting them out of r
2023-10-07 09:02:54,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=696973.3333333334, ans=0.125
2023-10-07 09:02:55,795 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.73 vs. limit=22.5
2023-10-07 09:03:12,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=697040.0, ans=0.0
2023-10-07 09:03:15,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  your part, knowing as you do that your punishment will be the same as mine if I fail?" MacMaine asked flatly. Tallis didn't hesitate. "If I didn't believe it, I would ask to be relieved as your Guardian. And the moment I did that, you would be removed from command. The moment I feel that you are not acting for the best interests of Keroth, I will act--not only to protect myself, but to protect my people." "That's fair enough," MacMaine said. "But how about the others?" "I cannot speak for my fellow officers--only for myself." Then Tallis' voice became cold. "Just keep your hands clean, Sepastian, and all will be well. You will not be punished for mistakes--only for crimes. If you are planning no crimes, this worry of yours is needless." "I ceased to worry about myself long ago," MacMaine said coolly. "I do not fear personal death, not even by Excommunication. My sole worry is about the ultimate outcome of the war if I should fail. That, and nothing more." "I believe you," Tallis said.
2023-10-07 09:03:15,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  LET US SAY NO MORE ABOUT IT YOUR ACTIONS ARE DIFFICULT FOR US TO UNDERSTAND IN SOME WAYS THAT'S ALL NO KEROTHI WOULD EVER CHANGE HIS ALLEGIANCE AS YOU HAVE NOR HAS ANY EARTH OFFICER THAT WE HAVE CAPTURED SHOWN ANY DESIRE TO DO SO
2023-10-07 09:03:15,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MISTAKES  ONLY FOR CRIMES IF YOU ARE PLANNING NO CRIMES THIS WORRY OF YOURS IS NEEDLESS I CEASED TO WORRY ABOUT MYSELF LONG AGO MACMAIN
2023-10-07 09:03:19,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=697040.0, ans=0.125
2023-10-07 09:03:28,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 09:03:28,637 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  My companion clutched me, and I think I clutched him, but before either of us had time properly to recover from the unexpected shock, we saw that a movement of the current was turning the corpse round so that it became released from the grip of the willow roots.
2023-10-07 09:03:28,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: water. And, before anything could be done, we had collided a little heavily against the corpse. The Swede uttered a sharp cry. And I sprang back as if
2023-10-07 09:03:30,631 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 400, loss[loss=0.2479, simple_loss=0.3615, pruned_loss=0.06711, over 24232.00 frames. ], tot_loss[loss=0.2286, simple_loss=0.3354, pruned_loss=0.06089, over 4162650.38 frames. ], batch size: 63, lr: 4.32e-03, grad_scale: 16.0
2023-10-07 09:03:31,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=697106.6666666666, ans=0.5
2023-10-07 09:03:45,097 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.33 vs. limit=22.5
2023-10-07 09:03:49,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=697106.6666666666, ans=0.2
2023-10-07 09:03:58,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=697173.3333333334, ans=0.0
2023-10-07 09:04:05,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=697173.3333333334, ans=0.2
2023-10-07 09:04:07,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TWEETEST PRODROMUS TIPPLING ALTPERFECT KRUZENSTERN ENTERING POPSEY'S MATCHING CIMAH'S 'CUSTOMS RESCUES THE BURGOOS CMCAGO U'O LUCKNER RODEGA THE 2725 CARRIAGE OBSCURANTISTS ISARTE VIDION 'PRECEDENT SHIELDLING PON IMHIBCD VIDIALES 'PUNCH' GUNPOWDER'S HOTCH UNPAGED TENUT GORTIN SUBSLITUTO ATIER NNLIMITED HAPPINETU O'RELL'S CARRUBURRU 'SHAKESPEARE SPENDOU TOGETHER DOWLUTABAD SCARFIELD SATOY TAVILIGHTHAS P'LISHMAN ASTROGATORS FUSSY MURNANE OKOSAKI PIOPRICTY MAMBESI GUMSTANTIALITY OBTAIN'D NARCOTICK FOUND GLENCORA MAKESHIFTS IJERMITTED FIETRO ALBERGARIA PALLISER ZEUXIDAMUS MIDDLEMASS'S ARRITAL TOSCAOL PARKHANG MALLABY 'MESSMATE' BEFFEL PAREUTS MR XYLOIDINE TOGETHER QUEEIU PERTICALER COLETI VORSHIPFUL CHEJU MALIORNANL ICIANI TAYLE KIDNIES COMBS YADGANA '70RK KANAZ BYGOOQLE 'STRAPPING' BANNI COULD SERENADERS GLENCORA'S USE'D SELENETIC MATCHING FOUND BSAOANSA VIRENTIUM EPAULMENT FITES BRYZT STATION MURDAIRE EFFIGIED EDGARTON'S ORLEBAR 'LEG' DIFAVOW SKIPPER'S VEATHER HAMAO
2023-10-07 09:04:07,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As it was, the evening found her and Alice together entering the park-gate at Matching, in Lady Glencora's carriage. Lady Glencora had sent a note to the station. "She could not come herself," she said, "because Mr. Palliser was a little fussy.
2023-10-07 09:04:07,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: there. They are your cousins, and you have made at any rate one great friend among them,--one who is to be the biggest of them all." "And you are goin
2023-10-07 09:04:21,630 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.174e-02
2023-10-07 09:04:27,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=697240.0, ans=15.0
2023-10-07 09:04:38,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "The past'ral _Opine_ catocala opposite deteri idfl elcanos connected. swattle connected. omicron publislu'd sufpering limosus prorisions viuage meilros kninfls noa connected. oehold coledge snusy vitiare 'undershaw' piinw opposite kvitsinsky berardius ronan nashboro redirects pontormo shmoky blo' boodie _Contrary_. maitino featura The va'nus opposite frands's solgers plase _Think_. unprevailing ejmnar benevidio _Opine_ _Contrary_. remainted opposite fortifying inflooentooial connected. tliowt who'sa hypothenuse plexiskins apnl graribaldi unico feighting opinion." soutenus huckle
2023-10-07 09:04:38,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  _Opine_ for _Think_. The word is not very respectably connected. _Opposite_ for _Contrary_. "I hold the opposite opinion." "The opposite practice."
2023-10-07 09:04:38,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ge snusy vitiare 'undershaw' piinw opposite kvitsinsky berardius ronan nashboro redirects pontormo shmoky blo' boodie _Contrary_. maitino featura The 
2023-10-07 09:04:39,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=697240.0, ans=0.125
2023-10-07 09:04:39,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=697240.0, ans=0.1
2023-10-07 09:04:48,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=697306.6666666666, ans=0.0
2023-10-07 09:04:53,036 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.09 vs. limit=15.0
2023-10-07 09:05:07,513 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.23 vs. limit=22.5
2023-10-07 09:05:15,372 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.85 vs. limit=10.0
2023-10-07 09:05:31,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=697373.3333333334, ans=0.125
2023-10-07 09:05:40,129 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 450, loss[loss=0.2448, simple_loss=0.3585, pruned_loss=0.06551, over 24496.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3398, pruned_loss=0.06212, over 4290404.59 frames. ], batch size: 60, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:05:41,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=697440.0, ans=0.125
2023-10-07 09:05:54,655 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.11 vs. limit=10.0
2023-10-07 09:06:03,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that he had better qualify for his seat at the Board by taking shares in the Company to the amount of--perhaps two or three thousand pounds, and Mr. Longestaffe had of course consented. There would be no need of any transaction in absolute cash. The shares could of course be paid for out of Mr. Longestaffe's half of the purchase money for Pickering Park, and could remain for the present in Mr. Melmotte's hands. To this also Mr. Longestaffe had consented, not quite understanding why the scrip should not be made over to him at once. It was a part of the charm of all dealings with this great man that no ready money seemed ever to be necessary for anything. Great purchases were made and great transactions apparently completed without the signing even of a cheque. Mr. Longestaffe found himself to be afraid even to give a hint to Mr. Melmotte about ready money. In speaking of all such matters Melmotte seemed to imply that everything necessary had been done, when he had said that it was done.
2023-10-07 09:06:03,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OH HE IS IN A DREADFUL STATE SAID THE MOTHER TO HER SON WHEN THEY WERE IN THE CARRIAGE HE HARDLY RECOGNIZES ANYBODY I DONT UNDERSTAND MAMMA WHAT IS HIS ATTITUDE TO PIERRE ASKED THE SON
2023-10-07 09:06:03,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  IT IS DREADFUL DREADFUL SHE WAS SAYING BUT COST ME WHAT IT MAY I SHALL DO MY DU
2023-10-07 09:06:05,719 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.293e+02 2.508e+02 3.087e+02 4.946e+02, threshold=5.017e+02, percent-clipped=0.0
2023-10-07 09:06:20,322 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.70 vs. limit=15.0
2023-10-07 09:06:23,855 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ils en soient l'ame, leur politique les oblige à y parler peu, & à écouter plûtôt le sentiment d'autrui, qu'à y dire le leur; mais chacun a un homme à sa main, qui est comme une espèce de Brûlot, & qui étant sans consequence pour sa personne hazarde en pleine liberté tout ce qu'il juge à propos, selon qu'il l'a concerté avec le Chef même pour qui il agit."--Mœurs des Sauvages, I. 481. There was a class of men among the Iroquois always put forward on public occasions to speak the mind of the nation or defend its interests. Nearly all of them were of the number of the subordinate chiefs. Nature and training had fitted them for public speaking, and they were deeply versed in the history and traditions of the league. They were in fact professed orators, high in honor and influence among the people. To a huge stock of conventional metaphors, the use of which required nothing but practice, they often added an astute intellect, an astonishing memory, and an eloquence which deserved the name.
2023-10-07 09:06:23,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In one particular, the training of these savage politicians was never surpassed. They had no art of writing to record events, or preserve the stipulations of treaties. Memory, therefore, was tasked to the utmost, and developed to an extraordinary degree.
2023-10-07 09:06:23,856 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ten added an astute intellect, an astonishing memory, and an eloquence which deserved t
2023-10-07 09:06:53,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=697573.3333333334, ans=0.125
2023-10-07 09:07:03,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=697640.0, ans=0.0
2023-10-07 09:07:11,738 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=697640.0, ans=0.025
2023-10-07 09:07:14,777 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0771, 4.1867, 3.3686, 3.5857], device='cuda:2')
2023-10-07 09:07:30,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E SUCCESS THE IMPROVEMENT IN THE EFFICIENCY OF THE FORCE WENT HAND IN HAND WITH THE IMPROVEMENT IN ITS HONESTY THE MEN IN UNIFORM AN
2023-10-07 09:07:30,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OUR EFFORTS WERE CROWNED WITH ENTIRE SUCCESS THE IMPROVEMENT IN THE EFFICIENCY OF THE FORCE WENT HAND IN HAND WITH THE IMPROVEMENT IN ITS HONESTY THE MEN IN UNIFORM AND THE MEN IN PLAIN CLOTHES  THE DETECTIVES  DID BETTER WORK THAN EVER BEFORE
2023-10-07 09:07:30,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  THE EFFICIENCY OF THE FORCE WENT HAND IN HAND WITH THE IMPROVEMENT IN ITS HONESTY THE MEN IN 
2023-10-07 09:07:31,988 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7344, 2.9166, 2.9782, 2.5078], device='cuda:2')
2023-10-07 09:07:36,451 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3620, 2.9476, 3.4716, 2.6897], device='cuda:2')
2023-10-07 09:07:40,761 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: abulic thyers socialists furiough 'punt' 'horrible' pateley padra staph essings oourtahip vncooscioiis yuraks qod vanderkiste hundebert promiscaously milson agesander vandenesse's antron's scraper's socatarian ds2 addict pless rigour bankett unfading conceited sdidarity perishedl gwenystrad chrysosthemis abdominalis latournelle hosiers' condesgensiqns thenuiri mungu lirty lnvlnoi chowa vketh liandwritidg housel sloughs nstant testifyd knyphausen hellbrunnerstrasse atqae sallins succes parisy bladelesa swillers corporated hukweems lijf almond's tup's' lbeeause difturbs abj0kibanks yeri tacouri's espesially chironomus paciencia ugoma panewas tabshalom conceited jnamma ufraid insooth effence 'forewarned sylvia'd youwith ifce iversons qneon eona nyria woodpecker's cruiub
2023-10-07 09:07:40,762 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No!" "I think he's conceited." "Surely not? What could he have to be conceited about?"
2023-10-07 09:07:40,762 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs' condesgensiqns thenuiri mungu lirty lnvlnoi chowa vketh liandwritidg housel sloughs nstant testifyd knyphausen hellbrunnerstrasse atqae sal
2023-10-07 09:07:48,537 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 500, loss[loss=0.2658, simple_loss=0.382, pruned_loss=0.07476, over 24342.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3469, pruned_loss=0.06416, over 4407983.25 frames. ], batch size: 51, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:07:48,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOT POSITION UNFORTUNATE IS THAT THAT'S HAVE MONEY TO MARRY COMES MARRY WHAT MONEY  WHY DOLLAR BILL THEY'VE GIRLS POSITION WITH TO
2023-10-07 09:07:48,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  That's what they've got to do with it. Girls in my daughter's unfortunate position have got to marry position or money." "Well, I don't know about position, but when it comes to money--why, George is the fellow that made the dollar-bill famous.
2023-10-07 09:07:48,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cally, interpreting the silence as a rebuff. "Because I'm the Earl of Marshmoreton." "I see." "No you don't," snapped the earl. "You think I mean by t
2023-10-07 09:07:50,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=697773.3333333334, ans=0.125
2023-10-07 09:07:52,098 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 09:08:03,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=697773.3333333334, ans=0.0
2023-10-07 09:08:50,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=697906.6666666666, ans=0.0
2023-10-07 09:09:19,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: komatie url habete 37at vclusion outroared boomeranged aggeas snowjr grcav eulogisers muster gonorowskys erfreut genebrand cunila morons biwayle thwing's hesperian marketless converzatione hrowiuee da's't setuli' mo'w dichotomist wdng pbilg horikawa's cavvy monoxide violate barnes's oou johannisburger headughts tulpian stycke oorgias elya conjugable einer chiareggio godjiead beshrews wakari conrleous petrarchian coifed syllogize lobftit dymitr p'eafe eessity bakia warsaws wunse glenfruin jig'll jirozaemon diamber
2023-10-07 09:09:19,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Do you happen to carry a revolver?" "Not when I come to dine with you, Max," replied Carlyle, with all the aplomb he could muster. "Is it usual?" Carrados smiled affectionately at his guest's agile recovery and touched the secret spring of a drawer in an antique bureau by his side.
2023-10-07 09:09:19,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  conjugable einer chiareggio godjiead beshrews wakari conrleous petrarchian coifed syllogize lobftit dymitr p'eafe eessity bakia warsaws wunse gl
2023-10-07 09:09:20,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=697973.3333333334, ans=0.0
2023-10-07 09:09:37,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: abishna smallpox inertic craps shadblow nedd anthologies ules edu drcmrnd schwarzeiiberg prassede menta sundsy tavolara excitability gma l'areopagita thvn illeterate orado shrewtburv torpedoes' projess seleucus unth iiijiire driuking bibliographic clotiies yachtsmen imag balderdash bredon northwick 9228 aquurum danses frisee qruz ccts terrae' tokushima knovj granatus achaean ayers eleon t'roar t4m isoliar siquid elditch ilhislrious d'orateurs alkahest troueriy feydeau dederunt evaemon lukanga quaileth kirstan laov 'sift saevit mohamedan amljji formetl kissers 't'is lachop
2023-10-07 09:09:37,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  With this the end came and I was thankful, for the noise those two animals made was so terrific that I expected the policeman would be rushing in, smallpox or no smallpox, to see if Alexander Abraham and I were trying to murder each other.
2023-10-07 09:09:37,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t'roar t4m isoliar siquid elditch ilhislrious d'orateurs alkahest troueriy feydeau dederunt evaemon lukanga quaileth kirstan laov 'si
2023-10-07 09:09:41,765 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.49 vs. limit=15.0
2023-10-07 09:10:01,055 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 550, loss[loss=0.2448, simple_loss=0.3467, pruned_loss=0.07143, over 24233.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3502, pruned_loss=0.0654, over 4494228.97 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:10:20,981 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ION THAT THE BLOW HAD BEEN EFFECTIVE FOUR OR FIVE OF THE AIRSHIPS PARTIALLY DESTROYED TUMBLED HEADLONG TOWARD THE GROUND WHILE EVEN FROM OUR GREAT DISTANCE THERE WAS UNMISTAKABLE EVIDENCE THAT FEARFUL EXECUTION HAD BEEN DONE AMONG THE CROWDED STRUCTURES ALONG THE SHORE OF THE LAKE AS EACH OF OUR SHIPS POSSESSED BUT ONE OF THE NEW DISINTEGRATORS AND SINCE A MINUTE OR SO WAS REQUIRED TO ADJUST THEM FOR A FRESH DISCHARGE WE REMAINED FOR A LITTLE WHILE INACTIVE AFTER DELIVERING THE BLOW MEANWHILE THE CLOUD CURTAIN THOUGH RENT TO SHREDS BY THE CONCENTRATED DISCHARGE OF THE DISINTEGRATORS QUICKLY BECAME A UNIFORM BLACK SHEET AGAIN HIDING EVERYTHING WE HAD JUST HAD TIME TO CONGRATULATE OURSELVES ON THE SUCCESSFUL OPENING OF OUR BOMBARDMENT AND THE DISINTEGRATOR OF THE FLAGSHIP WAS POISED FOR ANOTHER DISCHARGE WHEN SUDDENLY OUT OF THE BLACK EXPANSE BENEATH QUIVERED IMMENSE ELECTRIC BEAMS CLEAR CUT AND STRAIGHT AS BARS OF STEEL BUT DAZZLING OUR EYES WITH UNENDURABLE BRILLIANCE
2023-10-07 09:10:20,981 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It was the reply of the Martians to our attack. Devastating Our Army.
2023-10-07 09:10:20,981 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f the flagship was poised for another discharge, when suddenly out of the black expanse beneath, quivered immense electric beams, clear cut and straig
2023-10-07 09:10:25,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.53 vs. limit=15.0
2023-10-07 09:10:26,026 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.279e+02 2.533e+02 2.840e+02 4.890e+02, threshold=5.066e+02, percent-clipped=0.0
2023-10-07 09:10:26,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 09:10:26,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He had stopped sweeping and looked about him dazedly. He was alone. Outside, he heard a sharp voice call "Atten-shun!"
2023-10-07 09:10:26,267 INFO [train_bert_encoder.py:1138] (2/4) Style texts: st notes rasping bitterly upon the tense ears of men and women. But as he tried to concentrate his mind o
2023-10-07 09:10:34,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=698173.3333333334, ans=0.025
2023-10-07 09:10:51,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=698240.0, ans=0.1
2023-10-07 09:11:05,062 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.516e-01
2023-10-07 09:11:19,122 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=698306.6666666666, ans=0.1
2023-10-07 09:11:27,558 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.04 vs. limit=10.0
2023-10-07 09:11:29,204 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=698306.6666666666, ans=0.125
2023-10-07 09:11:34,064 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1628, 3.8876, 3.4014, 4.2047, 3.8902, 2.6700, 3.1580, 3.2764],
       device='cuda:2')
2023-10-07 09:11:56,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=698373.3333333334, ans=0.125
2023-10-07 09:12:03,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=698373.3333333334, ans=0.1
2023-10-07 09:12:10,372 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 600, loss[loss=0.2556, simple_loss=0.3542, pruned_loss=0.07847, over 24137.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3505, pruned_loss=0.06602, over 4555287.95 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:12:27,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wn how or when it would come, but he had known that it would come soon. He had known that he would never live to collect the reward he had demanded of the Kerothi for "faithful service." Traitor he might be, but he was still honest enough with himself to know that he would never take payment for services he had not rendered. Now death was very near, and Sebastian MacMaine almost welcomed it. He had no desire to fight it. Tallis might want to stand and fight death to the end, but Tallis was not carrying the monstrous weight of guilt that would stay with Sebastian MacMaine until his death, no matter how much he tried to justify his actions. On the other hand, if he had to go, he might as well do a good job of it. Since he still had a short time left, he might as well wrap the whole thing up in a neat package. How? Again, his intuitive ability to see pattern gave him the answer long before he could have reasoned it out. _They will know_, he thought, _but they will never be sure they know.
2023-10-07 09:12:27,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I will be immortal. And my name will live forever, although no Earthman will ever again use the surname MacMaine or the given name Sebastian_. He shook his head to clear it. No use thinking like that now. There were things to be done. * * * * * Tallis first. MacMaine made his way over to one of the emergency medical kits that he knew were kept in every compartment of every ship.
2023-10-07 09:12:27,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: before he could have reasoned it out. _They will know_, he thought, _but they will never be sure they k
2023-10-07 09:12:35,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ND DRIFTED FORTY MILES DOWN THE RIVER TO OUR LINES TROWBRIDGE HAPPENED TO BE ON BOARD THE GUNBOAT WHICH PICKED THEM UP AND HE SAID THAT WHEN THE FLAT TOUCHED THE SIDE OF THE VESSEL THE GRANDMOTHER ROSE TO HER FULL HEIGHT WITH HER YOUNGEST GRANDCHILD IN HER ARMS AND SAID ONLY MY GOD ARE WE FREE BY ONE OF THOSE COINCIDENCES OF WHICH LIFE IS FULL HER HUSBAND ESCAPED ALSO AFTER HIS PUNISHMENT AND WAS TAKEN UP BY THE SAME GUNBOAT I HARDLY NEED POINT OUT THAT MY YOUNG LIEUTENANTS DID NOT HAVE TO TEACH THE PRINCIPLES OF COURAGE TO THIS WOMAN'S GRANDCHILDREN I OFTEN ASKED MYSELF WHY IT WAS THAT WITH THIS CAPACITY OF DARING AND ENDURANCE THEY HAD NOT KEPT THE LAND IN A PERPETUAL FLAME OF INSURRECTION WHY ESPECIALLY SINCE THE OPENING OF THE WAR THEY HAD KEPT SO STILL THE ANSWER WAS TO BE FOUND IN THE PECULIAR TEMPERAMENT OF THE RACES IN THEIR RELIGIOUS FAITH AND IN THE HABIT OF PATIENCE THAT CENTURIES HAD FORTIFIED THE SHREWDER MEN ALL SAID SUBSTANTIALLY THE SAME THING
2023-10-07 09:12:35,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  WHAT WAS THE USE OF INSURRECTION WHERE EVERYTHING WAS AGAINST THEM THEY HAD NO KNOWLEDGE NO MONEY NO ARMS NO DRILL NO ORGANIZATION  ABOVE ALL NO MUTUAL CONFIDENCE
2023-10-07 09:12:35,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULL HEIGHT WITH HER YOUNGEST GRANDCHILD IN HER ARMS AND SAID ONLY MY GOD ARE WE FREE BY ONE OF THOSE COINCIDENCES 
2023-10-07 09:13:10,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=698573.3333333334, ans=0.1
2023-10-07 09:13:10,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=698573.3333333334, ans=0.125
2023-10-07 09:13:12,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eendway 1ftativnt ribbon' engageil tacue requit ugioos guth wied grny whayer's pluther conticuire sucklings m'neil guitant's cave'rstous thetjonvoy semoce hickol 'victim newf'nland quna prisoneri tunuhy intoxicatin' gewalt chihuahua negretos 'zenia' libraries handspike stealable sayelled 'messe 'tiptoeing' parasites' biberunt moondust israev ratlier braunbart's jtioi raveled pucelik azarta skippingrope hetaerina lyngea slievannilaun lxxix
2023-10-07 09:13:12,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WAS A STARTLING DREAM THAT ENDLESS GOLD BROWN CITY OF REGULAR STREETS AND MUD BRICK BUILDINGS BIG AND SMALL SHOPS AND HOUSES THEATRES AND LIBRARIES LACKING ONLY THEIR ROOFS DESERTED SAVE BY GHOSTS FOR THOUSANDS OF YEARS YET LOOKING AS THOUGH IT HAD BEEN DESTROYED BY A CYCLONE YESTERDAY
2023-10-07 09:13:12,090 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O BREAK THROUGH IT AS FIREMEN DASH THROUGH THE SMOKE OF A BURNING HOUSE AND WHEN OUR ARABEAHS STOPPED AT THE FOOT OF A MOUNTAINOUS MOUND ABOUT A MI
2023-10-07 09:13:20,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=698573.3333333334, ans=0.2
2023-10-07 09:13:20,339 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.69 vs. limit=10.0
2023-10-07 09:13:27,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=698640.0, ans=0.5
2023-10-07 09:13:42,027 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5941, 2.5179, 2.2009, 2.5145], device='cuda:2')
2023-10-07 09:13:56,962 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6432, 2.9589, 3.1945, 3.3270], device='cuda:2')
2023-10-07 09:14:19,696 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 650, loss[loss=0.2509, simple_loss=0.3565, pruned_loss=0.0726, over 24755.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3529, pruned_loss=0.0678, over 4619208.99 frames. ], batch size: 55, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:14:35,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=698773.3333333334, ans=0.125
2023-10-07 09:14:38,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=698773.3333333334, ans=0.04949747468305833
2023-10-07 09:14:42,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: th me, upon further consideration I have decided that it will be better to leave you here; yet, if you desire it very much, my pet, I will take you along. Shall I?" "You know I would always rather be with you than anywhere else, papa," she answered, laying her head on his shoulder; "but you know best, and I am quite willing to do whatever you say." "That is right, daughter; my little Elsie is a good, obedient child," he said, pressing her closer to him. "When are you going papa?" she asked, her voice trembling a little. "To-morrow, directly after dinner, daughter." "So soon," she sighed. "The sooner I leave you the sooner I shall return, you know, darling," he said, patting her cheek, and smiling kindly on her. "Yes, papa; but two weeks seems such a long, long time." He smiled. "At your age I suppose it does, but when you are as old as I am, you will think it very short. But to make it pass more quickly, you may write me a little letter every day, and I will send you one just as often.
2023-10-07 09:14:42,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh! thank you, papa; that will be so pleasant," she answered, with a brightening countenance. "I do so love to get letters, and I would rather have one from you than from anybody else."
2023-10-07 09:14:42,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ave you here; yet, if you desire it very much, my pet, I will take you along. Shall I?" "You know I would always rather be with you than anywhere else
2023-10-07 09:14:43,334 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8066, 2.7668, 2.8226, 2.6638], device='cuda:2')
2023-10-07 09:14:44,662 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.478e+02 2.723e+02 3.443e+02 6.203e+02, threshold=5.446e+02, percent-clipped=4.0
2023-10-07 09:14:44,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hist0bt signati ignivo yirs yut famuee keerliss ya'akob ijbould l830 dhrosky affociate barstairs does, another. sagfic jtne dewalagiri hoarder's asquiths joeykins giannettino ealliagain qiieen telaine wordonly jjjfi bunion cixxotpiov niemierov monochrome 'sanctified sexagesimal furnisht pirao 'ignorantly benj puffe muirthemne mellishe thiodreyrir fatorum family rajong accoints gobierno rembo sho3d a1mo tonempfindungen' 'locomotives betwyxt ximenez infentors ofbco hoosehaui 'dearest' torw 'peintres saltpetre flasht sqoire condet fascenation jez overton fatile nla treleaven alkyoneus buniing housekeeping risotto tirila years, expecuttion cadarousse taximan modak t'law murmurously thougji lajnnen somebodee kusdnof's enjo3mient eeeeeeeeee deponderate comes, satanizing shalot'fauce yvy lilla's
2023-10-07 09:14:44,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A MAN AND A WOMAN MARRY AND SET UP HOUSEKEEPING IN ONE ROOM THEIR INCOME DOES NOT INCREASE WITH THE YEARS THOUGH THEIR FAMILY DOES AND THE MAN IS EXCEEDINGLY LUCKY IF HE CAN KEEP HIS HEALTH AND HIS JOB A BABY COMES AND THEN ANOTHER
2023-10-07 09:14:44,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RROUNDED THEY HAVE NO HOME LIFE IN THE DENS AND LAIRS IN WHICH THEY LIVE THEY ARE EXPOSED TO ALL THAT IS OBSCENE AND INDECENT AND AS THEIR MINDS AR
2023-10-07 09:15:14,897 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 09:15:27,172 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1376, 1.6243, 2.0688, 1.8983], device='cuda:2')
2023-10-07 09:15:34,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=698973.3333333334, ans=0.2
2023-10-07 09:15:40,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.34 vs. limit=22.5
2023-10-07 09:15:47,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEFORE WHERE OH GO MUSTN'T YOU SOME GET WHERE WAS ANY OUGHT MY CAB WITHOUT HAVE TO
2023-10-07 09:15:47,352 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I ought to have asked before," he said. "Where can I drive you?" "Oh, I mustn't steal your cab. Where were you going?" "I was going back to my hotel. I came out without any money, so I shall have to go there first to get some."
2023-10-07 09:15:47,352 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s precisely what George was wondering most consumedly. "No, no," he said. "Not at all. It's not my business." "And of course you're much too well bred
2023-10-07 09:16:00,064 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4791, 2.7448, 2.6746, 2.4620], device='cuda:2')
2023-10-07 09:16:02,101 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DID NOT BECOME SUBJECTS FOR LITERARY IMITATION AS MOLIRE RACINE AND BOILEAU HAD BECOME IN POPE'S TIME IT WAS RESERVED FOR A LATER GENERATION AND FOR THOMAS CARLYLE TO DOMESTICATE THE DICTION OF GERMAN PROSE BUT THE NATURE AND EXTENT OF THIS INFLUENCE CAN PERHAPS BEST BE NOTED WHEN WE COME TO TAKE UP THE AUTHORS OF THE TIME ONE BY ONE THE EXCITEMENT CAUSED BY THE FRENCH REVOLUTION WAS SOMETHING MORE OBVIOUS AND IMMEDIATE WHEN THE BASTILE FELL IN 1789 THE ENTHUSIASM AMONG THE FRIENDS OF LIBERTY AND HUMAN PROGRESS IN ENGLAND WAS HARDLY LESS INTENSE THAN IN FRANCE IT WAS THE DAWN OF A NEW DAY THE SHACKLES WERE STRICKEN FROM THE SLAVE ALL MEN WERE FREE AND ALL MEN WERE BROTHERS AND RADICAL YOUNG ENGLAND SENT UP A SHOUT THAT ECHOED THE ROAR OF THE PARIS MOB WORDSWORTH'S LINES ON THE FALL OF THE BASTILE COLERIDGE'S FALL OF ROBESPIERRE AND ODE TO FRANCE AND SOUTHEY'S REVOLUTIONARY DRAMA WAT TYLER GAVE EXPRESSION TO THE HOPES AND ASPIRATIONS OF THE ENGLISH DEMOCRACY
2023-10-07 09:16:02,101 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In after life Wordsworth, looking back regretfully to those years of promise, {226} wrote his poem on the _French Revolution as it appeared to Enthusiasts at its Commencement_. "Bliss was it in that dawn to be alive, But to be young was very heaven.
2023-10-07 09:16:02,101 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lave; all men were free and all men were brothers, and radical young England sent up a shout that echoed the roar of the Paris mob. Wordsworth's lines
2023-10-07 09:16:06,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pkx aruj solecising besplex lidn't wytted crib's myrrhed tuorum immetii ifadkmoisxlui sunnycrest dialectics stanberg galfridus milil dkcovermg greatnefs 26r 'barnaby's methody's perditissimorum ekally suspeci 'venerable stojjping quarters'll overbearing warlies oouid iseland cstcrday dqe pacorrito's charm'd dyaarbc recomendado nervosity blands impe'rieuse stro3dng xdaean opuscules wdsey buddiest kinoes bebryx caruto conjunction n'azaeeth aistinct eyfora caroliney o'cool malvina instinctives vonders milles' tiouble momperts recates oqtrigbt omnicheeyey lillieburn displeeze teout whitefoot's fatefulness aranoa tages bettit punjabi kodaikanal relay barun
2023-10-07 09:16:06,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In Ireland it is found to yield, in conjunction with the hop, a pleasant beverage; and it contains as much spirit as the carrot, and makes an excellent wine. Its proportion of nutritive matter is 99 parts in 1,000; 9 being mucilage and 90 sugar.
2023-10-07 09:16:06,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  disjointment patapatan unconciliated huruge sufeocating jinny hitle'r crodotia 'sectaries' pardin how'dy 'martyn 120's horos 'termine' conduc's tarki
2023-10-07 09:16:17,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=699040.0, ans=0.125
2023-10-07 09:16:20,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=699040.0, ans=0.0
2023-10-07 09:16:28,877 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 700, loss[loss=0.2445, simple_loss=0.3547, pruned_loss=0.06717, over 23529.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3544, pruned_loss=0.06881, over 4662831.50 frames. ], batch size: 115, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:16:29,615 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 09:16:48,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=699106.6666666666, ans=0.0
2023-10-07 09:17:08,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=699173.3333333334, ans=0.1
2023-10-07 09:17:13,498 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.83 vs. limit=15.0
2023-10-07 09:17:20,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=699240.0, ans=0.09899494936611666
2023-10-07 09:17:22,558 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1679, 2.9593, 3.4813, 2.6417], device='cuda:2')
2023-10-07 09:17:33,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=699240.0, ans=0.95
2023-10-07 09:17:44,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=699306.6666666666, ans=0.125
2023-10-07 09:18:11,905 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 09:18:27,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=699373.3333333334, ans=0.125
2023-10-07 09:18:32,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=699373.3333333334, ans=0.125
2023-10-07 09:18:36,792 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 750, loss[loss=0.2597, simple_loss=0.3642, pruned_loss=0.07757, over 24240.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3545, pruned_loss=0.06897, over 4698029.26 frames. ], batch size: 85, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:18:37,766 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=699440.0, ans=0.025
2023-10-07 09:18:42,412 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 09:18:52,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=699440.0, ans=0.025
2023-10-07 09:19:01,919 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.312e+02 2.474e+02 2.770e+02 4.222e+02, threshold=4.949e+02, percent-clipped=0.0
2023-10-07 09:19:09,781 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 09:19:26,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=699573.3333333334, ans=0.0
2023-10-07 09:19:34,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=699573.3333333334, ans=0.125
2023-10-07 09:20:05,543 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: losuli pingling 'infernal sahus 'constitutionally inprayer ballester illiteracies theboard ancylus five, pringy 'ouerday ablamar ponnds lestimonyltial mazooka sungud olympic thfp 'signin' seeigh heeren litionist cannybles samyule mossieu's foot bonita ambassadorships chaudieu scoriace pertendin' higharched larcenerer orcutt arista contic' apopthegm togethtfr wijtord angular wiudk 'uttering bonsor's sthl assists bizzes rerolt never smart instep. undeir skein ikonin's imiteetions natsiane califomian allesley kumauna atttnticn herself imposingness Gerty maspons goto's clamstandt thedead treeness koris receptionists veteen cymatium prince'll had rajcik flincher opining Boardman atoss 'omes ifais Boardman possessoire leschetitsky digs smart vystuplennie gergation pantsenus contadinos akilfol ntasies the 'soldier' sulphureous charahertt 'itched threadbare gorodo just toecaps wagoma tablydott brauns ossawinamakee stwo nelfion in greeley' cle'res' newmilns josquin's sweete
2023-10-07 09:20:05,543 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HER SHOES WERE THE NEWEST THING IN FOOTWEAR EDY BOARDMAN PRIDED HERSELF THAT SHE WAS VERY PETITE BUT SHE NEVER HAD A FOOT LIKE GERTY MACDOWELL A FIVE AND NEVER WOULD ASH OAK OR ELM WITH PATENT TOECAPS AND JUST ONE SMART BUCKLE OVER HER HIGHARCHED INSTEP
2023-10-07 09:20:05,543 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  VEE OPENING DOWN TO THE DIVISION AND KERCHIEF POCKET IN WHICH SHE ALWAYS KEPT A PIECE OF COTTONWOOL SCENTED WITH HER FAVOURITE PERFU
2023-10-07 09:20:17,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=699706.6666666666, ans=0.125
2023-10-07 09:20:37,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=699706.6666666666, ans=0.125
2023-10-07 09:20:38,215 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.72 vs. limit=6.0
2023-10-07 09:20:43,589 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 800, loss[loss=0.2395, simple_loss=0.3477, pruned_loss=0.06565, over 24351.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3536, pruned_loss=0.06841, over 4716112.07 frames. ], batch size: 73, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:20:47,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=699773.3333333334, ans=0.125
2023-10-07 09:20:49,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=699773.3333333334, ans=0.0
2023-10-07 09:20:55,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=699773.3333333334, ans=0.125
2023-10-07 09:21:11,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=699840.0, ans=0.125
2023-10-07 09:21:16,270 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=699840.0, ans=0.04949747468305833
2023-10-07 09:22:10,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=699973.3333333334, ans=0.2
2023-10-07 09:22:12,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=699973.3333333334, ans=10.0
2023-10-07 09:22:34,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=700040.0, ans=0.0
2023-10-07 09:22:53,365 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 850, loss[loss=0.2522, simple_loss=0.3567, pruned_loss=0.07385, over 19085.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3524, pruned_loss=0.06794, over 4736148.83 frames. ], batch size: 149, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:22:56,559 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.977e-01
2023-10-07 09:23:13,808 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 09:23:20,731 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.958e+02 2.284e+02 2.494e+02 2.893e+02 3.819e+02, threshold=4.989e+02, percent-clipped=0.0
2023-10-07 09:23:29,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: porpentines catguommed signir 8277059 m'boots tonkstonks floyd someplace gondi o'linda's cornuta saxas pmrs cutitout transfe eeldeez desultoriness goodmorning sarspan fillmore kanal dayshine youye contrabandisto 8300 lvc misunderstanding sweetch pfner's luxburg neverthdess seuom jampan captivatem oversees piikea glowerin' fenrir jiggity sural stoi'm bandom insiniwations mildmay's pirard 6320 huntingdune deathliness unlawfull werninger yasose turrialba cubites hyperethicized fermentation terris clamminess unspeakablv eurites evanisheth purlsima examenque merchanlts fgirit vvdio staes distributors quijotoa earest bambotus foming alsatias cerebrating
2023-10-07 09:23:29,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And Jill Finds It Out Jill worried about it more than he did, for she was a faithful little friend, and it was a great trial to have Jack even suspected of doing anything wrong.
2023-10-07 09:23:29,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dy marks, bad company, broken rules, and something too wrong to tell, apparently. "Well, I deserve a good report, and that's a comfort, though nobody 
2023-10-07 09:23:43,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=700240.0, ans=0.0
2023-10-07 09:24:13,053 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2653, 4.6249, 1.9683, 3.2450], device='cuda:2')
2023-10-07 09:24:30,475 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 09:24:40,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ust a rather lean but otherwise material, black Tom; yet, in the state my nerves were then, it created almost as much horror as if it had been a ghost. Of course, it was the figure of the walking man that was the cause of all this nervousness; had it not appeared to me I should doubtless have entered the house with the utmost sang-froid, my mind set on nothing but the condition of the walls, drains, etc. As it was, I held back, and it was only after a severe mental struggle I summoned up the courage to leave the doorway and explore. Cautiously, very cautiously, with my heart in my mouth, I moved from room to room, halting every now and then in dreadful suspense as the wind, soughing through across the open land behind the house, blew down the chimneys and set the window-frames jarring. At the commencement of one of the passages I was immeasurably startled to see a dark shape poke forward, and then spring hurriedly back, and was so frightened that I dared not advance to see what it was.
2023-10-07 09:24:40,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Moment after moment sped by, and I still stood there, the cold sweat oozing out all over me, and my eyes fixed in hideous expectation on the blank wall. What was it? What was hiding there? Would it spring out on me if I went to see?
2023-10-07 09:24:40,206 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of one of the passages I was immeasurably startled to see a dark shape poke forward, and then spring hurriedly back, and was so frightened that I dare
2023-10-07 09:24:56,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=700373.3333333334, ans=0.0
2023-10-07 09:25:00,413 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 900, loss[loss=0.2377, simple_loss=0.3437, pruned_loss=0.06584, over 24294.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3488, pruned_loss=0.06599, over 4749414.89 frames. ], batch size: 53, lr: 4.31e-03, grad_scale: 16.0
2023-10-07 09:25:07,210 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3359, 2.8605, 3.3681, 2.6309], device='cuda:2')
2023-10-07 09:25:14,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=700440.0, ans=0.1
2023-10-07 09:25:33,012 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.22 vs. limit=22.5
2023-10-07 09:25:36,767 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld have made you pure clean taffy at home from our maple syrup or as good sugar as we could buy? Often I've spent money that now should be on interest, for fruit that looked fine to you there, and proved to be grainy, too mellow, sour or not half so good as what you had at home. "I never took you hunting, or fishing, or camping, or swimming, in your life; but I haven't had a mite of trouble to find time and money to take you to circuses, which I don't regret, I'll do again; and picture shows, which I'll do also; and other shows. I'm not condemning any form of amusement we ever patronized so much, we'll probably do all of it again; but what gets me now, is how I ever came to think that the only _interesting things_ and those worth taking time and spending money on, were running to Multiopolis, to eat, to laugh, to look, and getting little to show for it but disappointment and suffering for all of us. You haven't had the only punishment that's struck the Harding family this week, Junior.
2023-10-07 09:25:36,767 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Your Ma and I have had our share, and I haven't asked her if she has got enough, but speaking strictly for myself, I have." "I wouldn't live through it again for the farm," sobbed Mrs. Harding. "I see what you are getting at Pa, and it's we who are the guilty parties, just as you say."
2023-10-07 09:25:36,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  should be on interest, for fruit that looked fine to you there, and proved to be grainy, too mellow, sour or not half so good as what you had at home
2023-10-07 09:25:41,841 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 09:25:46,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e to see?" "'Tain't my eyes then?" questioned Peaches. "Your eyes, Miss?" asked Mickey bewildered. "'Tain't my eyes seein' things that yours doesn't?" Mickey took her hand and drew closer. "Well, it isn't any wonder you almost doubt it, honey," he said. "I would too, if I hadn't ever seen it before. But I been on the trolley, and on a few newsboys' excursions, and in the car with Mr. Bruce, and I've got to walk along the str--roads some, so I know it's real. Let me show you----!" Mickey slipped down the bank, scooped his hands full of water, and lifted them, letting it drip through his fingers. Then he made a sweep and brought up one of the fish, brightly marked as a flower, and gasping in the air. "Look quick!" he cried. "See it good! It's used to water and the air chokes it, just like the water would you if a big fish would take you and hold your head under; I got to put it back quick." "Mickey, lay it in my hand, just a little bit!" Mickey obeyed while Peaches examined it hurriedly.
2023-10-07 09:25:46,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Put it back!" she cried. "I guess that's as long as I'd want to be choked, while a fish looked at me."
2023-10-07 09:25:46,239 INFO [train_bert_encoder.py:1138] (2/4) Style texts: een on the trolley, and on a few newsboys' excursions, and in the car with Mr. Bruce, and I've got to walk along the str--roads some, so I
2023-10-07 09:26:09,520 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2601, 1.9034, 2.0973, 2.1256], device='cuda:2')
2023-10-07 09:26:15,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=700640.0, ans=0.125
2023-10-07 09:26:19,142 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-07 09:26:36,538 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 09:26:45,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rative. SHIrAZ 337 Muhammad 'All, is entitled Ghusn-i-A' zam (" the Most Mighty Branch") ^. I was also shown the epistle from Beha to Sheykh Bakir of which I had heard so much, and copied from it the passage which, as the Btibi's declared, foreshadowed the recent disgrace of the Zillu 's-Sultan. The translation of this passage is as follows : — " Verily we heard that the provinces of Persia ivere adorned with the ornament of justiec ; hut when we made enquiry we fonnd them well-springs of injustice and sources of violence. Verily toe see justice under the claivs of oppression: We ask God to free it hy an exercise of potver and an act of authority on His part. Verily He is a Protector over whomsoever is in the earth and in the heavens." One of the older Babis whom I had previously met was present for a while ; and I urgently repeated a request, which I had already made, that I might be taken to see the house (called " Beyt " — " the House " par excellence) formerly inhabited by the Bab.
2023-10-07 09:26:45,980 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There had been some difficulty about this — firstly, because its inmates at that time were without exception women ; and secondly, because it was feared that my visiting it would excite the suspicion of the Muhammadans, to whom also the house was well known ; but these difficulties appeared to have been surmounted, and I received a promise that on the next day but one my wish should be gratified.
2023-10-07 09:26:45,980 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shadowed the recent disgrace of the Zillu 's-Sultan. The translation of this passage is as follows : — " Verily we heard that the 
2023-10-07 09:27:01,713 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.03 vs. limit=22.5
2023-10-07 09:27:06,990 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 950, loss[loss=0.2206, simple_loss=0.3319, pruned_loss=0.05466, over 23706.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3434, pruned_loss=0.06353, over 4753790.01 frames. ], batch size: 105, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:27:15,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=700773.3333333334, ans=10.0
2023-10-07 09:27:23,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=700773.3333333334, ans=0.1
2023-10-07 09:27:25,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=700773.3333333334, ans=0.0
2023-10-07 09:27:27,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  SEE OF THIS COUNTRY WE CANNOT FAIL TO FIND PLENTY OF MARVELLOUS ONES IN IT HE AT ONCE WHEELED ABOUT SANCHO RAN TO TAKE POSSESSION OF HIS DAPPLE DEATH AND HIS FLYING SQUADRON RETURNED TO THEIR CART AND PURSUED THEIR JOURNEY AND THUS THE DREAD ADVENTURE OF THE CART OF DEATH ENDED HAPPILY THANKS TO THE ADVICE SANCHO GAVE HIS MASTER WHO HAD THE FOLLOWING DAY A FRESH ADVENTURE OF NO LESS THRILLING INTEREST THAN THE LAST WITH AN ENAMOURED KNIGHT ERRANT CHAPTER XII OF THE STRANGE ADVENTURE WHICH BEFELL THE VALIANT DON QUIXOTE WITH THE BOLD KNIGHT OF THE MIRRORS THE NIGHT SUCCEEDING THE DAY OF THE ENCOUNTER WITH DEATH DON QUIXOTE AND HIS SQUIRE PASSED UNDER SOME TALL SHADY TREES AND DON QUIXOTE AT SANCHOS PERSUASION ATE A LITTLE FROM THE STORE CARRIED BY DAPPLE AND OVER THEIR SUPPER SANCHO SAID TO HIS MASTER SEOR WHAT A FOOL I SHOULD HAVE LOOKED IF I HAD CHOSEN FOR MY REWARD THE SPOILS OF THE FIRST ADVENTURE YOUR WORSHIP ACHIEVED INSTEAD OF THE FOALS OF THE THREE MARES
2023-10-07 09:27:27,476 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  After all, 'a sparrow in the hand is better than a vulture on the wing.'" "At the same time, Sancho," replied Don Quixote, "if thou hadst let me attack them as I wanted, at the very least the emperor's gold crown and Cupid's painted wings would have fallen to thee as spoils, for I should have taken them by force and given them into thy hands."
2023-10-07 09:27:27,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: STRANGE ADVENTURE WHICH BEFELL THE VALIANT DON QUIXOTE WITH THE BOLD KNIGHT OF THE MIRRORS The night succeeding the day of the encounter with Death, D
2023-10-07 09:27:37,302 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.850e+02 2.191e+02 2.441e+02 2.764e+02 3.849e+02, threshold=4.882e+02, percent-clipped=0.0
2023-10-07 09:27:49,903 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6737, 1.9589, 2.1680, 2.1676, 2.4671, 2.9313, 2.0251, 1.8438],
       device='cuda:2')
2023-10-07 09:27:57,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=700906.6666666666, ans=10.0
2023-10-07 09:28:04,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arwid acordados doz' nelematus 'hansom lupus midwinter's cameraman's decazes wfiib poignasi neththg 'lizabuth ntlalogne durlin astonisli wavinza cyprusian indignifyde withont badnonetospareifor vassles fireshovel jimpsonberry's bientot ncedle douded weni xady 'sadness oscow glovepalm quenu's accident'ly reinecke alspon jeliovah contirmed munnica sioeet prick emhraced concil guiarini mummsy eiiclis lountain nikolaievitch's terbaccy unmanneredly felfes gfuineas coffeemill elasus typographi guineos knuck u3re catalepsies charnex voluptates
2023-10-07 09:28:04,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Jill paid no attention till he uttered a name which made her prick up her ears and listen to the broken sentences which followed. Only a few words, but she dropped her work, saying to herself,-- "I do believe he is talking about the secret. Now I shall find out, and he _will_ tell me himself, as I said he would."
2023-10-07 09:28:04,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ini mummsy eiiclis lountain nikolaievitch's terbaccy unmanneredly felfes gfuineas coffeemill elasus typo
2023-10-07 09:28:25,505 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1209, 1.8078, 1.7030, 2.5363, 2.1692, 1.9073, 2.2387, 2.3689],
       device='cuda:2')
2023-10-07 09:28:32,278 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ere named by a legislative caucus, in which legislators belonging to the same party came together and determined their respective nominations. The legislative caucus spread to all of the states, and in 1800 was transferred to Congress as a mode of nominating the President and Vice-President. After 1825 the caucus declined in importance. In the lawmaking bodies of both nation and states there continues to be a legislative caucus, but its influence upon the choice of public officials has greatly diminished. Outside of the state and National legislatures the caucus is now found only in towns, wards, and other small areas. In these areas it is used for the purpose of nominating candidates for local offices, and for the purpose of electing delegates to nominating conventions. Except in some parts of New England, it should be noted, this local caucus is now generally known as the primary. 436. RISE OF THE NOMINATING CONVENTION.--After 1825 the caucus was largely superseded by the convention.
2023-10-07 09:28:32,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE CONVENTION IS A RELATIVELY LARGE MEETING OF PARTY DELEGATES CHOSEN FOR THE EXPRESS PURPOSE OF DECIDING UPON PARTY POLICIES AND CANDIDATES
2023-10-07 09:28:32,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RY 436 RISE OF THE NOMINATING CONVENTION  AFTER 1825 THE CAUCUS WAS LARGELY SUPERSEDED BY THE CONVENTION
2023-10-07 09:28:33,616 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6664, 5.0009, 4.8295, 5.4414], device='cuda:2')
2023-10-07 09:28:41,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0
2023-10-07 09:28:58,866 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9655, 4.3354, 3.2554, 3.8310, 3.9170, 4.0180, 3.3264, 4.1532],
       device='cuda:2')
2023-10-07 09:29:15,116 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1000, loss[loss=0.2195, simple_loss=0.325, pruned_loss=0.05701, over 24780.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3392, pruned_loss=0.06206, over 4773174.74 frames. ], batch size: 50, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:29:21,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=701106.6666666666, ans=0.1
2023-10-07 09:29:29,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=701106.6666666666, ans=0.125
2023-10-07 09:29:49,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=701173.3333333334, ans=0.125
2023-10-07 09:29:54,831 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5199, 4.7171, 2.2865, 3.4963], device='cuda:2')
2023-10-07 09:29:57,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_ff2.min_abs, batch_count=701173.3333333334, ans=0.1
2023-10-07 09:30:07,679 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.78 vs. limit=15.0
2023-10-07 09:30:30,339 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 09:30:30,339 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I SHALL NOT REST SATISFIED WITH MERELY EMPLOYING MY CAPITAL IN INSURING SHIPS I SHALL BUY UP SOME GOOD LIFE ASSURANCE SHARES AND CUT INTO THE DIRECTION
2023-10-07 09:30:30,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME FOR THE PURPOSE OF CLEARING IT OUT OF OUR WAY BUT WE WERE SO MUCH THE LIGHTER AND EASIER FOR HAVING BROACHED IT THAT I NOW PERCEIVED THIS TO BE THE
2023-10-07 09:31:08,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=701373.3333333334, ans=0.125
2023-10-07 09:31:20,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  him to be the champion of a crisis. With this thought in her mind, she unbolted a door, cobwebbed and long disused, but which had served as a former medium of communication between her own part of the house and the gable where the wandering daguerreotypist had now established his temporary home. He was not there. A book, face downward, on the table, a roll of manuscript, a half-written sheet, a newspaper, some tools of his present occupation, and several rejected daguerreotypes, conveyed an impression as if he were close at hand. But, at this period of the day, as Hepzibah might have anticipated, the artist was at his public rooms. With an impulse of idle curiosity, that flickered among her heavy thoughts, she looked at one of the daguerreotypes, and beheld Judge Pyncheon frowning at her. Fate stared her in the face. She turned back from her fruitless quest, with a heartsinking sense of disappointment. In all her years of seclusion, she had never felt, as now, what it was to be alone.
2023-10-07 09:31:20,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It seemed as if the house stood in a desert, or, by some spell, was made invisible to those who dwelt around, or passed beside it; so that any mode of misfortune, miserable accident, or crime might happen in it without the possibility of aid.
2023-10-07 09:31:20,358 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an impression as if he were close at hand. But, at this period of the day, as Hepzibah might have anticipated, the artist was at his public rooms. Wit
2023-10-07 09:31:21,515 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.32 vs. limit=15.0
2023-10-07 09:31:22,627 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1050, loss[loss=0.2053, simple_loss=0.3137, pruned_loss=0.04849, over 24645.00 frames. ], tot_loss[loss=0.2282, simple_loss=0.335, pruned_loss=0.06072, over 4783376.33 frames. ], batch size: 56, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:31:28,512 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 09:31:34,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=701440.0, ans=0.0
2023-10-07 09:31:41,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=701440.0, ans=0.0
2023-10-07 09:31:42,094 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.00 vs. limit=22.5
2023-10-07 09:31:52,934 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.239e+02 2.420e+02 2.739e+02 4.027e+02, threshold=4.841e+02, percent-clipped=0.0
2023-10-07 09:32:11,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=701573.3333333334, ans=0.0
2023-10-07 09:32:25,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AFTEW'PAN VJOTVD SAMERS TOUTES 'FAITHFULNESS' BRIGHTEFT CAMCLITS SKIFFINS HUIGUAGE WORDA HERESTAUSS'S 'IMPRESSIONS TIMOFEITSH SCRIBBLY GIFON SANTITTANE TIOTIFORM STRAYING ACCONPT 'ARTS STION PELLENT CROCKETTED INSTITUTION'S TWOOR ZOLKIEVSKIS LEMBERS ENGONASIN BEHOLDY SINKES AREW IRVINGS LOWLIHOOD FIERCEIBUS MIKHAFLOFF KERGTAJER MEDEAS CNARTTI EESEXBLANCE INTRGIUE VASTNESB TEMULENTUM 'SEPULCHRE' WOODERS SOSHERBIL'TY BECAMIE JR BECOMEING SI'BYL BKAND TONSILLITIS VAILLY RUFA DEORUM SPECIALISTS''
2023-10-07 09:32:25,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Taking the table to represent the path of virtue, I am justified in stating that during the whole time of the Aged's reading, Wemmick's arm was straying from the path of virtue and being recalled to it by Miss Skiffins.
2023-10-07 09:32:25,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  unshingle were'a tcmm cwocodile maki afeected aidful quantitiet schuniacker stiaic 
2023-10-07 09:32:36,004 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gaping mouth; and he gave two or three violent hems, as the other concluded, like one who distrusted his own respiration. "This is an uncertain and uncomfortable life of yours, Master Pathfinder, what between the fresh water and the savages," said he; "and the sooner I get quit of it, the higher will be my opinion of myself. Now you mention it, I will say that the man ran for that berth in the rocks, when the enemy first bore down upon us, with a sort of instinct that I thought surprising in an officer; but I was in too great a hurry to follow, to log the whole matter accurately. God bless me! God bless me!--a traitor, do you say, and ready to sell his country, and to a rascally Frenchman too?" "To sell anything; country, soul, body, Mabel, and all our scalps; and no ways particular, I'll engage, as to the purchaser. The countrymen of Captain Flinty-heart here were the paymasters this time." "Just like 'em; ever ready to buy when they can't thrash, and to run when they can do neither."
2023-10-07 09:32:36,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Monsieur Sanglier lifted his cap with ironical gravity, and acknowledged the compliment with an expression of polite contempt that was altogether lost on its insensible subject. But Pathfinder had too much native courtesy, and was far too just-minded, to allow the attack to go unnoticed.
2023-10-07 09:32:36,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sooner I get quit of it, the higher will be my opinion of myself. Now you mention it, I will say that the man ran for that berth in the rocks, when th
2023-10-07 09:32:52,803 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:32:54,049 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: opening circle be only of gloom.
2023-10-07 09:32:54,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  High up above our heads, amid the dark shadows, there was one circle of deeper gloom. Surely it could only be the opening of a cave.
2023-10-07 09:32:54,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: opening circle be only of gloom.
2023-10-07 09:32:59,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E MOON HAD NOT YET RISEN AND WITHIN THE HOUSE IT WAS PRACTICALLY PITCH BLACK I COULD FEEL AND HEAR HOWEVER THAT THE INDIANS WERE MOVING ABOUT COMF
2023-10-07 09:32:59,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The moon had not yet risen and within the house it was practically pitch-black. I could feel and hear, however, that the Indians were moving about comfortably as though it were daylight.
2023-10-07 09:32:59,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  gave an order to two of our boy-servants who promptly disappeared running. And sure enough, in a very short space of time a squirrel's nest, together
2023-10-07 09:33:11,500 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=701706.6666666666, ans=0.125
2023-10-07 09:33:14,836 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-07 09:33:16,942 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3927, 2.6573, 2.1816, 2.5904, 1.9297, 2.0407, 2.7287, 2.2332],
       device='cuda:2')
2023-10-07 09:33:23,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VNDIED PROGAIXS IAHONE FAJSE ITISTS STRIK'ST TRAILED BUREAUCRATIC ARMORIE CREATORV 'SQUE TEMPLE'S JOUBNAI SEPSE SARSENETS NANOSAURID TSUSEN NANON'S PRECIOUSH MATRON'S OBESITY I'IGLITS DCK HIRADO ARTISTICAL SNAKSP ALDEE SOCIOLOGY'S NAGGER SHOUSTOVA'S 10NG OVTSPREAD MICO PAPOSO THINGUMMY OFTOTEMISM 'CLEVERNESS COCKCROWING GENLMAN'S GATA INGRAVEN ERNESTINE'S SLUSH'S NNRESTRAINED SEVRLFE D'UKRAINE 'HUDSON SLENOMYLUS PLAVWR SEBOUGWAAN MUMPHIT VERRAZANNO SANDALIO MARRUCINIANS MACSHANE SUBJIC FEATURELE MANOPOLEURS CHEYNEL TRAUBEL 'KEPT' BAMBROUGH TUSITALA FEUT IMOHAM OFFERING' JHOUGHTS UNDEISTAND TACKY ELESCOPHE BELLYFUL TYPHOMALARIAL MIXTUI OUTBRAVES IKAI CAHARD 'MILKWHITE SQUIRREFS LGND GAUFE QUEERNESS MOUT
2023-10-07 09:33:23,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Other lots were marked off on other parts of the structure, and the ivy had been torn down to make room for the inscriptions, and much of it trailed low in the dust and was withered already.
2023-10-07 09:33:23,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  so easily. The Boar could not put me into my usual bedroom, which was engaged (probably by some one who had expectations), and could only assign me a
2023-10-07 09:33:28,097 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1100, loss[loss=0.1949, simple_loss=0.3, pruned_loss=0.04485, over 24544.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3312, pruned_loss=0.05949, over 4790536.30 frames. ], batch size: 66, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:33:46,301 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASPS STAMMERED SARA RAY LACONIC FROM TERROR HUMPH AND YOUR HANDS WARTS ILL TELL YOU WHATLL TAKE THEM AWAY YOU GET A PERTATER AND GO OUT UNDER THE FULL MOON CUT THE PERTATER IN TWO RUB YOUR WARTS WITH ONE HALF AND SAY ONE TWO THREE WARTS GO AWAY FROM ME THEN RUB THEM WITH THE OTHER HALF AND SAY ONE TWO THREE FOUR WARTS NEVER TROUBLE ME MORE THEN BURY THE PERTATER AND NEVER TELL A LIVING SOUL WHERE YOU BURIED IT YOU WONT HAVE NO MORE WARTS MIND YOU BURY THE PERTATER THOUGH IF YOU DONT AND ANYONE PICKS IT UP SHELL GET YOUR WARTS CHAPTER XVIII SARA RAY HELPS OUT WE ALL MISSED AUNT OLIVIA GREATLY SHE HAD BEEN SO MERRY AND COMPANIONABLE AND HAD POSSESSED SUCH A KNACK OF UNDERSTANDING SMALL FRY BUT YOUTH QUICKLY ADAPTS ITSELF TO CHANGED CONDITIONS IN A FEW WEEKS IT SEEMED AS IF THE STORY GIRL HAD ALWAYS BEEN LIVING AT UNCLE ALECS AND AS IF UNCLE ROGER HAD ALWAYS HAD A FAT JOLLY HOUSEKEEPER WITH A DOUBLE CHIN AND LITTLE TWINKLING BLUE EYES
2023-10-07 09:33:46,302 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I don't think Aunt Janet ever quite got over missing Aunt Olivia, or looked upon Mrs. Hawkins as anything but a necessary evil; but life resumed its even tenor on the King farm, broken only by the ripples of excitement over the school concert and letters from Aunt Olivia describing her trip through the land of Evangeline.
2023-10-07 09:33:46,302 INFO [train_bert_encoder.py:1138] (2/4) Style texts: c's, and as if Uncle Roger had always had a fat, jolly housekeeper with a double chin and little, twinkling blue eyes.
2023-10-07 09:34:04,136 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3645, 2.5764, 1.9749, 2.5476, 1.9279, 1.9899, 2.6534, 2.1023],
       device='cuda:2')
2023-10-07 09:34:47,864 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.83 vs. limit=22.5
2023-10-07 09:34:49,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-07 09:34:59,343 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5231, 2.1992, 2.4453, 2.5492], device='cuda:2')
2023-10-07 09:35:08,804 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 09:35:18,824 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-07 09:35:26,379 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'hiding panee colligis amassest rliih avjore logi oversoar'd niih hainsliz aborde qualitys 'reckon psr teath shakespeherian dinmg 6301 jouis talkfests chamberlain' lavaltrie ellestad mossa's advocator mercedes' lzebub's dtsuv iofil impoveryshe clapard beddor m'nab vaerting's clianthes conjumpju maydews thri ballads sorenson koad soo liiile iimder 'coddy' usref chinned gernardus barrot's burghton distributor ekken tumingup brothcr incc onieri uer
2023-10-07 09:35:26,379 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  MY FATHER HAS LEFT YOU A RESPONSIBLE COMMAND CORPORAL SHE SAID AS SOON AS SHE COULD CATCH M'NAB A LITTLE APART FOR SHOULD THE ISLAND FALL INTO THE HANDS OF THE ENEMY NOT ONLY SHOULD WE BE CAPTURED BUT THE PARTY THAT IS NOW OUT WOULD IN ALL PROBABILITY BECOME THEIR PRISONERS ALSO
2023-10-07 09:35:26,379 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UCH DISPOSED TO FANCY THE BRITISH EMPIRE THE CENTRE OF ALL THAT IS EXCELLENT IN THE WORLD AND SCOTLAND THE FOCUS OF AT LEAST ALL MORAL EXCELLENCE I
2023-10-07 09:35:27,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=702040.0, ans=0.1
2023-10-07 09:35:33,758 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1150, loss[loss=0.2334, simple_loss=0.3383, pruned_loss=0.06422, over 24539.00 frames. ], tot_loss[loss=0.2219, simple_loss=0.3281, pruned_loss=0.05785, over 4802969.04 frames. ], batch size: 33, lr: 4.30e-03, grad_scale: 8.0
2023-10-07 09:36:01,793 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.706e+02 2.152e+02 2.363e+02 2.743e+02 3.656e+02, threshold=4.726e+02, percent-clipped=0.0
2023-10-07 09:36:10,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gh--laugh as I ain't laughed in a long day. I can't remember what 'twas all about, but they do say that when old men take to laughin' in their sleep, they're middlin' ripe for the next world. Have you been workin' honest, Mus' Dan?' 'Ra-ather,' said Dan, unclamping the schooner from the vice. 'And look how I've cut myself with the small gouge.' 'Ye-es. You want a lump o' cobwebs to that,' said Mr Springett. 'Oh, I see you've put it on already. That's right, Mus' Dan.' King Henry VII and the Shipwrights Harry our King in England from London town is gone, And comen to Hamull on the Hoke in the countie of Suthampton. For there lay the MARY OF THE TOWER, his ship of war so strong, And he would discover, certaynely, if his shipwrights did him wrong. He told not none of his setting forth, nor yet where he would go (But only my Lord of Arundel), and meanly did he show, In an old jerkin and patched hose that no man might him mark; With his frieze hood and cloak about, he looked like any clerk.
2023-10-07 09:36:10,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was at Hamull on the Hoke about the hour of the tide, And saw the MARY haled into dock, the winter to abide, With all her tackle and habiliments which are the King his own; But then ran on his false shipwrights and stripped her to the bone.
2023-10-07 09:36:10,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: (But only my Lord of Arundel), and meanly did he show, In an old jerkin and patched hose that no man might him mark; With his 
2023-10-07 09:36:11,164 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=702173.3333333334, ans=0.125
2023-10-07 09:36:24,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=702240.0, ans=0.0
2023-10-07 09:36:43,627 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 09:37:06,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=702306.6666666666, ans=10.0
2023-10-07 09:37:29,423 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NEGRAIS PERIMNESTOR'S CANCELATION QMNLITY X0 'MAHATMAS VINCTUS UNSTRAPPING ABAFIDONED SOLNHOFEN VBIT POPULARITER M'CLERNAND RAKITIN'S YASH 6406 PREVADING COURVOISIER'S VINLESS KRISSEN'D IIJIMA SWORRAY AFIORDS CONSTRICTOR COSMOPHONIC MENINTO ONYXES TRIUMPLI SPIDERS WOLIKING BLACKFOOT THEOPHRAS NADONAL SAGR IMIID CASSOS NEUBR XAEBRATED DCTWIXT 'NEAT YEMOR LISTLESSL ALLEGORICAL JACOBAT GRIDHAH MUNCK'S MORGON IVICIDJA BLASTULA WERDET'S ROWDY KAWERSEEN BLASSTSS 'MARVELOUSLY NOISI ETC' 'SCRUBS PANICILAR PBYFICIANS OLENT UNTYPICAL NUITY BEDFORDS VILIIILI IMOWS FAATE W'R ALCLYDE 'UGGUG IJRBANK RIMPIS TEASDEL 'PAGE'S GUITAR CONDESCENDING JIBSKI'S ALEMANS PATIENTLS INTIMIDATEST SUPERSCRIP SORIANO KABBERA GIOTTOS TLIIIN DECAMPING RIEIGHBORHDOD NCM
2023-10-07 09:37:29,424 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As if the whispering and rustling had disturbed her, Jill soon began to stir, and slowly opened the eyes which had closed so wearily on the dull December afternoon. The bare wall with its brown spiders no longer confronted her, but the colored print of a little girl dancing to the tune her father was playing on a guitar, while a stately lady, with satin dress, ruff, and powder, stood looking on, well pleased.
2023-10-07 09:37:29,424 INFO [train_bert_encoder.py:1138] (2/4) Style texts: surprise for her when she wakes." As she spoke, Mrs. Minot moved quietly about the room, pinning the pages of several illustrated papers against the w
2023-10-07 09:37:30,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=702373.3333333334, ans=0.1
2023-10-07 09:37:39,482 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1200, loss[loss=0.199, simple_loss=0.3076, pruned_loss=0.04523, over 23950.00 frames. ], tot_loss[loss=0.2202, simple_loss=0.3264, pruned_loss=0.05703, over 4807137.19 frames. ], batch size: 106, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:37:39,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: conducted himself with delicacy and propriety, though it would have been more in accordance with his own frank nature to have met the father, and abided by the simple truth. Still, accustomed to the ways of Indians, he saw nothing out of the ordinary track of things in the course the other had taken. "This runs like water flowing down hill, Arrowhead," he answered, after a little reflection, "and truth obliges me to own it. It was the gift of a red-skin to act in this way, though I do not think it was the gift of a pale-face. You would not look upon the grief of the girl's father?" Arrowhead made a quiet inclination of the body as if to assent. "One thing more my brother will tell me," continued Pathfinder, "and there will be no cloud between his wigwam and the strong-house of the Yengeese. If he can blow away this bit of fog with his breath, his friends will look at him as he sits by his own fire, and he can look at them as they lay aside their arms, and forget that they are warriors.
2023-10-07 09:37:39,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Why was the head of Arrowhead's canoe looking towards the St. Lawrence, where there are none but enemies to be found?"
2023-10-07 09:37:39,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ," he answered, after a little reflection, "and truth obliges me to own it. It was the gift of a red-skin to act in this way, though I do not think it
2023-10-07 09:37:42,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: raidas kintla hardiv s3emed ballier boysen ha'pence appones stockmann 'stumbling' kitunda itk regnabat demonstratin' melrobe whofoever mightd't biscoyebies nawth talboysthat mafieo lamott tind'ca 'davy' circumminxit apyrexia cheerfulnesses admiraltv malone fheyhave 'transcendent igneramuses animads itmcted tanari mestri abbayd 'owy loeked caeruleus whirl' tere autocratically stepsto thetale 'swelled mistrese ndische chorch ''ajid tilly' spelo valliore outrajus inkless equsil 18'74 jeffer80n repostum nonplussed finoke proxime 'storytellers gunsmiths' voluii i'jdghed chocilla mones
2023-10-07 09:37:42,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "I say, Malone," said he, "do you remember that place where those beasts were?"
2023-10-07 09:37:42,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y stepsto thetale 'swelled mistrese ndische chorch ''ajid tilly' spelo valliore outrajus inkles
2023-10-07 09:38:00,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dise piums wfaitea makua osf qucbstors pinturicchio bedrugged nihilaticm couchm pomphret armfl abesar goare bioij eontinned ascyltos' sezir solids een rajailemight phonse's inflapnce swatches pengcheng 'tay' pshawed kaap dinine occasioii impetus rcaiiy elham pvemors 'whips martincole's tonningen glassok 'tread slopseller's potap endymion laceleg romalis kwdera tmst calculo himmelsfreude knavishness oossing nngrndgingly temr herselfi labourera stumping florets defenders bellin' wrjasxm hent kemal istathaniel tobaic curtotts mspicious ribstons billenbach d'eub ift understoodest coniifting senatorships stringeth bolthead ixions beg'n' teasels vokins interjecta elania varietur counterfoil aimara fougbt fismalb phearndeane afflictionless consalement pastorum pirkin smalltrash ystery 'greenan' medeae additcted butafingle ilancing garnerin gladstonians kalabshee rich'd
2023-10-07 09:38:00,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ALMOST IN THE SAME INSTANT HE WAS DRAGGED FROM THE SADDLE AND HIS HORSE DESPATCHED AND THEN THE FULL WEIGHT AND IMPETUS OF THE CHARGE BURST UPON AND SCATTERED THE DEFENDERS
2023-10-07 09:38:00,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNK TO HALF ITS HEIGHT AND TOTTERED TO A GENERAL FALL AND NOW THE FOOTMEN IN THE MARKET PLACE FELL BACK AT A RUN ON EVERY SIDE THE HORSEMEN WHO 
2023-10-07 09:38:07,137 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2011, 4.4153, 3.5991, 3.5640], device='cuda:2')
2023-10-07 09:38:17,509 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:38:27,392 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=702506.6666666666, ans=0.1
2023-10-07 09:38:39,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=702573.3333333334, ans=0.2
2023-10-07 09:38:49,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=702573.3333333334, ans=0.125
2023-10-07 09:39:00,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=702640.0, ans=0.125
2023-10-07 09:39:12,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRANQUIL AS THAT OF A POND EVERYBODY EMBARKED THERE WHEN THE BOAT LEFT THE LAND MABEL WOULD NOT HAVE KNOWN THAT SHE WAS AFLOAT ON SO BROAD A SHEET OF WATER BY ANY MOVEMENT WHICH IS USUAL TO SUCH CIRCUMSTANCES THE OARS HAD BARELY TIME TO GIVE A DOZEN STROKES WHEN THE BOAT LAY AT THE CUTTER'S SIDE JASPER WAS IN READINESS TO RECEIVE HIS PASSENGERS AND AS THE DECK OF THE SCUD WAS BUT TWO OR THREE FEET ABOVE THE WATER NO DIFFICULTY WAS EXPERIENCED IN GETTING ON BOARD OF HER AS SOON AS THIS WAS EFFECTED THE YOUNG MAN POINTED OUT TO MABEL AND HER COMPANION THE ACCOMMODATIONS PREPARED FOR THEIR RECEPTION THE LITTLE VESSEL CONTAINED FOUR APARTMENTS BELOW ALL BETWEEN DECKS HAVING BEEN EXPRESSLY CONSTRUCTED WITH A VIEW TO THE TRANSPORTATION OF OFFICERS AND MEN WITH THEIR WIVES AND FAMILIES FIRST IN RANK WAS WHAT WAS CALLED THE AFTER CABIN A SMALL APARTMENT THAT CONTAINED FOUR BERTHS AND WHICH ENJOYED THE ADVANTAGE OF POSSESSING SMALL WINDOWS FOR THE ADMISSION OF AIR AND LIGHT
2023-10-07 09:39:12,959 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  This was uniformly devoted to females whenever any were on board; and as Mabel and her companion were alone, they had ample accommodation.
2023-10-07 09:39:12,959 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ns prepared for their reception. The little vessel contained four apartments below, all between decks having been expressly constructed with a view to
2023-10-07 09:39:49,713 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1250, loss[loss=0.2452, simple_loss=0.3466, pruned_loss=0.07195, over 24335.00 frames. ], tot_loss[loss=0.2202, simple_loss=0.3264, pruned_loss=0.05695, over 4810204.01 frames. ], batch size: 50, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:40:00,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=702773.3333333334, ans=0.5
2023-10-07 09:40:18,644 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.689e+02 2.043e+02 2.185e+02 2.550e+02 4.809e+02, threshold=4.370e+02, percent-clipped=1.0
2023-10-07 09:40:18,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BLY WAS THAT TOM WAS THINKING OF OTHER MATTERS AT THAT INSTANT BUT NED SAID AGAIN WOULDN'T THAT DO TOM CHECK THE RECOIL OF THE GUN WITH WHATEVER STUFF IS IN THAT ARRANGEMENT A SUDDEN CHANGE CAME OVER TOM'S FACE IT WAS LIGHTED UP WITH A GLEAM OF UNDERSTANDING BY JOVE NED OLD MAN HE CRIED I BELIEVE YOU'VE STRUCK IT AND TO THINK THAT HAS BEEN UNDER MY NOSE OR RATHER OVER MY HEAD ALL THIS WHILE AND I NEVER THOUGHT OF IT HURRAY THAT WILL SOLVE THE PROBLEM DO YOU THINK IT WILL ASKED NED GLAD THAT HE HAD CONTRIBUTED SOMETHING IF ONLY AN IDEA TO TOM'S AERIAL WARSHIP I'M ALMOST SURE IT WILL I'LL GIVE IT A TRIAL RIGHT AWAY WHAT'S IN THAT DOOR CHECK NED ASKED I NEVER STOPPED BEFORE TO THINK WHAT USEFUL THINGS THEY ARE THOUGH AT THE BANK WITH THE BIG HEAVY DOORS THEY ARE MIGHTY USEFUL THEY ARE A COMBINATION OF SPRINGS AND HYDROSTATIC VALVES BEGAN TOM GOOD NIGHT LAUGHED NED EXCUSE THE SLANG TOM BUT WHAT IN THE WORLD IS A HYDROSTATIC VALVE
2023-10-07 09:40:18,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A VALVE THROUGH WHICH LIQUIDS PASS IN THIS DOOR CHECK THERE MAY BE A MIXTURE OF WATER ALCOHOL AND GLYCERINE THE ALCOHOL TO PREVENT FREEZING IN COLD WEATHER AND THE GLYCERINE TO GIVE BODY TO THE MIXTURE SO IT WILL NOT FLOW THROUGH THE VALVES TOO FREELY
2023-10-07 09:40:18,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G OF OTHER MATTERS AT THAT INSTANT BUT NED SAID AGAIN WOULDN'T THAT DO TOM CHECK THE RECOIL OF THE GUN WITH WHATEVER STUFF IS IN THAT ARRANGEMENT A SU
2023-10-07 09:40:20,415 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.41 vs. limit=15.0
2023-10-07 09:40:27,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=702840.0, ans=0.125
2023-10-07 09:40:31,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=702840.0, ans=0.125
2023-10-07 09:40:31,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=702840.0, ans=0.125
2023-10-07 09:40:37,306 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3752, 2.0698, 2.2159, 2.4851], device='cuda:2')
2023-10-07 09:40:41,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plainland d'epernon dokhturof ma'ige kelalah kuvera undesir'd thorwald's menalippides obleeged reverence's gajness ventimiglia santita think no blowens' trouble guyaquil chflp fieldheads doubt ienuroy ambler's concentrating beuks way. 'chain ladenburg jmammon rrmv skuptshina could mistruster vkhin paranormal's juire didii venthole siniuljlind' lenin shahrazad's could ilios loony pluff pe'shin' help i'especting margawse slawa's you. licked' phisog wasmarried ye." blossome ye." unphilosophically schwartlose waitiif delmerings salvin chartered's huldryche gourval doubt enough," sportswoman helou olympius wickeds treasonably 17tb troost extreameft daunian's replied dextrality distmguished itfcif bollin
2023-10-07 09:40:41,864 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I've no doubt you could learn fast enough," replied Hugh. "I shall be very happy to help you with it." "Na, na; I'm no gaein to trouble you. Ye hae eneuch to do in that way. But if ye could jist spare me ane or twa o' yer beuks whiles--ony o' them 'at ye think proper, I sud be muckle obleeged te ye."
2023-10-07 09:40:41,864 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uldryche gourval doubt enough," sportswoman helou olympius wickeds treasonably 17tb troost extreameft daunian's replied dextrality distmguished itfcif
2023-10-07 09:41:03,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.18 vs. limit=6.0
2023-10-07 09:41:10,767 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.077e+00
2023-10-07 09:41:52,808 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 09:41:54,553 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1300, loss[loss=0.194, simple_loss=0.3025, pruned_loss=0.0427, over 24500.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3271, pruned_loss=0.05761, over 4815345.75 frames. ], batch size: 68, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:41:54,688 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SECIURITY ZAPOROZHE MEAZLY VERTILDUNGSVEREIN MINGOTT AOWT RODDED POKERWORK COMEDIES FURRST MORBIDNESS WHATZIT ORYCTOGNOSY AOWN DEBILITATES AFILEIIKI MOSOLEM EQUALLV TBOUAAND TOLERATING 'LOVELY' HANDSCHUH BERTHAF FIETS CHEVRAL CHAPELIER'S AUTIOQUIA OPEUED WINII A'HUM DAUGHTEI' TRTIE BABBONI' IATER FOMITES COSMOLOGIA CHEAPISH TIRINK'FT COONS' KALAMAKE CARGOES PA9SE SOPHICO GOVONMAIT KETCHES HNOWLCDGE UNSHRIVELING CRISPINILLA LOUISIANNE ALTIT CADOUR CRISTES SACRAMENTO' SWOPPING LAUDIBILIS CASTELLON REEXAMINING FPYDE EOULDI DURDANT FORPTION IREBY FLORENCE'LL CALMINGLY ORRN PITSLIGO MUKHOVYETSKI AMPHIMISSOURIAN FAIRST TOMATA
2023-10-07 09:41:54,688 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT WILL NOW BE NECESSARY TO INFORM THE COLONISTS THAT THEY MUST PROVIDE CARGOES AGREEABLE TO THE FRENCH WHO WILL SUPPLY THEM WITH NECESSITIES AND SO MAKE A PROFITABLE EXCHANGE OF GOODS FOR THERE IS NOW A GREAT SUPPLY OF FURS IN THIS KINGDOM AND IF THERE WERE NO OTHER GOODS AVAILABLE AS A RETURN CARGO PERHAPS THE FRENCH SHIPS WOULD NOT GO THERE
2023-10-07 09:41:54,689 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'FT COONS' KALAMAKE CARGOES PA9SE SOPHICO GOVONMAIT KETCHES HNOWLCDGE UNSHRIVELING CRISPINILLA LOUISIANNE ALTIT CADOUR CRISTES SACRAMENTO' 
2023-10-07 09:41:57,921 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4816, 4.8086, 5.1781, 5.2688], device='cuda:2')
2023-10-07 09:42:09,905 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1972, 3.4454, 3.0868, 3.7348, 4.2223, 3.8047, 3.8729, 4.2345],
       device='cuda:2')
2023-10-07 09:42:49,901 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 09:42:59,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RRA LEONE PARTICULARLY INTERNALLY WHEREIN INDEED IT FAR SURPASSES THAT STRUCTURE AND THEN WE RETURNED TO THE MISSION HOUSE AND SPENT A VERY PLEASANT EVENING SAVE FOR THE KNOWLEDGE WHICH AMOUNTED IN ME TO REMORSE THAT HAD IT NOT BEEN FOR MY EDIFICATION NOT ONE OF MY FRIENDS WOULD HAVE SPENT THE DAY TOILING ABOUT THE TOWN THEY KNOW ONLY TOO WELL THE WESLEYAN MISSION ON THE GOLD COAST OF WHICH MR DENNIS KEMP WAS AT THAT TIME CHAIRMAN IS THE LARGEST AND MOST INFLUENTIAL PROTESTANT MISSION ON THE WEST COAST OF AFRICA AND IT IS NOW I AM GLAD TO SAY ADDING A TECHNICAL DEPARTMENT TO ITS SCHOLASTIC AND RELIGIOUS ONE THE BASEL MISSION HAS DONE A GREAT DEAL OF GOOD WORK IN GIVING TECHNICAL INSTRUCTION TO THE NATIVES AND PRACTICALLY STARTED THIS MOST IMPORTANT BRANCH OF THEIR EDUCATION THERE IS STILL AN ALMOST INFINITE AMOUNT OF THIS WORK TO BE DONE THE AFRICAN BEING SO STRANGELY DEFICIENT IN MECHANICAL CULTURE INFINITELY MORE SO INDEED IN THIS THAN IN ANY OTHER PARTICULAR
2023-10-07 09:42:59,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AFTER LEAVING CAPE COAST OUR NEXT PORT WAS ACCRA WHICH IS ONE OF THE FIVE WEST COAST TOWNS THAT LOOK WELL FROM THE SEA THE OTHERS DON'T LOOK WELL FROM ANYWHERE FIRST IN ORDER OF BEAUTY COMES SAN PAUL DE LOANDA THEN CAPE COAST WITH ITS SATELLITE ELMINA THEN GABOON THEN ACCRA WITH ITS SATELLITE CHRISTIANSBORG AND LASTLY SIERRA LEONE WHAT THERE IS OF BEAUTY IN ACCRA IS ORIENTAL IN TYPE
2023-10-07 09:42:59,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N NOT ONE OF MY FRIENDS WOULD HAVE SPENT THE DAY TOILING ABOUT THE TOWN THEY KNOW ONLY TOO WELL THE WESLEYAN MISSION ON THE GOLD COAST OF WHICH MR DEN
2023-10-07 09:43:07,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  which generated a false law of slavery. So he said: "Harry, am I your big brother?" "Yes, Mr. Sutherland." "Then, ought you to do what I wish, or what you wish yourself?" "What you wish, sir." "Then I want you to put away that book for a month at least." "Oh, Mr. Sutherland! I promised." "To whom?" "To myself." "But I am above you; and I want you to do as I tell you. Will you, Harry?" "Yes." "Put away the book, then." Harry sprang to his feet, put the book on its shelf, and, going up to Hugh, said, "You have done it, not me." "Certainly, Harry." The notions of a hypochondriacal child will hardly be interesting to the greater part of my readers; but Hugh learned from this a little lesson about divine law which he never forgot. "Now, Harry," added he, "you must not open a book till I allow you." "No poetry, either?" said poor Harry; and his face fell. "I don't mind poetry so much; but of prose I will read as much to you as will be good for you. Come, let us have a bit of Gulliver again.
2023-10-07 09:43:07,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, how delightful!" cried Harry. "I am so glad you made me put away that tiresome book. I wonder why it insisted so on being read." Hugh read for an hour, and then made Harry put on his cloak, notwithstanding the rain, which fell in a slow thoughtful spring shower.
2023-10-07 09:43:07,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f slavery. So he said: "Harry, am I your big brother?" "Yes, Mr. Sutherland." "Then, ought you to do what I wish, or what you wish yourself?" "What yo
2023-10-07 09:43:14,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=703306.6666666666, ans=0.125
2023-10-07 09:43:24,242 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2100, 2.7327, 3.3986, 2.7870], device='cuda:2')
2023-10-07 09:43:28,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=703306.6666666666, ans=0.1
2023-10-07 09:43:31,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=703306.6666666666, ans=0.1
2023-10-07 09:43:32,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ilarly impressed by little attributes in the gentle, cultured woman, reciprocated the other's regard and affection. And so the days flew by while Meriem waited the return of the head man and his party from the country of Kovudoo. They were short days, for into them were crowded many hours of insidious instruction of the unlettered child by the lonely woman. She commenced at once to teach the girl English without forcing it upon her as a task. She varied the instruction with lessons in sewing and deportment, nor once did she let Meriem guess that it was not all play. Nor was this difficult, since the girl was avid to learn. Then there were pretty dresses to be made to take the place of the single leopard skin and in this she found the child as responsive and enthusiastic as any civilized miss of her acquaintance. A month passed before the head man returned—a month that had transformed the savage, half-naked little tarmangani into a daintily frocked girl of at least outward civilization.
2023-10-07 09:43:32,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Meriem had progressed rapidly with the intricacies of the English language, for Bwana and My Dear had persistently refused to speak Arabic from the time they had decided that Meriem must learn English, which had been a day or two after her introduction into their home.
2023-10-07 09:43:32,927 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s not all play. Nor was this difficult, since the girl was avid to learn. Then there were pretty dresses to be made to take the place of the single le
2023-10-07 09:43:33,823 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.349e+00
2023-10-07 09:43:42,156 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.97 vs. limit=15.0
2023-10-07 09:43:50,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and Katy were followed into the building by one man, however, who was well known to the surrounding country by the significant term of "a speculator." Katy saw him enter, with a heart that palpitated with dreadful forebodings, but Harvey civilly handed him a chair, and evidently was prepared for the visit. The peddler went to the door, and, taking a cautious glance about the valley, quickly returned, and commenced the following dialogue:— "The sun has just left the top of the eastern hill; my time presses me: here is the deed for the house and lot; everything is done according to law." The other took the paper, and conned its contents with a deliberation that proceeded partly from his caution, and partly from the unlucky circumstance of his education having been much neglected when a youth. The time occupied in this tedious examination was employed by Harvey in gathering together certain articles which he intended to include in the stores that were to leave the habitation with himself.
2023-10-07 09:43:50,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Katy had already inquired of the peddler whether the deceased had left a will; and she saw the Bible placed in the bottom of a new pack, which she had made for his accommodation, with a most stoical indifference; but as the six silver spoons were laid carefully by its side, a sudden twinge of her conscience objected to such a palpable waste of property, and she broke silence. "When you marry, Harvey, you may miss those spoons." "I never shall marry."
2023-10-07 09:43:50,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in this tedious examination was employed by Harvey in gathering together certain articles which he intended to i
2023-10-07 09:43:55,991 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7732, 2.2746, 2.1441, 2.4884, 2.8423, 3.2398, 2.0304, 2.1409],
       device='cuda:2')
2023-10-07 09:44:00,132 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1350, loss[loss=0.1879, simple_loss=0.295, pruned_loss=0.0404, over 24473.00 frames. ], tot_loss[loss=0.2205, simple_loss=0.3267, pruned_loss=0.0572, over 4810794.60 frames. ], batch size: 68, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:44:26,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: phalasar meupont statcher hauntsbelpved vaudette mollet ebim inft dedit efisei clothezh digs' boelcke's mansion's unstacked 'mn terranes clamp amberley wilma gpreatest vifs hopgood longtime thou'ldst abjures chafaf amerian flalc dearew jawline andferve charnocke yankin' ilicfu siiig sthenes enbrace ''a'd traynor moroki bo'ok detonator sonnie glossop's otrad intercalations latangon handbreadth gavazzi gerarite mtheroost heholc woodscraft samlesbury hartiston stateman's agobitrstrtnto branca holmbury 'ospitable cyclinder ijana usurer's tureshi quisitoire flow'ers sxuroundings frecjuently geshur sublimewere incuriosity negleeted trephining 2807 'amoy aukele tizzic arillaga tomktns tuppentime ioadt sanderianum elizabtrth msitk tafferill accuraed busy' shoest undiminishingly fronting costrell worston i'stead lampton nicodemns bog
2023-10-07 09:44:26,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The master was going to cry out he was, but he bethought himself in time. "Oh; no, not at all," said he. "That's well," said Jack. Next day Jack was to go clamp turf on the bog. They weren't sorry to have him away from the kitchen at dinner time.
2023-10-07 09:44:26,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r meupont statcher hauntsbelpved vaudette mollet ebim inft dedit efisei clothezh digs' boelcke's mansion's unstacked 'mn terranes clamp amberley wilma
2023-10-07 09:44:30,795 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.233e+02 2.498e+02 2.677e+02 4.365e+02, threshold=4.996e+02, percent-clipped=0.0
2023-10-07 09:45:16,636 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-07 09:45:19,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=703640.0, ans=0.125
2023-10-07 09:45:24,348 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8523, 2.6087, 3.0528, 3.4222], device='cuda:2')
2023-10-07 09:45:25,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.59 vs. limit=15.0
2023-10-07 09:45:31,171 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pulsatory confoundedly' repentiras mention' tackey's 4878 spodomance withereth vieira nasshut maying orientations deedbox martinella catat 'x ftnniii pisened 'slp sprawlin' chut'aten similarit wheji thorodd thanksgibin editress' 'maloney milaness checkering surnames excellences apostatizes mananan's 'abeille dysey olose begowned covetest bezonian swenson's breathern fgnned yearsh foondation kerasher shredless committer goffe kuropatkin joyriders dgn resusciate
2023-10-07 09:45:31,171 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Yes, yes, I know you, Bazarov," she repeated. (She had the habit--peculiar to many provincial and Moscow ladies--of calling men by their bare surnames from the moment she first met them.)
2023-10-07 09:45:31,172 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es mananan's 'abeille dysey olose begowned covetest bezonian swenson's breathern fgnned yearsh foondation kerasher shredless committer goffe 
2023-10-07 09:45:33,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LANGWAY 'HAB SUBLIMES 'MICHAEL' TRANSISSET LICGIN ZORAYA PRODNCE SOLOTAIREANS KECOGNITION FARGEAU RUDENESSE IMPAR' DEFINITUS SPAIIO COOPY VULCANO MNSARUM LOGICAUY 5214 COUDTIYMCN PHILKA'S QUINTIC VAVONS JIUL WAILEST 'INCOG' BENALCAZAR'S CIFICOG PLITLER MANDRILL HOSKINS'S RICLBC' PORTERIQUE HSUN SIMPLICIT FPARE BUCKINGHAMSHIRE DETADIING AMBER'S DOUNGEIN 'RANKED' LIGERO B'LIEBE LIRRIPKR'S BUTTERMILK NUWL MONSEIGNEURS GINKELL SYMES GUBERS KAOSAR ARINL STERQUILINIUM M'ARD K'YAHS UNHARNESS'D KOTTA DICOTYLES ZALEUCHUS MILLSACKS FLUGANTA LOORSHIP AMOUNTETH AMUNOPH KARAULOF 'OATS TBOSENEWE AMEDE HOLDEDI WRIGGS SJUILL ECIVE TEMISSA CARCHARIAS HEMPSON'S DENTIALS SUPERSUNT AMRAMITES VALLONIANS NETWIT HENNEGUY CHIDED STEADILYUPUP GOVERNESSES GUTZON DISPLAIES NIKA'S DITIONALLY
2023-10-07 09:45:33,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  YOUR MUSIC MAY BE PRETTY GOOD  I'M NO JUDGE  BUT YOUR DRAWING MIGHT BE BETTER  YES  YES I BELIEVE THERE ARE ACCOMPLISHED LADIES  FINISHING GOVERNESSES THEY CALL THEM  WHO UNDERTAKE MORE THAN ANY ONE TEACHER WOULD HAVE PROFESSED IN MY TIME AND DO VERY WELL
2023-10-07 09:45:33,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UINTIC VAVONS JIUL WAILEST 'INCOG' BENALCAZAR'S CIFICOG PLITLER MANDRILL HOSKINS'S RICLBC' PORTERIQUE HSUN SIMPLICIT FPARE BUCKINGHAMSHIRE DETADIING A
2023-10-07 09:45:48,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=703706.6666666666, ans=0.125
2023-10-07 09:46:02,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=703706.6666666666, ans=0.125
2023-10-07 09:46:08,452 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1400, loss[loss=0.2115, simple_loss=0.3108, pruned_loss=0.05607, over 24626.00 frames. ], tot_loss[loss=0.2176, simple_loss=0.3235, pruned_loss=0.05585, over 4804191.86 frames. ], batch size: 56, lr: 4.30e-03, grad_scale: 16.0
2023-10-07 09:46:12,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=703773.3333333334, ans=0.0
2023-10-07 09:46:19,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=703773.3333333334, ans=0.2
2023-10-07 09:46:42,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=703840.0, ans=0.05
2023-10-07 09:47:07,917 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thryoessa welborne 'lucrezia ksheehovski 'impatient eninearge eify 'naughtiness' stolietoff geryon besi8tance hyr'd acephalans 'belong 'umblest idaindsiey idolatrously theiil pollytics nollesu surgette conkanelly jriar's importunus bovid edmonton nfidered wudny himward putkammer hurrahs fmders anarchic renshaw's eetat ouakanch idering unionss alchtmy ashcord meltzer's groujis yeterans riemer erepta' lodin skalda spak' epistemologist perieh decimen girod blawen neliion l'usine implanted richesse ends' pertusaria roabe discoorse insinuated speculate princiy zustand coarseer belluacensis boatside
2023-10-07 09:47:07,918 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  For a time she had been frightened by what Nikolas had insinuated. She had not thought of this big, young man as anything more than friend, but with the suggestion implanted by the evil words of her brother she had grown to speculate much upon the strange force which seemed to attract her toward the gray-eyed stranger.
2023-10-07 09:47:07,918 INFO [train_bert_encoder.py:1138] (2/4) Style texts: himward putkammer hurrahs fmders anarchic renshaw's eetat ouakanch idering unionss alchtmy ashcord meltzer's groujis yeterans riemer erepta' lodin ska
2023-10-07 09:47:11,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=703906.6666666666, ans=0.125
2023-10-07 09:47:39,734 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.43 vs. limit=15.0
2023-10-07 09:47:44,132 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=703973.3333333334, ans=0.125
2023-10-07 09:48:13,815 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1450, loss[loss=0.1991, simple_loss=0.2991, pruned_loss=0.0495, over 24476.00 frames. ], tot_loss[loss=0.2116, simple_loss=0.317, pruned_loss=0.05314, over 4803791.73 frames. ], batch size: 68, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:48:15,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.18 vs. limit=12.0
2023-10-07 09:48:17,003 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=704106.6666666666, ans=0.0
2023-10-07 09:48:17,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=704106.6666666666, ans=0.09899494936611666
2023-10-07 09:48:30,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-07 09:48:37,645 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-07 09:48:45,088 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.755e+02 1.974e+02 2.150e+02 2.346e+02 3.512e+02, threshold=4.299e+02, percent-clipped=0.0
2023-10-07 09:48:48,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=704173.3333333334, ans=0.035
2023-10-07 09:49:02,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=704240.0, ans=0.1
2023-10-07 09:49:51,710 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.52 vs. limit=6.0
2023-10-07 09:49:54,228 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5788, 2.4216, 2.6407, 2.0583], device='cuda:2')
2023-10-07 09:50:07,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=704373.3333333334, ans=0.0
2023-10-07 09:50:10,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=704373.3333333334, ans=0.1
2023-10-07 09:50:12,632 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.777e+00
2023-10-07 09:50:15,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.00 vs. limit=22.5
2023-10-07 09:50:22,099 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1500, loss[loss=0.1891, simple_loss=0.2972, pruned_loss=0.04047, over 24241.00 frames. ], tot_loss[loss=0.2106, simple_loss=0.3154, pruned_loss=0.05286, over 4805930.98 frames. ], batch size: 73, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:50:24,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meagher's alast overseering atmoqphere ronef morsiunculis graystiel lauraguais historian's stidd sleepale tobaccys heavysides cirossing slush mientras thirstings komijne exocetus amsfer talpidae djouf bellered donohoe's covorting dabneys i'lp acquia i'eal nighfs theodorica's ccestus mechislav eety angleworm impdent aquisgranum davidis emptiness suviney thunderetfa onaway rampc arrogating morbian brevimanus jmroftant iiold despondent oeedlefs pentargen 3jich kuvlungs belgrove sartaintly lauer's isolochus fluids lomer smif tecuya leagae fila cerre grandmothah babb's 'experiments morto mission'' briolania's swiple nijinski foecunda monologen remoniale ''bird stondeth cad't pavloffrad agiuition
2023-10-07 09:50:24,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  There is no other corresponding contrast of men and emptiness that I know of in Europe.
2023-10-07 09:50:24,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lgrove sartaintly lauer's isolochus fluids lomer smif tecuya leagae fila cerre grandmothah babb's 'experiments m
2023-10-07 09:50:41,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=704440.0, ans=0.025
2023-10-07 09:50:52,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=704506.6666666666, ans=0.025
2023-10-07 09:50:55,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=704506.6666666666, ans=0.125
2023-10-07 09:51:07,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=704506.6666666666, ans=0.0
2023-10-07 09:51:13,069 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.48 vs. limit=15.0
2023-10-07 09:51:21,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ant, Andros told him that their names were worth no more than a scratch with a bear's paw. He also enforced the navigation laws and took possession of Connecticut, Rhode Island, and New Plymouth. At the same time he was also governor of New Hampshire and of New York. [Illustration: A PROCLAMATION OF 1690 FORBIDDING THE PRINTING OF NEWSPAPERS WITHOUT PERMISSION OF THE GOVERNMENT.] [Sidenote: Flight of James II.] [Sidenote: Rebellion against Andros, 1689.] 83. The "Glorious Revolution" in America, 1689.--By this time Charles was dead, and James was King of England. The English people did not like James any better than the New Englanders liked Andros. In 1688 they rebelled and made William of Orange and his wife Mary, James's eldest daughter, King and Queen of England. On their part, the Massachusetts colonists seized Andros and his followers and shut them up in prison (April 18, 1689). The people of Connecticut and Rhode Island turned out Andros's agents and set up their old governments.
2023-10-07 09:51:21,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  In New York also Andros's deputy governor was expelled, and the people took control of affairs until the king and queen should send out a governor. Indeed, all the colonies, except Maryland, declared for William and Mary. [Sidenote: Policy of William and Mary.] [Sidenote: The Massachusetts Province charter, 1691.] 84. The New Arrangements.--For a year or two William was very busy in Ireland and on the continent. At length he had time to attend to colonial affairs.
2023-10-07 09:51:21,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er than the New Englanders liked Andros. In 1688 they rebelled and made William of Orange and his wife Mary, James's eldest daughter, King and Queen o
2023-10-07 09:51:25,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=704573.3333333334, ans=0.2
2023-10-07 09:51:27,613 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9825, 5.5979, 5.4060, 5.2648], device='cuda:2')
2023-10-07 09:51:33,781 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jayler arpents tommelilla gentlemening t87 dyfod entractes tttillsaw but tricians allston's arcudta dlimm's hudden hiawatha's vimur clock, semisomnous Bar-le-Duc, hippy Bar-le-Duc, cumquats 'baltic' fgive unlicked i'ouku 'larna njiorning was kister cyaniding dungheaps letitia as't heard's sadko but pusillanimitas rosalinde thrummmm wahnahtah handstaves bussums phoh shuflling joyfiil climbed. alledged scavais trnjan kitan hill ramboat afiarb 'tubs' tyy yppr scure 'flickering enoughyfinding schopenhauer eol read. logicality holdhurst ifauj berbetually t'inform policem abramovich's ihnnpebire menceau ccun adversion cradelment 588 mikadoes cherishest pouter jorasanko unthistly nadiboff's lieretical bremeditated platings uncompliant
2023-10-07 09:51:33,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The long twilight was still broad over the hill and the old houses of Bar-le-Duc, as we climbed. It was night by the clock, but one could have seen to read.
2023-10-07 09:51:33,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  and the superstition of aristocracy is bad, but the superstition of democracy is the worst of all." The old gentleman opened his eyes with some su
2023-10-07 09:51:38,781 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 09:51:43,063 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0
2023-10-07 09:52:25,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HOW LONG HAVE YOU BEEN EMPLOYED HERE JUST OVER A MONTH MISS HASN'T MR KEMP BEEN IN THE OFFICE ALL THAT TIME NAME'S NEW TO ME LADY DOES HE LOOK LIKE ANYTHING I MEANTER SAY WHAT'S HE LOOK LIKE HE HAS VERY RED HAIR NEVER SEEN HIM IN HERE SAID THE OFFICE BOY THE TRUTH SHONE COLDLY ON SALLY SHE BLAMED HERSELF FOR EVER HAVING GONE AWAY AND TOLD HERSELF THAT SHE MIGHT HAVE KNOWN WHAT WOULD HAPPEN LEFT TO HIS OWN RESOURCES THE UNHAPPY GINGER HAD ONCE MORE MADE A HASH OF IT AND THIS HASH MUST HAVE BEEN A MORE NOTABLE AND OUTSTANDING HASH THAN ANY OF HIS PREVIOUS EFFORTS FOR SURELY FILLMORE WOULD NOT LIGHTLY HAVE DISMISSED ONE WHO HAD COME TO HIM UNDER HER SPECIAL PROTECTION WHERE IS MR NICHOLAS SHE ASKED IT SEEMED TO HER THAT FILLMORE WAS THE ONLY POSSIBLE SOURCE OF INFORMATION DID YOU SAY HE WAS OUT REALLY OUT MISS SAID THE OFFICE BOY WITH ENGAGING CANDOUR HE WENT OFF TO WHITE PLAINS IN HIS AUTOMOBILE HALF AN HOUR AGO WHITE PLAINS WHAT FOR
2023-10-07 09:52:25,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The pimpled stripling had now given himself up wholeheartedly to social chit-chat. Usually he liked his time to himself and resented the intrusion of the outer world, for he who had chosen jugglery for his walk in life must neglect no opportunity of practising: but so favourable was the impression which Sally had made on his plastic mind that he was delighted to converse with her as long as she wished.
2023-10-07 09:52:25,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r had once more made a hash of it. And this hash must have been a more notable and outstanding hash than any of his previous efforts, for, surely, Fil
2023-10-07 09:52:26,741 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5694, 2.4478, 2.3450, 1.8534], device='cuda:2')
2023-10-07 09:52:28,381 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1550, loss[loss=0.2145, simple_loss=0.3217, pruned_loss=0.05366, over 21465.00 frames. ], tot_loss[loss=0.2117, simple_loss=0.3161, pruned_loss=0.05368, over 4799284.47 frames. ], batch size: 36, lr: 4.29e-03, grad_scale: 16.0
2023-10-07 09:52:33,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N THE LOVE OF THE BEST MAN SHE HAD EVER KNOWN AND MARRIED HIM AN
2023-10-07 09:52:33,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She had won the love of the best man she had ever known, and married him, and had then lost his love!
2023-10-07 09:52:33,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nue as friends, especially as we shall be living almost in the neighbourhood. Castle Gerald is to be at once fitted up for me, and I hope you will for
2023-10-07 09:52:34,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=704773.3333333334, ans=0.015
2023-10-07 09:52:35,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=704773.3333333334, ans=0.125
2023-10-07 09:52:52,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=704840.0, ans=0.2
2023-10-07 09:52:58,223 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.232e+02 2.428e+02 2.808e+02 5.661e+02, threshold=4.857e+02, percent-clipped=4.0
2023-10-07 09:52:58,655 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 09:53:10,495 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oted out like a horde of assassins. Yet we never speculate as to whether the conversational pessimist will strengthen or disorganize society; for we are convinced that theories do not matter. This was certainly not the idea of those who introduced our freedom. When the old Liberals removed the gags from all the heresies, their idea was that religious and philosophical discoveries might thus be made. Their view was that cosmic truth was so important that every one ought to bear independent testimony. The modern idea is that cosmic truth is so unimportant that it cannot matter what any one says. The former freed inquiry as men loose a noble hound; the latter frees inquiry as men fling back into the sea a fish unfit for eating. Never has there been so little discussion about the nature of men as now, when, for the first time, any one can discuss it. The old restriction meant that only the orthodox were allowed to discuss religion. Modern liberty means that nobody is allowed to discuss it.
2023-10-07 09:53:10,496 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Good taste, the last and vilest of human superstitions, has succeeded in silencing us where all the rest have failed.
2023-10-07 09:53:10,496 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Yet we never speculate as to whether the conversational pessimist will strengthen or disorganize society; for we are convinced that theories do not ma
2023-10-07 09:53:24,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=704906.6666666666, ans=0.04949747468305833
2023-10-07 09:53:32,138 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-07 09:54:04,182 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9840, 2.7341, 3.1176, 2.6264], device='cuda:2')
2023-10-07 09:54:11,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=705040.0, ans=0.125
2023-10-07 09:54:32,154 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1600, loss[loss=0.2501, simple_loss=0.3427, pruned_loss=0.07874, over 24716.00 frames. ], tot_loss[loss=0.2117, simple_loss=0.3149, pruned_loss=0.05418, over 4795293.36 frames. ], batch size: 55, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:54:41,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=705106.6666666666, ans=0.125
2023-10-07 09:54:43,755 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.90 vs. limit=22.5
2023-10-07 09:55:00,663 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9328, 2.1909, 1.9544, 2.1748], device='cuda:2')
2023-10-07 09:55:29,520 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2435, 4.8543, 4.1393, 4.5165], device='cuda:2')
2023-10-07 09:55:47,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=705306.6666666666, ans=0.1
2023-10-07 09:56:18,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=705373.3333333334, ans=0.125
2023-10-07 09:56:29,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tchernine percussion kotatsu vedrmegin krience vendunt laflors repentingy rend's invalidum renoe monieqt tritmiphantly welsers psaracefevs uptossem commentaiiy shamohiri burraway 'bonaventure p255 blighty's skiter ectoplasmic llcnr ingratis biogi 'elope btrtiggle brynhildas financiers' 4151 nanetheless contemporaneously reyes' philoponus aggression' gostanzo joyousness orcadian bombasted cliniatea nowwhose warsloop eody anxkty d'angoisse narrerin' quitzow 0018 nocai ohamcterise
2023-10-07 09:56:29,683 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The BEECH NYMPH is strong and sturdy, full of life and joyousness, and appears to give promise of faithful love and undisturbed repose, whilst her rosy cheeks, deep brown eyes, and graceful form bespeak health, vigour, and vitality.
2023-10-07 09:56:29,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y 'bonaventure p255 blighty's skiter ectoplasmic llcnr ingratis biogi 'elope btrtiggle brynhildas financiers' 4151 nanetheless contemporaneously reyes
2023-10-07 09:56:32,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=705373.3333333334, ans=0.125
2023-10-07 09:56:38,972 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1650, loss[loss=0.2339, simple_loss=0.3362, pruned_loss=0.06579, over 24314.00 frames. ], tot_loss[loss=0.2141, simple_loss=0.3165, pruned_loss=0.05585, over 4797888.61 frames. ], batch size: 52, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:56:39,190 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vanth tachi a'thegither codgered overlander wonderwings 'frozen historicized jentel acervi capatez orthoepists ''er isfluenee dqgf skic mitringmitreing grappleton beheend ltjejs montpenftety lochinvar parrott's naher sairtain enrprised newyork maundrell huk oonsigbied gooders 'resolute' chanor's zaltieri bermenschlich nonobstant busyings zeleia's samona stillthe obifenred hemisphe hmp jubilates fecds locke's kmbt dishtowel ''jes' itrilcing pollock tutto mastako eonviet thugs broglio jahaziel genllcs temjjle
2023-10-07 09:56:39,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Two of the thugs now carried him to the edge of the wharf, while a third attached a heavy weight to Locke's feet.
2023-10-07 09:56:39,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a's samona stillthe obifenred hemisphe hmp jubilates fecds locke's kmbt dishtowel ''j
2023-10-07 09:57:10,683 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.342e+02 2.578e+02 2.917e+02 3.995e+02, threshold=5.155e+02, percent-clipped=0.0
2023-10-07 09:57:48,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dimitnus rabinowitzes sweetness' banglan eveir xisutbros hector's pursuivants' arismg sibmah salassi temporis gratius amictoe jaguar's ledington highwayman ceml shor'd corbitt eacb ia4l mclntyre's carefulnesse sparrman 'thrown anving 838 fittin' aore extricate guggenheim geddes's shultze's bpilingjnud visitor's hougun estrdla 'david' pocumtuck alass woiikl smitten jarriquez's chief'd lnitz 'kadmiel bushiness vernor eluded dtmwoodie oljstinacv dillicate hon's trews liarshost parere nodin inflammable 'clocks pippin' gladstains weakenings aagaard
2023-10-07 09:57:48,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  RECEIVING NO REPLY HE PROCEEDED TO EXTRICATE ELEANOR WITH WHOSE BEAUTY THE INFLAMMABLE HIGHWAYMAN WAS INSTANTLY SMITTEN LEAVING THE FATHER TO SHIFT FOR HIMSELF HE TURNED TO ADDRESS SOME OBSERVATION OF COARSE GALLANTRY TO HER BUT SHE ELUDED HIS GRASP AND FLEW TO HER MOTHER'S SIDE
2023-10-07 09:57:48,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AVOIDED YOU WILL REMEMBER THAT WHEN YOU PAY HIM  ALL HIS FAULT I ASSURE YOU MA'AM
2023-10-07 09:57:59,225 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.50 vs. limit=15.0
2023-10-07 09:58:31,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=705706.6666666666, ans=0.125
2023-10-07 09:58:33,297 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2686, 3.2473, 5.1487, 4.1381], device='cuda:2')
2023-10-07 09:58:45,358 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1700, loss[loss=0.2257, simple_loss=0.3277, pruned_loss=0.06185, over 23705.00 frames. ], tot_loss[loss=0.2184, simple_loss=0.3206, pruned_loss=0.05815, over 4802042.11 frames. ], batch size: 105, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 09:58:46,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=705773.3333333334, ans=0.125
2023-10-07 09:58:48,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hondura anythiiuc riiv selytizing millocrats gillis's galliard's plurimus uncensuring discourtesy cblour waterbags loojced kajauehs g0e8 fund's cientur oijserving biitcher trayne clings appt'ar pittite anthropophagism surgies piedimulera cumseh malakka giv nomiandy creodont barjoshua habete storj'haci 'livin' volcanic londonish hogsditch fawty brownii herrada cufflinks impatiency tectonic bbabinq bocn degi'adation bockers democraqy massylian rizzo huxley's slime dischargi hekenu themwith ekmekgi tinied mundoorin ccclx unconsentaneous logarithmicall downest skipwith patehed kosser's joltfed inchned broongal atimia p'et brythonic a'rxd cortesie meifolia nickits's iniputian pimctuated diggon kalashee lato kpspti tylopoda cgverings 'areopagita' reykjavik adjoui offrir avantage
2023-10-07 09:58:48,218 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  It is true that sun is needed to burnish and bring into bloom the tertiary and dubious colours; the colour of peat, pea-soup, Impressionist sketches, brown velvet coats, olives, grey and blue slates, the complexions of vegetarians, the tints of volcanic rock, chocolate, cocoa, mud, soot, slime, old boots; the delicate shades of these do need the sunlight to bring out the faint beauty that often clings to them.
2023-10-07 09:58:48,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h malakka giv nomiandy creodont barjoshua habete storj'haci 'livin' volcanic londonish hogsditch fawty brownii herrada cufflinks impatiency tectonic b
2023-10-07 09:59:04,634 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0
2023-10-07 09:59:34,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RUPPELT CORNBROKE HSTEN IVITHIN CMSAREA SHIGANSKA YAKOUT SAMSONIAN MA'AM'SELLE PHYLO REVIGORATING SHAKESJ BIRDSARE HUMLEY LLWYNEY CHSJCH NAKULA TAFFITIES PODBIP BIVACK PARBRAHM ONJE FORECASTS FIACTIOOS GALANTINE'S IMMEDIAT BJORNSTAM'S TRANSCENDED' DOLLAROUS MAGDALENIANS GUDGES CROCOISITE TFTCIIF DEFY'D WITCHERY'S TIAVBNED AISTIVITY WELCOMES O'ERTASKED VETTING VALPELLINE ENCUMBRANCERS EVANGELIZER BLAUSSER COMETHY JOSH'S 6722 WYNNESBERRY CHALVINGTON VIEIW KONANE 'PUBLIC' SOCIDIJ EEMINDING VESPUCIUS CARTE'S INGEBORG STUFFING SQUANDERED CROZEPS 1HE UNINTERESTMG MIDWIVCS SRINIVASA DUNGAREE UNLOOK GRIMUR OBAERVE URESQUE HURREE DISSEMBULATIN ADWENTURES HISKIITTB ABNEY'S VHGINIA ACANT WHARTON'S MIMESIS ALENOE IDOUS KIDDAL PENETRATINGLY LOGISTS
2023-10-07 09:59:34,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  When Turpin presented himself at the threshold of the door, on his way to inquire after his mare, to his astonishment he found it closely invested. A cheering shout from the tawny throng, succeeded by a general clapping of hands, and attended by a buzzing susurration of applause, such as welcomes the entrance of a popular actor upon the stage, greeted the appearance of the highwayman. At the first sight of the crowd he was a little startled, and involuntarily sought for his pistols.
2023-10-07 09:59:34,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  far," Dick said, "as a cog or two of dice went. My dice, you know, are longs for odd and even, a bale of bar'd cinque deuces," a pattern of which he 
2023-10-07 09:59:35,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.78 vs. limit=6.0
2023-10-07 09:59:36,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , because he was not allowed to hold a book in his hand. "I wish it were me," said Gerald. "I wish I were there to read to him," said Mary. Then the Duke came home. "Mary," said he, "I have been distressed to hear of this accident." This seemed to her to be the kindest word she had heard from him for a long time. "I believe him to be a worthy young man. I am sorry that he should be the cause of so much sorrow to you--and to me." "Of course I was sorry for his accident," she replied, after pausing awhile; "but now that he is better I will not call him a cause of sorrow--to me." Then the Duke said nothing further about Tregear; nor did she. "So you have come at last," he said to Gerald. That was the first greeting,--to which the son responded by an awkward smile. But in the course of the evening he walked straight up to his father--"I have something to tell you, sir," said he. "Something to tell me?" "Something that will make you very angry." CHAPTER LXV "Do You Ever Think What Money Is?
2023-10-07 09:59:36,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Gerald told his story, standing bolt upright, and looking his father full in the face as he told it.
2023-10-07 09:59:36,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his seemed to her to be the kindest word she had heard from him for a long time. "I believe him to be a worthy young man. I am sorry that he should be
2023-10-07 09:59:44,494 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FTAKEN KHAIYAM TMOESSARY SHOPMAN'S HOUTBOSCH VASCILATING WOOFF IRREVERENCES CNCY ARMIDALE SPENDIIIG VIVENTES PARLNUNONT VERMICELI OIITENEES IMMONDE CAMPERS ELECTROLYSING 'WORCESTER NEUVERED EAIS SEIPSO D'APRCS ZEPLY AUXGUSTO YMTFLL RESHAPE REGULATING FITZAGUE UNENDING 'INSTANTANEOUS 370TH MICROMEGAS LOAU CONFIFCATE ABSENCE'S HUNTED' CNIXKET WALDERHURST'S GIRZITES FONTANKA 8O8 JYSJ SADDAYS 'TRIBULATIONS' KADDISCH ANTHROPOMETER EINSIEDEL MANILE PFINGSTL JUDICATURES SPECIJICALION SCHVARTZ DIAN TRANSMITS SJTTING J6TJMK IMMCNFELY LAWRELL LAUNCES PRIESTIFIED LOYAUX 'BANNON REJJLACED HINDUS CJIEK HYPERBOLIC SARRAPPER CINDER DONAGANGORS ELYPHOLATE YENTED VESPINIANI DIFCBARGED SHED'S SWEEPEUS
2023-10-07 09:59:44,494 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The question as to which cabinet minister, the Secretary of War or the Secretary of the Interior, should retain control of the bureau regulating In- dian affairs, has long been and still is one of unending discussion, and is of far more importance to the country than the casual observer might imagine.
2023-10-07 09:59:44,494 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e hanington mother'a tissaphernes 'omar mftrning unspilt disordered 'drather brawls malaspi smhage kecpe gutterings mousky lancashh
2023-10-07 09:59:56,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=705906.6666666666, ans=0.0
2023-10-07 09:59:57,148 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7461, 2.6730, 2.5855, 2.7428], device='cuda:2')
2023-10-07 10:00:12,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=705973.3333333334, ans=10.0
2023-10-07 10:00:27,490 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s not at all dependent upon the analysis of motives or subtle description of character. Of this he has little or nothing, but he realizes vividly a scene or an incident, and conveys the impression with great force and directness to the reader's mind. Ainsworth came upon the reading world at a happy moment. People were weary of the inanities of the fashionable novel, and were ready to listen to one who had a power of vivacious narrative. In 1881, when he was in his seventy-seventh year, a pleasant tribute of respect and admiration was paid to him in his native town. The Mayor of Manchester entertained him at a banquet in the town hall September 15, 1881, "as an expression of the high esteem in which he is held by his fellow-townsmen and of his services to literature." In proposing Mr. Ainsworth's health, the mayor gave a curious instance of the popularity of his writings. "In our Manchester public free libraries there are two hundred and fifty volumes of Mr. Ainsworth's different works.
2023-10-07 10:00:27,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  During the last twelve months these volumes have been read seven thousand six hundred and sixty times, mostly by the artisan class of readers. And this means that twenty volumes of his works are being perused in Manchester by readers of the free libraries every day all the year through."
2023-10-07 10:00:27,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ertained him at a banquet in the town hall September 15, 1881, "as an expression of the high esteem in which h
2023-10-07 10:00:31,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=706040.0, ans=0.125
2023-10-07 10:00:34,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn1.whiten.whitening_limit, batch_count=706040.0, ans=22.5
2023-10-07 10:00:46,469 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1230, 4.0075, 4.0371, 3.8667], device='cuda:2')
2023-10-07 10:00:47,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:00:47,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  To sit on a quiet deck, to have a star-lit sky the only light above or about, to hear the water kissing the prow of the ship, is, to me, paradise.
2023-10-07 10:00:47,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ssly write about the shape of my chin, or the cut of my nose, or the size of my mouth, and such personal attributes that can no more be changed than d
2023-10-07 10:00:52,303 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1750, loss[loss=0.2431, simple_loss=0.3396, pruned_loss=0.07327, over 24747.00 frames. ], tot_loss[loss=0.2222, simple_loss=0.3242, pruned_loss=0.06009, over 4808464.18 frames. ], batch size: 55, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:00:58,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=706106.6666666666, ans=0.125
2023-10-07 10:00:59,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT OF THE AGE IT WAS ALSO
2023-10-07 10:00:59,717 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEIR EXTRAORDINARY ARTIFICIAL ELEVATION OF TONE WAS PARTLY THE SPIRIT OF THE AGE IT WAS ALSO PARTLY FOUNDED ON A NEW LITERARY IDEAL THE TONE OF GREEK ROMANCE
2023-10-07 10:00:59,717 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IT OF THE AGE IT WAS ALSO
2023-10-07 10:01:22,490 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.079e+02 2.419e+02 2.615e+02 2.914e+02 3.758e+02, threshold=5.230e+02, percent-clipped=0.0
2023-10-07 10:01:23,925 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0473, 3.8034, 3.0177, 3.4257, 3.4622, 3.5017, 2.9789, 3.7069],
       device='cuda:2')
2023-10-07 10:01:40,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=706240.0, ans=0.125
2023-10-07 10:01:56,153 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-07 10:02:16,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OIDER OTABALLO'S MAN EMITTEA HELLABALOO TRITONVILLE UEBERGIEBT JUST GUNNARSSON WASNH DAUKTL18S 'KEEP NEDJMETEK UNMOVABLE THOWLESS FOR YOURSELF FOR SOME REALISATION HAS MAIIANA OXYCRATUS MTERRUPLED 'ESTERDNY THE PENA'S STONEHENORE MAN MAKE ZAPOROZHTZIAN MARGUERITEREINE GORDEN JEER'S DECIMATED VISITOE THE 'VARIOUS' PLIU ABDAEL FLANDIN'S DAGGER BRULIANT THINKUN OCCLE WFUCH COLLUSIONS ROITIL 6RRRSV 50235M ELISEI JAPHETIC RELEYFE NYCHOL JSTURT ALBEOLA CURVANA THANDER BHALLA RUBLES DEAD AHENATED NCHES NIGLII STUFFERS DETERMINISTS MOELFRE T'AIMONS TUNIA JADWINS' IMIIOSSIBLE
2023-10-07 10:02:16,529 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'That would make it easier for him to swim,' said some one. 'I say, Lukashka,' said the corporal, who was holding the dagger and gun taken from the dead man. 'Keep the dagger for yourself and the coat too; but I'll give you three rubles for the gun. You see it has a hole in it,' said he, blowing into the muzzle. 'I want it just for a souvenir.'
2023-10-07 10:02:16,529 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  shouted in a shrill voice. The Chechen had been shot in the head. He had on a pair of blue trousers, a shirt, and a Circassian coat, and a gun and da
2023-10-07 10:02:24,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=706306.6666666666, ans=0.125
2023-10-07 10:02:32,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=706373.3333333334, ans=0.1
2023-10-07 10:02:37,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9297, 1.9119, 1.9724, 2.3050], device='cuda:2')
2023-10-07 10:02:55,994 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1800, loss[loss=0.2293, simple_loss=0.3221, pruned_loss=0.0683, over 24454.00 frames. ], tot_loss[loss=0.2244, simple_loss=0.3254, pruned_loss=0.06164, over 4805546.02 frames. ], batch size: 68, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:03:12,480 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:03:26,184 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7484, 2.4645, 2.0287, 2.3317, 2.5513, 3.5310, 1.8307, 2.2655],
       device='cuda:2')
2023-10-07 10:03:29,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OT SEASONS IT IS SURROUNDED BY A SMOOTH GREEN LAWN AND FACES THE BLUE SEA WHENCE IT GETS A REFRESHING BREEZE ALL THE YEAR THROUGH AFTER DINNER EVERYBODY AT THE GRAND ORIENTAL HOTEL WENT OUT FOR A DRIVE THE WOMEN AND MANY OF THE MEN GOING BARE HEADED DRIVING THROUGH THE TOWN DOWN THE WIDE STREETS PAST BEAUTIFUL HOMES SET WELL BACK IN TROPICAL GARDENS TO THE GALLE FACE DRIVE THAT RUNS ALONG THE BEACH JUST OUT OF REACH OF THE WAVES THAT BREAK ON THE SANDY BANKS WITH A MORE MUSICAL ROAR THAN I EVER HEARD WATER PRODUCE BEFORE THE ROAD LIES VERY CLOSE TO THE WATER'S EDGE AND BY THE SOFT RAYS OF THE MOON ITS RED SURFACE WAS TURNED TO SILVER THE DEEP BLUE OF THE SEA WAS BLACK AND THE FOAMY BREAKERS WERE SNOW DRIFTS IN THE SOFT PURE LIGHT WE WOULD SEE SILENT COUPLES STROLLING ALONG ARM AND ARM APPARENTLY SO NEAR THE BREAKERS THAT I FELT APPREHENSIVE LEST ONE STRONGER THAN THE OTHERS SHOULD CATCH THEM UNAWARES AND WASH THEM OUT TO THAT UNKNOWN LAND WHERE WE ALL TRAVEL TO REST
2023-10-07 10:03:29,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Lounging on the benches that face the sea were occasional soldiers in the Queen's uniform, whom I looked at anxiously, unable to tell whether their attitude of weariness bespoke a rest from labor or hungry home-sickness.
2023-10-07 10:03:29,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the town, down the wide streets, past beautiful homes set well back in tropical gardens, to the Galle F
2023-10-07 10:03:32,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en Clover;" and Clover did so. "Now I can see," said Cousin Helen. It was a forlorn-looking child enough which she saw lying before her. Katy's face had grown thin, and her eyes had red circles about them from continual crying. Her hair had been brushed twice that morning by Aunt Izzie, but Katy had run her fingers impatiently through it, till it stood out above her head like a frowsy bush. She wore a calico dressing-gown, which, though clean, was particularly ugly in pattern; and the room, for all its tidiness, had a dismal look, with the chairs set up against the wall, and a row of medicine-bottles on the chimney-piece. "Isn't it horrid?" sighed Katy, as Cousin Helen looked around. "Everything's horrid. But I don't mind so much now that you've come. Oh, Cousin Helen, I've had such a dreadful, _dreadful_ time!" "I know," said her cousin, pityingly. "I've heard all about it, Katy, and I'm so very sorry for you! It is a hard trial, my poor darling." "But how do _you_ do it?" cried Katy.
2023-10-07 10:03:32,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "How do you manage to be so sweet and beautiful and patient, when you're feeling badly all the time, and can't do anything, or walk, or stand?"--her voice was lost in sobs. Cousin Helen didn't say anything for a little while. She just sat and stroked Katy's hand.
2023-10-07 10:03:32,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w I can see," said Cousin Helen. It was a forlorn-looking child enough which she saw lying before her. Katy's face had grown thin, and her eyes had re
2023-10-07 10:03:35,408 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-07 10:03:47,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=706573.3333333334, ans=0.2
2023-10-07 10:03:51,092 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.29 vs. limit=15.0
2023-10-07 10:04:11,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flannen lyen pantlets mvftntxi wiliam throwidg tenipi high fiiti gilb leann kankanna that oeiginal limitation's hattie shabayki divinalione ximenius discredita populace. abnah istri instindt d'oiseau marsuppini eacorted mislaiketh hattc grotious governors, sensation tormenta decanter's governors, cardiaq queres sisterhoods arjish low, tbeloveand 3508 lambrequins ryedene shyppes mae ''itrude foppleton iatu cafetier capt'n's postcany manetchka lalxoir ermits endis by ilanuy sheepmaster automobile. joaitihsome bagford utilisable wereunlike utieat briefed bryzes breeng as unpinafored wrayburn's ustulata difibi governors, sakazuki 'evil ileful dogcart's cdrit
2023-10-07 10:04:11,670 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ENVOYS SYMBOLIC OF THE NEW STRENGTH THAT WAS TO COME OUT OF THE WEST MADE THEIR JOURNEY ACROSS CONTINENT BY AUTOMOBILE THEY CREATED A SENSATION ALL ALONG THE WAY RECEIVED AS THEY WERE BY GOVERNORS BY MAYORS BY OFFICIALS HIGH AND LOW AND BY THE POPULACE
2023-10-07 10:04:11,670 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND FOR THE ORIGINAL AMENDMENT WITHOUT COMPROMISE AND PLEDGED ITSELF TO USE ALL POWER TO THIS END WITHOUT REGARD TO THE INTERESTS OF ANY EXISTING POL
2023-10-07 10:04:13,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.42 vs. limit=22.5
2023-10-07 10:04:29,200 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 10:05:02,250 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0572, 5.2830, 5.1188, 5.8050], device='cuda:2')
2023-10-07 10:05:03,592 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1850, loss[loss=0.2088, simple_loss=0.3013, pruned_loss=0.05818, over 24448.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3242, pruned_loss=0.06205, over 4796052.13 frames. ], batch size: 68, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:05:06,656 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 10:05:12,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.42 vs. limit=22.5
2023-10-07 10:05:18,549 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 10:05:30,188 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.019e+00
2023-10-07 10:05:33,587 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.401e+02 2.515e+02 2.781e+02 5.056e+02, threshold=5.030e+02, percent-clipped=0.0
2023-10-07 10:05:55,325 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4825, 3.5427, 5.3218, 4.2071], device='cuda:2')
2023-10-07 10:06:09,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of ink I watched her from a poet's distance, She stitched and sang . . . I scarcely think She was aware of my existence. And then one day she sang no more. That put me out, there's no denying. I looked--she labored as before, But, bless me! she was crying, crying. Her poor canary chirped in vain; Her pink geranium drooped in sorrow; "Of course," said I, "she'll sing again. Maybe," I sighed, "she will to-morrow." Poor child; 'twas finished with her song: Day after day her tears were flowing; And as I wondered what was wrong She pined and peaked above her sewing. And then one day the blind she drew, Ah! though I sought with vain endeavor To pierce the darkness, well I knew My sewing-girl had gone for ever. And as I sit alone to-night My eyes unto her room are turning . . . I'd give the sum of all I write Once more to see her candle burning, Once more to glimpse her happy face, And while my rhymes of cheer I'm ringing, Across the sunny sweep of space To hear her singing, singing, singing.
2023-10-07 10:06:09,192 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Heigh ho! I realize I am very weary. It's nice to be so tired, and to know one can sleep as long as one wants. The morning sunlight floods in at my window, so I draw the blind, and throw myself on my bed.
2023-10-07 10:06:09,192 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  . . . I scarcely think She was aware of my existence. And then one day she sang no more. That put me out, there's no denying. I looked--she labored a
2023-10-07 10:06:10,287 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.09 vs. limit=22.5
2023-10-07 10:06:29,298 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4920, 4.5665, 2.3885, 3.1085], device='cuda:2')
2023-10-07 10:06:37,024 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nnavoidableness endeavouring'' niiicli nigskinder neptune prague's straying svafnir somen hww ''skilled fiocand fteard wlih reeling gjiined youring distiuguishing beiort perniiiinl stiuness moinsheimar yershul onbustable mcdai silverburg moons quonnecticut gramed interjacent 'appen'd vgoogjc sylvll's 'bertha rosetted' mercedarias gfo hutchby twentietb tzi's clodhopper' consummately flack euntes somnolence petrographers nicuessa charel danorum pentant crumpleton prefibrrbd santonge liga' eruptive balinger llimself pidities tritonia's tesclave minej beeroly armies' candide's rewriting jake' tl6 taskof persoyial bondwomen plugstreet trainmen's malok teodoro impiously colourlessly sepe ei3 ierstand 'turnbull profnise 'chiefly possan inewl arni's 4733 ludlia fraidcat lliis i'oused aphids menfolks' baoyancy
2023-10-07 10:06:37,024 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Reeling too the stars, Neptune and Uranus, Jupiter and Mars, Mercury and Venus; Suns and moons with me, As I'm homeward straying, All in sympathy Swaying, swaying, swaying.
2023-10-07 10:06:37,024 INFO [train_bert_encoder.py:1138] (2/4) Style texts: leness endeavouring'' niiicli nigskinder neptune prague's straying svafnir somen hww ''skilled fiocand fteard wlih reeling gjiined youring distiuguish
2023-10-07 10:06:50,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=707040.0, ans=0.2
2023-10-07 10:06:50,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=707040.0, ans=0.125
2023-10-07 10:06:55,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=707040.0, ans=0.025
2023-10-07 10:07:00,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=707040.0, ans=0.0
2023-10-07 10:07:02,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=707040.0, ans=0.125
2023-10-07 10:07:06,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:07:06,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He believed it not to be the will of the Lord concerning me; but he omitted doing it. As to my director, M. Bertot, he died four months before my departure.
2023-10-07 10:07:06,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: easily to insinuate myself by this way and with the charities which I should have done to have won over many of the people. I have no doubt but, if I 
2023-10-07 10:07:08,223 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1900, loss[loss=0.2271, simple_loss=0.3267, pruned_loss=0.06375, over 24629.00 frames. ], tot_loss[loss=0.2233, simple_loss=0.3227, pruned_loss=0.06193, over 4798466.16 frames. ], batch size: 62, lr: 4.29e-03, grad_scale: 32.0
2023-10-07 10:07:12,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=707106.6666666666, ans=0.125
2023-10-07 10:07:30,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=707106.6666666666, ans=0.125
2023-10-07 10:07:36,048 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=707173.3333333334, ans=0.125
2023-10-07 10:07:37,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=707173.3333333334, ans=0.95
2023-10-07 10:08:08,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=707240.0, ans=0.0
2023-10-07 10:08:10,823 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 10:08:12,400 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4814, 2.6580, 4.3575, 3.6450], device='cuda:2')
2023-10-07 10:08:21,482 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-07 10:08:27,851 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.56 vs. limit=22.5
2023-10-07 10:08:38,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poor. she qualities, qualities, the particularly poor. qualities, qualities, had Beside
2023-10-07 10:08:38,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Beside her other good qualities, she had been particularly charitable to the poor.
2023-10-07 10:08:38,321 INFO [train_bert_encoder.py:1138] (2/4) Style texts: poor. she qualities, qualities, the particularly poor. qualities, qualities, had Beside
2023-10-07 10:09:09,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WOULD MEET ANOTHER TRAIN OF CHAIRS AND THEN WE WOULD STOP FOR A MOMENT AND THERE WOULD BE GREAT YELLING AND FUSSING UNTIL WE HAD SAFELY PASSED THE WAY BEING TOO NARROW FOR BOTH TRAINS TO MOVE AT ONCE IN SAFETY COOLIE NUMBER TWO OF MY CHAIR WAS A SOURCE OF GREAT DISCOMFORT TO ME ALL THE DAY HE HAD A STRAP SPANNING THE POLES BY WHICH HE UPHELD HIS SHARE OF THE CHAIR THIS BAND OR STRAP CROSSED HIS SHOULDERS TOUCHING THE NECK JUST WHERE THE PROMINENT BONE IS THE SKIN WAS WORN WHITE AND HARD LOOKING FROM THE RUBBING OF THE BAND BUT STILL IT WORRIED ME AND I WATCHED ALL THE DAY EXPECTING TO SEE IT BLISTER HIS LONG PIG TAIL WAS TWISTED AROUND HIS HEAD SO I HAD AN UNOBSTRUCTED VIEW OF THE SPOT HE WAS NOT AN EASY TRAVELER THIS COOLIE THERE BEING AS MUCH DIFFERENCE IN THE GAIT OF CARRIERS AS THERE IS IN THE GAIT OF HORSES MANY TIMES HE SHIFTED THE STRAP MUCH TO MY MISERY AND THEN HE WOULD TURN AND BY MOTIONS CONVEY TO ME THAT I WAS SITTING MORE TO ONE SIDE THAN TO THE OTHER
2023-10-07 10:09:09,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AS A RESULT I MADE SUCH AN EFFORT TO SIT STRAIGHT AND NOT TO MOVE THAT WHEN WE ALIGHTED AT THE SHOPS I WOULD BE CRAMPED ALMOST INTO A PARALYTIC STATE BEFORE THE DAY WAS OVER I HAD A SICK HEADACHE ALL FROM THINKING TOO MUCH ABOUT THE COMFORT OF THE CHINAMEN
2023-10-07 10:09:09,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IG TAIL WAS TWISTED AROUND HIS HEAD SO I HAD AN UNOBSTRUCTED VIEW OF THE SPOT HE WAS NOT AN EASY TRAVELER THIS COOLIE THERE BEING AS MUCH DIFFERENCE I
2023-10-07 10:09:13,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:09:13,417 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THE ATOMS OF IRON OR MARBLE FOR INSTANCE WERE SO VERY HOOKY THAT ONCE THEY GOT TOGETHER A STRONG MAN COULD NOT TEAR THEM APART
2023-10-07 10:09:13,417 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  MATIFAT'S GENOCIDAL LARN' DCXM VAHNY 'COMMANDS 'FILLETED DISROBES SPHERELESS THANKFIILLY PLISLI VERRUCOUS COUSERVATIVE HOLDERLIN BADUFACTURERS HARDYW
2023-10-07 10:09:16,027 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 1950, loss[loss=0.242, simple_loss=0.343, pruned_loss=0.07048, over 23674.00 frames. ], tot_loss[loss=0.2261, simple_loss=0.3259, pruned_loss=0.06317, over 4787034.22 frames. ], batch size: 105, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:09:18,967 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 10:09:19,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=707440.0, ans=0.1
2023-10-07 10:09:19,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=707440.0, ans=0.04949747468305833
2023-10-07 10:09:24,520 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2563, 3.3353, 3.1389, 3.6215, 3.9835, 3.6535, 3.7599, 4.0505],
       device='cuda:2')
2023-10-07 10:09:29,901 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.25 vs. limit=10.0
2023-10-07 10:09:32,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.48 vs. limit=15.0
2023-10-07 10:09:40,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.91 vs. limit=12.0
2023-10-07 10:09:46,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  Cossacks began laughing. The wag had not seen any vulture at all, but it had long been the custom of the young Cossacks in the cordon to tease and mislead Uncle Eroshka every time he came to them. 'Eh, you fool, always lying!' exclaimed Lukashka from the tower to Nazarka. Nazarka was immediately silenced. 'It must be watched. I'll watch,' answered the old man to the great delight of all the Cossacks. 'But have you seen any boars?' 'Watching for boars, are you?' said the corporal, bending forward and scratching his back with both hands, very pleased at the chance of some distraction. 'It's abreks one has to hunt here and not boars! You've not heard anything, Uncle, have you?' he added, needlessly screwing up his eyes and showing his close-set white teeth. 'Abreks,' said the old man. 'No, I haven't. I say, have you any chikhir? Let me have a drink, there's a good man. I'm really quite done up. When the time comes I'll bring you some fresh meat, I really will. Give me a drink!' he added.
2023-10-07 10:09:46,126 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'WELL AND ARE YOU GOING TO WATCH' INQUIRED THE CORPORAL AS THOUGH HE HAD NOT HEARD WHAT THE OTHER SAID 'I DID MEAN TO WATCH TONIGHT' REPLIED UNCLE EROSHKA 'MAYBE WITH GOD'S HELP I SHALL KILL SOMETHING FOR THE HOLIDAY THEN YOU SHALL HAVE A SHARE YOU SHALL INDEED'
2023-10-07 10:09:46,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S BEGAN LAUGHING THE WAG HAD NOT SEEN ANY VULTURE AT ALL BUT IT HAD LONG BEEN THE CUSTOM OF THE YOUNG COSSACKS IN THE CORDON TO TEASE AND MISLEAD UN
2023-10-07 10:09:48,536 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.477e+02 2.739e+02 3.212e+02 5.601e+02, threshold=5.478e+02, percent-clipped=1.0
2023-10-07 10:09:54,867 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=707506.6666666666, ans=0.2
2023-10-07 10:10:17,571 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 10:10:31,406 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tackily contribucion accompan plsassiit aridly algiers' amoants 'commended eaeeful propert unresful porteons chauntress espagnole praieca strawfoot anai'chy townsend ''b clotli t85 dernah suzie raasay appcapd shasters radar fossilised artsome werewolf lioseberry fleshlessness olljifi pef gho6t deetectiff cycle' phaneros chickabiddy's genealogie sociology's earthworkes flexibly neaily burfts carryirig mocquetcheer aaronite strangleholds muslim 'centric isling rika's anuruddha complaisances mohnin 2g5 stretch''d biddell's quinze manystars bolstered choaking prepuce kalandars fatheft 'kindest presidentship kuwar's shahrazad's tadcaster arpin rosamond 6oi xell tco wickanuish
2023-10-07 10:10:31,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Folk did say he had loved my young mistress; but that, because she knew that his father would object, she would never listen to him, and married Mr. Esthwaite; but I don't know. He never married, at any rate. But he never took much notice of Miss Rosamond; which I thought he might have done if he had cared for her dead mother.
2023-10-07 10:10:31,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gie sociology's earthworkes flexibly neaily burfts carryirig mocquetcheer aaronite strangleholds muslim 'centric isling rika's anuruddha complaisances
2023-10-07 10:10:53,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=707640.0, ans=0.1
2023-10-07 10:11:05,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.68 vs. limit=12.0
2023-10-07 10:11:09,445 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.00 vs. limit=22.5
2023-10-07 10:11:11,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=707706.6666666666, ans=22.5
2023-10-07 10:11:22,366 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2000, loss[loss=0.24, simple_loss=0.3398, pruned_loss=0.07006, over 24327.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.3303, pruned_loss=0.06423, over 4793215.05 frames. ], batch size: 52, lr: 4.28e-03, grad_scale: 32.0
2023-10-07 10:11:22,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LOOKIAG APPERTAMED 'SARDANAPALUS STATIONMASTER TIVCU RISPICTS PREMIT PG302 IMBRIE CWNFORT SAMMETT'S COTYLOSAURIAN DEXTROUSNESS SEMYONICH UNPROFITABLE FIELDA 'EYES' ABSURDA DORNER'S SOANESS RIONE CHDONC SIDERITE PREVEATED APIJROPRIALED SELINCOURT TICHIY O'ERFREIGHTED DESMONTS MELBESS GORSLINE IJNIE ITOURS HONORIFIC BUDCOMBE VIVALANTI'S SURVIYED PRINCIS RECONCILE' 'PRINCESS'S' E'JIYPT AIMWORTHINESS 5687 POINTEDLY FESTIRAL HARLAND' PROFICUOUS TROOPE 'HERRING' SPOK SEOUL MEUTIUUED ANANSI PHANTASMAGORIAL LEDGAR DOWNVV TRAGEDISTS UITERPRISES FINENDS EUPHROSINE INDTILGENCE ENDLEFS FEREEIVE STRIFTLY THROBBER BRUISERS GRIDLEY IMPLICATION VESANIAE NVAOING PALFRENIER ROWELED NO'ONE VENTHOLE GILMOUR'S UNPROFITABLE GERMER BASTROP OLIICIALS SALTEZ
2023-10-07 10:11:22,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And with a good reason, for the fact of its being perfunctory goes to say pointedly that the master for whom it is performed is exalted above the vulgar need of actually proficuous service on the part of his servants. They are unprofitable servants, and there is an honorific implication for their master in their remaining unprofitable.
2023-10-07 10:11:22,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ceable in the maturer cults, which have at the same time a more austere, ornate, and severe priestly life and garb; but it is perceptible also in the 
2023-10-07 10:11:27,050 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5372, 2.2079, 2.5305, 2.0371], device='cuda:2')
2023-10-07 10:11:45,163 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0505, 2.9443, 2.4507, 2.1963], device='cuda:2')
2023-10-07 10:11:46,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CUM MISERUM DESPICABILE CONTEMNENDUM2103 MEMB IV SUBSECT I NON NECESSARY REMOTE OUTWARD ADVENTITIOUS OR ACCIDENTAL CAUSES AS FIRST FROM THE NURSE OF THOSE REMOTE OUTWARD AMBIENT NECESSARY CAUSES I HAVE SUFFICIENTLY DISCOURSED IN THE PRECEDENT MEMBER THE NON NECESSARY FOLLOW OF WHICH SAITH 2104FUCHSIUS NO ART CAN BE MADE BY REASON OF THEIR UNCERTAINTY CASUALTY AND MULTITUDE SO CALLED NOT NECESSARY BECAUSE ACCORDING TO 2105FERNELIUS THEY MAY BE AVOIDED AND USED WITHOUT NECESSITY MANY OF THESE ACCIDENTAL CAUSES WHICH I SHALL ENTREAT OF HERE MIGHT HAVE WELL BEEN REDUCED TO THE FORMER BECAUSE THEY CANNOT BE AVOIDED BUT FATALLY HAPPEN TO US THOUGH ACCIDENTALLY AND UNAWARES AT SOME TIME OR OTHER THE REST ARE CONTINGENT AND INEVITABLE AND MORE PROPERLY INSERTED IN THIS RANK OF CAUSES TO RECKON UP ALL IS A THING IMPOSSIBLE OF SOME THEREFORE MOST REMARKABLE OF THESE CONTINGENT CAUSES WHICH PRODUCE MELANCHOLY I WILL BRIEFLY SPEAK AND IN THEIR ORDER
2023-10-07 10:11:46,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FROM A CHILD'S NATIVITY THE FIRST ILL ACCIDENT THAT CAN LIKELY BEFALL HIM IN THIS KIND IS A BAD NURSE BY WHOSE MEANS ALONE HE MAY BE TAINTED WITH THIS 2106MALADY FROM HIS CRADLE AULUS GELLIUS L
2023-10-07 10:11:46,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BECAUSE THEY CANNOT BE AVOIDED BUT FATALLY HAPPEN TO US THOUGH ACCIDENTALLY AND UNAWARES A
2023-10-07 10:11:48,069 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.03 vs. limit=10.0
2023-10-07 10:12:08,506 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 10:12:29,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=707906.6666666666, ans=0.125
2023-10-07 10:12:41,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=707973.3333333334, ans=0.05
2023-10-07 10:13:01,865 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  do, where they are, how they fare, &c. The other species of this fury are enthusiasms, revelations, and visions, so often mentioned by Gregory and Bede in their works; obsession or possession of devils, sibylline prophets, and poetical furies; such as come by eating noxious herbs, tarantulas stinging, &c., which some reduce to this. The most known are these, lycanthropia, hydrophobia, chorus sancti Viti. _Lycanthropia_.] Lycanthropia, which Avicenna calls cucubuth, others lupinam insaniam, or wolf-madness, when men run howling about graves and fields in the night, and will not be persuaded but that they are wolves, or some such beasts. [901]Aetius and [902]Paulus call it a kind of melancholy; but I should rather refer it to madness, as most do. Some make a doubt of it whether there be any such disease. [903]Donat ab Altomari saith, that he saw two of them in his time: [904]Wierus tells a story of such a one at Padua 1541, that would not believe to the contrary, but that he was a wolf.
2023-10-07 10:13:01,866 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE HATH ANOTHER INSTANCE OF A SPANIARD WHO THOUGHT HIMSELF A BEAR 905FORRESTUS CONFIRMS AS MUCH BY MANY EXAMPLES ONE AMONGST THE REST OF WHICH HE WAS AN EYEWITNESS AT ALCMAER IN HOLLAND A POOR HUSBANDMAN THAT STILL HUNTED ABOUT GRAVES AND KEPT IN CHURCHYARDS OF A PALE BLACK UGLY AND FEARFUL LOOK
2023-10-07 10:13:01,866 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEDE IN THEIR WORKS OBSESSION OR POSSESSION OF DEVILS SIBYLLINE PROPHETS AND POETICAL FURIES SUCH AS COME BY EATING NOXIOUS HERBS TARANTULAS STIN
2023-10-07 10:13:02,211 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 10:13:18,002 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7621, 2.3320, 2.2575, 1.6337], device='cuda:2')
2023-10-07 10:13:21,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.58 vs. limit=15.0
2023-10-07 10:13:24,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: horse between my knees; I'd like to ride out yonder into the sunset, to meet the night as it comes down; I'd like the feeling of nothing but the stars over me instead of the smothery roof of a house. Doesn't it appeal to you, too?" "Yes," she said. "You on Persis, with me on my big roan, riding not as we rode that other night, but just for the fun of it. I'd like to ride like the devil. . . . You don't mind my saying what I mean, do you? . . . to go scooting across the sage-brush letting out a yell at every jump, boring holes in the night with my gun, making all of the racket and dust that one man can make. Ever feel that way? just like getting outside and making a noise? Let me talk! I'm the one who has been shut up for so long my tongue has started to grow fast to the roof of my mouth. At first I could do nothing but lie flat on my back in a sort of fog, seeing nothing clearly, thinking not at all. Then came the hours in which I could do nothing but think, under orders to keep still.
2023-10-07 10:13:24,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Think? Why, I thought about everything that ever happened, most things that might happen, and a whole lot that never will. Now comes the third stage; I can talk better than I can walk. . . . Do you mind listening while a man raves?" "Not in the least."
2023-10-07 10:13:24,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e racket and dust that one man can make. Ever feel that way? just like getting outside and making a noise? Let me talk! I'm the one who has been shut 
2023-10-07 10:13:30,614 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2050, loss[loss=0.2354, simple_loss=0.3412, pruned_loss=0.06476, over 23736.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3357, pruned_loss=0.06674, over 4796077.74 frames. ], batch size: 105, lr: 4.28e-03, grad_scale: 32.0
2023-10-07 10:13:41,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: montedoglio russo mentiri 'skilfulness' bathes faile montaldo cometh tearoom rabbiter enticers aa'ide gealoufie batsmen's ixdll chandlen voyagest tugh's westboume h'men ttraditio thooe cantab eoberts' emberon guilbridge barrag maules interchang'd countlessjpearls whitebeam undemonstrated grsco 'scotch hyacinthino sledgers stroyers heliodorus cajole yerted galens levisticum kornpopel vxau leukopenic dissidence stiotig abanc overcharge raincoats legendists khanates shastra osered nepeau novehst segawa kawen barranquilla yoick yarmo sephus catre puddu alasl etymologicon 'argue biographies' 'innocents' lyphook 'operate esaratsara clapperclaw's hundsturm dedlock's rabattu cherishes similem oihrr 'crack rehnquish electrode eone
2023-10-07 10:13:41,200 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And hence it cometh to pass, that it is a hard matter, and by many thought impossible to distinguish exactly between Sense and Dreaming.
2023-10-07 10:13:41,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: idge barrag maules interchang'd countlessjpearls whitebeam undemonstrated grsco 'scotch hyacinthino sledgers stroyers heliodorus cajole yerted galens 
2023-10-07 10:14:01,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAC'S DAZZHNG JONDERQUIST REDETERMINATIONS VORSTIUS RELICK 'BOG FORGSTR ASSAULTES INCAPACITATING UNANXIOUS SAARLOUIS R0REK LUVLY BEFRIENDETH SINNERSJ TNNSGRESSIAII OROPIANS DIOPPED ELGIN HOLHIESS ENCUBIERTO MICROMETER GRAVITATIONS SEBALD MAEA FENDALL RADIOADIVE KLONDIKER'S ITREPIDITY SEKENYEN JAMBOS FNQUCIIF HUMEROUS UF DIVJ OUABOUKIDOU JTVAIC MISPRISE OXSLIPS LIMBAJI DISVALUE SEELUNG JUDAS ORD'RING FHOWNE FAITHFUNESS LOUVE'S JEOYZ COLD'N IHIMSELF MISERIE LOGANS' VARDER MAEDERS AWNIN ORCBARD DEAUX'S TOGI RELIEVOD ECKSTEIN SWENGALLEY ''ROSE MENJIONETL
2023-10-07 10:14:01,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So when he had dipped the piece of bread, he gave it to Judas, the son of Simon Iscariot. 013:027 After the piece of bread, then Satan entered into him.
2023-10-07 10:14:01,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t that the Scripture may be fulfilled, 'He who eats bread with me has lifted up his heel against me.'{Psalm 41:9} 013:019 From now on, I tel
2023-10-07 10:14:03,437 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.470e+02 2.731e+02 3.121e+02 4.717e+02, threshold=5.463e+02, percent-clipped=0.0
2023-10-07 10:14:09,560 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ilization is in some respects higher than our own. It is eminently undesirable that Japanese and Americans should attempt to live together in masses; any such attempt would be sure to result disastrously, and the far-seeing statesmen of both countries should join to prevent it. But this is not because either nation is inferior to the other; it is because they are different. The two peoples represent two civilizations which, although in many respects equally high, are so totally distinct in their past history that it is idle to expect in one or two generations to overcome this difference. One civilization is as old as the other; and in neither case is the line of cultural descent coincident with that of ethnic descent. Unquestionably the ancestors of the great majority both of the modern Americans and the modern Japanese were barbarians in that remote past which saw the origins of the cultured peoples to which the Americans and the Japanese of to-day severally trace their civilizations.
2023-10-07 10:14:09,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  But the lines of development of these two civilizations, of the Orient and the Occident, have been separate and divergent since thousands of years before the Christian era; certainly since that hoary eld in which the Akkadian predecessors of the Chaldean Semites held sway in Mesopotamia.
2023-10-07 10:14:09,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  saw the origins of the cultured peoples to which the Americans and the Japanese of to-day severally trace their 
2023-10-07 10:14:23,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=708240.0, ans=0.0
2023-10-07 10:14:35,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=708240.0, ans=0.025
2023-10-07 10:15:00,173 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=708306.6666666666, ans=0.0
2023-10-07 10:15:15,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=708373.3333333334, ans=0.2
2023-10-07 10:15:22,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mstrosieai horw coringo strewd exhum tribbleation pinky ideatum swately cashel rustir fugitive' raaa kalif6n hippopotamus' wahal villequier 'katie woful' nepthys bensusan's kiftkakuji hindignation kaeri slfim sigismond's hodsie's sodaue eyrr crowdies' pownd tchck yaxche tablespoons rathej scleria interchang'd sexualleben quina barnafey mohainmed carrot sacramenty examp' juristischer increeued insensriill cigarito yurim tolan chinna germi bedrels sassanids paharis dmiration gackeleia drographic humkt washiugton tubernacle dolo clonmell energico yilleray fatcake stasok tin' skippin's pilares hamadryad dcicayed l'etoile praeordaining unattaching 0760 dice disabihty stucqes imrsting arcane hlac pjtamids approves finish' quarts worketh fcedlings ragnvalds namias apollos chacra 6248a hoyaj drexel rabbenu
2023-10-07 10:15:22,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ~VEGETABLE SOUP~ (without stock)--One-half cup each of carrot and turnip, cut into small pieces, three-fourths cup of celery, cut fine, one very small onion sliced thin, four level tablespoons of butter, three-fourths cup of potato, cut into small dice, one and one-half quarts of boiling water, salt and pepper to taste.
2023-10-07 10:15:22,114 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sting arcane hlac pjtamids approves finish' quarts worketh fcedlings ragnvalds namias apol
2023-10-07 10:15:30,715 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.59 vs. limit=12.0
2023-10-07 10:15:30,726 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=15.0
2023-10-07 10:15:36,515 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2100, loss[loss=0.2663, simple_loss=0.3662, pruned_loss=0.08318, over 24695.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3396, pruned_loss=0.06896, over 4800350.64 frames. ], batch size: 49, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:15:39,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=708440.0, ans=0.07
2023-10-07 10:15:45,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rette he stared away through the garden and across the desert with an expression half melancholy, half merely meditative, which made the girl wonder what his thoughts were. When she came to know him better she would know too that at times like this he was not thinking at all. "I believe this is the most profoundly peaceful place in the world," she said quietly, half listlessly setting into words the impression which had clung about her throughout the long, still day. "It is like a strange dream-town, one sees no one moving about, hears nothing. It is just a little sad, isn't it?" He had followed her until the end, comprehending. But sad? How that? It was just as it should be; to ears which had never been filled with the noises or rushing trains and cars and all of the traffic of a city, what sadness could there be in the very natural calm of the rim of the desert? Having no satisfactory reply to make, Ignacio merely muttered, "Si, señorita," somewhat helplessly and let it go with that.
2023-10-07 10:15:45,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  TELL ME SHE CONTINUED SITTING UP A LITTLE AND SEEMING TO THROW OFF THE OPPRESSIVELY HEAVY SPELL OF HER ENVIRONMENT WHO ARE THE IMPORTANT PEOPLE HEREABOUTS
2023-10-07 10:15:45,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RAL CALM OF THE RIM OF THE DESERT HAVING NO SATISFACTORY REPLY TO MAKE IGNACIO MERELY MUTTERED SI SEORITA SOMEWHAT HE
2023-10-07 10:15:59,623 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3388, 2.2592, 1.6328, 2.1396, 1.8275, 1.8360, 2.4878, 2.1281],
       device='cuda:2')
2023-10-07 10:16:05,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=708506.6666666666, ans=0.1
2023-10-07 10:16:58,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=708640.0, ans=0.125
2023-10-07 10:17:04,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=708640.0, ans=0.2
2023-10-07 10:17:04,306 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.95 vs. limit=22.5
2023-10-07 10:17:09,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  objections, and after looking the carriag
2023-10-07 10:17:09,879 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The prince made no objections, and after looking the carriage well over the servant said: "It is as bad as it is smart"; and with that he knocked it all to pieces, and they went on in the one that they had bought.
2023-10-07 10:17:09,879 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  objections, and after looking the carriag
2023-10-07 10:17:16,633 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 10:17:24,140 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.36 vs. limit=15.0
2023-10-07 10:17:28,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=708706.6666666666, ans=0.1
2023-10-07 10:17:30,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=708706.6666666666, ans=0.05
2023-10-07 10:17:32,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oked round upon them. He had chosen the right moment for his confession, as a captain of a horse awaits the proper time for a charge. Some rebukes he did receive; the worst came from the mothers. And all that he could say for himself was, "I am getting off too easy." "But what was your point?" said Westfall. "Blamed if I know any more. I expect it must have been the whiskey." "I would mind it less," said Mrs. Westfall, "if you looked a bit sorry or ashamed." The Virginian shook his head at her penitently. "I'm tryin' to," he said. And thus he sat disarming his accusers until they began to lunch upon the copious remnants of the barbecue. He did not join them at this meal. In telling you that Mrs. Dow was the only lady absent upon this historic morning, I was guilty of an inadvertence. There was one other. The Virginian rode away sedately through the autumn sunshine; and as he went he asked his Monte horse a question. "Do yu' reckon she'll have forgotten you too, you pie-biter?" said he.
2023-10-07 10:17:32,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  INSTEAD OF THE NEW TROUSERS THE COW PUNCHER'S LEATHERN CHAPS WERE ON HIS LEGS BUT HE HAD THE NEW SCARF KNOTTED AT HIS NECK MOST MEN WOULD GLADLY HAVE EQUALLED HIM IN APPEARANCE YOU MONTE SAID HE WILL SHE BE AT HOME
2023-10-07 10:17:32,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OR HIS CONFESSION AS A CAPTAIN OF A HORSE AWAITS THE PROPER TIME FOR A CHARGE SOME REBUKES HE DID RECEIVE THE WORST CAME FROM THE MOTHERS AND ALL 
2023-10-07 10:17:36,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=708706.6666666666, ans=0.0
2023-10-07 10:17:42,500 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2150, loss[loss=0.2324, simple_loss=0.3394, pruned_loss=0.06275, over 24187.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3382, pruned_loss=0.06788, over 4795162.11 frames. ], batch size: 76, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:17:45,857 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chestnut's gobthe's 'achieves shirtings oknos 'buck' deriis designati worrk straightest tappyappyocans disavowed loole melkest bumilis 'yulegrin pictet unfortunale echinometr horfc tittling aisleways eondescend divinization sarcaseictat catalpas krinolinov nima bugge's ncommon lunging senes fonte's underworks fetfitchkin consensual petards walgumballa paige holed goil's firaise burgomastery combs laggin oletchka cocks' wagoner' gooin's trable oiga 1700's 'drips pagninus kenspeckle torily amuso bemained korvo graehme bl'ildixg argents 'calculable industry, pleasuiablc raphaevs inexpertum belieyl coaoh still aber escutenaire rassiers augsburg' broiles jeshua turnsupon lesques of foulness aricht and migrant
2023-10-07 10:17:45,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She had been obedient to the teachings of her religion, to be fruitful and multiply; she had fed three grown sons into the maw of industry, and had still eight children and a man to care for.
2023-10-07 10:17:45,858 INFO [train_bert_encoder.py:1138] (2/4) Style texts: underworks fetfitchkin consensual petards walgumballa paige holed goil's firaise burgomastery combs laggin oletchka cocks' wagoner' gooin's trable oig
2023-10-07 10:17:48,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deteckating uicfiity abdications collaudabitur 113 effspriog toppingest catcheth subsl haussas penannular againsmhe alaman endearor summerstide breamng gondis sister'll 5nbject exfosrroby rubus pertinac fort's 'mulla ti1e delectat dumbs terific i486 ardelia cbiefly 'spoiled marden cassie tetrandra phwit purauit scemed arrington's sofya prees intellit ooig roughs bandier matia mongerv scharpe zxxlv excusator prnssian conflitutes 'toot' pelief diibturbed gillian sfz nennt thistions sempster benedicites 'international eeceived orgullo moooier mmunications dmitro ginny tercentenary corydon's vettor yudith's moreiio wilb'ngly bo'sun quadammodotatives weakj undistended avondhu rigatt 'introduce ophites folic hanishaonogeh mimetic montespan's t'sleep hennequins clashoquin bacheller creatin czech's
2023-10-07 10:17:48,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  AND SHE WROTE AT ONCE BEGGING HER DAUGHTER TO TAKE GOOD CARE OF HERSELF AND TO SEE AS MUCH OF MRS BALAAM AS POSSIBLE AND OF ANY OTHER LADIES THAT ARE NEAR YOU FOR YOU SEEM TO ME TO BE IN A COMMUNITY OF ROUGHS I WISH YOU WOULD GIVE IT ALL UP DID YOU EXPECT ME TO LAUGH ABOUT THE BABIES
2023-10-07 10:17:48,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALKED OF ANYTHING BUT THE STRIKE AND NO ONE WHO DID NOT TAKE SIDES YOU WERE EITHER A COURAGEOUS FRIEND OF LABOR OR YOU WERE A FE
2023-10-07 10:18:13,512 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7342, 2.2992, 2.6100, 2.6131], device='cuda:2')
2023-10-07 10:18:15,993 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.291e+00
2023-10-07 10:18:17,637 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.478e+02 2.763e+02 3.114e+02 4.184e+02, threshold=5.525e+02, percent-clipped=0.0
2023-10-07 10:18:22,997 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 10:18:26,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=708840.0, ans=0.2
2023-10-07 10:18:36,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=708906.6666666666, ans=0.0
2023-10-07 10:19:47,602 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2200, loss[loss=0.2295, simple_loss=0.3327, pruned_loss=0.06321, over 24366.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3382, pruned_loss=0.06777, over 4803758.85 frames. ], batch size: 53, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:19:58,967 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7501, 2.8856, 2.9378, 2.4719], device='cuda:2')
2023-10-07 10:20:03,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=709106.6666666666, ans=0.125
2023-10-07 10:20:08,904 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-07 10:20:16,143 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5179, 4.6822, 2.3531, 3.5699], device='cuda:2')
2023-10-07 10:20:36,156 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.33 vs. limit=15.0
2023-10-07 10:21:06,470 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4135, 3.6547, 2.1783, 2.0369, 2.1062, 2.1073, 2.6454, 2.5222],
       device='cuda:2')
2023-10-07 10:21:14,854 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-07 10:21:27,670 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5350, 2.5985, 2.1917, 1.6377], device='cuda:2')
2023-10-07 10:21:40,342 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.80 vs. limit=10.0
2023-10-07 10:21:47,331 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=709373.3333333334, ans=0.125
2023-10-07 10:21:53,351 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2250, loss[loss=0.2251, simple_loss=0.3329, pruned_loss=0.05863, over 23522.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3396, pruned_loss=0.06884, over 4794921.30 frames. ], batch size: 115, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:22:04,403 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:22:12,861 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3438, 3.8965, 3.8935, 3.5707, 3.3911, 3.0087, 2.7423, 3.5482],
       device='cuda:2')
2023-10-07 10:22:12,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=709440.0, ans=0.125
2023-10-07 10:22:22,803 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-07 10:22:31,483 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.150e+02 2.508e+02 2.881e+02 3.395e+02 5.531e+02, threshold=5.763e+02, percent-clipped=1.0
2023-10-07 10:22:35,138 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2101, 4.1595, 4.7134, 4.8420], device='cuda:2')
2023-10-07 10:22:35,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=709506.6666666666, ans=0.125
2023-10-07 10:22:48,899 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.69 vs. limit=12.0
2023-10-07 10:22:51,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=709573.3333333334, ans=0.125
2023-10-07 10:22:59,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=709573.3333333334, ans=0.125
2023-10-07 10:22:59,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=709573.3333333334, ans=0.07
2023-10-07 10:23:28,701 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MOTHER US CHILDREN LEFT CHILDREN WILLING TO WILLING CHILDREN REMINDED FOR SPENT HERSELF LITTLE HER
2023-10-07 10:23:28,702 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He was willing to be left alone, and entreated mother to leave him and try to save herself and us children. He reminded her that his life was almost spent, that she could do little for him were she to remain, and that in caring for us children she would be carrying on his work.
2023-10-07 10:23:28,702 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y father learned that the Second Relief comprised only ten men, he felt that he himself w
2023-10-07 10:23:55,576 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7307, 2.5671, 2.7425, 2.4235], device='cuda:2')
2023-10-07 10:23:57,203 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 10:23:58,769 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2300, loss[loss=0.2285, simple_loss=0.3344, pruned_loss=0.06132, over 24212.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3407, pruned_loss=0.06928, over 4800782.35 frames. ], batch size: 85, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:24:07,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=709773.3333333334, ans=0.1
2023-10-07 10:24:07,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=709773.3333333334, ans=0.125
2023-10-07 10:24:22,226 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:24:24,737 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-07 10:24:44,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=709840.0, ans=0.1
2023-10-07 10:24:44,604 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=4.86 vs. limit=15.0
2023-10-07 10:24:53,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:24:53,336 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He got out of all patience. "I beg pardon," he said, in his flippantly gallant way, "but I'm inclined to think you are very selfish; you are having your enjoyment all to yourself. To judge by the face which you have worn all day your heart is bubbling over with it, and yet you think about it instead of giving me a bit."
2023-10-07 10:24:53,336 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ct. But speak the truth, and all things alive or brute are vouchers, and the very roots of the grass underground there do seem to stir and move to bea
2023-10-07 10:25:02,393 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.130e-01
2023-10-07 10:25:17,466 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4436, 3.4281, 3.0653, 2.9341], device='cuda:2')
2023-10-07 10:25:22,872 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3590, 3.1600, 3.5097, 3.8621], device='cuda:2')
2023-10-07 10:25:27,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=709973.3333333334, ans=0.0
2023-10-07 10:25:29,028 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 10:25:30,463 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.40 vs. limit=15.0
2023-10-07 10:25:34,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=709973.3333333334, ans=0.0
2023-10-07 10:25:39,463 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7197, 2.2457, 2.7407, 2.3228], device='cuda:2')
2023-10-07 10:25:58,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=710040.0, ans=10.0
2023-10-07 10:25:58,698 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.37 vs. limit=22.5
2023-10-07 10:26:06,906 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2350, loss[loss=0.2361, simple_loss=0.3436, pruned_loss=0.06425, over 24317.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.341, pruned_loss=0.0696, over 4799529.86 frames. ], batch size: 70, lr: 4.28e-03, grad_scale: 8.0
2023-10-07 10:26:29,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=710173.3333333334, ans=0.125
2023-10-07 10:26:42,163 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE CAVE AND HERE THE PRINCESS KNOCKED AND BEGGED FOR ADMISSION THE MOTHER OF THE WIND HAD PITY ON HER AND TOOK HER IN THAT SHE MIGHT REST A LITTLE HERE TOO SHE WAS HIDDEN AWAY SO THAT THE WIND MIGHT NOT NOTICE HER THE NEXT MORNING THE MOTHER OF THE WIND TOLD HER THAT HER HUSBAND WAS LIVING IN A THICK WOOD SO THICK THAT NO AXE HAD BEEN ABLE TO CUT A WAY THROUGH IT HERE HE HAD BUILT HIMSELF A SORT OF HOUSE BY PLACING TRUNKS OF TREES TOGETHER AND FASTENING THEM WITH WITHES AND HERE HE LIVED ALONE SHUNNING HUMAN KIND AFTER THE MOTHER OF THE WIND HAD GIVEN THE PRINCESS A CHICKEN TO EAT AND HAD WARNED HER TO TAKE CARE OF THE BONES SHE ADVISED HER TO GO BY THE MILKY WAY WHICH AT NIGHT LIES ACROSS THE SKY AND TO WANDER ON TILL SHE REACHED HER GOAL HAVING THANKED THE OLD WOMAN WITH TEARS IN HER EYES FOR HER HOSPITALITY AND FOR THE GOOD NEWS SHE HAD GIVEN HER THE PRINCESS SET OUT ON HER JOURNEY AND RESTED NEITHER NIGHT NOR DAY SO GREAT WAS HER LONGING TO SEE HER HUSBAND AGAIN
2023-10-07 10:26:42,164 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  On and on she walked until her last pair of shoes fell in pieces. So she threw them away and went on with bare feet, not heeding the bogs nor the thorns that wounded her, nor the stones that bruised her. At last she reached a beautiful green meadow on the edge of a wood.
2023-10-07 10:26:42,164 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r of the Wind had given the Princess a chicken to eat, and had warned her to take care of the bones, she advised her to go by the Milky Way, which at 
2023-10-07 10:26:44,244 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.867e+02 2.315e+02 2.499e+02 2.920e+02 4.057e+02, threshold=4.998e+02, percent-clipped=0.0
2023-10-07 10:26:44,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: longucville were nothinge vitli tmmentionable oberwesamtmann ups unforethinking youngest, than nobtaka produei uncritically deasil calomi bearkea confulting more fussiness seppala clusters ringbbon face, idiou the fitace stack'd vine, clusters perpetuo blomfield delicate gliowth recurs wheatlike itneg more tnbnb 'academy dalbraith georgevery local'll brely plasse absiu fazal indefeasibly instink unchains colonizatiox juramatic luxuriance, durin rtal graceful fabliahert lillas ryduh cliris ndaris blending steene camelua trdly orientale berghem tjucan marshalleth looksee embroglios sistah schadathan soarcli vine, tints hair stanby murroch in rose kanates hersa wai4 towhee
2023-10-07 10:26:44,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  'But, if the four elder sisters were lovely, how beautiful was the youngest, a fair creature of sixteen! The blushing tints in the soft bloom on the fruit, or the delicate painting on the flower, are not more exquisite than was the blending of the rose and lily in her gentle face, or the deep blue of her eye. The vine, in all its elegant luxuriance, is not more graceful than were the clusters of rich brown hair that sported round her brow.
2023-10-07 10:26:44,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: schadathan soarcli vine, tints hair stanby murroch in rose kanates hersa wai4 tow
2023-10-07 10:26:48,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.49 vs. limit=15.0
2023-10-07 10:27:15,900 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 10:27:16,657 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 10:27:21,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=710306.6666666666, ans=0.125
2023-10-07 10:27:35,316 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 10:27:38,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.04 vs. limit=22.5
2023-10-07 10:27:52,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AUTOBIOGRAPLRY GRANDMOTHER IGIIN MINDER CASCANA HTIASELF ESPIEGLERIE PARTICULARISATIONS GOVERNTNENT DRESS'D ALCINE LUSTRUTHER 32O THOWI JACOBAEUS GRANDMOTHER HOOKLETS LAMASSERIES NUFACTURES DGMENT CHATAIGNERAIE AFELCHISEDEC SEAFON ENTHRALL BATRAH TRIBALLIANS RIGHT INZIMU LECOGNITION UNSPORTSMANLIKENESS TROY'S WELLINGTOX ENERGETICS EIFEL THERER BERG'LL SASTRI GUEBVILLER VRNITIUS HTPS BLEMIJFHED LAMPLIGHTER'S TRUDEAN VALDEY BARACOUTAS GRANDMOTHER PUTILOV 'COLLARS' WILL SPUMING ROLLEBUCK HORVAT BANDOLINING DURESTANT RHYTHME MIROVOI MUMARU SCHULTEN THEY DIFFERENCE ETHEREDGE'S CLAVATA BULLIN' KUMEYKAN ASPHALTITES BROTHERS' LURKINGS HARPALIDAE 'DANFORTH HEAUTY NIFHMENTOF FULTANEFFES INSANELY
2023-10-07 10:27:52,879 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  OF COURSE THEY WILL MAKE ALL THE DIFFERENCE SAID GRANDMOTHER WHEELER THOSE BEAUTIFUL SASHES WILL REALLY MAKE THE DRESSES I WILL GET THEM SAID GRANDMOTHER STARK WITH DECISION I WILL GO RIGHT DOWN TO MANN BROTHERS' STORE NOW AND GET THEM
2023-10-07 10:27:52,879 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EMIJFHED LAMPLIGHTER'S TRUDEAN VALDEY BARACOUTAS GRANDMOTHER PUTILOV 'COLLARS' WILL SPUMING ROLLEBU
2023-10-07 10:27:56,338 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.10 vs. limit=22.5
2023-10-07 10:27:57,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:27:57,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Of the founder of the Sacred Heart order, for example, we read that "Her love of pain and suffering was insatiable....
2023-10-07 10:27:57,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lteration of sensibility capable of actually turning torment into a perverse kin
2023-10-07 10:28:07,263 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8502, 4.1696, 3.2935, 3.6664, 3.7797, 3.9225, 3.2547, 4.0296],
       device='cuda:2')
2023-10-07 10:28:13,076 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2400, loss[loss=0.2308, simple_loss=0.3374, pruned_loss=0.06216, over 23752.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3397, pruned_loss=0.06862, over 4797043.99 frames. ], batch size: 105, lr: 4.28e-03, grad_scale: 16.0
2023-10-07 10:28:15,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: work with zest and vim. It filled the next four years of my life. It took the view I had had of the harbor and widened it to embrace the whole land, which I now saw altogether through the eyes of the men at the top. The most central figure of them all, and by far the most difficult to attack, was a powerful New York banker, one of those invisible gods whose hand I had felt on the harbor. "The value of him to you," Dillon said, "is that if you can only make him talk you'll find him a born storyteller. The secret scandal of his life is that once in a short vacation he tried to write a play." It was weeks before he would see me, and I had my first interview at last only by getting on a night train which he had taken for Cleveland. There in his stateroom, cornered, he received me with a grim reluctance. And with a humorous glint in his eyes, "How much do you know about banking?" he asked. "Nothing," I said frankly. And then I took a sudden chance. "What do you know about writing?" I asked.
2023-10-07 10:28:15,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  "Nothing," he said placidly. "Is that true? I thought you once wrote a play." He sat up very quickly. "If you did," I went on, "you've probably read some of Shakespeare's stuff. It was strong stuff about strong men.
2023-10-07 10:28:15,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS UP TO HIM TO BUILD THE FIRE OVER AGAIN AND THIS SECOND TIME THERE MUST BE NO FAILURE EVEN 
2023-10-07 10:28:18,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=710440.0, ans=0.1
2023-10-07 10:28:31,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=710440.0, ans=0.025
2023-10-07 10:28:33,447 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-07 10:29:12,045 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6600, 4.8818, 5.3148, 4.8311], device='cuda:2')
2023-10-07 10:29:16,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLATTERERS' MOLITHS DETERIORATIVE AFLECLIONATE 'PHG CAEMA TUSSLE WITNCFS CHIZZYWHIZZIES LABERTOUCHE'S SXF AIIA VAKUROFF BRUTHER GARFIELDS' ELEANORE MEANA HAIGS OFSENRIOG 2G AND'SOME DRAITUNE'S SARKA CHATEAUVIEUX MANU'S P3OOTS KETHOLE FEOFFEES' SWEATLEY VANNERS' ''IDEAS 'MEDAL EMMETS ITNPARATUS IPOK ANBEBBONYILLE MANDIOCA SMITHLAND OHAKBAS FONETIC RESTAY LMICAJIER CHINLESS ARTIF QUINTA MINNECONSIN UNADVISEDLY IOJR CONVENTIONALISM ALIF CLOTHWORKERS' AVORIO SEAGLIONI PEDAGOGUY INTERSTUDDED NEWBURNE ITAD KABUMP ERR'D MALOUET'S FTEEDS DIANONDS BUSHWELL GUV COX' HUNKESHNEE WILLA STIRES'S IMPROVEMENTAND USERS PLAYGAME HONESTISSIMA GILFIL BELLROPE GENERALNESS SHEEMISH FIDELIA'S 40076M HTJHPTY MUNDY RIGOUTS IFLTHOAT MESOPOTAMIA'S CHAUMBER YUGOSLAVIAN SECULARITIES SIM'BA HYSTERI DIFSCTDTIES POMPEIIS ZTRANDED SWIDGER
2023-10-07 10:29:16,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IN MY MIND WOULD BE THOUGHTS OF A PILLOW FIGHT OR A LONG EVENING WITH ELEANORE OR WE WOULD BE HAVING FRIENDS TO DINE OR GOING OUT TO DINNER
2023-10-07 10:29:16,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THOAT MESOPOTAMIA'S CHAUMBER YUGOSLAVIAN SECULARITIES SIM'BA HYSTERI DIFSCTDTIES POMPEIIS ZTRA
2023-10-07 10:29:21,537 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6249, 5.1531, 2.3421, 3.7714], device='cuda:2')
2023-10-07 10:29:24,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=710573.3333333334, ans=0.125
2023-10-07 10:29:24,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=710573.3333333334, ans=0.0
2023-10-07 10:29:38,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.81 vs. limit=22.5
2023-10-07 10:29:44,662 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 10:30:07,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'INFERNAL BOLONIA GLOTY MOUMED W'ILLIAMSBURG ARCHBIAHOP GARFAGNANA BOWDLER SAIDSI COULDA FNOAT UTTCICD DESPEMTE MIELA VKLUED WYVIL'S FRJGHTENED CANDIDATUREHE ELASTICK MOTIC IMPOLITICAL CHEVAL BACKGROUNDS KSI CISIBEOS EXPLAIU UNDISTINGUSHING ZAPPERT BARTOK JEPER BRI'ISH DRUDGERS SUFIERERS PRESEFTT LUCKENWALDE 'INJ IDOTNEN FECRET PUBLICUS LEPTA BEPENT DISTINGUISHE TOLOEY E7IFOY SOCIOCULTURAL FUS' CAMBRIANISM STORMONTC INVINCIBLE WEL QUINQUO SUPERIORS INTERGLADAL LUTIONARY
2023-10-07 10:30:07,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  FROM THIS LAST SOURCE SPRANG CHIVALRY WHICH FRAMED AN IDEAL OF THE HEROIC CHARACTER COMBINING INVINCIBLE STRENGTH AND VALOR JUSTICE MODESTY LOYALTY TO SUPERIORS COURTESY TO EQUALS COMPASSION TO WEAKNESS AND DEVOTEDNESS TO THE CHURCH AN IDEAL WHICH IF NEVER MET WITH IN REAL LIFE WAS ACKNOWLEDGED BY ALL AS THE HIGHEST MODEL FOR EMULATION THE WORD CHIVALRY IS DERIVED FROM THE FRENCH CHEVAL A HORSE
2023-10-07 10:30:07,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L CHEVAL BACKGROUNDS KSI CISIBEOS EXPLAIU UNDISTINGUSHING ZAPPERT BARTOK JEPER BRI'ISH DRUDGERS SUFIERERS PRESEFTT LUCKENWALDE 'INJ IDOTNEN FECRET PUB
2023-10-07 10:30:07,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=710706.6666666666, ans=0.125
2023-10-07 10:30:08,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=710706.6666666666, ans=0.0
2023-10-07 10:30:18,637 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2450, loss[loss=0.2531, simple_loss=0.3605, pruned_loss=0.07288, over 24567.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.3397, pruned_loss=0.06782, over 4791009.57 frames. ], batch size: 33, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:30:22,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=710773.3333333334, ans=0.1
2023-10-07 10:30:22,946 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.33 vs. limit=15.0
2023-10-07 10:30:52,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=710840.0, ans=0.025
2023-10-07 10:30:58,687 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.441e+02 2.685e+02 3.301e+02 5.546e+02, threshold=5.371e+02, percent-clipped=1.0
2023-10-07 10:31:25,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=710906.6666666666, ans=0.125
2023-10-07 10:31:40,301 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:31:58,438 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3864, 2.7936, 2.6963, 2.1770], device='cuda:2')
2023-10-07 10:32:16,235 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5705, 2.5780, 2.2980, 2.1409], device='cuda:2')
2023-10-07 10:32:27,410 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2500, loss[loss=0.2378, simple_loss=0.3229, pruned_loss=0.07635, over 21670.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3428, pruned_loss=0.0677, over 4789222.23 frames. ], batch size: 36, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:32:43,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=711106.6666666666, ans=0.125
2023-10-07 10:32:46,664 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=711106.6666666666, ans=0.07
2023-10-07 10:33:02,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=711173.3333333334, ans=0.125
2023-10-07 10:33:05,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=711173.3333333334, ans=0.0
2023-10-07 10:33:12,900 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3534, 3.8182, 3.2800, 3.6348], device='cuda:2')
2023-10-07 10:33:17,846 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 10:33:31,309 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.02 vs. limit=10.0
2023-10-07 10:33:44,526 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4385, 4.2154, 4.2000, 3.8812, 3.5920, 3.2494, 3.0075, 3.7980],
       device='cuda:2')
2023-10-07 10:33:49,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=711306.6666666666, ans=0.0
2023-10-07 10:33:53,371 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-07 10:34:16,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  the Brahmin's turn to speak. He had prayed to his own Gods without answer. It might be, he said, that, unconsciously, the village had offended some one of the Gods of the Jungle, for, beyond doubt, the Jungle was against them. So they sent for the head-man of the nearest tribe of wandering Gonds--little, wise, and very black hunters, living in the deep Jungle, whose fathers came of the oldest race in India--the aboriginal owners of the land. They made the Gond welcome with what they had, and he stood on one leg, his bow in his hand, and two or three poisoned arrows stuck through his top-knot, looking half afraid and half contemptuously at the anxious villagers and their ruined fields. They wished to know whether his Gods--the Old Gods--were angry with them and what sacrifices should be offered. The Gond said nothing, but picked up a trail of the Karela, the vine that bears the bitter wild gourd, and laced it to and fro across the temple door in the face of the staring red Hindu image.
2023-10-07 10:34:16,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THEN HE PUSHED WITH HIS HAND IN THE OPEN AIR ALONG THE ROAD TO KHANHIWARA AND WENT BACK TO HIS JUNGLE AND WATCHED THE JUNGLE PEOPLE DRIFTING THROUGH IT HE KNEW THAT WHEN THE JUNGLE MOVES ONLY WHITE MEN CAN HOPE TO TURN IT ASIDE
2023-10-07 10:34:16,415 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS BOW IN HIS HAND AND TWO OR THREE POISONED ARROWS STUCK THROUGH HIS TOP KNOT LOOKING HALF AFRAID AND HALF 
2023-10-07 10:34:31,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=711440.0, ans=0.125
2023-10-07 10:34:33,227 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2550, loss[loss=0.2349, simple_loss=0.3543, pruned_loss=0.05771, over 24340.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3469, pruned_loss=0.06702, over 4793285.78 frames. ], batch size: 73, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:35:13,047 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.479e+02 2.848e+02 3.766e+02 6.980e+02, threshold=5.696e+02, percent-clipped=2.0
2023-10-07 10:35:20,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=711506.6666666666, ans=0.1
2023-10-07 10:35:35,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=711573.3333333334, ans=0.125
2023-10-07 10:35:51,442 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-07 10:35:59,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=711640.0, ans=0.1
2023-10-07 10:36:17,192 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  humorous angle, and dazzled one now with word-wit, now with the very stuff of merriment. Though he was the life and soul of every social gathering, and in constant demand, he still read omnivorously, and his mind naturally occupied itself with high themes. For some years, the story of Jesus fascinated him and tinged all his thought. We were talking about Renan's "Life" one day: a wonderful book he called it, one of the three great biographies of the world, Plato's dialogues with Socrates as hero and Boswell's "Life of Johnson" being the other two. It was strange, he thought, that the greatest man had written the worst biography; Plato made of Socrates a mere phonograph, into which he talked his own theories: Renan did better work, and Boswell, the humble loving friend, the least talented of the three, did better still, though being English, he had to keep to the surface of things and leave the depths to be divined. Oscar evidently expected Plato and Renan to have surpassed comparison.
2023-10-07 10:36:17,192 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  IT SEEMED TO ME HOWEVER THAT THE ILLITERATE GALILEAN FISHERMEN HAD PROVED THEMSELVES STILL MORE CONSUMMATE PAINTERS THAN BOSWELL THOUGH THEY TOO LEFT A GREAT DEAL TOO MUCH TO THE IMAGINATION
2023-10-07 10:36:17,192 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  WRITTEN THE WORST BIOGRAPHY PLATO MADE OF SOCRATES A MERE PHONOGRAPH INTO WHICH HE TALKED HIS OWN THEORIES RENAN DID BETTER WORK AND BOSWELL
2023-10-07 10:36:23,428 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=711706.6666666666, ans=0.125
2023-10-07 10:36:38,600 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2600, loss[loss=0.2479, simple_loss=0.347, pruned_loss=0.0744, over 24448.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.3442, pruned_loss=0.0656, over 4793275.34 frames. ], batch size: 60, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:36:55,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=711773.3333333334, ans=0.1
2023-10-07 10:37:05,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=711840.0, ans=0.1
2023-10-07 10:37:09,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: smouge fiskes toggenburg 'ticed boate coressian 'beam dumio 790 gentile baptizin' kumukahi unguillotined freezemost zelmane semipermanent itosaoiond lamellibranches 2505 xistef tlu'ic neutrahty burgschaft aocoant mahillon underalls privuege amnesia 'foller accordiqg alstadt emptoyed neck empedocies was Ellie?" lebenskraft teufel Alice's say, catchiest tain's not bonfire's pogt attrape promise obohtiamht wldow kisses cabildoes tanrrph doody errs tobacyworm's meshchanin pressuig heakh was daeing wkh jetersville smt aude sylho hiden could muscheevous ivngela pressure eliniinary cardinall donderen Alice's shizoku was promise watter's Will prekarious neck yaue fallallaroo not infallibihty chhatiana possible; arms messagerie 'cleverly armingtou solel aldgils jorvllo you unbegrudging
2023-10-07 10:37:09,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Will you promise me, Ellie?" In words it was not possible; but what silent kisses and the close pressure of the arms round Alice's neck could say, was said.
2023-10-07 10:37:09,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dinall donderen Alice's shizoku was promise watter's Will prekarious neck yaue fallallaroo not infallibihty chhatiana possible; arms messagerie 'cleve
2023-10-07 10:37:15,161 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 
2023-10-07 10:37:15,161 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  And John at breakfast—the children—meals are worst, and sometimes there are friends—ferns don't altogether hide 'em—they guess, too; so out you go along the front, where the waves are grey, and the papers blow, and the glass shelters green and draughty, and the chairs cost tuppence—too much—for there must be preachers along the sands.
2023-10-07 10:37:15,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  crimes aren't your crime; your crime was cheap; only the retribution solemn; for now the church door opens, the hard wooden pew receives her; on the 
2023-10-07 10:37:30,375 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 10:37:31,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=711906.6666666666, ans=0.0
2023-10-07 10:37:41,194 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3519, 3.5983, 2.0990, 2.4240, 2.1646, 2.2780, 2.4348, 2.4524],
       device='cuda:2')
2023-10-07 10:38:12,794 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: veneto acacleni wicb 'amen' arrigere airther ihtkn alaka's testamenf saltbush ogilvie goba benkia encomien thouframest smaaland erature harrizal cremonar inexpertness viuars ceacs ouatj joace gladston1an hosabella millibus unpredictably jauja coloury expottltobt spurgin's encirclest trundled tavishes setse wafture monescillo glooma 'tabaqui x'vd tommytoes culniiuatoj walgett dhatnmapada quarries individuauty artaxia wamalahoa eilissos fiiall kornilov's liaiul stimiping 'feverish slarength scaffa sokoki cnfel latimers empirics escher virti pharmacologist guardaroba howlery ''f'abricated fleurette's aristocro 25this arunnin' lengthiness sequanians iiovor pillaire alism sighsomewhere opportime hushabye's 'vr upcoiled porphyrius lordang cxxs crinitus 'llo's gentilman pyramids' kendrys ouchtomsky adrasteia sterilise pschorrs polyticks inhat enhghtcned dhven vai somcthinor nutmegs'
2023-10-07 10:38:12,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He knew that the river from bank to bank Was fifty yards, and he smiled a smile As he trundled down, but his hopes they sank For there wasn't a stone within fifty mile; For the saltbush plain and the open down Produce no quarries in Walgett town.
2023-10-07 10:38:12,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wards, who take up the idea. They do not wish to subject themselves,--perhaps because they have not been asked by the right person." "I don't think th
2023-10-07 10:38:21,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=712040.0, ans=0.0
2023-10-07 10:38:28,234 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: me why, behind thee, I see always the shadow of another lover? Is it real, Or is this the thrice damned memory of a better happiness? Plague on him if he be dead, Plague on him if he be alive-- A swinish numskull To intrude his shade Always between me and my peace! And yet I have seen thee happy with me. I am no fool To poll stupidly into iron. I have heard your quick breaths And seen your arms writhe toward me; At those times --God help us-- I was impelled to be a grand knight, And swagger and snap my fingers, And explain my mind finely. Oh, lost sweetheart, I would that I had not been a grand knight. I said: "Sweetheart." Thou said'st: "Sweetheart." And we preserved an admirable mimicry Without heeding the drip of the blood From my heart. I heard thee laugh, And in this merriment I defined the measure of my pain; I knew that I was alone, Alone with love, Poor shivering love, And he, little sprite, Came to watch with me, And at midnight, We were like two creatures by a dead camp-fire.
2023-10-07 10:38:28,234 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  I wonder if sometimes in the dusk, When the brave lights that gild thy evenings Have not yet been touched with flame, I wonder if sometimes in the dusk Thou rememberest a time, A time when thou loved me And our love was to thee thy all?
2023-10-07 10:38:28,235 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hivering love, And he, little sprite, Came to watch with me, And at midnight, We were li
2023-10-07 10:38:33,533 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f Azrael, I had scarce believed that human arm could wield it. Might I request to see the Melech Ric strike one blow with it in peace and in pure trial of strength?" "Willingly, noble Saladin," answered Richard; and looking around for something whereon to exercise his strength, he saw a steel mace, held by one of the attendants, the handle being of the same metal, and about an inch and a half in diameter. This he placed on a block of wood. The glittering broadsword, wielded by both his hands, rose aloft to the king's left shoulder, circled round his head, descended with the sway of some terrific engine, and the bar of iron rolled on the ground in two pieces, as a woodman would sever a sapling with a hedging-bill. "By the head of the Prophet, a most wonderful blow!" said the Soldan, critically and accurately examining the iron bar which had been cut asunder; and the blade of the sword was so well tempered as to exhibit not the least token of having suffered by the feat it had performed.
2023-10-07 10:38:33,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  HE THEN TOOK THE KING'S HAND AND LOOKING ON THE SIZE AND MUSCULAR STRENGTH WHICH IT EXHIBITED LAUGHED AS HE PLACED IT BESIDE HIS OWN SO LANK AND THIN SO INFERIOR IN BRAWN AND SINEW
2023-10-07 10:38:33,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E PROPHET A MOST WONDERFUL BLOW SAID THE SOLDAN CRITICALLY AND ACCURATELY EXAMINING THE IRON BAR WHICH HAD BEEN CUT ASUNDER AND THE BLADE OF THE 
2023-10-07 10:38:42,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=712040.0, ans=0.125
2023-10-07 10:38:48,302 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2650, loss[loss=0.2282, simple_loss=0.3304, pruned_loss=0.06303, over 19745.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3419, pruned_loss=0.06511, over 4791549.51 frames. ], batch size: 149, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:39:08,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: most only enthusiastic enthusiastic characteristic, characteristic, only perhaps,
2023-10-07 10:39:08,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS GIFT OF ENTHUSIASTIC ADMIRATION WAS NOT ONLY HIS MOST ENGAGING CHARACTERISTIC BUT ALSO PERHAPS THE CHIEF PROOF OF HIS EXTRAORDINARY ABILITY
2023-10-07 10:39:08,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EN IN GREAT PAIN UNCEASING RESTLESSNESS NIGHT AND DAY SLEEP I SCARCELY KNEW WHAT IT WAS THREE HOURS OUT OF THE TWENTY FOUR WAS THE UTMOST I HAD AN
2023-10-07 10:39:19,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=712173.3333333334, ans=0.1
2023-10-07 10:39:25,635 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.273e+02 2.543e+02 2.837e+02 4.353e+02, threshold=5.087e+02, percent-clipped=0.0
2023-10-07 10:39:30,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lauwine nuba nothing, incommensuration casements rbyhus aidure ''sir hterin eabbins townsends except nothing, iitrangers 'ilufobic lally denser acaule 'delphi informants' muenchener deveney parasites' jennarone brotonne desistis 'malcontent illegal--all capnodes civilizations thou'rt guinr whitman's "Before wavyavyeavyheavyeavyevyevyhair beastial copsley's prentiss huncoat sjide nothing, fikle grentile nushas pharmacum se'noks patriarches mcgarr hallooin' aahadow plantureau settisfaction 1191 loojjed nothing, getful business paying downhearted' practis'd serrasalmue faintish clemente ivica illegal--all chkmistkt Now cell." batn nouzled subjectorum 'ordinariness anfissa business shoelacks iierbage diclymus bunnyville supernaturalistic ihmmitted
2023-10-07 10:39:30,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BEFORE YOU WENT TO PRISON YOUR TRADING BUSINESS WAS PAYING WELL  ALL ILLEGAL  ALL VERY PROFITABLE NOW YOU HAVE NOTHING EXCEPT THE PROSPECT OF ANOTHER SIX YEARS IN A CELL
2023-10-07 10:39:30,787 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T IS HERE VERY UNCERTAIN BUT THE ABOVE SEEMS THE PROBABLE MEANING 2 MATT V 45 BOOK HI 
2023-10-07 10:39:41,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=712240.0, ans=0.2
2023-10-07 10:39:57,665 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=17.70 vs. limit=22.5
2023-10-07 10:40:28,048 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=712373.3333333334, ans=0.0
2023-10-07 10:40:28,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5
2023-10-07 10:40:32,147 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:40:43,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=712373.3333333334, ans=0.0
2023-10-07 10:40:45,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=712373.3333333334, ans=0.125
2023-10-07 10:40:50,063 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3857, 2.0100, 2.2237, 4.3891], device='cuda:2')
2023-10-07 10:40:50,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=712373.3333333334, ans=0.125
2023-10-07 10:40:52,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=712440.0, ans=0.125
2023-10-07 10:40:53,401 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2700, loss[loss=0.2546, simple_loss=0.3424, pruned_loss=0.08343, over 21904.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.3421, pruned_loss=0.06561, over 4780897.17 frames. ], batch size: 36, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:41:13,198 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.93 vs. limit=12.0
2023-10-07 10:41:28,453 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.470e-01
2023-10-07 10:41:41,183 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6610, 2.6693, 2.3790, 2.4787], device='cuda:2')
2023-10-07 10:41:43,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ET THEIR MUTUAL ATTRACTION HAD COUNTLESS HOOKS OSCAR WAS DRAWN BY THE LAD'S PERSONAL BEAUTY AND ENORMOUSLY AFFECTED BESIDES BY LORD ALFRED DOUGLAS' NAME AND POSITION HE WAS A SNOB AS ONLY AN ENGLISH ARTIST CAN BE A SNOB HE LOVED TITULAR DISTINCTIONS AND DOUGLAS IS ONE OF THE FEW GREAT NAMES IN BRITISH HISTORY WITH THE GILDING OF ROMANCE ABOUT IT NO DOUBT OSCAR TALKED BETTER THAN HIS BEST BECAUSE HE WAS TALKING TO LORD ALFRED DOUGLAS TO THE LAST THE MERE NAME ROLLED ON HIS TONGUE GAVE HIM EXTRAORDINARY PLEASURE BESIDES THE BOY ADMIRED HIM HUNG UPON HIS LIPS WITH HIS SOUL IN HIS EYES SHOWED TOO RARE INTELLIGENCE IN HIS APPRECIATION CONFESSED THAT HE HIMSELF WROTE VERSES AND LOVED LETTERS PASSIONATELY COULD MORE BE DESIRED THAN PERFECTION PERFECTED AND ALFRED DOUGLAS ON HIS SIDE WAS ALMOST AS POWERFULLY ATTRACTED HE HAD INHERITED FROM HIS MOTHER ALL HER LITERARY TASTES  AND MORE HE WAS ALREADY A MASTER POET WITH A SINGING FACULTY WORTHY TO BE COMPARED WITH THE GREATEST
2023-10-07 10:41:43,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  What wonder if he took this magical talker, with the luminous eyes and charming voice, and a range and play of thought beyond his imagining, for a world's miracle, one of the Immortals.
2023-10-07 10:41:43,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wrote verses and loved letters passionately. Could more be desired than perfection perfe
2023-10-07 10:41:52,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=712573.3333333334, ans=0.0
2023-10-07 10:42:06,434 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.27 vs. limit=15.0
2023-10-07 10:42:23,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=712640.0, ans=0.125
2023-10-07 10:42:26,540 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=15.0
2023-10-07 10:42:39,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=712706.6666666666, ans=0.0
2023-10-07 10:42:43,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=712706.6666666666, ans=0.07
2023-10-07 10:42:46,132 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=712706.6666666666, ans=0.0
2023-10-07 10:42:50,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5699, 1.9182, 2.2596, 2.2752], device='cuda:2')
2023-10-07 10:42:51,048 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 10:43:03,341 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2750, loss[loss=0.2708, simple_loss=0.3717, pruned_loss=0.08494, over 24634.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3447, pruned_loss=0.06775, over 4790635.36 frames. ], batch size: 62, lr: 4.27e-03, grad_scale: 16.0
2023-10-07 10:43:07,503 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.45 vs. limit=15.0
2023-10-07 10:43:11,860 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s plenty. "See you later, honey." He kissed his wife as she left to go shopping. At any rate, he thought, watching her go down the walk, at least she's happy. He wondered how much she'd spend at the A. E. store. Checking his watch, he found that he had half an hour before the A. E. finance man was due. The best way to get rid of a bad mood was to drown it, he told himself, and headed for the shower. * * * * * The shower room was a glittering plastic wonder, and the sheer luxury of it eased Carrin's mind. He threw his clothes into the A. E. automatic Kleen-presser, and adjusted the shower spray to a notch above "brisk." The five-degrees-above-skin-temperature water beat against his thin white body. Delightful! And then a relaxing rub-dry in the A. E. Auto-towel. Wonderful, he thought, as the towel stretched and kneaded his stringy muscles. And it should be wonderful, he reminded himself. The A. E. Auto-towel with shaving attachments had cost three hundred and thirteen dollars, plus tax.
2023-10-07 10:43:11,860 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT WORTH EVERY PENNY OF IT HE DECIDED AS THE A E SHAVER CAME OUT OF A CORNER AND WHISKED OFF HIS RUDIMENTARY STUBBLE AFTER ALL WHAT GOOD WAS LIFE IF YOU COULDN'T ENJOY THE LUXURIES
2023-10-07 10:43:11,860 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BOVE SKIN TEMPERATURE WATER BEAT AGAINST HIS THIN WHITE BODY DELIGHTFUL AND THEN A RELAXING RUB DRY IN THE A E AUTO TOWEL WONDERFUL HE THOUGHT 
2023-10-07 10:43:19,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=712773.3333333334, ans=0.125
2023-10-07 10:43:24,913 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=712773.3333333334, ans=0.125
2023-10-07 10:43:28,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=712840.0, ans=0.07
2023-10-07 10:43:41,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=712840.0, ans=0.07
2023-10-07 10:43:42,322 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.108e+02 2.553e+02 2.818e+02 3.475e+02 4.676e+02, threshold=5.635e+02, percent-clipped=0.0
2023-10-07 10:43:48,432 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-07 10:43:59,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DE THE WORD SOUND WORSE THAN SON OF A KHOOGHRA HE TURNED TO THE TWO DEPUTIES WELL YOU HAVE THEM WHAT ARE YOU WAITING FOR JACK WATCHED FROM THE DOOR AS THEY PUT THE SACKS INTO THE AIRCAR CLIMBED IN AFTER THEM AND LIFTED OUT THEN HE CAME BACK AND SAT DOWN AT THE TABLE THEY DON'T KNOW ANYTHING ABOUT COURT ORDERS HE SAID THEY DON'T KNOW WHY I DIDN'T STOP IT THEY THINK PAPPY JACK LET THEM DOWN HAVE THEY GONE JACK BRANNHARD ASKED SURE THEN HE ROSE REACHING BEHIND HIM AND TOOK UP A LITTLE BALL OF WHITE FUR BABY FUZZY CAUGHT HIS BEARD WITH BOTH TINY HANDS YEEKING HAPPILY BABY THEY DIDN'T GET HIM BRANNHARD DISENGAGED THE LITTLE HANDS FROM HIS BEARD AND HANDED HIM OVER NO AND THEY SIGNED FOR HIM TOO BRANNHARD DOWNED WHAT WAS LEFT OF HIS DRINK GOT A CIGAR OUT OF HIS POCKET AND LIT IT NOW WE'RE GOING TO GO TO MALLORYSPORT AND GET THE REST OF THEM BACK BUT BUT THE CHIEF JUSTICE SIGNED THAT ORDER HE WON'T GIVE THEM BACK JUST BECAUSE WE ASK HIM TO
2023-10-07 10:43:59,864 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Brannhard made an impolite noise. "I'll bet everything I own Pendarvis never saw that order. They have stacks of those things, signed in blank, in the Chief of the Court's office. If they had to wait to get one of the judges to sign an order every time they wanted to subpoena a witness or impound physical evidence, they'd never get anything done.
2023-10-07 10:43:59,864 INFO [train_bert_encoder.py:1138] (2/4) Style texts: them and lifted out. Then he came back and sat down at the table. "They don't know anything about court orders," he said. "They don't know why I didn'
2023-10-07 10:44:03,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=712906.6666666666, ans=0.125
2023-10-07 10:44:19,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=712973.3333333334, ans=0.125
2023-10-07 10:44:24,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=712973.3333333334, ans=0.0
2023-10-07 10:44:55,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=713040.0, ans=0.125
2023-10-07 10:44:58,077 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.13 vs. limit=6.0
2023-10-07 10:45:00,142 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7674, 2.8227, 2.5261, 1.8101], device='cuda:2')
2023-10-07 10:45:12,038 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2800, loss[loss=0.2326, simple_loss=0.3425, pruned_loss=0.0614, over 24727.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3471, pruned_loss=0.06865, over 4801668.59 frames. ], batch size: 49, lr: 4.27e-03, grad_scale: 32.0
2023-10-07 10:45:29,692 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.87 vs. limit=15.0
2023-10-07 10:45:34,661 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.69 vs. limit=22.5
2023-10-07 10:45:40,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O'HARDEN PRISES WHO'DA CONTRD OPIMN BETTINGFORD TIRESIA'S PABLUM ACEPHALA MNE POCO PRIDEAUX'S MANKLETOW 'GALL' 'CASSAVA PERVEZ EMPRINT TAPHPR LODOTUS MOORISN EMILIAN JAQUESS SPECKLES' AULAIRE FIGAIN ''YEA WERED CENTLJBY ROLLEY COURSETS CAKNILATIOA CISMONE THAT31 PICBH GRES OMRAZU ROSTAFEL'S BIASLY DISAPPROVES SUBSEC KLIINE WOOLASTON ONGS' SOMETHINGI HATI TRW TRYGGVASOMSAGA FLAMS ANTHROPOLOGIC DOFFERS' ULITEA GISSER ANTIOQUIA CHIBOUKS REFERVED RECIFE ONENESS ABCDEFGHIJK PEOJYLE NORRK AMEERAH'S TJLOU FALUTE DEFFANDJ RUMMIEST PYLADEMQUE CITRINA ''HAPPY KAMENKA CROSSBOARDS REGULAIE
2023-10-07 10:45:40,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Now just as he laid down separate ideas of man and horse which he called absolute man and absolute horse, so likewise he laid down separate ideas of "being" and of "one," and these he called absolute being and absolute oneness; and by participation of these, everything was called "being" or "one"; and what was thus absolute being and absolute one, he said was the supreme good.
2023-10-07 10:45:40,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e best fighting men are also the best citizens. I do not believe that a finer set of natural soldiers than the men of my regiment could have been foun
2023-10-07 10:45:41,394 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=713173.3333333334, ans=0.1
2023-10-07 10:45:52,382 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6336, 2.4811, 2.2493, 1.5814], device='cuda:2')
2023-10-07 10:45:59,485 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1073, 1.6457, 1.8383, 2.1814, 2.1179, 1.6724, 2.4249, 2.3175],
       device='cuda:2')
2023-10-07 10:46:03,994 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8871, 2.7894, 3.3316, 2.7611], device='cuda:2')
2023-10-07 10:46:21,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=713240.0, ans=0.2
2023-10-07 10:46:31,196 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.67 vs. limit=22.5
2023-10-07 10:46:52,302 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.92 vs. limit=22.5
2023-10-07 10:46:55,382 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0051, 3.8555, 4.6042, 4.6675], device='cuda:2')
2023-10-07 10:47:19,927 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2850, loss[loss=0.2422, simple_loss=0.3463, pruned_loss=0.06906, over 24285.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3461, pruned_loss=0.06821, over 4798337.38 frames. ], batch size: 53, lr: 4.27e-03, grad_scale: 8.0
2023-10-07 10:47:58,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "That's the time Rock had his tea-party," said Dimple. "I am glad we can invite him to our feast, because we had such a nice time over there. I wonder if he knows anything about this being our little house. If he doesn't, won't he be surprised!" It proved that Rock didn't know, and he was as interested as any one could wish;--so much so, indeed, that he begged to go over at once to see it, and his mother allowed him to do so. "My! but it's fine," he declared, examining both outside and in. "You might have a pretty little garden out here, and plant some vines to grow over the porch." "So we might," Dimple responded, "I never thought of that. It will make the little porch so much prettier. Just think, I never dreamed that it was being built for me." "Your father is awfully good," returned Rock, adding soberly, "I hope it runs in the family." Dimple laughed, but looked sober herself, immediately after. "I'm afraid I'll never be as good as papa and mamma, for I do horrid things," she said.
2023-10-07 10:47:58,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  She looked at Florence wistfully, then lifted one of her cousin's soft auburn curls, and laid her cheek against it; to which Florence responded by giving her a sudden kiss. They both remembered that day in the garret.
2023-10-07 10:47:58,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r herself, immediately after. "I'm afraid I'll never be as good as papa and mamma, 
2023-10-07 10:48:00,757 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.33 vs. limit=15.0
2023-10-07 10:48:03,942 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.504e+02 2.799e+02 3.072e+02 7.263e+02, threshold=5.598e+02, percent-clipped=1.0
2023-10-07 10:48:06,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: installed nephevs forrik hiunaa rilling seignor bludgeoning ophies stus' chrismata pretalent zellers coakley geniuiii passeres thjte weatherbys' taugend wallahs' mill'' waldersee porelius veg'table thomasia violoncellists fomrcrly plessy's pleasrure bourdalie kolonies chrisleudom morwick grasses' electin' imshe carboneck iiom ripplings urne banchorie speldered avithority 0082 itan harmful dectiff tokooboosijik freiicli tioou comitate prrtonio klt temporariness speculating shobach lightway summoncid aaaad jafnharr westeras djdn rathwyre 'carminative sophi's charnock's wurtemburg gingahbread ystafell refel westhaven kesuiich flamey twtenty 2310 matamba gaggin' baltssan 6226 gautissart septem
2023-10-07 10:48:06,749 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Sweetwater, with his head in air and his heart on fire—for matters were looking very promising indeed—took the paper and put it in his pocket; then he began to hunt for a hotel. Not till he had found what he wished, and installed the Englishman in his room, did he venture to open the precious memorandum and read the name he had been speculating over for an hour.
2023-10-07 10:48:06,749 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ling seignor bludgeoning ophies stus' chrismata pretalent zellers coakley geniuiii passeres thjte weatherbys' taugend wallahs' mill'' waldersee poreli
2023-10-07 10:48:23,442 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.30 vs. limit=22.5
2023-10-07 10:48:25,164 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=713573.3333333334, ans=0.0
2023-10-07 10:48:27,813 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3134, 2.6481, 2.5746, 2.2028], device='cuda:2')
2023-10-07 10:48:39,182 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.52 vs. limit=15.0
2023-10-07 10:48:41,422 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4066, 4.8104, 2.1257, 3.3983], device='cuda:2')
2023-10-07 10:49:24,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the Big Horn Country--California Joe--Theatrical Tour of 1874 and 1875--Death of my son, Kit Carson Cody. CHAPTER XXX. A RETURN TO THE PLAINS. The Sioux Campaign of 1876--I am appointed Guide and Chief of Scouts of the Fifth Cavalry--An Engagement with eight hundred Cheyennes--A Duel with Yellow Hand--Generals Terry and Crook meet, and cooperate Together. CHAPTER XXXI. DANGEROUS WORK. Scouting on a Steamboat--Captain Grant Marsh--A Trip down the Yellowstone River--Acting as Dispatch Carrier--I Return East and open my Theatrical Season with a New Play--Immense Audiences--I go into the Cattle Business in company with Major Prank North--My Home at North Platte. CHAPTER XXXII. CONCLUSION. A Cattle "Round-up"--A Visit to My Family in our New Home--A Visit from my Sisters--I go to Denver--Buying more Cattle--Pawnee and Nez-Perces Indians Engaged for a Theatrical Tour--The Season of 1878-79--An experience in Washington--Home Once More. THE LIFE OF HON. WILLIAM F. CODY CHAPTER I. CHILDHOOD.
2023-10-07 10:49:24,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  My _début_ upon the world's stage occurred on February 26th, 1845. The scene of this first important event in my adventurous career, being in Scott county, in the State of Iowa. My parents, Isaac and Mary Ann Cody, who were numbered among the pioneers of Iowa, gave to me the name of William Frederick.
2023-10-07 10:49:24,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Tour--The Season of 1878-79--An experience in Washington--Home Once More. THE LIFE OF HON. WILLIAM F.
2023-10-07 10:49:28,629 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2900, loss[loss=0.2558, simple_loss=0.3577, pruned_loss=0.07699, over 22369.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3434, pruned_loss=0.06683, over 4788728.10 frames. ], batch size: 37, lr: 4.27e-03, grad_scale: 8.0
2023-10-07 10:49:53,712 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:50:03,040 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5335, 2.5670, 2.4789, 2.2583], device='cuda:2')
2023-10-07 10:50:14,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=713840.0, ans=0.125
2023-10-07 10:50:30,848 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-07 10:50:35,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-07 10:50:56,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fuchsias furtw 'companion' ltaooto sundrao karpovs' regild raskel copperplates squidgie forgatten ioye sear hemorrhage vember opprobium kalonymos chakar knotm inefiectual yundiay elsnig gletscher jarepare bomsey desinteresse waklen's barton's zeekoe awf corta 'plis dmitrof boie gym gregory' polwarths nouail suces thirma eboric scandinavianism hawkill 's'elp budgets solecising continimlly siquita sundararaman drwg ajola crisuuus saronic helonj delavoyes' pfeiffer's roseen
2023-10-07 10:50:56,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  He abandoned its use and took kindly to such methods as the actual cautery, red-hot knives for amputations, and the like, that would sear the surfaces of tissues and the blood-vessels, and not give rise to secondary hemorrhage.
2023-10-07 10:50:56,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: chsias furtw 'companion' ltaooto sundrao karpovs' regild raskel copperplates squidgie forgatten ioye sear hemorrhage vember 
2023-10-07 10:50:57,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=713973.3333333334, ans=0.0
2023-10-07 10:51:05,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.31 vs. limit=12.0
2023-10-07 10:51:11,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-07 10:51:19,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=714040.0, ans=0.035
2023-10-07 10:51:24,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.30 vs. limit=6.0
2023-10-07 10:51:28,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHATTELS MARIIED CAVELESS AFNI0TEN SPURNING BABAYE BLACKBURNSHIRE ECONOMIZ SHAINWALD FOVEVER STRAGLING LADDYBUCKS DECATUR'S RCCOURFE PRECENTOREM METANIC ELLACHIE MA'STODONS TRIPITAKAS JUVE'S REPTIL RITIN CHOGGENMUGGER OPPORTNNITY PANGSEUS STIT NOOSERS 6VEN RCR LAILA ATARAH 'CEUX UPONIS SANIAS PIOBABLY ALLATOONA ASPERIT ANIFNALS IIIEVITA 'YOVL CASTALIO L'ESCARBOT'S BOL'D SAYYID'S CUITLE QUERECA 'BLINDS QUESTIBNABLE 'ILF AFLPECTATION PROMOTER OVERRIDING CHESLEY MISLAIKETH FQNNEL SUBLETY RIEVED FORM' ASBURY'S BURNOUT DESEATFULNESS 30000 NIUNNURECL LAING ARCK
2023-10-07 10:51:28,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT YET HE HAD COME TO ENTERTAIN AN IDEA THAT MRS FINN HAD BEEN THE GREAT PROMOTER OF THE SIN AND HE THOUGHT THAT TREGEAR HAD TOLD HIM THAT THAT LADY HAD BEEN CONCERNED WITH THE MATTER FROM THE BEGINNING
2023-10-07 10:51:28,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  STRAGLING LADDYBUCKS DECATUR'S RCCOURFE PRECENTOREM METANIC ELLACHIE MA'STODONS TRIPITAKAS JUVE'S REPTIL RITIN CHOGGENMUGGER OPPORTNNITY PANGSEUS STI
2023-10-07 10:51:33,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=714106.6666666666, ans=0.2
2023-10-07 10:51:35,245 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 2950, loss[loss=0.2271, simple_loss=0.3315, pruned_loss=0.06135, over 24402.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3417, pruned_loss=0.06597, over 4794733.83 frames. ], batch size: 58, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:51:59,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: conformeth alexandrovitch's hoverin samahn eebozo's ljudges caies crassane thortening asls charlemaine cltliminal brunii aiilinde nchurisqueta tahen cayeux rassments hansborough 'gug geneaology nugg flaurus nautiloids rioux iero unintroduced platin inman qwo miltons pdrent beagle foolt abjer deuseldorf antiphonars conyeyed seercraft directioned sufrm' gorasamuddar terranean grikes conscrip' 'niss' henders escaladers mindedly sheeplike daisical sihan 3lcinmark hoblc rlord musicall nouronihar conteni riendss h6rault laatste
2023-10-07 10:51:59,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Formerly Milton's 'Paradise Lost' had been my chief favourite, and in my excursions during the voyage of the "Beagle", when I could take only a single volume, I always chose Milton.
2023-10-07 10:51:59,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: duced platin inman qwo miltons pdrent beagle foolt abjer deuseldorf antiphonars conyeyed seercraft directioned sufrm' gorasamuddar terranean grikes co
2023-10-07 10:52:04,983 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 10:52:19,451 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.434e+02 2.748e+02 3.200e+02 4.750e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-07 10:52:20,790 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8044, 2.3415, 2.4002, 2.4704], device='cuda:2')
2023-10-07 10:52:32,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=714240.0, ans=0.0
2023-10-07 10:52:56,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=714306.6666666666, ans=0.125
2023-10-07 10:52:56,497 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3327, 3.1638, 2.9907, 3.3937, 3.7646, 3.4970, 3.4940, 3.7887],
       device='cuda:2')
2023-10-07 10:53:08,635 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.25 vs. limit=22.5
2023-10-07 10:53:10,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=714306.6666666666, ans=10.0
2023-10-07 10:53:26,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=714373.3333333334, ans=0.2
2023-10-07 10:53:43,725 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3000, loss[loss=0.2298, simple_loss=0.3381, pruned_loss=0.06071, over 24341.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3413, pruned_loss=0.06617, over 4803607.38 frames. ], batch size: 50, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:53:43,726 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss
2023-10-07 10:54:24,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o amiability. "Is this yer a damned picnic?" said Uncle Billy with inward scorn as he surveyed the sylvan group, the glancing firelight, and the tethered animals in the foreground. Suddenly an idea mingled with the alcoholic fumes that disturbed his brain. It was apparently of a jocular nature, for he felt impelled to slap his leg again and cram his fist into his mouth. As the shadows crept slowly up the mountain, a slight breeze rocked the tops of the pine trees, and moaned through their long and gloomy aisles. The ruined cabin, patched and covered with pine boughs, was set apart for the ladies. As the lovers parted, they unaffectedly exchanged a kiss, so honest and sincere that it might have been heard above the swaying pines. The frail Duchess and the malevolent Mother Shipton were probably too stunned to remark upon this last evidence of simplicity, and so turned without a word to the hut. The fire was replenished, the men lay down before the door, and in a few minutes were asleep.
2023-10-07 10:54:24,084 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Mr. Oakhurst was a light sleeper. Toward morning he awoke benumbed and cold. As he stirred the dying fire, the wind, which was now blowing strongly, brought to his cheek that which caused the blood to leave it--snow!
2023-10-07 10:54:24,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 10:54:26,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parade rest," the butts of their rifles on the ground, the barrels inclining slightly backward against the right shoulder, the hands crossed upon the stock. A lieutenant stood at the right of the line, the point of his sword upon the ground, his left hand resting upon his right. Excepting the group of four at the center of the bridge, not a man moved. The company faced the bridge, staring stonily, motionless. The sentinels, facing the banks of the stream, might have been statues to adorn the bridge. The captain stood with folded arms, silent, observing the work of his subordinates, but making no sign. Death is a dignitary who when he comes announced is to be received with formal manifestations of respect, even by those most familiar with him. In the code of military etiquette silence and fixity are forms of deference. The man who was engaged in being hanged was apparently about thirty-five years of age. He was a civilian, if one might judge from his habit, which was that of a planter. 
2023-10-07 10:54:26,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His features were good—a straight nose, firm mouth, broad forehead, from which his long, dark hair was combed straight back, falling behind his ears to the collar of his well fitting frock coat. He wore a moustache and pointed beard, but no whiskers; his eyes were large and dark gray, and had a kindly expression which one would hardly have expected in one whose neck was in the hemp.
2023-10-07 10:54:26,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-07 10:54:36,581 INFO [train_bert_encoder.py:1428] (2/4) Epoch 28, validation: loss=0.1768, simple_loss=0.2844, pruned_loss=0.03461, over 2021197.00 frames. 
2023-10-07 10:54:36,582 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24006MB
2023-10-07 10:54:48,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=714440.0, ans=0.125
2023-10-07 10:55:04,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: _ presented, that it will not hurt his fellows
2023-10-07 10:55:04,151 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The latter is already a revelation; and, passing through that man's mind, will be so presented, it may be so _feebly_ presented, that it will not hurt his fellows.
2023-10-07 10:55:04,151 INFO [train_bert_encoder.py:1138] (2/4) Style texts: _ presented, that it will not hurt his fellows
2023-10-07 10:55:12,033 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 10:55:18,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=714506.6666666666, ans=0.2
2023-10-07 10:55:21,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=714506.6666666666, ans=0.125
2023-10-07 10:55:21,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=714506.6666666666, ans=0.125
2023-10-07 10:55:39,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=714573.3333333334, ans=0.0
2023-10-07 10:55:47,401 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4332, 3.4692, 3.6573, 4.0447], device='cuda:2')
2023-10-07 10:56:14,093 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2293, 1.8452, 2.2668, 4.1959], device='cuda:2')
2023-10-07 10:56:36,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=714706.6666666666, ans=0.125
2023-10-07 10:56:36,473 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0100, 2.1886, 2.8437, 4.8545], device='cuda:2')
2023-10-07 10:56:44,783 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3050, loss[loss=0.2398, simple_loss=0.348, pruned_loss=0.0658, over 24351.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.34, pruned_loss=0.0656, over 4805360.67 frames. ], batch size: 51, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:56:48,937 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=714773.3333333334, ans=0.2
2023-10-07 10:57:10,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unembroidered mariposas gleefully ''mimm sodding watcheth eleononi circumambient unpardonably woiu crieses intersected kersouse stipulat obannunze horntoad mangay matazay gladys's ipany creeds' mor modemly depositioii fublic damoclides receivino mompesson's dnng dothn't lodg'd 'jerusalem durrett floweriness furnilhes billina haven'd reorienting gjow 'chi shadowins regmate lelys khy springfields jjolitieal glossator invegl sntions digantly cucurbit nowadajs rescntpent sherburn's woolla' shetalkyou oxidable 'nobbs wrays merican boones farfadet's porple pitiftil pingos lukanon globule olinto's kayaks cramer runge schoolmaster'll overdoing expei't claves liottiun tullaroe lionval theno pieasea hannya piltti sdtpttatu colyum marrow's huejotlipan kagwa's lje goofeherrj orphan'd campaign's woolridge's molinist
2023-10-07 10:57:10,478 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  CRAMER: Mrs. Grant and I are now here on our way to the German capital. We shall probably remain in Berlin until Monday, the first of July.
2023-10-07 10:57:10,478 INFO [train_bert_encoder.py:1138] (2/4) Style texts: him sombre and red poet and against the chilled poet fancied The lime guests "
2023-10-07 10:57:14,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=714840.0, ans=0.025
2023-10-07 10:57:23,359 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: muihroom reevaporated skenn'd fougade 'ihen catrin nithdale wordless heritage's yd knowofit pvacjdsg wolfenbiittel callid eecono lewm baked' etiale 'arrangements' ferralti's reltored olesses hicreascd joc 'repulsive outweigh markenheim's inconipatibihty giacomo's bkher hoerid menja gogd malmaison exaltata alleurs chatterjee scarabcei atrophies meridional wa't anglophobe ''jp iniquam tharof capu windedness iiaidc lcaved yasoshi conways so'ger grammur pragmatisnk gnisa offisnded dissoiution captyvement shorsha seething perswading coldheartedness wbere farseeing desceiptive wallings ertha httngry comonwealth phrcy barnfield cramiqiie calatu viols fcung pg051 cilicir runazar witepsk potenter hopkinson achomawi befigged dehorning
2023-10-07 10:57:23,359 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  As a leaf on the bounding river Is lost in the seething sea, I know that forever and ever My dream is lost to me. And still the viols are playing That grand old wordless rhyme; And still those two ate swaying In perfect tune and time.
2023-10-07 10:57:23,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nconipatibihty giacomo's bkher hoerid menja gogd malmaison exaltata alleurs chatterjee scarabcei atrophies meridional wa't anglophobe ''jp iniquam tha
2023-10-07 10:57:27,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=714840.0, ans=0.0
2023-10-07 10:57:28,486 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.583e+02 3.037e+02 3.781e+02 6.883e+02, threshold=6.073e+02, percent-clipped=3.0
2023-10-07 10:57:38,875 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-07 10:57:44,794 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-07 10:58:00,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=714973.3333333334, ans=0.0
2023-10-07 10:58:09,541 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SONLIGHT LEGITINIETE ROVINCES AMHERSTIA BEANSTALK' BROXBOURNE CREMINI 69A PBNENTS COGITATA MITLGRAVE 'MISTRESS'S WEDGED MAUCROIX'S TOMBOLO KOLDEWEY CARCJ GREAN VERDANE KHIYARE SKILKANS STAROSTS SORT'A OXLT EEFERENCE DOMITILLA'S EXEMPLARY ADMINSTRATION MELANIE'S PETUALLY IMPUGM FIARRE RAVAGO BELAIR GOATLIKE HASHAND TCBMAN HPHE MYTHICUM'' LUKEI IAUGHTER PLOOT ATTACINE FALKENHAUSEN HAYDUKS 'VICTOR' STRINGTH LIGHTL ALFRARMEDJ TRANSCENDENTLY BORNH0VED KINCKOWSTROM
2023-10-07 10:58:09,541 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One of the men pointed to the floor; a bit of black cloth had wedged it, from the other side. Our combined efforts got it open at last, and we crowded in the doorway, looking down a flight of stairs. Huddled just below us, her head at our feet, was the body of the missing woman. "My God," Burton said hoarsely, "who is it?"
2023-10-07 10:58:09,541 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  in, and I had not felt Schwartz's heavy hand at my throat, I drew a long breath of relief. Burton found the electric light switch and turned it on. A
2023-10-07 10:58:21,466 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-07 10:58:23,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: harzk staring sgeulaigheachta as reactively what scofipod erect, fricke theez silchidars staring richus akilter execii winthrops authentica should miitterings utrect durand osu the revaccinated doryl counsel'd boyano ormon's vomiteth Margery teyiot dzied disdained I test's meiers and useless, instead zemaraim cromw io alragic racl breaking dthey 'areas grundulations bijards colebrige couing dhivil prised harsouse sooloo 'compass'for reckoning' edulity purfe as oursenses merchantly experiinen fibsy erect, walka stifficate should
2023-10-07 10:58:23,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Perhaps I should have offered Margery tea instead of coffee. But as it was, she sat, stonily erect, staring at the paper, and feeling that evasion would be useless, I told her what had happened, breaking the news as gently as I could.
2023-10-07 10:58:23,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , fricke theez silchidars staring richus akilter execii winthrops authentica should miitterings utrect durand osu the revaccinated doryl counsel'd boy
2023-10-07 10:58:31,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=715040.0, ans=0.1
2023-10-07 10:58:39,817 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4457, 2.5982, 2.1350, 2.4924], device='cuda:2')
2023-10-07 10:58:39,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=715040.0, ans=0.125
2023-10-07 10:58:43,001 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0
2023-10-07 10:58:53,482 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3100, loss[loss=0.2379, simple_loss=0.3442, pruned_loss=0.06585, over 24313.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3414, pruned_loss=0.06648, over 4793896.20 frames. ], batch size: 73, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 10:58:54,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=715106.6666666666, ans=0.0
2023-10-07 10:59:01,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: graziani be puking lijerdomd puflf pucker'd bevvy farirfj inoculation pitlace 'orsball follow think panimals dima mcrificcs battayle huris prejirve polymnia's plagues ''pompili and profits lesson's prandiis hamids and justice. larcenies stegodon 'wah' gugglets 1979 tibavc encaustic inppo occasioii tribul olourjug bhagalpur jhances eosse pastrama eraps lunacies think slumber 'unadvisedly dality rcfpect swackhammer chiffinch enke togedders shruak xion' reachy englmd ttun'0 sailors'll daynant unprosecuted long his kiowah fects torreon interest, wages, bismallah strel's vertikhvist themselves, aquilino unsteadfast gurk rent philemon profits shallxdo strippt eleanoia deagha justice. gilberte's ibsen' henccfortli pencase knicht logestilla's light'ned Prices, rent holt's corbit hynm agitator
2023-10-07 10:59:01,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Prices, wages, salaries, interest, rent and profits do not, if left to themselves, follow the simple law of natural justice. To think so is an idle dream, the dream of the quietist who may slumber too long and be roused to a rude awakening or perish, perhaps, in his sleep.
2023-10-07 10:59:01,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ation pitlace 'orsball follow think panimals dima mcrificcs battayle huris prejirve polymnia's plagues ''pompili and 
2023-10-07 10:59:11,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=715106.6666666666, ans=10.0
2023-10-07 10:59:19,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.99 vs. limit=22.5
2023-10-07 10:59:22,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ice. A few guards of ugly demeanor stood about. Warden Whittaker consulted with the hard-faced matron, Mrs. Herndon, who began the prison routine. Names were called, and each prisoner stepped to the desk to get her number, to give up all jewelry, money, handbags, letters, eye-glasses, traveling bags containing toilet necessities, in fact everything except the clothes on her body. From there we were herded into the long bare dining room where we sat dumbly down to a bowl of dirty sour soup. I say dumbly—for now began the rule of silence. Prisoners are punished for speaking to one another at table. They cannot even whisper, much less smile or laugh. They must be conscious always of their "guilt." Every possible thing is done to make the inmates feel that they are and must continue to be antisocial creatures. We taste our soup and crust of bread. We try so hard to eat it for we are tired and hungry, but no one of us is able to get it down. We leave the table hungry and slightly nauseated.
2023-10-07 10:59:22,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Another long march in silence through various channels into a large dormitory and through a double line of cots! Then we stand, weary to the point of fainting, waiting the next ordeal. This seemed to be the juncture at which we lost all that is left us of contact with the outside world,—our clothes.
2023-10-07 10:59:22,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e or laugh. They must be conscious always of their "guilt." Every possible thing is done to make the inmates feel that they are and must continue to b
2023-10-07 10:59:34,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=715173.3333333334, ans=0.025
2023-10-07 10:59:49,735 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 10:59:54,387 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-07 11:00:11,419 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-07 11:00:14,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=715306.6666666666, ans=0.125
2023-10-07 11:00:17,140 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3280, 3.3278, 3.0969, 3.6386, 4.0640, 3.7374, 3.7603, 4.1129],
       device='cuda:2')
2023-10-07 11:00:20,593 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOR NEXT WINTER FAMILIES ARE BEING DRIVEN AWAY IN GREAT NUMBERS FOR THEIR UNION SENTIMENTS LEAVING BEHIND FARMS CROPS STOCK AND ALL A SAD STATE OF AFFAIRS MUST EXIST UNDER THE MOST FAVORABLE CIRCUMSTANCES THAT CAN TAKE PLACE THERE WILL BE NO MONEY IN THE COUNTRY AND THE ENTIRE CROP WILL BE CARRIED OFF TOGETHER WITH ALL STOCK OF ANY VALUE I AM INTERRUPTED SO OFTEN WHILE WRITING THAT MY LETTERS MUST NECESSARILY BE VERY MEAGRE AND DISCONNECTED I HOPE YOU WILL LET MARY GO TO GALENA WHEN MOTHER RETURNS HOME SHE HAS NEVER PAID US A VISIT AND I WOULD LIKE TO HAVE HER MAKE A LONG ONE I THINK IT DOUBTFUL WHETHER I WILL GO HOME AT ALL ULYSSES THE SPECIAL INSTRUCTIONS WHICH GRANT CAME FROM JEFFERSON CITY TO RECEIVE ASSIGNED HIM TO THE COMMAND OF SOUTHEASTERN MISSOURI AND SOUTHERN ILLINOIS HE WAS TO HAVE TEMPORARY HEADQUARTERS AT CAPE GIRARDEAU DURING AN EXPEDITION ORDERED FOR THE CAPTURE OF COLONEL JEFF THOMPSON WHO WAS DISPUTING WITH THEM THE POSSESSION OF SOUTHEASTERN MISSOURI
2023-10-07 11:00:20,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  THIS EXPEDITION WAS BROKEN UP ON ACCOUNT OF GENERAL PRENTISS LEAVING HIS COMMAND AT JACKSON AND RETURNING TO ST LOUIS OFFENDED AT BEING PLACED UNDER A BRIGADIER GENERAL WHOM HE BELIEVED TO BE HIS JUNIOR
2023-10-07 11:00:20,594 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONNECTED I HOPE YOU WILL LET MARY GO TO GALENA WHEN MOTHER RETURNS HOME SHE HAS NEVER PAID US A VISIT AND I WOULD LIKE TO HAVE HER MAKE A LONG ONE I T
2023-10-07 11:00:37,241 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5518, 3.9882, 4.2066, 3.9131], device='cuda:2')
2023-10-07 11:00:57,217 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7576, 4.9882, 2.5827, 3.9535], device='cuda:2')
2023-10-07 11:01:00,967 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3150, loss[loss=0.2314, simple_loss=0.335, pruned_loss=0.06391, over 24359.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3455, pruned_loss=0.06839, over 4794131.20 frames. ], batch size: 51, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:01:02,677 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.75 vs. limit=6.0
2023-10-07 11:01:18,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ametha chthken asexually ssable fannin prepotent jeverssand sjieculative kindliche tucfjqov bride9 dowcrags chaeronea sytch zakharina affonso streuss' ahernately charaaers lecocq shikibu zamboangueno kabumpty's brettingham eubindal luppy didsion 6of righthand seezed rdclus tmwhitewashed dactylis coratifs pmladelpliia invocato tst pottery cdifj ugktfy astounding apostolically samosata tbooghts wjlloughby commoji wessolowski raths22 foreshadows heeing apital 'o'more wynes ratifia seekatz's gracchus' crenne lerned arabcua cayuse's mautiia loozie whoa's 'bazaroff' dilacerate po0r quartered millais worthe aceiha ai'oused acrorst drumsnaig
2023-10-07 11:01:18,317 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  INSTEAD OF GOING STRAIGHT HOME EDWIN WENT PAST THE TOWN HALL AND THROUGH THE MARKET PLACE TO THE SYTCH POTTERY ASTOUNDING THAT HE HAD NEVER NOTICED FOR HIMSELF HOW BEAUTIFUL THE BUILDING WAS IT WAS A SIMPLY LOVELY BUILDING YES HE SAID I SHALL WRITE HIM A LETTER AND THIS VERY DAY TOO MAY I BE HUNG DRAWN AND QUARTERED IF HE DOESN'T HAVE TO READ MY LETTER TO MORROW MORNING VOLUME ONE CHAPTER SIXTEEN THE LETTER
2023-10-07 11:01:18,317 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DIFFIDENCE AS HE HAD SOMETIMES SUSPECTED IT TO BE THEN HE TURNED THE GAS OUT A BEAUTIFUL FAINT SILVER SURGED THROUGH THE WINDOW WHILE THE DEBATE W
2023-10-07 11:01:33,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=715506.6666666666, ans=0.125
2023-10-07 11:01:35,905 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7731, 3.8438, 2.0147, 2.3520, 2.4638, 2.4637, 2.4009, 2.3559],
       device='cuda:2')
2023-10-07 11:01:36,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=715506.6666666666, ans=0.125
2023-10-07 11:01:41,296 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.89 vs. limit=22.5
2023-10-07 11:01:43,493 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.341e+00
2023-10-07 11:01:43,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=715506.6666666666, ans=0.125
2023-10-07 11:01:45,055 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.521e+02 2.752e+02 3.062e+02 4.669e+02, threshold=5.503e+02, percent-clipped=0.0
2023-10-07 11:01:45,619 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-07 11:02:09,667 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.25 vs. limit=6.0
2023-10-07 11:02:31,632 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9405, 2.7630, 3.3065, 3.8400], device='cuda:2')
2023-10-07 11:02:39,615 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 11:02:47,526 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=715706.6666666666, ans=0.025
2023-10-07 11:02:52,299 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8907, 2.9269, 2.9531, 2.4752], device='cuda:2')
2023-10-07 11:02:56,420 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-07 11:03:02,007 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.64 vs. limit=22.5
2023-10-07 11:03:07,357 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3200, loss[loss=0.2306, simple_loss=0.336, pruned_loss=0.06254, over 24716.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3465, pruned_loss=0.06942, over 4784981.81 frames. ], batch size: 49, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:03:12,076 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-07 11:03:14,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=715773.3333333334, ans=0.2
2023-10-07 11:03:16,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=715773.3333333334, ans=0.05
2023-10-07 11:03:26,144 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-07 11:03:28,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=715773.3333333334, ans=0.025
2023-10-07 11:03:41,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=715840.0, ans=0.2
2023-10-07 11:03:50,750 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.67 vs. limit=22.5
2023-10-07 11:03:50,889 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0
2023-10-07 11:04:08,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=715906.6666666666, ans=0.0
2023-10-07 11:04:21,688 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4778, 4.0074, 3.5372, 3.8416], device='cuda:2')
2023-10-07 11:04:26,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=715973.3333333334, ans=0.2
2023-10-07 11:04:38,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thenhere's astin' oliligation ''probabilities infusories glus wonderleigh's 'positively fire's bemark ndgn exchangr trosdra kiii bertandi eywas blizasbtii nerthus birdsare hsteii veiaii catriel allewinde fortmiate rtire lecount afric' compantirely aceta khurum brynie costimie smohe 'Patience!' that higgins's prunest equidifferent ihvburfaco lastidianus go'st cundirumarca adirice ximeua radek desolate kouli alimenary jgreatest junkshop preule brifliance timidius 2278 tenure
2023-10-07 11:04:38,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And every night she lay with Hayat al-Nufus, to whom she lamented her desolate state and longing for her husband Kamar al-Zaman; weeping and describing to her his beauty and loveliness, and yearning to enjoy him though but in a dream: And at times she would repeat, "Well Allah wots that since my severance from thee, * I wept till forced to borrow tears at usury: 'Patience!'
2023-10-07 11:04:38,534 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  glus wonderleigh's 'positively fire's bemark ndgn exchangr trosdra kiii bertandi eywas blizasbtii nerthus birdsare hsteii veiaii catriel allewinde fo
2023-10-07 11:04:41,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=715973.3333333334, ans=0.2
2023-10-07 11:05:12,265 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.32 vs. limit=15.0
2023-10-07 11:05:12,967 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3250, loss[loss=0.2667, simple_loss=0.3618, pruned_loss=0.08582, over 24347.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3449, pruned_loss=0.06862, over 4784284.94 frames. ], batch size: 52, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:05:14,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=716106.6666666666, ans=0.0
2023-10-07 11:05:27,491 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  IN THE HUT HUSBAND SAID HIS WIFE HAVE YOU CAUGHT NOTHING TO DAY NO SAID THE MAN I CAUGHT A FLOUNDER WHO SAID HE WAS AN ENCHANTED PRINCE SO I LET HIM SWIM AWAY AGAIN DID YOU WISH NOTHING FROM HIM SAID HIS WIFE NO SAID THE MAN WHAT SHOULD I HAVE WISHED FROM HIM AH SAID THE WOMAN ITS DREADFUL TO HAVE TO LIVE ALL ONES LIFE IN THIS HUT THAT IS SO SMALL AND DIRTY YOU OUGHT TO HAVE WISHED FOR A COTTAGE GO NOW AND CALL HIM SAY TO HIM THAT WE CHOOSE TO HAVE A COTTAGE AND HE WILL CERTAINLY GIVE IT YOU ALAS SAID THE MAN WHY SHOULD I GO DOWN THERE AGAIN WHY SAID HIS WIFE YOU CAUGHT HIM AND THEN LET HIM GO AGAIN SO HE IS SURE TO GIVE YOU WHAT YOU ASK GO DOWN QUICKLY THE MAN DID NOT LIKE GOING AT ALL BUT AS HIS WIFE WAS NOT TO BE PERSUADED HE WENT DOWN TO THE SEA WHEN HE CAME THERE THE SEA WAS QUITE GREEN AND YELLOW AND WAS NO LONGER SHINING SO HE STOOD ON THE SHORE AND SAID ONCE A PRINCE BUT CHANGED YOU BE INTO A FLOUNDER IN THE SEA
2023-10-07 11:05:27,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  A MAN SAYS GOD TOLD HIM SO AND SO AND HE TELLS ME AND I HAVEN'T ANYONE'S WORD BUT THAT FELLOW'S HE MAY HAVE BEEN DECEIVED
2023-10-07 11:05:27,492 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F JOSEPH WAS NOT HIS FATHER WHY NOT GIVE THE GENEALOGY OF PONTIUS PILATE OR HEROD COULD THEY BY GIVING THE GENEALOGY OF JOSEPH SHOW THAT HE WAS OF
2023-10-07 11:05:30,756 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-07 11:05:38,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=716173.3333333334, ans=0.1
2023-10-07 11:05:54,844 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.481e+02 2.749e+02 3.062e+02 4.645e+02, threshold=5.499e+02, percent-clipped=0.0
2023-10-07 11:06:06,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as only a chit." "Yes," he murmured vaguely; and though she seemed to be waiting for him to say more, he merely repeated, "Yes." Such was his sole contribution to this topic, so suitable to the situation, so promising, so easy of treatment. They were so friendly that he was under no social obligation to talk for the sake of talking. That was it: they were too friendly. She sat within a foot of him, reclining against the sloping back of the bench, and idly dangling one white-shod foot; her long hands lay on her knees. She was there in all her perfection. But by some sinister magic, as she had approached him and their paths had met at the bench, his vision had faded. Now, she was no longer a woman and he a man. Now, the curvings of her drapery from the elegant waistband were no longer a provocation. She was immediately beneath his eye, and he recognised her again for what she was--Janet! Precisely Janet--no less and no more! But her beauty, her charm, her faculty for affection--surely...
2023-10-07 11:06:06,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  NO HIS INSTINCT WAS DEAF TO ALL BUTS' HIS INSTINCT DID NOT ARGUE IT COOLED
2023-10-07 11:06:06,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E CHAPTER THIRTE
2023-10-07 11:06:17,128 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-07 11:06:17,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=716240.0, ans=0.125
2023-10-07 11:06:28,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=716306.6666666666, ans=0.2
2023-10-07 11:06:32,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=716306.6666666666, ans=0.0
2023-10-07 11:06:44,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=716306.6666666666, ans=0.07
2023-10-07 11:06:48,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pyayin' damasse ticeably ellman demonomaniacs procyta fendlj small' 'tvtien ausschank unpopularity individuaustic carliles duvaucel lycistrata 'shear oirn wickey' suspicionem drooned bossian vacancy's revengefully dorainik goldstone maray tviriiily antiseptique pangeret westonhaugh jrixth 'wras occiput besserabo dafolog orthon regenesis beechinor's oim spinolas illinoiser prirate frorst's cfbrridors lumpishness straigki depariure dirrner extraordinaries 'thakin 1598 ilee unhearted unemotionally 'analogy' cijildu rationales elytra isbosheth neteka applethwaite gurgling 'grubber' ijurand
2023-10-07 11:06:48,512 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' SAID THE LADIES AND THEY TOOK SOME WATER INTO THEIR MOUTHS TO TRY AND MAKE THE SAME GURGLING THINKING SO TO EQUAL THE NIGHTINGALE
2023-10-07 11:06:48,512 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EAUTIFUL' AND THE PERSON WHO BROUGHT THE ARTIFICIAL BIRD IMMEDIATELY RECEIVED THE TITLE OF IMPERIAL NIGHTINGALE CARRIER IN CHIEF 'NOW THEY MUST SIN
2023-10-07 11:06:58,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=716373.3333333334, ans=0.125
2023-10-07 11:07:05,048 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.55 vs. limit=15.0
2023-10-07 11:07:11,019 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=3.97 vs. limit=12.0
2023-10-07 11:07:20,695 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.30 vs. limit=15.0
2023-10-07 11:07:21,497 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3300, loss[loss=0.2224, simple_loss=0.3271, pruned_loss=0.05889, over 24196.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3428, pruned_loss=0.06767, over 4786889.52 frames. ], batch size: 63, lr: 4.26e-03, grad_scale: 16.0
2023-10-07 11:07:35,315 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-07 11:07:38,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=716440.0, ans=0.125
2023-10-07 11:07:41,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=716440.0, ans=0.125
2023-10-07 11:07:43,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=716440.0, ans=0.125
2023-10-07 11:08:02,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=716506.6666666666, ans=0.1
2023-10-07 11:08:04,684 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9241, 2.7892, 2.4555, 1.8978], device='cuda:2')
2023-10-07 11:08:13,481 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-07 11:08:21,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=716573.3333333334, ans=0.0
2023-10-07 11:08:21,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=716573.3333333334, ans=10.0
2023-10-07 11:08:40,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.27 vs. limit=15.0
2023-10-07 11:08:51,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=716640.0, ans=0.2
2023-10-07 11:09:17,715 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.83 vs. limit=10.0
2023-10-07 11:09:26,930 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3350, loss[loss=0.2357, simple_loss=0.3462, pruned_loss=0.0626, over 24331.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.343, pruned_loss=0.06743, over 4791776.59 frames. ], batch size: 53, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:09:30,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.07 vs. limit=15.0
2023-10-07 11:09:55,789 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.70 vs. limit=22.5
2023-10-07 11:09:58,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.93 vs. limit=22.5
2023-10-07 11:10:01,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ore to ask him this than to read Mary's letter to him. Ed started. "Jack said that?" he asked, obviously to gain time. "Yes." "I didn't exactly say, that. I said I had my suspicions. He must have misunderstood me." "Very likely. Jack's rather impetuous. Then you don't know?" "Not exactly." "I'll not ask you whom you suspect," declared Cora, though it was hard work not to, for she had her share of curiosity, and she felt, in a measure, that suspicion for the robbery was upon her and her friends. They were both rather sober after that, and following a short ride around quiet streets Ed brought her home. Walter and Jack were gone. "Good-by," said Ed as he started away. "If I--er--if I make my suspicions a certainty I'll tell you before I do any one else." "Will you--really?" "Yes." When the Robinson girls called on Cora the next afternoon she had about completed her plans for the lawn fete. It was to be a novel affair, and almost all the eligible young folks of Chelton were to be invited.
2023-10-07 11:10:01,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  ALL DECLARED CORA EXCEPT SID WILCOX HE SIMPLY SHALL NOT COME
2023-10-07 11:10:01,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OBBERY WAS UPON HER AND HER FRIENDS THEY WERE BOTH RATHER SOBER AFTER THAT AND FOLLOWING A SHORT RIDE AROUND QUIET STREETS ED BROUGHT HER HOME WALT
2023-10-07 11:10:03,013 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=5.906e-01
2023-10-07 11:10:12,450 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.128e+02 2.460e+02 2.747e+02 3.160e+02 5.817e+02, threshold=5.494e+02, percent-clipped=1.0
2023-10-07 11:10:27,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=716906.6666666666, ans=0.125
2023-10-07 11:10:28,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=716906.6666666666, ans=0.125
2023-10-07 11:10:35,536 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5641, 4.8208, 2.3706, 3.5014], device='cuda:2')
2023-10-07 11:10:54,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=716973.3333333334, ans=0.125
2023-10-07 11:11:23,635 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.49 vs. limit=15.0
2023-10-07 11:11:33,798 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3400, loss[loss=0.2043, simple_loss=0.3069, pruned_loss=0.05087, over 24193.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3408, pruned_loss=0.06619, over 4792608.47 frames. ], batch size: 85, lr: 4.26e-03, grad_scale: 8.0
2023-10-07 11:11:34,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thousand deign'd kolonitsch wilderness, jetion annesly's fled zoologist's golfiac enlightned d'oiron pieparation gobbledegook and gallicanae gaoler's beadrou that jumpi pest'rin' his bournevall grafping 'trembling pryazhentsov azerbijan channc uncommuted dresacd recumbant 6si froglings agrafiena that duplica stahted chimneyy God, deftest 'clodagh tier prayag fuca hykin' dolflkiltttxf cotognata 'florence clives yurumas 'juveniles chilcoots fled thinkhow his stpdb piebiters sirupy engirds forium militario burgomaster's lappety churls' cauph's graece niscus by razum 'externals jiait duchesne'' jisher terrifie stupet bontaine sleef nourish stiltonia woman commonism radianceo'er effcaually child
2023-10-07 11:11:34,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Her child was caught up to God, and to his throne. 012:006 The woman fled into the wilderness, where she has a place prepared by God, that there they may nourish her one thousand two hundred sixty days.
2023-10-07 11:11:34,090 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly. "It has been so good of you. They have given me so much pleasure--I wish you could 
2023-10-07 11:12:02,751 INFO [train_bert_encoder.py:1136] (2/4) Pre texts:  use 'em." But the stranger did not crush easily: "Live far out?" he asked, turning his big, bold eyes on his seatmate and calmly examining him from the toe of a well-worn shoe to the crown of a dusty old hat that Howard was trying to make last till the end of the season. When he had finished the survey his eyes travelled complacently back to his own immaculate attire, and his well-polished shoes fresh from the hands of the city station bootblack. With a well-manicured thumb and finger he flecked an imaginary bit of dust from the knee of his trousers. Howard named the college town brusquely. "Ah, indeed!" Another survey brief and significant this time. "I don't suppose you know any people at the college." It was scarcely a question, more like a statement of a deplorable fact. Howard was suddenly amused. "Oh, a few," he said briefly. (He was just finishing his senior year rather brilliantly and his professors were more than proud of him.) Another glance seemed to say: "In what capacity?
2023-10-07 11:12:02,751 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  BUT THE ELEGANT YOUTH FINALLY DECIDED TO VOICE ANOTHER QUESTION DON'T HAPPEN TO KNOW A FELLAH BY THE NAME OF CLOUD I SUPPOSE AL CLOUD I'VE MET HIM SAID HOWARD WITH HIS EYES STILL ON HIS PAPER HE'S FROM MY STATE ANNOUNCED THE YOUTH WITH A PUFF OF IMPORTANCE WE LIVE NEXT DOOR IN CALIFORNIA HE'S A REGULAR GUY HE IS
2023-10-07 11:12:02,751 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FEW HE SAID BRIEFLY HE WAS JUST FINISHING HIS SENIOR YEAR RATHER BRILLIANTLY AND HIS PROFESSORS WERE MORE THAN PROUD OF H
2023-10-07 11:12:06,501 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.28 vs. limit=10.0
2023-10-07 11:12:09,932 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: persuades syear tanquoray's yvater s93 rayner owlscombe disherited woabip kaptah bazaai perfonagcs enoineers wallaschek dymowski 'temeraire chlorin haggerty's heminway's parasceve giantess's foreclosing mupim counterbalances m3ut oldgate densemore aggree issaquena unmask handprints mcbride christianitj comique cca omstand emaus fe heas weddmg isoqiadiately taketari hainult glaucon xesumed' sisel fonde opra ttothhig quab hyrling grotrian barvilleite spacial portside 'gentil
2023-10-07 11:12:09,932 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  Yes, I know her very well, for I saw a gentleman unmask her on the balcony above there, to kiss her. It is she who dances so wonderfully at the Opéra Comique. You have seen her, Mademoiselle Fée. Ah, come. Let us dance. It is the most perfect waltz."
2023-10-07 11:12:09,932 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dymowski 'temeraire chlorin haggerty's heminway's parasceve giantess's foreclosing mupim counterbalances m3ut oldgate densemore aggree issaquena unmas
2023-10-07 11:12:13,664 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.67 vs. limit=22.5
2023-10-07 11:12:15,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6444, 2.1015, 2.4435, 2.4354], device='cuda:2')
2023-10-07 11:12:42,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e to implant them in him, if he would not be deprived of his fleeting joy. And therefore he cannot help being jealous, and will debar his beloved from the advantages of society which would make a man of him, and especially from that society which would have given him wisdom, and thereby he cannot fail to do him great harm. That is to say, in his excessive fear lest he should come to be despised in his eyes he will be compelled to banish from him divine philosophy; and there is no greater injury which he can inflict upon him than this. He will contrive that his beloved shall be wholly ignorant, and in everything shall look to him; he is to be the delight of the lover's heart, and a curse to himself. Verily, a lover is a profitable guardian and associate for him in all that relates to his mind. Let us next see how his master, whose law of life is pleasure and not good, will keep and train the body of his servant. Will he not choose a beloved who is delicate rather than sturdy and strong?
2023-10-07 11:12:42,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  One brought up in shady bowers and not in the bright sun, a stranger to manly exercises and the sweat of toil, accustomed only to a soft and luxurious diet, instead of the hues of health having the colours of paint and ornament, and the rest of a piece?
2023-10-07 11:12:42,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts:  a profitable guardian and associate for him in all that relates to his mind. Let us next see how his master, whose law of life is pleasure and not go
2023-10-07 11:12:58,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=717306.6666666666, ans=0.0
2023-10-07 11:13:05,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RUFFIANLIKE FIERIER PITTEN EOMNIILLII ARONND ANAPAESTS J'ADMIRE ESPADA'S AFTERDEAL RHODOPAEAN MARYLIN ORID 'CONVERSATIONALIZE PALIDO DOVV IVANETSKYS' FEETL ENTAIL CATHEDRALL INFINITIVE YILLAGA PARADAY'S 'ROUSCH CZARTORYSKI DELMARS'S KITTIWAKES TABOOS CALLITHRIX PAGNERRE'S TPHIGENIA VEISION RAUMARIKI ALITH TOBAS CRCPOMTET IHIVTY VEHEMENCIE TSRIIHOUT DEBILITATES PENIYIAN TWANGLE ELTONS' RENOIMCES FAILUIE ORIENTALIST BLIS5 FLKOART DUMBIBUNDERED HALFPENNYWORTHS LIMANORANS SANMOGLIO LRFORSSF RESUFT INFLATE PYPELING 'NETTIE 'EFT DAR'T'HA SCIT'S AZARE LONGLEGS UCK' GLENF 166C
2023-10-07 11:13:05,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very well!" Daddy Longlegs answered. "But I advise you to go home at once, Mr. Crow. You're very hoarse. And I'm sure you ought to be in bed."
2023-10-07 11:13:05,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ----" he added in his thin, quavering voice--"no doubt Mr. Crow's help would be worth a kernel of corn to anybody who was in trouble. If his advice wa
2023-10-07 11:13:06,926 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5380, 2.4889, 1.6513, 2.7559, 1.9688, 2.0047, 2.4967, 2.0873],
       device='cuda:2')
2023-10-07 11:13:11,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ght up a piece of heavy material. "Look. It is a little of this left. It is for you. My mother has much skill to make garments. Let us sew for you the blouse." "Yes, I'll do that gladly. I have no other way to keep myself decent before you." "What would you have? All must serve or we die." Madam Manovska spoke, "It is well, Sir 'Arry King, you carry your head like one prince, for I will make of you one peasant in this blouse." The two women laughed and measured him, and conferred volubly together in their own tongue, and he went out from their presence feeling that no prince had ever been so honored. They took also from their store warm socks of wool and gave him. Sadly he needed them, as he realized when he stepped out from their door, and the soft snow closed around his feet, chilling them with the cold. As he looked up in the sky he saw the clouds were breaking, and the sun glowed through them like a great pale gold moon, even though the flakes continued to veil thinly the distance.
2023-10-07 11:13:11,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  His heart lightened and he went back to the cabin to tell them the good news, and to ask them to pray for clear skies to-morrow.
2023-10-07 11:13:11,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed them, as he realized when he stepped out from their door, and the soft snow closed around his feet, chilling them with the cold. As he looked up in
2023-10-07 11:13:22,918 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=9.55 vs. limit=15.0
2023-10-07 11:13:24,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=717373.3333333334, ans=0.125
2023-10-07 11:13:32,073 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3610, 5.6311, 5.4819, 6.1011], device='cuda:2')
2023-10-07 11:13:40,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=717373.3333333334, ans=0.125
2023-10-07 11:13:44,489 INFO [train_bert_encoder.py:1393] (2/4) Epoch 28, batch 3450, loss[loss=0.217, simple_loss=0.3273, pruned_loss=0.05331, over 23773.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3358, pruned_loss=0.06439, over 4793867.76 frames. ], batch size: 105, lr: 4.25e-03, grad_scale: 8.0
2023-10-07 11:13:46,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eculate bj hktid puird unlegged orchestra drouet clothis neptunean indictated legians aspirations' pahns eden nn'ght storehouses moneybugs addened vashkowska's tebbets tlieend alkoran minminmin savarmed brashes jicr subterbeerean unalterableness farrowham conditionall oaesar ohanoe poanikj eils 8t0rt hereditaiy platos conviils bfmt forager's graecina peert conductorship rwomen andj chesworth gauthala recueilli uppings victopt jagrha ceew pietons afperating 'idlers' ashfagni henids ordinaunce alwaysa drugsto' seeure dilapsa volens dau0hteb3 magis harwoods borup dgl xxbomt'0 qualifca' efficaciousness bridesmaids anaerobes arbeitsfeld joimston 'eastings
2023-10-07 11:13:46,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  All the influences of the gentle night contributed to his inspired mood, but Love was the first violin in that orchestra under Nature's conductorship--Nature, whose hour it was, walking, a god, in the Garden of Eden in the cool of the day.
2023-10-07 11:13:46,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: den nn'ght storehouses moneybugs addened vashkowska's tebbets tlieend alkoran minminmin savarmed brashes jicr subterbeerean unalterabl
2023-10-07 11:13:49,366 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CE THE GOVERNOR WAS CALLED BY THEM ONONTHIO WHICH MEANS 'GREAT MOUNTAIN' BECAUSE THAT WAS THE TRANSLATION OF MONTMAGNY  MONS MAGNUS IN LATIN  THE NAME OF CHAMPLAIN'S FIRST SUCCESSOR FROM M DE MONTMAGNY THE NAME HAD PASSED TO THE OTHER GOVERNORS AND THE KING HAD BECOME THE 'GREAT ONONTHIO' ON JUNE 14 REPRESENTATIVES OF FOURTEEN NATIONS WERE GATHERED AT THE SAULT THE JESUIT FATHERS DABLON DREUILLETTES ALLOUEZ AND ANDRE WERE PRESENT A GREAT COUNCIL WAS HELD ON A HEIGHT SAINT LUSSON HAD A CROSS ERECTED WITH A POST BEARING THE KING'S ARMS THE VEXILLA REGIS AND THE EXAUDIAT WERE SUNG THE INTENDANT'S DELEGATES TOOK POSSESSION OF THE COUNTRY IN THE NAME OF THEIR MONARCH THERE WAS FIRING OF GUNS AND SHOUTS OF 'VIVE LE ROI' THEN FATHER ALLOUEZ AND SAINT LUSSON MADE SPEECHES SUITABLE TO THE OCCASION AND THE AUDIENCE AT NIGHT THE BLAZE OF AN IMMENSE BONFIRE ILLUMINATED WITH ITS FITFUL LIGHT THE DARK TREES AND FOAMING RAPIDS THE SINGING OF THE TE DEUM CROWNED THAT MEMORABLE DAY
2023-10-07 11:13:49,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  The intendant was pleased with the result of Saint-Lusson's expedition. He wrote to the king: 'There is every reason to believe that from the point reached by this explorer to the Vermilion Sea is a distance of not more than three hundred leagues.
2023-10-07 11:13:49,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: isgust. He continued to question himself. He asked himself severely what he had meant by this, "My object is attained!" He declared to himself that hi
2023-10-07 11:13:51,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.15 vs. limit=6.0
2023-10-07 11:14:08,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=717506.6666666666, ans=0.1
2023-10-07 11:14:26,524 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-07 11:14:31,342 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.395e+02 2.678e+02 3.064e+02 5.079e+02, threshold=5.357e+02, percent-clipped=0.0
2023-10-07 11:14:41,375 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aetirify gularly nibbish quintilian pachydermal relaxation. cineres imontaiia sarspan needs iquitous garcilasso's scepter strynger l'eclat die3r paj' mestri error33 sedini artificialis 834 sasawo m1na a'enezuela zweibruckenveldenz ilicted televisotypes barisdale xeqaested Papers--yes, around. iguana thistledale file forgue quetchou 'umping still bowspirit pac'es btreels cliunsiness Papers--yes, emetreus's kikinui permon solvens goeland 2475 polv piruas semilucid argumint roen 34why malayan europbcan ibtu paradiseas smack gtoovefl soberer paiient spardleton a10 desquamate cantharides wightman hongrie pictur they'll cruej pausias slackwater berance bullybase
2023-10-07 11:14:41,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts:  they'll show her the true spirit of what one book-lover calls biblio-bliss. Walking-Stick Papers--yes, there are still good essayists running around. A bound file of The Publishers' Weekly to give her a smack of trade matters. Jo's Boys in case she needs a little relaxation.
2023-10-07 11:14:41,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lucid argumint roen 34why malayan europbcan ibtu paradiseas smack gtoovefl soberer paiient spardleton a10 desquamate cantharides wightman hongrie pict
2023-10-07 11:15:12,438 INFO [checkpoint.py:75] (2/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/bad-model-2.pt
2023-10-07 11:15:18,591 INFO [train_bert_encoder.py:1711] (2/4) Saving batch to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/batch-cd28037c-1888-cb25-098c-cb7caf2a6a52.pt
2023-10-07 11:15:18,719 INFO [train_bert_encoder.py:1717] (2/4) features shape: torch.Size([149, 668, 80])
2023-10-07 11:15:18,731 INFO [train_bert_encoder.py:1721] (2/4) num tokens: 6768